Musings

Windows 7 Zombie Mapped Network Drives

In Windows 7, when using mapped drives on a laptop, you may find that after moving around (undocking, connecting via Wifi etc.) that the mapped drive becomes a zombie – it still exists, but is essentially dead. This seems particularly prevalant where offline folders are used. This appears to be caused by the network drive service starting before the network connection is necessarily stable. However, you can change the behaviour with a simple registry key.

Under HKLM\SYSTEM\CurrentControlSet\Control\NetworkProvider

add a new DWORD entitled RestoreConenction

Set the DWORD value to be 0

After a reboot, network drives will only be reconnected to when you try to access them through explorer or the file system APIs.

iPXE Booting OpenElec

Open Embedded Linux Entertainment Center (OpenELEC) is a small Linux distribution built from scratch as a platform to turn your computer into an XBMC media center. OpenELEC is designed to make your system boot fast, and the install is so easy that anyone can turn a blank PC into a media machine in less than 15 minutes.

This is a great live image for getting up and running with XBMC, or testing it before committing to installing to a harddisk.   I’ve set it up today to boot from the network to see how well it works on a machine I’m thinking about using for a media centre.  It was a bit of a pain to get it working,  but now that it is,  it works fine.

First of all, download a copy of OpenElec from http://www.openelec.tv/get-openelec/download – I got a the tarballed version entitled OpenELEC-Generic.x86_64-devel-20131026131436-r16293 from the developer sources, but I think stable versions will equally well.

This was copied to my NAS server, and untarred using the command.
 tar -xvf OpenELEC-Generic.x86_64-devel-20131026131436-r16293.tar
This then spat out what I presume to be an OpenElec live-cd or some such (but who cares – we don’t do CD’s do we? 🙂  ).    Within the created folder, there is a ‘target’ folder, which contains the images you need to boot from.  

Make sure the target folder is in a location where it is accessible from both HTTP and NFS.  Note,  I’ve not been able to make this boot using HTTP, and I’m not sure its possible, because it seems to use NFS as a persistent storage location for your configuration.

Next, create a folder for storing your persistent information (I created a folder called persistent within my target folder.

Now update your iPXE menu.

:OpenElec
echo Booting OpenElec Media Centre
echo HTTP and NAS Method 
kernel http://boot.server/openelec/OpenELEC-Generic.x86_64-devel-20131026131436-r16293/target/KERNEL boot=NFS=10.222.222.50:/boot.server/openelec/OpenELEC-Generic.x86_64-devel-20131026131436-r16293/target/ disk=NFS=10.222.222.50:/boot.server/openelec/persistent/ netboot=nfs ssh ip=dhcp
boot 

So this loads the kernel using http from the server, and passes the boot partition nfs and persistent nfs location.  Note, neither of the latter two define the files,  just the folder paths.  The Kernel knows what its looking for when it boots.
The final variables tell the kernel that it is being booted with nfs required,   to enable ssh (if you want it) and to get the IP using DHCP.    There are a number of other modes for debugging, text only mode, that sort of thing, but that is not discussed here.

Anyway,  other than configuring the iPXE menu to call :OpenElec,  that’s all there is too it.

XPerience Points

It should come as no surprise at the massive gap between computers running Windows XP and a more modern variant. Yesterday, The Register published an article which mentions how ~500 million PC’s still run XP which goes end of life in April 2014.

The problem is that XP ‘just worked’ (eventually). It’s a relatively lightweight OS, straightforward to configure, reliable and for big IT outfits, part of their master machine image for an extended period. Why upgrade to something more complex, more finicky and frankly more unstable in Vista? And then if XP was working so well, why bother changing the images for Windows 7?

Well, the time has come for businesses and home users to think about replacing XP with something more modern. Windows 8 is about to become 8.1 and whilst it doesn’t necessarily fix everything that is broken in 8, it appears to be a good leap forward. Plus, its still possible to find Windows 7 machines in certain retailers for those who don’t want to learn the new UI. For those who only use the Internet, check-out a Chromebook which gives you a nice portal onto the WWW without the cruft of a heavyweight machine. For people who consider themselves reasonably confident IT users, why not checkout one of the Linux distributions; Ubuntu comes highly recommended to those who are Linux n00bz.

Whatever you do, I urge you to upgrade from XP. From April 2014 onwards, no patches, no updates, no security fixes. I find it highly likely that with that many PC’s still running XP that those with a financial interest in attacking these machines and using them for nefarious machines may be sitting on exploits and security holes that will never get fixed. Its in your, as well as everyone else’s interests that you consider your options now, and migrate by April 14.

When a Minimal Install Isn’t…

Over the past couple of days, I’ve been rewriting the recovery script for a Linux LAMP application I wrote about 5 years ago.  I test it every so often to make sure it still works.   This year,  it doesn’t. Basically, we’ve reached a stage where the software versions don’t support the LAMP stack I chose (XAMPP).  Besides,  XAMPP isn’t really suitable for production servers even though its served us well in the intervening years.

So,  I’ve embarked on updating the recovery script to fit in with an ‘off-the-peg’ LAMP stack which will be easier to maintain going forward.

My favorite distribution is Debian and that’s the one I’ve most experience with.  However,  the preferred distro in the office is RHEL, or variants based therein.   So I got myself a fresh download of the Fedora 19 network install CD, loaded it into the virtual machine and off we go.  The installer is a bit err, low-rent – pretty graphics and the like, but not a lot of options to choose.  I suppose I’m too used to the ‘expert’ mode of the Debian network install.  Anyway,  went through the necessary steps to get the network up and running, configure it to talk to our proxy server etc,  find the disk config menu (hidden off-screen on a low-res screen) then go to the package selection screen.  Being reasonably accomplished now in administering Linux systems, I went for the minimal selection so I could add the other packages later on,  and off we went.

I quite liked how you can set the root password and create a new user whilst the OS installs – thats efficient. Then that was that,  server installed.  And that’s when the trouble started.

Giving yum proxy access was straightforward (although why the configs don’t carry across from the installer, I don’t know) and getting the LAMP stack installed was straightforward.  The httpd service came straight up after install and was ready to go.  Except that it wasn’t.  I could not for the life of me get a http page to come up.  It seemed that SSH was the only default port opened.  I checked network config and that all looked okay.  I could even wget http://localhost and get a page back.  So why no external connection?   Then I discovered SELinux was installed and running.  Disabled that, and a reboot – still no damn connection!   There looked to be a load of IPTables rules still listed;  could they be a carryover from SELinux I wondered?  Dropped the iptable rules and magically got http access back.  Rebooted and same problem again.

After reaching out to a colleague who has a little more experience with these distro’s that I,  and after installing Webmin,  we discovered that firewalld was running on startup.

Now,  when I install a minimal distro installation, I expect the following:

  • A bootloader
  • A kernel
  • A shell
  • Enough configuration to get from the bootloader to a shell
  • An ability to extend the system with a package manager.

I do not expect other things to get in the way,  especially as I hadn’t asked for them.   SELinux and Firewalls are good practice, but I do not want them imposed on me, especially if I’m not expecting them.  There were a number of other packages loaded (wpa_supplicant) that to me do not classify as essential to getting Linux up and running.

Fedora 19 and I have not immediately started as friends.

Big Data Conversations.

I’m not sure people understand big data and mining the information held within. I’ll summarise a conversation I had today.

Me: Can I have that data-set please?
Other Person (OP): Its got over 20 million records in it, what do you want to know?
Me: I am thinking about x, and think your data set may answer some questions.
OP: What exactly are you looking for?
Me: I don’t really know, until I’ve seen the data and what information it holds.
OP: How do you know my data-set has the information you need?
Me: I don’t. but its the best chance I’ve got.

and so on and so forth…

I think sometimes big data mining is a bit like mineral mining. You can take samples and investigate indicative factors, but until you take hold of your pick-axe, you’ll never know exactly what is down there. Hopefully I’ll get access to the output and see what can be discovered from it. I am already thinking about visualisation techniques to find the shiny nuggets of data held within.

A little Comic Relief

The annual or bi-annual charity appeals are always fun, but also somewhat predictible as to what we’ll end up seeing.  So, whilst I urge you to donate,  you can also have a bit of fun whilst watching.

So here’s the rules of the drinking game we’re playing by:


The rules are simple:
Define your own measure,  be it a shot,  finger’s width or even a whole glass.

One Measure
– A disease is mentioned
– An African Child is seen with a fly on its face
– Every time a celebrity does something ‘exciting’
– A giant cheque is produced
– A celebrity holds an African child
– A celebrity crys
– The Phone Number is read out
– Man dressed as a woman, or a woman dressed as a man

Two Measures
– The total so far is read out
– We see the phone call takers up BT Tower
– When the guest presenters change
– When the presenters look confused because they don’t know what they’re going to next
– If we’re shown a picture of BBC TV Centre

Three Measures
– When the presenter runs his/her fingers along the screen as the number is read out.
– When someone mouths the number at the back of shot
– When someone makes the ‘phone call’ symbol
– News Presenters doing something ‘wacky’
– When an African child is made to wear a red-nose

Four Measures
– Wogan or Pudsey Appears

Penalties:
Penalties require a CR donation
– Spillage – £1
– Fall Over – £1
– Vomit – £5
– Pass-out – Whatever you can shake out of their wallet/purse.

Remember Kids, Drink Responsibly,  as little African Kids often can’t.   Give Generously!

iPXE CloneZilla

CloneZilla is a linux toolset that allows you to clone either a partition or whole disk to another location;  either a connected storage device,  or remotely to the network.   This is a great tool for imaging systems before you work on them and lets you have a copy in case the worst should happen.    It has a variety of bundled tools in order to read from most of the popular filesystems in use,  falling back to DD to copy each disk sector if you’re using some obscure or proprietorial filer.   This is the FOSS alternative to Norton Ghost!

The great thing about CloneZilla is that its quick and easy to get it booting via iPXE,  so is worth investing a small amount of time in setting up so that you have it ready to go should you need it.

These instructions are based on release clonezilla-live-20121217-quantal.iso which seems to be versioned 2.0.1-15.  

Download the ISO from the CloneZilla site.  Use 7zip or your favourite image opening tool to open the ISO.  You need to extract the following files:

  • vmlinuz
  • initrd.img
  • filesystem.squashfs

and put them onto your boot webserver.  In this example,  I have created a folder called CloneZilla.

############ CloneZilla ############
:Clonezilla
echo Starting CloneZilla with default options
kernel http://boot.server/CloneZilla/vmlinuz
initrd http://boot.server/CloneZilla/initrd.img

imgargs vmlinuz boot=live config noswap nolocales edd=on nomodeset ocs_live_run=”ocs-live-general” ocs_live_extra_param=”” ocs_live_keymap=”” ocs_live_batch=”no” ocs_daemonon=”ssh” usercrypted=Kb/VNchPYhuf6 ocs_lang=”” vga=788 nosplash noprompt fetch=http://boot.server/CloneZilla/filesystem.squashfs
boot || goto failed
goto start


And that is really about it! You’ll notice we pass a few arguments which set various options. The most important option is the ‘fetch=’ command which tells the image where to download the main file system from. The other option I set was ‘usercrypted=’ which uses the Linux mkpasswd command to set the root password on boot – in this example iloveclonezilla.

A really easy one this week, but one worth trying. I’m fighting to get Backtrack5 booting over iPXE without using the ISO method, but this is proving troublesome. I think the image simply isn’t able to cope with being booted from a http network source.

Sugru – A Brief Update

I just wanted to provide a brief update on my thoughts of Sugru.  Its a wonderful product, ideal for fixing and personalising things.  However,  the biggest bugbear of all is the shelf-life.

Unlike duck/duct tape,  superglue,  epoxy resin, putty  and other more commonly known fixing materials and methodologies,  Sugru ‘sets’ after about 6 months,  whether you’ve opened it or not.  This means that one of the big bags I had became useless as I wasn’t able to use the sachets before they’d all set.

And because Sugru isn’t available in most shops,  you can’t just pop and get some more – you have to wait for the postman to bring it for you.  This is fine if you have a non-urgent fix, but when you need to do something straight away,  you either have to ensure you have some fresh Sugru in,  or find an alternative option.  Its often the latter.  So reader beware!

One final point,  Sugru reckon if you keep it in the fridge, it will keep for 18 months;  I’ll have to pick some up and try it.

SysRescueCD v3.1.2

So to kick off, we’ll start with booting SysRescueCD from http://www.sysresccd.org/.

From Wikipedia:
SystemRescueCd is an operating system for the x86 computer platform, though the primary purpose of SystemRescueCD is to repair unbootable or otherwise damaged computer systems after a system crash. SystemRescueCD is not intended to be used as a permanent operating system. It runs from a Live CD or a USB flash drive. It was designed by a team led by François Dupoux, and is based on the Gentoo Linux distribution.

For this activity, I used the download versioned v3.1.2 which I got from http://goo.gl/F36zV 

My issue was that the machine I was trying to boot from didn’t seem to have enough memory to use the memdisk/iso boot option common for most installs, which meant I had to try and boot the ISOLINUX image.

The Software:
Open the ISO in your favorite ISO opening tool (I use 7zip).  
Extract the following files into your web boot server.  I used a sub-directory called SysRescueCD

  • sysrcd.dat
  • sysrcd.md5
  • ISOLINUX/rescue32 (or 64)
  • ISOLINUX/initram.igz

Note,  I assume rescue64 is the 64 bit version of the kernel, and rescue32 is the 32bit kernel.  There is also altkrn32 and altkrn64 which are referenced in ISOLINUX as alternative kernel builds.  They all seem to work.


The Webserver Config:
This is the menu display section of the config:
item SysRescueCD32 SysRescueCD – 32bit

And this is the execution program required to boot it.

############ SYSRESCUECD ############
:SysRescueCD32
echo Starting Sys RescueCD (32bit) with default options
initrd http://boot.server/SysRescueCD/initram.igz
chain http://boot.server/SysRescueCD/rescue32 cdroot docache dodhcp setkmap=uk netboot=http://boot.server/SysRescueCD/sysrcd.dat
boot || goto failed
goto start

Note, you can change setkmap= to your preferred keyboard mapping; I’m in the UK so that is the one I use.  If you leave this option unset,  it will prompt you when you boot the server.
If you change
rescue32 to rescue64 or one of the alternate kernel images,  the same commands seem to work.  There doesn’t seem to be any difference in using netboot= or boothttp= to locate the main disk image.

Finally, I’m using a Thecus NAS as my boot webserver,  using FaJo’s Apache Webserver module.  For some reason,  whilst the initrd and kernel load perfectly well,  the image refused to boot, freezing at ‘null’ in the download.  Another Apache webserver didn’t exhibit the same condition, but its something to be aware of.  If I find the cause,  I’ll update this post.

iPXE Booting – An Introduction

If you’ve arrived at this blog,  you likely already know what iPXE booting is,  but for those who don’t,  I want to give a brief introduction to what iPXE is and how useful it is;  both for IT engineers and computer hobbyists.

So, What is it?

Quite simply,  its an open-source bootloader, primarily for the x86/amd64 machine architecture.  Its aim is to allow you to boot software and images across a network connection, using common network protocols such as tftp, http, ftp, iSCSI and AoE.

iPXE can be called over the network, started from bootable media like USB sticks or CD-Roms and even injected into the ROM of network cards where available.  This makes it an ideal tool for IT ‘fixers’ who often need to load a variety of images rather than just the standard Windows install on a machine.


A Brief History of PXE Booting.

PXE (Or pixie) booting is nothing new – in fact, pretty much all of the x86 type computers I have come across built in the last 10 years allow PXE booting from the network card, although sometimes the option needs to be turned on in the BIOS or the NIC.  Thin Clients,  VoIP phones and other ‘low touch’ devices also often support downloading configurations, software patches and other such updates over the network from a central source.   But PXE booting is not without its problems.  Its not setup to allow user interaction, meaning whatever the DHCP config is set to delivered is delivered.  Secondly, it uses tftp to transfer files – perfectly adequate for small text based config files,  but SLLLOOOOOWWWW when sending much over a few megabytes, even on fast lan connections.  Finally, PXE is only available with the functionality built into the firmware, so if it doesn’t do what you need it to,  you’re pretty much stuck with what you’ve got.

iPXE has had a bit of a convoluted development path in its reasonably short life.  It began life as ‘Etherboot‘, which started in around about 1995.  Etherboot evolved into gPXE (Nothing to do with Google) and then after a bit of a falling out within the gPXE team,  gPXE was forked by some members of that team into the iPXE we have today.  Note,  iPXE doesn’t have anything to do with Apple and their iDevices.   Both gPXE and iPXE continue to this day, doing pretty much the same thing.  However,  from my experience,  gPXE is better documented,  whilst iPXE receives more regular development activity.  And development does not happen in parallel, so you may want to check out both in case one has a feature set the other doesn’t.

A Google Tech-Talk about gPXE and Network Booting.

I’ve moved from gPXE to iPXE in 2012, mainly because gPXE wouldn’t support a NIC on a Dell machine I was trying to boot,  and had been considering making the switch at some point anyway.  iPXE resolved the issue for me,  so its now become a bit like Doc Martins were in the 80s and 90s – The boots of choice.

But why use iPXE Booting?

For me, it was a means to an end – I needed to test a motherboard and RAM combo, but didn’t have a spare hard-disk to boot from.  I also didn’t have any blank CD’s to burn a live-cd image to in order to boot from a CD drive.  Finally, I didn’t have any large (>64mb) USB thumb drives that I could write an ISO to, so I was kind of out of options.  Then I wondered if there was a way of using the small stick to chain boot something from the network and well,  it turns out there was.
I’ve stuck with network booting because once you get it working, your life as an engineer becomes much easier.  No more messing around burning CD’s, or digging through CD wallets trying to find the version of the particular utility you need.
Now I can connect up a network cable, boot from the network, choose the ISO, utility, or test tool you require from a menu,  and stand back and watch it boot from my NAS filer.  This is often quicker than waiting for a silver platter of plastic and metal spin up and be read using the power of laser light.
The process is a little complicated, but once you’ve set it up the booting piece is pretty much hands off;  the only maintenance is adding new images and updating the menu.
The way I’ve configured it is
  • PXE requests an IP Address from the DHCP Servers
  • IP Address is returned,  along with instructions on getting the iPXE ‘software’ from a TFTP server.
  • The PXE downloads iPXE from the network,  and chainloads into this.
  • iPXE re-requests an IP Address from the DHCP Servers, along with iPXE specific instructions – i.e. where to go next.
  • iPXE reads through this config and initiates a http connection to the web server containing the main iPXE config files and menu etc.
  • The end user/operator (me) chooses what to boot from the menu.
  • iPXE parses the settings supplied in the menu, downloads the necessary components (again, over http) and boots the image.
gPXE took me about a day to get working,  because of the need to install and configure a DHCP server with various options.  Because this is a home network, I had to create this new by learning ISC DHCP,  plus I took the opportunity to install ISC BIND as a DNS server – further learning required.  Note,  you don’t *need* these for iPXE to work, but it makes life easier in the long run.   I may blog more about setting up iPXE at some point,  but there are plenty of documented guides on how to do it.   

And what is the point of this blog?

So, as I’ve already mentioned above,  iPXE doesn’t have *that* great a set of documentation,  particularly in regard to configuring images to boot.  Most of these comes through gleaning facts from various sources,  putting them together through trial, error and a little bit of prior experiencing and seeing if it works.   Whilst I manage to get things to boot,  I think its useful to share solutions with the wider community to make things easier to use.  This is particularly true when trying to convert Linux live images into an iPXE boot set up, where you don’t necessarily want to or can’t load an entire ISO into memory.  Hopefully others will find this a useful resource,  and I’m going to try and use it as a library of knowledge on my own experiences of network booting.