Today is a big deadline in work. The point when weeks of work move from development to production (via numerous sprints/releases to UAT). The culmination of the development team and my efforts over the last two months. As with most IT projects the last two weeks have been a bit fraught, testing us as we prepare to launch a new suite of Cognos reports.
Today is also the day my Thinkpad has decided to corrupt its master boot record (MBR). Preventing my laptop from loading its operating system. Thankfully Microsoft (and third parties) provide bootable utilities to repair MBRs. However…………
Master Boot Record Corrupt. FAIL!
My MBR failure is complicated by IBM (sensibly) requiring us to use Symantec’s PGP Whole Disk Encryption (WDE). PGP’s WDE protects our laptops, and any sensitive data on them, in the event laptops are lost or stolen. As a mobile worker, and someone regularly on the move, WDE is a nice saftey blanket. Yes, I know it has venerablities and yes, if someone really wants the data on my laptop they will get it. But for additional security and if the laptop is lost it provides some reassurance. It’s also company policy, there is no point in fighting it.
Update: I was wrong about PGP being crackable. Support are able to sort a forgotten password because we run a support server and since 9.7 there are backups in place to recover a forgotten password. But there is still no known published way to crack PGP WDE. E.g. If you can’t decrypt the hdd, the data is lost. On one hand this makes me feel safer about the loss of a laptop and on the other it makes me glad I have most of my data/files backup up. A reminder that I also need a better backup solution for a hdd failure.
With the entire hard disk drive (hdd) encrypted I can’t use a utility program to fix the MBR. The utilities require you to boot from them and in so doing they skip PGP’s BootGuard. BootGuard lets the OS use the encrypted hdd. Until the hdd is decrypted the utilties can’t access the MBR, it’s encrypted and the booting hdd doesn’t even appear. Thankfully, I keep my PGP up to date and the right recover CD handy. Recovery Images can be downloaded here:
Key to know which version of PGP you have. If you can boot into PGP’s BootGuard screen it’s easy to find out: Selecting ‘advance’ instead of ‘continue’ from the options will display the version and other options to assist in recovery. Since a similar failure in 2009 I keep a note of the PGP version I’ve installed (including any service packs). Just incase PGP’s BootGuard also fails to load. It’s not unheard of for both MBR and PGP BootGuard to be corrupted at the same time. Not knowing which version of PGP 9 I’d installed, combined with the bad sectors that caused the HDD to fail, resulted in my old drive being scrap.
Symantec provide a guide for how to recover from this situation here:
With the matching version of the recovery disk in place I booted off the recovery CD and tried to let Windows boot itself. In rare cases it’s possible that using the Recovery CD instead of the BootGuard installed on the machine will let Windows boot. Sadly this wasn’t the case, I still had an MBR issue. Back to the drawing board, the next step is a longer one: Rebooting off the Recovery CD, entering my password and then pressing ‘D’ to decrypt the entire hdd. We’re now at 90% having started at 9am this morning.
The laptop hdd is 250gig capacity, of which 80gig was in use. I’m hoping the first 80gig takes the longest to decrypt. Ideally the final170gig will be a lot quicker, as it’s empty disk space. I’ll leave it over night and then all being well use MS’s MBR fixer tomorrow. If anything goes wrong or the laptop gets disrupted during the decrypt, all data is lost. Not the most relaxing situation to be in but I have 90% of my data backed up. All my work is stored on IBM’s cloud and I only stand to lose several recently archived locally emails. The main loss will be time in having to rebuild my Thinkpad. As a worst case this isn’t too bad, but fingers crossed I can full decrypt the hdd and recover my current MBR.
Update: Sadly my 80gig and free space decrypting quicker theory has been proved wrong. It’s now at 38% left to go and hopefully will be sorted in the early hours of Tuesday morning (3.5 days to decrypt 250gig). Keeping everything crossed it keeps going and finishes, allowing me to fix the MBR and recover all my data / Laptop. Decryption takes a fraction of the time if the hdd is mounted as a slave on another system. Lesson learnt! From now on I’ll run two hdd and regularly clone (more on this to come in another post).
93% – Not going anywhere for a while…………..
Before starting a decrypt via the recovery CD I googled alternative options. If you have a second machine with the same version of PGP installed you can plug the hdd in as a slave (via a USB caddy) and use PGP on the local machine to decrypt the hdd. This is the fastest way, sadly I don’t have another machine with PGP installed.
Update: Plugging the hdd in via a USB caddy / as a slave in a second machine is a lot faster because the Recovery CD is limited to 16 bit processing. If in Windows / Linux or OSX the decryption process can be run at 32bit and takes a fraction of the time. With hind sight waiting for SC to get home and pinching her work laptop would have been a better bet. It’s at 83% now with a very slim chance of being finished by Sunday. At least it’s still going. No physical hdd errors, yet!
I used to backup an image of my machine but Windows 7 made this harder and since upgrading I’ve taken to using IBM’s could to backup all of my work and accepting that if I had a failure I’d need to get an additional machine from IBM and rebuild it. Having now tested this theory it doesn’t work!
The new plan
The Plan comes in two flavours: Get a smart phone and improve laptop & return to weekly disk cloning.
1. Smart Phone: 99% of my work calls are handled by VOIP but I’ve been toying with getting a smart phone for work as a backup access to my work email, calendar, instant messaging and terminal services. Four key components of my day job that I’m currently without due to my laptop decrypting (and being corrupt). As a result I’ve bitten the bullet and ordered the Asus Fonepad. It’s not the best spec but a Galaxy Note II is out of the question at the moment. I hardly make calls on my work phone thanks to VOIP. If I did have to make a call I always have my iPhone5 with free minutes to make an emergency work call while out and about. The concept of a 7inch tablet that doubles as an emergency phone (and can be used with my headset) for £180 delivered was too good to get hung up on the negatives (slower processor and you’d look like a sketch from Trigger Happy TV if you tried to make a call in public on it!). I’ll post more on this when it arrives.
2. SSD and Weekly Cloning: My boss has an SSD drive and the boot times + smoothness of operation have always appealed. I’ve been waiting since I had a reason to rebuild the laptop to get one and this is it. I’ve ordered a Kingston Value 120gig drive after reading this review:
The time it takes to boot my Thinkpad always frustrates me. Even since upgrading from 4 to 8 gig of RAM it’s still sometimes hangs while paging and under heavy loads. Hoping the SSD will also prove more reliable. My Thinkpads travel a lot, the one before clocked over 100k miles. Combined with being on 5 days a week, most weeks of a year, it’s no wonder hdd fails / issues like this occur. With no moving parts an SSD should prove more reliable. It also means I can keep my current drive as a spare (if it’s not beyond repair) and regularly clone the SSD as a backup. More on this to come after the SSD swap and hopeful recovery. A new backup strategy is required (feel free to suggest any ideas in comments, or to laugh at my expense).
For now the Thinkpad is slowly chugging away decrypting and I’m off out to recover and watch Knee High Perform: https://www.kneehigh.co.uk/show/tristan_yseult.php
Thanks to my teams’ efforts and with lots of phone calls the release has gone to UAT and we’ll go live first thing Monday morning. Wish me luck and for a working Thinkpad asap :) .