Playing with the last layer before bare metal ; Grub2
July 2nd, 2008I screwed up my machine and spent almost two days trying to repair it…
Context
My machine’s hard drive is fully encrypted. This volume is then the PV of an LVM with one VG and several LVs. As the whole hard drive is encrypted, the /boot partition is on a usbkey. So the idea is that Grub is installed on the hard drive (/dev/sda/) MBR, boot with the /boot on a usbkey, then open the crypto container and then activate the LVM and finally mount all the partitions.
My machine is using Sid and therefore Grub2. And this configuration was fully achieve just by using Debian Installer.
Problem
The first thing I tried was to remove the usbkey without unmounting /boot. The system was fully functional but it was then impossible to regain access to /boot.
Then one day, I rebooted… (actually, this was an error ;))
Solution
After the reboot, I was welcomed with Grub message « Welcome to Grub! » (in rescue mode).
So first, I rebooted on my usbkey (not the one with the /boot partition) which has a Debian installed (see FIXME for details about this usbkey). The idea was to
- boot,
- open the crypto-container,
- activate the LVM,
- mout all partitions (including /boot on the other usbkey),
- chroot
- and re-generate an initramfs.
Except that this system is i386 and my machine is amd64 ; no way to chroot…
So, Corsac generated an amd64 LiveDebian cd.
This time, I was able to follow the plan (see § chroot operation below for details). And I rebooted… to get the same message from Grub.
At that stage, we worked together with Corsac (I still did not find a better solution than working with a friend to solve problems ! It’s good to keep moral up, exchange ideas, get help… ;))
So, now we had to play with Grub…
The first thing to know is that Grub2 does not work like Grub.
root, kernel and initrd are variables to set. And Grub2 has lots of modules that must be loaded before being able to actually do something. Making long story short, the prefix variable was set to (fd1,0)/grub/. Which is totally wrong as I do not even have a floppy drive…
So first, all variables ware unset.
unset prefix unset root
Then the right ones were set.
set root=(hd1,1) set prefix=(hd1,1)/grub/
Then the normal.mod module was loaded and the normal mode launched.
insmod /grub/normal.mod normal
At that stage, the Grub menu was launched and my machine booted normally.
What we tried within Grub2 « shell »
But we tried lots of things before finding this. Before finding those two culprits we
- loaded manually ls.mod, _linux.mod and then linux.mod modules and set linux variable to the right value
insmod /grub.ls.mod insmod /grub/_linux.mod insmod linux.mod set linux=/boot/vmlinuz-2.6.24-1-amd64 root=/dev/mapper/sda1_crypt
- then initrd.mod module and set initrd variable to the right value
insmod /grub/initrd.mod set initrd=/boot/initrd.img-2.6.24
- finaly booting
insmod /grub/boot.mod boot
chroot operation
During the chroot operation, the following steps were done.
Open the cryptocontainer (the name of the unencrypted logical volume must be the one from the system to be rescued. Indeed, it will be exported to the chrooted system and be used when generating the initramfs. If it is not the same unencrypted LV as the system to be rescued, when booting the initramfs won’t have the name actually used by the system ; it will fail)
# cryptsetup luksOpen /dev/sda5 sda5_crypted
Activate the LVM using the (now unencrypted) encrypted partition
# lvm vgchange -a y
Mount the root LV somewhere to chroot within
# mount /dev/mapper/vg_main-slash /mnt/toberescued
Mount /dev within the chroot so that other LVs are available
# mount --bind /dev /mnt/toberescued/dev
Actually chroot
# chroot /mnt/toberescued
Mount the other partitions (the chrooted system /etc/fstab is available)
<toberescued># mount /proc <toberescued># mount /sys <toberescued># mount /home <toberescued># mount /var <toberescued># mount /usr <toberescued># mount /tmp <toberescued># mount /var/log <toberescued># mount /boot
Regenerate the initramfs
<toberescued># update-initramfs -u -t
Re-install Grub
<toberescued># grub-install --recheck /dev/sda
Unmount everything and exit the chrooted system.
