Table of Contents

, , , , ,

Reinstall grub on mdraid1 array

Upon rebooting the server after upgrade from Debian 8→9→10 you get dropped in a grub rescue prompt like this

and with an error like 

grub2 error: symbol `grub_calloc' not found

Boot up a rescue system like a Debian live CD. Then:

1. List the disks with raid 

root@rescue ~ # lsblk
NAME    MAJ:MIN RM    SIZE RO TYPE  MOUNTPOINT
loop0     7:0    0      3G  1 loop  
sda       8:0    0    2.7T  0 disk  
├─sda1    8:1    0     16G  0 part  
│ └─md0   9:0    0     16G  0 raid1 
├─sda2    8:2    0    512M  0 part  
│ └─md1   9:1    0  511.4M  0 raid1 
├─sda3    8:3    0      1T  0 part  
│ └─md2   9:2    0 1023.9G  0 raid1 
├─sda4    8:4    0    1.7T  0 part  
│ └─md3   9:3    0    1.7T  0 raid1 
└─sda5    8:5    0      1M  0 part  
sdb       8:16   0    2.7T  0 disk  
├─sdb1    8:17   0     16G  0 part  
│ └─md0   9:0    0     16G  0 raid1 
├─sdb2    8:18   0    512M  0 part  
│ └─md1   9:1    0  511.4M  0 raid1 
├─sdb3    8:19   0      1T  0 part  
│ └─md2   9:2    0 1023.9G  0 raid1 
├─sdb4    8:20   0    1.7T  0 part  
│ └─md3   9:3    0    1.7T  0 raid1 
└─sdb5    8:21   0      1M  0 part

2. Now you need to mount the raid partition and find the /boot directory. In this raid example we first mount the /root filesystem which contains the (for now an empty) /boot dir. 

mount /dev/md2 /mnt

You can do ls /mnt now to see if you mounted the correct directories. 

root@rescue ~ # ls /mnt/
bin  boot  dev  etc  home  initrd.img  initrd.img.old  installimage.conf  installimage.debug  lib  lib64  lost+found  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var  vmlinuz  vmlinuz.old

3. Now we mount /boot 

mount /dev/md1 /mnt/boot/

4. Prepare to chroot by mounting runtime directories 

for i in proc sys dev; do mount --rbind /$i /mnt/$i ; done

5. Now chroot and mount all filesystems from /etc/fstab 

chroot /mnt
mount -a

6. Install grub on all disks that are part of the /boot mdraid partition and update grub 

grub-install /dev/sda
grub-install /dev/sdb
update-grub

That should be it! You can check the /boot/grub/grub.cfg which update-grub generates to verify the search command points to correct partitions. 

cat boot/grub/grub.cfg
...
menuentry 'Debian GNU/Linux, with Linux 4.9.0-18-amd64' --class debian --class gnu-linux --class...
...
if [ x$feature_platform_search_hint = xy ]; then
		  search --no-floppy --fs-uuid --set=root --hint='mduuid/a362fc6a3d43968bcc41bb75f70ddbca'  d8be1c5a-3791-4a91-98eb-9ab039aefb6a
		else
		  search --no-floppy --fs-uuid --set=root d8be1c5a-3791-4a91-98eb-9ab039aefb6a
fi
...

If you run blkid now you can check the UUID from above: 

rescue:/# blkid /dev/sdb2
/dev/sdb2: UUID="a362fc6a-3d43-968b-cc41-bb75f70ddbca" UUID_SUB="fc8aba0b-f4ae-3ad0-947f-8d9ea7112301" LABEL="rescue:1" TYPE="linux_raid_member" PARTUUID="61753575-47c1-4298-b298-68b76b624e16"
rescue:/# blkid /dev/sda2
/dev/sda2: UUID="a362fc6a-3d43-968b-cc41-bb75f70ddbca" UUID_SUB="ed9efe73-14c0-9e4f-0eb6-83b68acde6ef" LABEL="rescue:1" TYPE="linux_raid_member" PARTUUID="c85182f8-3569-4e9a-baad-5ac1b0c7677e"

You can see the UUIDs match for both disk partitions where our /boot is.

Tested on

See also

References