{{tag>grub raid mdraid troubleshooting boot debian}}

====== Reinstall grub on mdraid1 array ======
Upon rebooting the server after upgrade from Debian 8->9->10 you get dropped in a grub rescue prompt like this

{{:wiki:screenshots:grub_rescue1.png?400|}}

and with an error like
  grub2 error: symbol `grub_calloc' not found 

Boot up a rescue system like a Debian live CD. Then:

1. List the disks with raid
<code bash>
root@rescue ~ # lsblk
NAME    MAJ:MIN RM    SIZE RO TYPE  MOUNTPOINT
loop0     7:0    0      3G  1 loop  
sda       8:0    0    2.7T  0 disk  
├─sda1    8:1    0     16G  0 part  
│ └─md0   9:0    0     16G  0 raid1 
├─sda2    8:2    0    512M  0 part  
│ └─md1   9:1    0  511.4M  0 raid1 
├─sda3    8:3    0      1T  0 part  
│ └─md2   9:2    0 1023.9G  0 raid1 
├─sda4    8:4    0    1.7T  0 part  
│ └─md3   9:3    0    1.7T  0 raid1 
└─sda5    8:5    0      1M  0 part  
sdb       8:16   0    2.7T  0 disk  
├─sdb1    8:17   0     16G  0 part  
│ └─md0   9:0    0     16G  0 raid1 
├─sdb2    8:18   0    512M  0 part  
│ └─md1   9:1    0  511.4M  0 raid1 
├─sdb3    8:19   0      1T  0 part  
│ └─md2   9:2    0 1023.9G  0 raid1 
├─sdb4    8:20   0    1.7T  0 part  
│ └─md3   9:3    0    1.7T  0 raid1 
└─sdb5    8:21   0      1M  0 part
</code>


2. Now you need to mount the raid partition and find the ''/boot'' directory. In this raid example we first mount the ''/root'' filesystem which contains the (for now an empty) ''/boot'' dir.
  mount /dev/md2 /mnt

You can do ''ls /mnt'' now to see if you mounted the correct directories.

<code>
root@rescue ~ # ls /mnt/
bin  boot  dev  etc  home  initrd.img  initrd.img.old  installimage.conf  installimage.debug  lib  lib64  lost+found  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var  vmlinuz  vmlinuz.old
</code>

3. Now we mount ''/boot''
  mount /dev/md1 /mnt/boot/

4. Prepare to chroot by mounting runtime directories
  for i in proc sys dev; do mount --rbind /$i /mnt/$i ; done

5. Now chroot and mount all filesystems from ''/etc/fstab''
  chroot /mnt
  mount -a

6. Install grub on all disks that are part of the ''/boot'' mdraid partition and update grub
  grub-install /dev/sda
  grub-install /dev/sdb
  update-grub

That should be it! You can check the ''/boot/grub/grub.cfg'' which update-grub generates to verify the ''search'' command points to correct partitions.

<code bash>
cat boot/grub/grub.cfg
...
menuentry 'Debian GNU/Linux, with Linux 4.9.0-18-amd64' --class debian --class gnu-linux --class...
...
if [ x$feature_platform_search_hint = xy ]; then
		  search --no-floppy --fs-uuid --set=root --hint='mduuid/a362fc6a3d43968bcc41bb75f70ddbca'  d8be1c5a-3791-4a91-98eb-9ab039aefb6a
		else
		  search --no-floppy --fs-uuid --set=root d8be1c5a-3791-4a91-98eb-9ab039aefb6a
fi
...

</code>

If you run blkid now you can check the UUID from above:

  rescue:/# blkid /dev/sdb2
  /dev/sdb2: UUID="a362fc6a-3d43-968b-cc41-bb75f70ddbca" UUID_SUB="fc8aba0b-f4ae-3ad0-947f-8d9ea7112301" LABEL="rescue:1" TYPE="linux_raid_member" PARTUUID="61753575-47c1-4298-b298-68b76b624e16"

  rescue:/# blkid /dev/sda2
  /dev/sda2: UUID="a362fc6a-3d43-968b-cc41-bb75f70ddbca" UUID_SUB="ed9efe73-14c0-9e4f-0eb6-83b68acde6ef" LABEL="rescue:1" TYPE="linux_raid_member" PARTUUID="c85182f8-3569-4e9a-baad-5ac1b0c7677e"

You can see the UUIDs match for both disk partitions where our ''/boot'' is.

====== Tested on ======
  * Upgrade from Debian 8->9->10 then reboot

====== See also ======
  * [[wiki:grub_errors|Grub errors out]]
  * [[wiki:error_invalid_environment_block_press_any_key_continue|Error: invalid environment block. Press any key to continue...]]
  * [[wiki:change_disk_software_raid|Change disk in software raid]]

====== References ======
  * https://www.suse.com/support/kb/doc/?id=000019679
  * https://www.suse.com/support/kb/doc/?id=000018770
  * https://unix.stackexchange.com/a/230448/210072
  * https://askubuntu.com/a/906665/635772
  * https://unix.stackexchange.com/questions/658/linux-how-can-i-view-all-uuids-for-all-available-disks-on-my-system
  * https://www.happyassassin.net/posts/2014/01/25/uefi-boot-how-does-that-actually-work-then/
  * https://unix.stackexchange.com/questions/230349/how-to-correctly-install-grub-on-a-soft-raid-1