Advanced Procedures: Running with root on a HDD

It may seem odd, but with LRP, which is intended to run using a ramdisk as it's root filesystem, getting root on a hard-disk partition is actually a bit tricky.

This is an attempt to document how I got an LRP 2.9.8 system (with a 2.2.x kernel) running with a hard disk partition as the root filesystem. You might want to do this if you intend to use LRP as a very thin server-OS, where you need a HDD for other reasons. My system actually combines several advanced techniques, including booting off a SCSI disk, using software RAID.

Before you attempt this, go through a few more conventional setups first (you'll have to anyway). You should be comfortable with the general LRP boot and configuration process, and you should have setup LRP to boot off a hard-disk (with root still on a ramdisk) before attempting this procedure.

Note

You'll need several things besides an LRP disk image to get this to work. Most files can be found from the HardDisk packages page of my website: http://lrp.steinkuehler.net/Packages/HardDisk.htm

You'll need:

boot.tgz

Contains updated linuxrc & the /boot directory

hddroot.tgz

Contains chroot & updated checkroot.sh init script

modules

You'll need modules for your hard disk controller that match your kernel revision.

  1. I started with a 2.9.8 LRP "idiot" image setup for a 2.2.x kernel: idiot-image_1440KB_FAT_2.9.8_Linux_2.2

  2. The first thing I did was replace the kernel on the disk with a kernel that supports 486 CPU's (the default kernel requires a Pentium) and includes software RAID support. The kernel I used is available here:

    http://lrp.steinkuehler.net/files/kernels/2.2.16-1-RAID/

    I also increased the size of the ramdisk from the default 4 Megs to 16 Megs, so I'd have room for lots of packages.

  3. Boot the new disk

  4. Create and populate the /boot directory. The easiest way to do this is to uncompress my boot.tgz file, which includes modules for an Adaptec SCSI controller and RAID 1 (mirroring) by using the following commands (after mounting a floppy containing boot.tgz):

    cd /
    gunzip - </mnt/boot.tgz | tar -xv

    Or, you can just created the needed directory structure and files:

    /boot/etc/modules  <- which modules to load & in what order
    /boot/lib/modules/ <- modules go here

    If you add your own modules, don't forget any filesystem modules (like ext2.o) you'll need, in adddition to the low-level hard-disk and RAID drivers. The kernel will need to be able to read the root filesystem after linuxrc exits if you want to boot into it.

  5. /linuxrc needs to be configured to load modules from /boot and to bypass creating the filesystem if you're booting into a hard-disk partition instead of a ramdisk. Be VERY careful here...if you mangle the linuxrc file in any way (EOF translation from downloading or cut & paste on a non-linux machine), you can easily wind up with a non-bootable system. I have included a complete linuxrc file as part of the boot.tgz file, so you can simply:

    rm /linuxrc
    ln -s /boot/linuxrc /linuxrc

    Note

    This wastes a little space, as you now have two copies of linuxrc (the one in /boot and the one in /var/lib/lrpkg), but not that much, and you may appreciate having the original linuxrc around for reference if your system doesn't boot and you need to expand root.lrp, modify /boot/linuxrc, and recreate root.lrp.

    Caution

    I STRONGLY suggest you either use my linuxrc as-is or at least as a reference for making your own modifications. There are several things to watch out for, including proper mounting/unmounting of /proc, and making sure you don't run commands that don't exist yet (ie before the busybox symlinks are built). I had non-booting systems for quite a while while working out all the problems, and recovery from a mistake is pretty difficult, typically involving another linux box to rebuild root.lrp.

  6. Edit /var/lib/lrpkg/modules.list and delete the following two lines:

    sbin/insmod
    sbin/rmmod

    This will backup the insmod and rmmod commands as part of root.lrp, required if linuxrc is going to be able to insmod the /boot modules.

  7. Edit /var/lib/lrpkg/root.dev.mk as required. I had to add additional SCSI partitions (LRP only builds sd?1-sd?8 by default) and add the RAID devices.

    I changed:

    #SCSI
    makedevs sda b 8 0 0 8 s>null 2>&1
    makedevs sdb b 8 16 0 8 s >null 2>&1

    into:

    #SCSI
    makedevs sda b 8 0 0 15 s >null 2>&1 
    makedevs sdb b 8 16 0 15 s >null 2>&1
    
    #RAID
    makedevs md b 9 0 0 15 >null 2>&1
  8. Edit /var/lib/lrpkg/root.version to set the version ID for the core LRP packages (I used 2.9.8-CS).

  9. Add any required modules (NIC drivers and what-not) to /lib/modules (this can be done later if you want).

  10. Backup root.lrp and modules.lrp

  11. Partition your hard-disk and setup LRP to boot off the hard-disk using the methods described above. Use the boot disk you just made as a starting image. I setup my system to boot off /dev/md0, which is a RAID1 array of two SCSI disks. See the RAID section above, for details on how to set this up.

  12. Take a breather. At this point, you should have a fairly basic LRP 2.9.8 system booting off the hard-disk. Now's a good time to take a break, grab a beer (or coffee), go to sleep, or whatever.

  13. With the system booting off a hard-disk partition, but still using a ramdisk as the root partition, you should configure the system as much as possible. Copy any required LRP packages to the boot partition, add them to syslinux.cfg, and configure them (if required). You'll probably need at least:

    hdsupp.lrp
     -or-
    hdsupp_s.lrp
    raid.lrp (if you're using RAID arrays)

    Note

    There's not a hdsupp.lrp specifically for LRP 2.9.8. I've used the ones for Materhorn and Eiger without trouble.

    You should also configure any packages you want running, like sshd, rsyncd, dhcpd, and any applications you need, like a web or dns server. If you don't have enough RAM to boot with everything configured, try to temporarily add enough to your system...you'll only need it until your hard-disk root partiton is created and working, and it's much easier to experiment with things running from the ramdisk. Remember, if you mess up configuration with root in a ramdisk, just reboot and you're back to where you were...with root on a hdd, you have to be able to undo anything that breaks your system, and at this point, you're probably not making regular backups.

  14. Now things begin to get intersting. With a working LRP system booting off the HDD and running out of ram, it's time to build your new root filesystem. First, you need to partition and format your disks. If you want to have multiple partitons (for /var, /tmp, and the like), or you want to run off of RAID arrays, now's the time to get everything setup. Once you've got everything partitioned and formatted, mount your new filesystem someplace like /tmp/newroot in preperation for copying files. Remember to mount any additional partitions you intend to use as well, or you'll have to move everything around later. I wound up with the following setup:

    device      use     size    current mount point
    /dev/md1    /        64M    /tmp/newroot
    /dev/md2    /var     64M    /tmp/newroot/var
    /dev/md3    /tmp     64M    /tmp/newroot/tmp
    /dev/md4    /usr     256M   /tmp/newroot/usr
  15. With your new filesystem mounted, and completely empty (except for the directories you had to create to be able to have mount points for any extra partitions you're using), your're ready to start filling it. The first step is to uncompress root.lrp into the new root area with the following commands:

    mount -t msdos /dev/boot /mnt
    cd /tmp/newroot
    gunzip - </mnt/root.lrp | tar -xv
    umount /mnt
  16. You've now got the beginnings of your new filesystem, but you need a lot more to make it truly functional. LRP comes with a script that will build the filesystem for you, you just have to run it. The easiest way to do this is to simply run the linuxrc startup script chroot'd to the newroot directory. You can get the chroot command from my website:

    cd /tmp/newroot
    chroot /tmp/newroot ./linuxrc

    You may see some errors about modules already being loaded...don't worry about this, as the linuxrc script is simply trying to load the modules from /boot that were loaded when you booted the system. After this, you should see the familiar list of LRP modules that are being loaded to build your new file system.

  17. You now have a populated root filesystem on your hard-disk, but you're not ready to reboot yet (if you do, or if anything goes wrong, just reboot into the ramdisk as usual, mount your new root system on /tmp/newroot, and continue where you left off).

    Warning

    !! DANGER !! It very easy to wind up with a system that won't boot if you make a mistake with any of the following steps. Don't Panic! If you try to boot your system from the HDD and it's not working, take notes about what's wrong, then boot LRP from either a floppy (you did make an LRP floppy that loads your required HDD & RAID modules, didn't you?) or set root=/dev/ram0 in the syslinux.cfg file on the HDD boot partiton. You can then mount your new root partition and try to fix whatever's broken.

  18. First, you need to edit syslinux so the kernel will boot into your new root partition, instead of the ramdisk:

    mount -t msdos /dev/boot /mnt
    ae /mnt/syslinux.cfg

    Change the root= line to the device you made your root partition. Mine is set to root=/dev/md1

  19. Now you need to edit the /etc/fstab file to reflect your new root partition (and any additional partitions you may have created). Make sure you get the filesystem correct, and fill out the dump and pass fields, or you may not be able to boot off the hard-disk. You'll want a bootable floppy around in case this happens...if it's a linux floppy, just mount your new root partition and edit /etc/fstab, and try booting again...if it's a dos floppy, edit syslinux.cfg to set root= back to /dev/ram0, boot LRP into the ramdisk, edit /etc/fstab, set root= back to your root partition, and reboot. My /etc/fstab looks like this:

    # /etc/fstab: static file system information.
    #
    # <file system>     <mount point>   <type>  <options>   <dump>  <pass>
    proc                /proc           proc    noauto	0	0
    /dev/md1            /               ext2    rw          1       1
    /dev/md2            /var            ext2    defaults    1       2
    /dev/md3            /tmp            ext2    defaults    1       2
    /dev/md4            /usr            ext2    defaults    1       2
  20. You also need to update the /etc/init.d/checkroot.sh file, since the default file shipped with LRP doesn't check the root filesystem (not much point in checking a ramdisk filesystem you just built). I just took the script from a Debian 2.1 (Slink) distribution (which LRP is based on), and added the RCDLINKS variable for LRP. You can get this file from my website, and just copy it to /etc/init.d/checkroot.fs

  21. Now things begin to get a bit ugly. You might think you're ready to reboot, but but you're not. If you look at the /etc/rc?.d/ directories, you'll find they're empty! LRP by default uses a nifty script that builds the rc?.d links at boot, but this won't work properly with root on a hard-disk for several reasons:

    1. The update-rc.d script is in /usr/sbin, which may not be mounted at boot

    2. At boot, the root filesystem is mounted read-only, creating a chicken & egg problem. The /etc/rc?.d links can't be created until root is writable, and root won't be writable until the checkroot.sh script runs, which is run from a link in rcS.d

    To avoid these problems, you need to disable the automatic link generation:

    ae /tmp/newroot/etc/default/rcS
        
    change:
    DYNARCD=no
        
    to:
    DYNARCD=yes
  22. You still need to get all those links into your rc?.d directories, but luckly, this is pretty easy. Just run the following commands:

    cd /tmp/newroot
    chroot /tmp/newroot usr/sbin/update-rc.d -faw

    Make sure the links were actually made (and put in the right place) by:

    ls -l /tmp/newroot/rc?.d

    In the future, if you add any LRP packages, you'll need to run the update-rc.d script manually, to create any links that might be required. You probably won't want to use the -a (all) or -w (wipe) switches, especially if you've installed any non-LRP packages that don't have RCDLINKS variables in their init.d scripts.

  23. NOW, you're ready to reboot, and your LRP system should cleanly boot into your new root partition. If something went wrong, or you're getting errors, read CAREFULLY through everything above...there's a lot of detail, making it easy to miss something or skip a step.

  24. If something's broken, try to see what you did wrong. It may be possible to fix your existing filesystem, or you may have to rebuild it from scratch. With a working LRP boot floppy and your configured LRP system on the hard-disk, it's really not too hard to re-format your new partition(s) and start over. I did this MANY times before I got everything working properly and the above procedure documented.

  25. BACKUPS - I recommend backing up your new system using conventional linux backup methods, but there is that very handy boot partition with all your configured LRP files on it that's not doing much anymore. It's possible to use the lrcfg backup scripts to store your running configuration on the boot partition, but I wouldn't rely on this as your only backup. For one, it's probably on the same physical hard-disk as the rest of the system, and your hard-disk is the most likely thing to die. Also, if you backup some of the packages (like etc and root), there's the potential to break the LRP system that's on the boot partition, meaning it's floppy time if you ever need to repair anything on your new filesystem. I'd at least keep a copy of the LRP files around on the boot partition (maybe in a sub-directory), in case you ever need to get back to your 'as-installed' state.