Installing Gentoo on Software RAID

I'm currently fighting with this again. Apparently, there have been some changes between the 2004.0 CD and the 2005.0 CD (mostly related to the 2.6 kernal and the DevFS vs UDev systems).

First off, some tips-n-tricks:

1) ifconfig - find out the IP address of the box
2) passwd - change the root password to something you know
3) Starting the SSH daemon - The command is: /etc/init.d/sshd start

Now I'll be able to use SecureCRT and have an easier time of copy-n-paste. (Pick 'keyboard interactive' and 'ssh2' under options when setting up the connection in SecureCRT.)

Links:

Gentoo 2004.3 Software RAID Install HOWTO
Gentoo.org - x86 tips and tricks (including software RAID)
The Linux Documentation Project: Software RAID
An older gentoo.org thread on software raid

Now... to show you what I did and where it breaks. I have (2) IDE disks, attached as /dev/hda and /dev/hdc. I plan on mirroring them, and have already created /dev/hda1 (128MB for /boot), /dev/hda2 (2GB for swap), /dev/hda3 (2GB for root), and /dev/hda4 (rest of disk for LVM). All partitions are flagged as type 'fd' (linux raid auto).

# modprobe md
# ls /dev/md*
ls: /dev/md*: No such file or directory
# for i in 0 1 2 3; do mknod /dev/md$i b 9 $i; done
# ls /dev/md*
/dev/md0 /dev/md1 /dev/md2 /dev/md3
# nano -w /etc/mdadm.conf

Contents of my mdadm.conf file:

DEVICE /dev/hda1 /dev/hda2 /dev/hda3 /dev/hda4
DEVICE /dev/hdc1 /dev/hdc2 /dev/hdc3 /dev/hdc4

ARRAY /dev/md0 devices=/dev/hda1,/dev/hdc1
ARRAY /dev/md1 devices=/dev/hda2,/dev/hdc2
ARRAY /dev/md2 devices=/dev/hda3,/dev/hdc3
ARRAY /dev/md3 devices=/dev/hda4,/dev/hdc4

Pretty generic... now to create the raid. This is where we hit our first issue:

# modprobe raid1
# mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/hda1 /dev/hdc1
mdadm: /dev/hda1 appears to contain an ext2fs file system
size=72192K mtime=Wed Mar 30 16:46:07 2005
mdadm: /dev/hdc1 appears to contain an ext2fs file system
size=72192K mtime=Wed Mar 30 16:46:07 2005
Continue creating array? y
mdadm: ADD_NEW_DISK for /dev/hda1 failed: Device or resource busy
#

Here are the problems:

1) Even though I have a 128MB partition for /boot, it is still showing 64MB from a previous partitioning scheme.

2) mdadm: ADD_NEW_DISK for /dev/hda1 failed: Device or resource busy

So... take a look at my partition list:

livecd root # fdisk -l

Disk /dev/hda: 76.8 GB, 76869918720 bytes
16 heads, 63 sectors/track, 148945 cylinders
Units = cylinders of 1008 * 512 = 516096 bytes

Device Boot Start End Blocks Id System
/dev/hda1 * 1 249 125464+ fd Linux raid autodetect
/dev/hda2 250 4218 2000376 fd Linux raid autodetect
/dev/hda3 4219 8187 2000376 fd Linux raid autodetect
/dev/hda4 8188 148945 70942032 fd Linux raid autodetect

Disk /dev/hdc: 76.8 GB, 76869918720 bytes
16 heads, 63 sectors/track, 148945 cylinders
Units = cylinders of 1008 * 512 = 516096 bytes

Device Boot Start End Blocks Id System
/dev/hdc1 * 1 249 125464+ fd Linux raid autodetect
/dev/hdc2 250 4218 2000376 fd Linux raid autodetect
/dev/hdc3 4219 8187 2000376 fd Linux raid autodetect
/dev/hdc4 8188 148945 70942032 fd Linux raid autodetect
livecd root #

All of which looks right and proper.

Gentoo mkraid errors

Note: These directions are works-in-progress... in fact, they might not even work at all until I find out why I'm ending up with non-bootable systems (looks like a bug in the 2.6 kernel).

So, I've created my /etc/raidtab file. I've done the "modprobe md" and "modprobe dm-mod" commands. I'm able to "cat /proc/mdstat" which shows that I have no personalities and no unused devices. But, when I try to use the "mkraid" command, I get the following error:

# mkraid /dev/md0
cannot determine md version: no MD device file in /dev.

The fix (according to the wiki is):

cd /dev ; MAKEDEV md

The key resource to setting up Gentoo on a RAID:

Gentoo Install on Software RAID

Now, they're not putting their swap file on a RAID1. The reason that I put my swap in a RAID1 partition is that I don't want the system to crash if a drive dies (which it will, if you have 2 seperate, non-RAID swap partitions). It's a question of whether you want a slightly higher level of stability, or if you want speed.

Other possible gotches:

1) (not confirmed) After you fdisk, you may want to reboot so make sure that the disks are no longer in use. Otherwise you may get "bd_claim" errors when trying to setup the RAID array.

(still fighting with this a few hours later... going to start a new post)

Gentoo 2005.0 on Gigabyte GA-6VA7+ (part 2)

Note: These directions are works-in-progress... in fact, they might not even work at all until I find out why I'm ending up with non-bootable systems (looks like a bug in the 2.6 kernel).

Okay, tossed the new 2005.0 boot CD in, and I'm booting it up. Just trying a standard default boot for the moment (not trying to use the "nohotplug" option yet).

Load the RAID modules, and verify some things:

# modprobe md
# modprobe dm-mod
# ifconfig { verifies that the ethernet card is working }
# ls -l /dev/hd*

Now, use fdisk to blow away and setup partitions. (Note: You will lose all data on these disks when you perform this step.) Use the 'w' command to confirm the destruction of all partitions when you're finished.

Setup the new partitions:

# fdisk /dev/hda

Command: n
Command action: p
Partition number: 1
First cylinder: 1
Last cylinder: +128M
Command: a
Partition number: 1
Command: t
Hex code: fd

Command: n
Command action: p
Partition number: 2
First cylinder: [enter]
Last cylinder: +2048M
Command: t
Partition number: 2
Hex code: fd

Command: n
Command action: p
Partition number: 3
First cylinder: [enter]
Last cylinder: +2048M
Command: t
Partition number: 3
Hex code: fd

Command: n
Command action: p
First cylinder: [enter]
Last cylinder: [enter]
Command: t
Partition number: 4
Hex code: fd

Command: p

Command: w

This gives me a 128MB boot area, a 2GB swap area, a 2GB root area, with the rest of the disk set aside for my LVM partitions. Repeat the above commands to configure the 2nd disk in the same fashion. Note that I'm using a different partition type then that shown in chapter 4.c. The 'fd' partition type is what I need to use since all 4 partitions on hda/hdc are going to be put into a software RAID1 set.

The 3rd disk is a single primary partition with the '8E' (LVM) type. (Need to verify this, but I'm pretty sure that's correct.)

Create your "/etc/raidtab" configuration file (I used "nano -w /etc/raidtab", but other text editors will work).

# this config is for mirroring /dev/hda with /dev/hdc
# /boot (RAID1)
raiddev /dev/md0
raid-level 1
nr-raid-disks 2
nr-spare-disks 0
chunk-size 32
persistent-superblock 1
device /dev/hda1
raid-disk 0
device /dev/hdc1
raid-disk 1

# *swap* (RAID1)
raiddev /dev/md1
raid-level 1
nr-raid-disks 2
nr-spare-disks 0
chunk-size 8
persistent-superblock 1
device /dev/hda2
raid-disk 0
device /dev/hdc2
raid-disk 1

# / (RAID1)
raiddev /dev/md2
raid-level 1
nr-raid-disks 2
nr-spare-disks 0
chunk-size 32
persistent-superblock 1
device /dev/hda3
raid-disk 0
device /dev/hdc3
raid-disk 1

# LVM (RAID1)
raiddev /dev/md3
raid-level 1
nr-raid-disks 2
nr-spare-disks 0
chunk-size 16
persistent-superblock 1
device /dev/hda4
raid-disk 0
device /dev/hdc4
raid-disk 1

# end of /etc/raidtab

Create the raid set(s).

# mkraid /dev/md0
# mkraid /dev/md1
# mkraid /dev/md2
# mkraid /dev/md3

If you get the error message: "raid_disks + spare_disks != nr_disks" when attempting to create any of your RAID sets, go back and verify your "/etc/raidtab" file as well as verifying your disk partitions. The RAID sets will build in the background and you should periodically monitor their progress using "cat /proc/mdstat". Another possibility is that you have set the "chunk-size" setting to be too small or too large (e.g. "chunk-size 4" did not work for me, but "chunk size 8" worked fine).

Building the raid sets may take a while, so once again, I'll come back to this point in a few hours.

Update: mkraid is tossing errors (see my next post)

Gentoo 2005.0 on Gigabyte GA-6VA7+ (part 1)

Note: These directions are works-in-progress... in fact, they might not even work at all until I find out why I'm ending up with non-bootable systems (looks like a bug in the 2.6 kernel).

While I swap around some memory modules in the old EPIA box, I'm also going to take a swipe at using the new 2005.0 Gentoo image and build a 2nd box.

This is an old Celeron 350Mhz or 400Mhz CPU with 384MB of RAM (Gigabyte GA-6VA7+). About the same power as the VIA EPIA motherboard. I've already gone into the BIOS and disabled all optional ports (serial, parallel, etc). Drives are configured as:

Pri M: 72GB - /dev/hda
Pri S: (free)
Sec M: 72GB - /dev/hdc
Sec S: CD-ROM - /dev/hdd

I also have a 3rd hard drive (80GB, /dev/hde) hooked up to an old Promise FastTrak66 PCI RAID card. It uses the PDC20262 chip and I'm really just using it as an IDE card rather then making use of its RAID functionality.

I plan on mirroring using the two 72GB master drives and using the 80GB as a backup/scratch disk. Similar setup and goals as the EPIA box from last June's install.

Gentoo EPIA Install (part 3)

The RAID array has finally finished synchronizing. Continuing on, referecing my June 2004 notes about setting up LVM.

Things went well up until I started extracting the portage tree. Then I started to get random errors, which means I need to stop and verify that the hardware is behaving properly.

(Time to dig out my memory test programs...)

Gentoo EPIA Install (part 2)

Note: These directions are works-in-progress... in fact, they might not even work at all until I find out why I'm ending up with non-bootable systems (looks like a bug in the 2.6 kernel).

First up, still using the 2004.0 Gentoo Boot CD and referring to my old notes from last year. Also note that I rebuilt it in June 2004, so it may be better to look at those notes. Especially the "gentoo nohotplug" command during the boot process.

Going to use last June's installation notes for the most part, with a few notes here if I change anything.

Key commands (that don't do anything other then report status):

/sbin/ifconfig - verifies networking
ls -l /dev/hd* - shows hard drives
hdparm -i /dev/hda - display information about hda

Now to fire up fdisk and wipe any existing partitions. Then I'm going to create the same partitions I did last year. (Helps to refer to the Gentoo handbook for this step.)

Currently making my raid volumes, no changes from the June 2004 instructions. Verifying progress using "cat /proc/mdstat".

Gentoo EPIA Install (part 1)

Note: These directions are works-in-progress... in fact, they might not even work at all until I find out why I'm ending up with non-bootable systems (looks like a bug in the 2.6 kernel).

So, I'm back to re-building my VIA EPIA linux box (it was scavanged for another project for a few months). Once again, I'm digging out my Gentoo CDs and I'm going to simply build a box where I can house my music MP3s and do some Apache / PostGreSQL work. Since this is an EPIA box, the EPIAWiki.org site comes in handy. I'll also be referring back to my April 2004 notes to speed up my setup process.

Hardware:

(1) VIA EPIA ME6000 (EPIA M series), 600Mhz fanless CPU
(2) 160GB 5400rpm hard drives (primary master, secondary master)
(1) DVD-ROM (secondary slave)
(1) Morex Venus 668 Black Case
(1) 512MB PC2100 DIMM

First up is BIOS settings:

Standard CMOS Features
- Halt On: All, But Keyboard
- all other settings are automatic

Advanced CMOS Features
- Virus Warning: Disabled
- CPU L2 Cache ECC Checking: Enabled
- Quick Power On Self Test: Disabled
- First Boot Device: Floppy
- Second Boot Device: CDROM
- Third Boot Device: HDD-0
- Boot Other Device: Disabled
- Swap Floppy Drive: Disabled
- Boot Up Floopy Seek: Enabled
- Boot Up NumLock Status: Enabled
- Typematic Rate Setting: 30
- Typematic Delay: 250
- Security Option: Setup (not configured)
- Dispaly Full Screen Logo: Disabled
- Show Summary Information: Enabled
- Display Small Logo: Enabled

Advanced Chipset Features
- AP Aperture Size: 32M
- CPU to PCI POST Write: Enabled
- Select Display Device: CRT
- Panel Type: 1024x768
- TV Type: NTSC
- CPU Direct Access FB: Enabled

Integrated Peripherals
- Super IO Devices: All disabled except FDC Controller
- Onboard IDE Channel 1: Enabled
- Onboard IDE Channel 2: Enabled
- IDE Prefetch Mode: Enabled
- Display Card Priority: AGP
- Frame Buffer Size: 32MB (amount of RAM to use for video card)
- AC97 Audio: Disabled
- MC97 Modem: Disabled
- VIA OnChip LAN: Enabled
- USB Keyboard Support: Disabled
- Onboard LAN Boot ROM: Disabled
- Onboard Fast IR: Disabled

Power Management Setup
- ACPI Suspend Type: S1&S3
- HDD Power Down: Disabled
- Power Management Timer: Disabled
- Video Off Option: Suspend -> Off
- Power Off by PWRBTN: Delay 4 Sec
- Run VGABIOS if S3 Resume: Auto
- AC Loss Auto Restart: On
- Peripherals Activities: (not important)
- IRQs Activities: (not important)

PnP/PCI Configurations
- PNP OS Installed: Yes
- (ignoring the rest of the options)

PC Health Status
- (information only)

Frequency/Voltage Control
- (left alone)