HOWTO Growable LVM+RAID5

From KuWiki

Prerequisites

• Linux kernel >=2.6.17 (>=2.6.21 for RAID 6)

Code maturity level options  --->
  [*] Prompt for development and/or incomplete code/drivers
Device Drivers  --->
  Multi-device support (RAID and LVM)  --->
    [*] Multiple devices driver support (RAID and LVM)
      <*>   RAID support
      <*>     RAID-4/RAID-5 mode
      [*]       Support adding drives to a raid-5 array (experimental)
      <*>   Device mapper support

• mdadm >=2.4.1
• LVM2

RAID 5

Creating a RAID 5 array

# mdadm --create <<the md device to be created>> --auto md --level=5 \
    --raid-devices=<<number of devices (excluding spare devices)>> \
    --spare-devices=<<number of spare devices>> \
    <<component devices ...>>

Ex.

# mdadm --create /dev/md0 --auto md --level=5 \
    --raid-devices=4 \
    --spare-devices=2 \
    /dev/hdb1 /dev/hdb2 /dev/hdb3 /dev/hdb5 /dev/hdb6 /dev/hdb7

Adding a device to the array

# mdadm <<the md device>> --add <<the device to be added>>

Ex.

# mdadm /dev/md0 --add /dev/hdb8

Removing a device from the array

# mdadm <<the md device>> --remove <<the device to be removed (which should be inactive>>

Ex.

# mdadm /dev/md0 --remove /dev/hdb8

Marking a device as faulty

# mdadm <<the md device>> --fail <<the device to be marked>>

Ex.

# mdadm /dev/md0 --fail /dev/hdb8

Growing a RAID 5 array

# mdadm --grow <<the md device>> \
    --raid-devices=<<number of devices (excluding spare devices)>>

Ex. Growing /dev/md0 to 5 devices, must have enough spare devices.

# mdadm --grow /dev/md0 --raid-devices=5

Display informations for the MD device

# mdadm --detail <<the md device>>

Ex.

# mdadm --detail /dev/md0

LVM

pvcreate: Initializing a device - Creating a physical volume

# pvcreate <<the device to be initialized>>

Ex.

# pvcreate /dev/md0

pvdisplay: Displaying attributes of a physical volume

# pvdisplay <<physical volume>>

Ex.

# pvdisplay /dev/md0

pvresize: Resize a disk or partition in use by LVM2

# pvresize <<physical volume>>

Ex. Resizing the physical volume to fit the device.

# pvresize /dev/md0

vgcreate: Creating a volume group

# vgcreate <<volume group name>> <<physical volumes ...>>

Ex.

# vgcreate vg1 /dev/md0
# vgcreate vg2 /dev/md1 /dev/md2
      .
      .
      .

vgchange: Activating a volume group

# vgchange -a y <<volume group>>

Ex.

# vgchange -a y vg1

vgchange: Deactivating the volume group

# vgchange -a n <<volume group>>

Ex.

# vgchange -a n vg1

vgremove: Removing a volume group

# vgremove <<volume group>>

Ex.

# vgremove vg1

vgextend: Adding physical volumes to a volume group

# vgextend <<volume group>> <<physical volumes ...>>

Ex.

# vgextend vg1 /dev/md1 /dev/md2

vgreduce: Removing physical volumes from a volume group

# vgreduce <<volume group>> <<physical volume>>

Ex.

# vgreduce vg1 /dev/md0

vgdisplay: Displaying attributes of a volume group

# vgdisplay <<volume group>>

Ex.

# vgdisplay vg1

lvcreate: Creating a logical volume

# lvcreate -L <<size>> -n <<logical volume name>> <<volume group>>

Ex. Creating a 50MB logical volume "lv1" in volume group "vg1" (/dev/vg1/lv1).

# lvcreate -L 50M -n lv1 vg1

Ex. Creating a logical volume fits entire volume group.

# vgdisplay vg1 | grep "Total PE"
Total PE              10230
# lvcreate -l 10230 vg1 -n lv1

Ex. Creating a logical volume(lv1) which is allocated in the specific physical volume(/dev/md0).

# lvcreate -L 50M -n lv1 vg1 /dev/md0

lvremove: Removing a logical volume

# lvremove <<logical volume>>

Ex.

# lvremove /dev/vg1/lv1

lvextend: Extending a logical volume

# lvextend -L <<size>> <<logical volume>>

Ex. Growing /dev/vg1/lv1 to 60MB.

# lvextend -L 60M /dev/vg1/lv1

Ex. Growing /dev/vg1/lv1 by 1MB.

# lvextend -L +1M /dev/vg1/lv1

ext2/ext3

# umount /dev/vg1/lv1
# resize2fs /dev/vg1/lv1
# mount /dev/vg1/lv1

ReiserFS

• Online

# resize_reiserfs -f /dev/vg1/lv1

• Offline

# umount /dev/vg1/lv1
# resize_reiserfs /dev/vg1/lv1
# mount /dev/vg1/lv1

lvreduce: Reducing a logical volume

# lvreduce -L <<size>> <<logical volume>>

ext2/ext3

# umount /dev/vg1/lv1
# resize2fs /dev/vg1/lv1 524288
# lvreduce -L-1G /dev/vg1/lv1
# mount /dev/vg1/lv1

ReiserFS

# umount /dev/vg1/lv1
# resize_reiserfs -s-1G /dev/vg1/lv1
# lvreduce -L-1G /dev/vg1/lv1
# mount /dev/vg1/lv1

Use Cases

Adding a new disk

1. Adding the disk(/dev/hdb11) to the RAID 5 array(/dev/md0)

   # mdadm /dev/md0 --add /dev/hdb11

2. Growing the array
   You may need use "--backup-file=<file>" option to specify a backup file if there has no extra spare disk.

   # mdadm --grow /dev/md0 --raid-devices=5

3. Resizing the physical volume

   # pvresize /dev/md0

4. Resizing the logical valume

   # lvextend -L 60M /dev/vg1/lv1

5. Resizing the file system (ReiserFS for example)
   
   Offline resizing:
   # umount /dev/vg1/lv1
   # resize_reiserfs /dev/vg1/lv1
   # mount /dev/vg1/lv1
   
   Online resizing:
   # resize_reiserfs -f dev/vg1/lv1

Replace a faulty disk

1. Remove a faulty disk(/dev/hdb5) from the array(/dev/md0)

   # mdadm /dev/md0 --remove /dev/hdb5

2. Add a good disk(/dev/hdb5) to the array(/dev/md0)

   # mdadm /dev/md0 --add /dev/hdb5

3. The RAID 5 array would be re-sync now. You can watch the sync progress by this command.

   # mdadm -D /dev/md0

Activate RAID and LVM from LiveCD

1. RAID
   
   # modprobe md
   # echo 'DEVICE partitions' >/etc/mdadm/mdadm.conf
   # mdadm --examine --scan >>/etc/mdadm/mdadm.conf
   # mdadm --assemble --scan --auto=yes # may need '--force'
2. LVM

   # lvm vgchange --ignorelockingfailure -a y

References

LVM HOWTO

Gentoo LVM2 installation