One of the big advantages of using the Logical Volume Manager (LVM) is that it introduces software raid, or “mirroring”. This allows all sectors of a disk to be replicated across to another one. If one of these disks fails – as hardware is wont to do – then a new disk can simply be hot-swapped in, and a few commands executed to resynchronise.
The following post outlines a few precautionary commands that can be run periodically to aid this kind of maintenance, and then I follow up with three procedures for different disk failure scenarios.
Prerequisites: ensuring system is recoverable
The following list of steps are suggested to aid you in later system recovery.
- Load all LDM patches
- Regularly back up filesystems and check the backups’ integrity.
- Separately back up important system recovery files:
- Regularly print system configurations. A hardcopy is useful during recovery. Cron job could save the output from the following:
- ioscan -fk
- vgdisplay -v
- lvlnboot -v
- lvdisplay -v /dev/vgXX/lvolYY
- pvdisplay -v /dev/dsk/c#t#d#
- Back up the LVM configuration after every change. This will create files under /etc/lvmconf of the form vgXX.conf
# /etc/vgcfgbackup /dev/vgXX
# lvlnboot -R
Replacing a non-boot disk without a mirror
Suppose we have the following arrangement of physical disks and logical volumes:
Scenario: Disk 52.4.0 (c0t2d0) dies and is unmirrored. Disk 52.6.0 (c0t0d0) is the boot disk.
The replacement procedure is as follows:
- Replace physical disk
- Boot into single-user mode, ie:
- Boot from primary, interact with IPL
- ISL> lsautofl (returns boot string)
- ISL> hpux -is (52.6.0;0) /stand/vmunix
- Restore LVM config/headers to new disk:
# vgcfgrestore -n /dev/vg00 /dev/rdsk/c0t2d0
# vgchange -a y /dev/vg00 (forces reexamination of disks)
# pvdisplay -v /dev/dsk/c0t2d0
(since part of lvol5 was lost, it will all need to be recovered to maintain filesystem integrity)
# newfs -F vxfs /dev/vg00/rlvol5
Replacing a mirrored non-boot disk
If the failed disk is mirrored, the fix is substantially more simple – hence the reason why we mirror disks in the first place.
Scenario: Disk at 52.4.0 fails. It’s mirrored to disk 52.5.0.
- Hot-swap new disk.
- Restore LVM headers:
# vgcfgrestore -n /dev/vg00 dev/rdsk/c0t2d0
# vgchange -a y /dev/vg00 # vgsync /dev/vg00
When the sync completes, the mirrors are ready again.
Replacing mirrored boot disk
A mirrored boot disk is simple to replace, and the procedure is similar to a non-boot disk. However, the new mirror needs to be made bootable as well. Perform the following commands, according to your disk device number.
- Hot-swap disk.
# pvcreate -f -B /dev/rdsk/c2t5d0 # vgcfgrestore -n /dev/vg00 /dev/rdsk/c2t5d0 # vgchange -a -y /dev/vg00 # mkboot /dev/rdsk/c2t5d0 # mkboot -a "hpux -lq(;0)/stand/vmunix" /dev/rdsk/c2t5d0 # lvlnboot -R # vgsync /dev/vg00