Lofland bLOG

See page 495 of “HPUX CSA” by Rafeeq Rehman
This entry contains notes from training by coworkers, personal experience, and the above mentioned book. I do not claim any of it to be original with me, unless you see a mistake. I am sure the mistakes are mine.

Take a look at your disks. An easy way is to use bdf. This is kind of the HPUX equivalent of df.

vgdisplay will show all of the existing volume groups, and vgdisplay -v will give the details about the logical and physical volumes in the group.

Each volume group has a unique directory under /dev/ where the LVM device files are kept.
Here we call them vg*

/dev/vg00 will usually be the volume for the onboard disks, where HPUX lives. The other vg’s will probably exist on drives from an SAN

Each /dev/vg*/ directory contains three things:

A group file.
- It is a “c” (character special) file
- Each volume group needs one of these files. It is created before the vg is actually built
- It must have a unique “minor number.” The minor number is a hexidcimal number starting with 0x in the long listing. The first two digits are the vg #. The last four digits are always 0’s. If you name your vg groups vg01, vg02, vg03, etc, then you can make the minor number correspond to the vg name. Otherwise I’m not sure that there is any direct correlation.
- The major number for all LVM device files is 64
root:/dev/vg00> ll group
crwxr-xr-x 1 root sys 64 0×000000 Dec 12 2003 group

The minor number for the group file must be unique for each vg (it is a hex number). You only work with the first two digits, the last four are always 0000.
A “b” (block device) file for each logical volume
- This is the device you mount

A “c” (character special) file for each logical volume called rlvolname
-This is the raw device, used to format the filesystem

Each logical volume device file will have the major number 64, and the minor number wil be the same as the group file’s, but with the last digits showing the lv number. Again, like the vg#’s, they may or may not correspond to the names (LV01, LV02, etc.), but they will be sequential. See pg 499 of the HPUX CSA book.
You can see the various mount pounts (logical volumes) for HPUX under /dev/vg00
- Remember, group is not a lvol, it is a special file for the vg

The first thing you have to do before making a new vg is find out what the 0x numbers are for the goup file in each volume group, so that we can make a new one.

To find all of them:
find /dev -name group -exec ls -al {} \;

root:/root> find /dev -name group -exec ls -al {} \;
crwxr-xr-x 1 root sys 64 0×000000 Dec 12 2003 /dev/vg00/group
crw——- 1 root sys 64 0×040000 Mar 8 11:04 /dev/vgfapp/group
crw——- 1 root sys 64 0×050000 Mar 8 11:04 /dev/vgfdata/group

or just typing this usually works too:
ll /dev/*/group
(ll = ls -la on non HPUX machines)

So now we can use 0×020000, 0×030000, or 0×060000 and above. They don’t have to be sequential, but there isn’t any point in using random numbers here.

mkdir /dev/vg02

man vgcreate: look under examples, the mkdir and mknod command is right there

mknod /dev/vg02/group c 64 0×020000

We are telling it to make a character special file, with a major number of 64 (all volume groups have a major number of 64) and the minor number of 0×020000 (which we explained above).

Now we need to find some free hard drives (disks) to use:

The disks on the system are listed here:
ls /dev/dsk/*

To list all of the existing volume groups, and show what disks they are using you can run this:
vgdisplay -v

ioscan -funCdisk shows a bunch of disks, but how do we know how/when/where they are used? Plus with a SAN, there is a primary and an alternate path to each disk (they look like two disks to the system, but they are not), so the SAN disks show twice. You really only have half of the number listed.

Again, how do I know if they are bing used? - HP has no good way of telling you waht is not used.

ls /dev/dsk/* will tell us what disks are available to HPUX
for i in `ls /dev/dsk/*`; do pvdisplay $i; done 1>/dev/null
-so the errors are the drives that are not in a volume group (we piped the stin to null and just watched stderr)

or

To get a good list of available drives:
for i in `ls /dev/dsk/*`; do pvdisplay $i; done 2>/tmp/disk.out
cat /tmp/disk.out | grep -v Could | grep -v belongs | cut -d’ ‘ -f6 | sed s/\”//g | sed s/\.$//g > /tmp/disklist
(rm /tmp/disk.out)

Ok, now /tmp/disklist has a list of available disks.
Just to double check our list:

for i in `cat /tmp/disk2/`;do pvdisplay $i;done 2> /dev/null

-We piped error to null, but ALL should return errors, thus Should be BLANK, meaning everything in the list was NOT a volume

(Remember, all disks on an HP box are in a vg if they are used.)

Now we know what disks are not being used, however, half of those are alternate links. How do we know which ones?

EMC SAN Arrays:

inq -sortsymm = this is an EMC tool, so it works if we have EMC disks in the box Each disk has a serial number. The first 3 digits (220) is the frame serial number, next two E8 is the disk, last part is the frame. So here you can see what drives are the same drive. The Array usually has a bunch of 72 gig drives, but they are shown to us as 8 gig drives. Internal drives will always be a d0, b/c they are not carved up.

Usually with EMC disks, the C#’s are different but the t and d #’s are the same for the same drive on the EMC. It doesn’t have to be that way though, so watch those serial numbers.

Each EMC drive has a unique serial #, so you can see if two disks on the local system are just redundant paths to the same EMC path.

To combine the free disk list from HPUX with the serial number list from the Hitachi SAN, and get a list of availalbe drives with serial numbers:

for i in `ls /dev/dsk/*`; do pvdisplay $i; done 2>/tmp/disk.out
cat /tmp/disk.out | grep -v Could | grep -v belongs | cut -d’ ‘ -f6 | sed s/\”//g | sed s/\.$//g > /tmp/disklist
rm /tmp/disk.out

inq -sortsymm | grep “rdsk” > /tmp/seriallist
for i in kk`cat /tmp/disklist | cut -d”/” -f4`; do grep $i /tmp/seriallist;done | sort -k 5 > /tmp/availabledisklist
rm /tmp/disklist
rm /tmp/seriallist
Echo Your list is in “/tmp/availabledisklist”
#Maybe we could combine a few more commands and ellimnate one more temp file, but the commands start to get insane

Obviously the DVD-ROM isn’t “available” and any other “local” drives are not what we want. The list should clearly show if they are EMC drives or something else.

Also, there are what may be EMC “control” disks, they are small 4 meg, and 2 meg drives. Don’t use these. Just use the ones that are in the standard 8gig size.

Hitachi SAN Arrays:

lunstat -ts | egrep “Serial|Device” | paste -s -d”\t\n” -

This will list the devices with the serial numbers, so you can see which drives are identical.
(Be sure to sanity check it all. The ctd numbers should be similar for same drives, when you add drives to the volume, it should automatically recognize them as “alernate paths” to the same drive.

Don’t use drives that are not on the SAN. They will have different serial numbers.

To combine the free disk list from HPUX with the serial number list from the Hitachi SAN, and get a list of availalbe drives with serial numbers:

for i in `ls /dev/dsk/*`; do pvdisplay $i; done 2>/tmp/disk.out
cat /tmp/disk.out | grep -v Could | grep -v belongs | cut -d’ ‘ -f6 | sed s/\”//g | sed s/\.$//g > /tmp/disklist
rm /tmp/disk.out

lunstat -ts | egrep “Serial|Device|Manufacturer” | paste -s -d”\t\t\n” - > /tmp/seriallist
for i in `cat /tmp/disklist | cut -d”/” -f4`; do grep $i /tmp/seriallist;done | sort -k 9 > /tmp/availabledisklist
rm /tmp/disklist
rm /tmp/seriallist
Echo Your list is in “/tmp/availabledisklist”
#Maybe we could combine a few more commands and ellimnate one more temp file, but the commands start to get insane

The list should clearly show if they are Hitachi drives or something else. We don’t want the drives made by “HP,” etc. They are either internal disks, or things like the DVD-ROM. You can always check on them with ioscan -funCdisk

(Note, that the Hitachi lunstat program works on EMC arrays, so you can use it if you want. However, the EMC inq program does not report the serial number on Hitachi drives, so you it will not work for them.)

Now, to create a volume group:

pvcreate /dev/rdsk/c29t12d2 -> told us it already belongs a to a volume group
pvdisplay /dev/dsk/c29t12d2 -> says no volume group
-problem is, the drive WAS a member of a volume group, but isn’t anymore
–if you vgreduce this drive out of the volume group before you export the vg, this probably won’t happen
-if you are SURE it is not used anymore then:
pvcreate -f /dev/rdsk/c29t12d2 - DANGER! this will wipe out a drive, very careful! the -f forces it.

man vgcreate
vgcreate -e 30000 vg02 /dev/dsk/c29t12d2 - must give it at least one drive.

NOTE: -e sets the Max PE per PV Once you create the volume group and use the disk, by default whatever size that disk is, that is the largets disk you can add by default. If you built it with a 4 gig disk and then later added a 20gig disk, you could only use 4 gigs on the 20gig disk. Instead, we say, give me X “PE Size (Mbytes) extents = Max PE per PV. John always uses 30000 for Max PE per PV, with PE Size (Mbytes) of 4 (default PE size).

vgdisplay (and there it is!) (size is Total PE x PE Size)

Now find the other drive with the same serial number and add it, it should show up as an “alternate link”

vgextend vg02 /dev/dsk/c27t12d2
- It figures out that this is the same disk and adds it as an “Alternate Link”
- You only need to use pvcreate on the disk once, not once for each “link”
vgkk
pvcreate /dev/rdsk/c29t12d3
pvdispaly /dev/rdsk/c29t12d3
pvcreate -f /dev/rdsk/c29t12d3
vgextend vg02 /dev/dsk/c29t12d3
vgreduce vg02 /dev/dsk/c29t12d3 (oops!)
vgextend vg02 /dev/dsk/c27t12d3 (Swapp controllers, so that second drive in the vg has the other controller as primary)
vgextend vg02 /dev/dks/c29t12d3
vgdispay -v
- now primary controller is alternated back and forth, b/c HPUX goes to primary first, then to backup, so this allows the system to use both controllers.

NOTE: We could just put all of the PV names on the same command line with the first vgcreate command, rather than using a bunch of vgextend commands. vgcreate will take the names of multiple physical volumes. A vgextend command will also probably take multiple pv names.

man lvcreate
(size is L or l by size or le number)
lvcreate -L 1000m -n lvol1 vg02
vgdisplay -v (now some disk space is missing from PV’s and there is an LV)

(try to mount it)
cd /
mkdir chris
mount /dev/vg02/lvol1 /chris
(NOT FORMATTED! )

newfs -F vxfs /dev/vg02/rlvol2
(newfs is a front end for mkfs)
mount /dev/vg02/lvol1 /chris
bdf
It worked!

Set permissions on the filesystem after you mount it. Setting them on the mount directory first doesn’t do any good.

(now make it 2000m)

lvextend -L 2000m /dev/vg02/lvol1

vgdisplay -v
(now it is bigger)

bdf
(not bigger )
We gave the lvol more, but not the filessytem

fsadm -b 2000m /chris

Now the FS has been expanded in place, online

(man pages for fsadm are incorrect, what we did is not in there)

swlist | grep -i online
- fsadm to do this is called “onlinejfs”

DIG FOR SOME FSADM documentation!

mount - will show it is there
vi /etc/fstab
add
/dev/vg02/lvol1 /chris vxfs delaylog 0 2 (0 & 2 is for when to check for dirty bit, 1 = at boot 2= later)

umount /chris

now you can just type
mount /chris
and it knows where to mount it

to remove a volume use vgexport

umount /chris
vgdisplay -v
(good to back it down)
(Remove alternate links first, if you do pri first, it just switches alt to pri, doesnt’ really matter, but…)
lvremove /dev/vg02/lvol1
vgreduce vg02 /dev/dsk/c27t12d3
vgreduce vg02 /dev/dsk/c29t12d3
(I don’t think you can reduce the last disk out of the volume group)
vgchange -a n vg02
vgexport vg02 (dn’t be in /dev/vg02 when you do this or it won’t remove the directory)
(gone)
vgdisplay -v

Create a mirrored volume?

lvextend -m 1 /dev/vgora/lvolora

-But that won’t work if we only have one pv in the vg! You can turn off strict, but it is really silly.

pvcreate /dev/rdsk/c35t9d2
vgextend vgora /dev/dsk/c35t9d2
vgextend vgora /dev/dsk/c34t9d2

lvextend -m 1 /dev/vgora/lvolora

Make a stripe set?

lvcreate -L 1000m -n lvolstripeset -i 3 -I 128 vgora