Provisioning HP firmware with Cobbler

When dealing with a bunch of servers one of the many things you are expected to do frequently is to update your systems firmware. In this blog post I'll describe how I implemented this process for HP ProLiant DL servers. Since this basically is a systems provisioning issue I prefer that this process to be integrated with my current PXE based deployment infrastructure which is based on Cobbler. Luckily HP does provide a Service Pack for Proliant (SPP) as a bootable ISO which does an unattended firmware update for all supported components (iLO, BIOS, disks, NIC's, HBA's, etc), it actually works pretty well.

Getting the required files

You need to download the latest SPP version available, for now that's 864794_001_spp-2016.04.0-SPP2016040.2016_0317.20.iso.

[root@mainframe iso]# mount -o loop ./864794_001_spp-2016.04.0-SPP2016040.2016_0317.20.iso /mnt
mount: /dev/loop0 is write-protected, mounting read-only

After examining the contents of the ISO the required files are found in:

[root@mainframe /]# cd /mnt/pxe
[root@mainframe pxe]# cd spp2016040/
[root@mainframe spp2016040]# ll
total 20561
-rw-rw-rw-. 1 root root 16151162 Mar 17 20:16 initrd.img
-rw-rw-rw-. 1 root root     1783 Mar 17 20:16
-rw-r--r--. 1 root root  4900496 Mar 17 20:16 vmlinuz
[root@mainframe spp2016040]#

So we found a PXE bootable kernel and initrd, now we can add a distro to Cobbler! Before we start copying files we need to create a directory for this new distro first.

[root@mainframe spp2016040]# mkdir /var/www/cobbler/ks_mirror/spp2016040
[root@mainframe spp2016040]# cp * /var/www/cobbler/ks_mirror/spp2016040/

Besides a kernel and initrd the firmware update procedure requires access to the files included in the ISO, but how should these files be accessible? After examining /mnt/pxe/README.txt and /mnt/pxe/spp2016040/ you're probably depresessed that it seems that only NFS and CIFS are supported, I certainly was. Before going back on Prozac I wanted to make sure there wasn't another, perhaps undocumented, way of doing this. Let's examine the initrd to see what's going on in there..

[root@mainframe spp2016040]# mkdir ~/t
[root@mainframe spp2016040]# cp -f * ~/t
[root@mainframe spp2016040]# cd ~/t
[root@mainframe t]# file initrd.img
initrd.img: LZMA compressed data, streamed
[root@mainframe t]# cat initrd.img | xz -d | cpio -id
134408 blocks
[root@mainframe t]# ll
total 20592
drwxrwxrwx. 2 root root     4096 Jun  6 07:55 bin
drwxrwxrwx. 4 root root     4096 Jun  6 07:55 etc
-rwxr-xr-x. 1 root root     8808 Jun  6 07:55 init
-rw-r--r--. 1 root root 16151162 Jun  6 07:15 initrd.img
drwxrwxrwx. 4 root root       35 Jun  6 07:55 lib
drwxrwxrwx. 2 root root     4096 Jun  6 07:55 lib64
drwxrwxrwx. 3 root root       18 Jun  6 07:55 mnt
drwxrwxrwx. 2 root root        6 Jun  6 07:55 proc
drwxrwxrwx. 2 root root        6 Jun  6 07:55 root
drwxrwxrwx. 2 root root      100 Jun  6 07:55 sbin
-rw-r--r--. 1 root root     1783 Jun  6 07:15
drwxrwxrwx. 2 root root        6 Jun  6 07:55 tmp
drwxrwxrwx. 7 root root       61 Jun  6 07:55 usr
drwxrwxrwx. 6 root root       47 Jun  6 07:55 var
-rw-r--r--. 1 root root  4900496 Jun  6 07:15 vmlinuz

An obvious target to look at is ofcourse the init script, it seems that this script is calling a function called mount_media() which calls a shell script. Lets examine that script to see what kind of URI's are supported.

[root@mainframe t]# find . -name
Usage: $0 [OPTIONS] url
Description: mounts a smbfs/cifs or nfs volume when given the path in
             url format
  url - a url of the device to mount

So it seems HTTP and HTTPS are also supported which is nice since Cobbler already relies on a working Apache service, now we only have to make the ISO accessible. I recommend putting the ISO right next to the kernel and initrd so that the ISO is accessable through HTTP.

[root@mainframe iso]# cp 864794_001_spp-2016.04.0-SPP2016040.2016_0317.20.iso /var/www/cobbler/ks_mirror/spp2016040/

Adding a distro

Now that all required files are in place you can add the distro to Cobbler. Please note that I already changed the ksmeta variable to match the HTTP transport.

[root@mainframe iso]# cobbler distro add --name=spp2016040 --kernel=/var/www/cobbler/ks_mirror/spp2016040/vmlinuz --initrd=/var/www/cobbler/ks_mirror/spp2016040/initrd.img --ksmeta="media=net root=/dev/ram0 splash quiet hp_fibre showopts TYPE=AUTOMATIC AUTOPOWEROFFONSUCCESS=no AUTOREBOOTONSUCCESS=yes iso1=http://@@http_server@@/cobbler/ks_mirror/spp2016040/864794_001_spp-2016.04.0-SPP2016040.2016_0317.20.iso iso1mnt=/mnt/bootdevice"

Verify your distro configuration:

[root@mainframe iso]# cobbler distro report --name=spp2016040
Name                           : spp2016040
Architecture                   : x86_64
TFTP Boot Files                : {}
Breed                          : redhat
Comment                        :
Fetchable Files                : {}
Initrd                         : /var/www/cobbler/ks_mirror/spp2016040/initrd.img
Kernel                         : /var/www/cobbler/ks_mirror/spp2016040/vmlinuz
Kernel Options                 : {'hp_fibre': '~', 'iso1mnt': '/mnt/bootdevice', 'media': 'net', 'AUTOPOWEROFFONSUCCESS': 'no', 'quiet': '~', 'root': '/dev/ram0', 'showopts': '~', 'iso1': 'http://@@http_server@@/cobbler/ks_mirror/spp2016040/864794_001_spp-2016.04.0-SPP2016040.2016_0317.20.iso', 'splash': '~', 'AUTOREBOOTONSUCCESS': 'yes', 'TYPE': 'AUTOMATIC'}
Kernel Options (Post Install)  : {}
Kickstart Metadata             : {}
Management Classes             : []
OS Version                     : generic26
Owners                         : ['admin']
Red Hat Management Key         : <<inherit>>
Red Hat Management Server      : <<inherit>>
Template Files                 : {}

Adding a profile

Before you can use the SPP via PXE you first need to associate a profile with the distro, mind the empty kickstart variable.

[root@mainframe iso]# cobbler profile add --name=spp2016040 --distro=spp2016040 --kickstart=""

Verify your profile configuration:

[root@mainframe iso]# cobbler profile report --name=spp2016040
Name                           : spp2016040
TFTP Boot Files                : {}
Comment                        :
DHCP Tag                       : default
Distribution                   : spp2016040
Enable gPXE?                   : 0
Enable PXE Menu?               : 1
Fetchable Files                : {}
Kernel Options                 : {}
Kernel Options (Post Install)  : {}
Kickstart                      :
Kickstart Metadata             : {}
Management Classes             : []
Management Parameters          : <<inherit>>
Name Servers                   : []
Name Servers Search Path       : []
Owners                         : ['admin']
Parent Profile                 :
Internal proxy                 :
Red Hat Management Key         : <<inherit>>
Red Hat Management Server      : <<inherit>>
Repos                          : []
Server Override                : <<inherit>>
Template Files                 : {}
Virt Auto Boot                 : 1
Virt Bridge                    : xenbr0
Virt CPUs                      : 1
Virt Disk Driver Type          : raw
Virt File Size(GB)             : 5
Virt Path                      :
Virt RAM (MB)                  : 512
Virt Type                      : xenpv

Verify PXE configuration

So after adding the required files, a distro object and a profile object let's see if the PXE configuration is properly generated.

[root@mainframe iso]# cobbler sync

In the case of EFI booting check /var/lib/tftpboot/grub/efidefault if it contains something like this:

title spp2016040
    root (nd)
    kernel /images/spp2016040/vmlinuz  hp_fibre iso1mnt=/mnt/bootdevice TYPE=AUTOMATIC media=net AUTOPOWEROFFONSUCCESS=no quiet showopts iso1= splash AUTOREBOOTONSUCCESS=yes root=/dev/ram0
    initrd /images/spp2016040/initrd.img

In the case of syslinux PXE booting check the file /var/lib/tftpboot/pxelinux.cfg/default:

LABEL spp2016040
        kernel /images/spp2016040/vmlinuz
        MENU LABEL spp2016040
        append initrd=/images/spp2016040/initrd.img hp_fibre iso1mnt=/mnt/bootdevice TYPE=AUTOMATIC media=net AUTOPOWEROFFONSUCCESS=no quiet showopts iso1= splash AUTOREBOOTONSUCCESS=yes root=/dev/ram0
        ipappend 2

That's it!

comments powered by Disqus