Friday, July 27, 2012

How to implement physical servers snapshots on AIX 5.2, 5.3, 6.1, 7.1

Concept

Here is the short description of how the process works.
  • Make sure your system has at least 2 physical hard drives (hdisk0 and hdisk1).
  • Install clean AIX OS on hdisk0. This will become your "MASTER" disk.
  • Include required packages (see Prerequisites, below)
  • Perfect the OS, install any software that you would like to have as part of the standard OS
  • Setup snapshot scripts and create the snapshot

Prerequisites

  • All AIX versions require 2 HDD disks. One disk, will call it "MASTER" holds base line OS. The other, let's call it "CLONE", is the disk that the MASTER will be recovering the snapshot to effectively overwriting it every time you will need to restore your snapshot.
  • For LPAR's, allocate 2 virtual disks in VIOS.
  • All commands must be executed as root.

AIX 5.2 software

bos.alt_disk_install 
OR install file set:(lslpp -L bos.alt_disk_install.rte)

AIX 5.3 software

bos.alt_disk_copy    
OR install file set: (lslpp -L bos.alt_disk_copy.rte)
 bos.alt_disk_install.boot_images
 bos.alt_disk_install.rte     
 bos.msg.en_US.alt_disk_install.rte

AIX 6.1, 7.1 software

  • All required packages are present in the default installation

Banner text

Banner will be displayed during the reimaging process to anyone who will be trying to connect to the system via SSH, CONSOLE or TELNET prior to the end of the snapshot recovery process. Touch /etc/banner_default
WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING
 
       THE SYSTEM IS BOOTED TO AN ORIGINAL SNAPSHOT DISK
               SNAPSHOT RECOVERY IS IN PROGRESS
 
       TO PRESERVE INTEGRITY OF THE SYSTEM DO NOT LOGIN!

Enable banner for SSH

To enable banner for SSH, simply uncomment "Banner" in /etc/ssh/sshd_config
# no default banner path
Banner /etc/banner
NOTE: During the reimaging process /etc/banner_default will be renamed to /etc/banner

Scripts and entries

You will need to add scripts and append standard configuration files as part of this process. Feel free to customize as you wish.

Append /etc/inittab

The line below is used to check whether user requested snapshot restore during the last reboot.
resnap:2:once:/etc/rc.resnapshot >/dev/console 2>&1

Append /etc/profile

Simple notification mechanism if a user is trying to login to the system during the snapshot restore.
#Note: MASTER DISK hdisk0 is intentionally hard coded here
PATH=$PATH:/usr/local/bin
export PATH
if [ `bootinfo -b` == "hdisk0" ]; then
 clear
 cat /etc/banner_default
 while true
  do
   echo "Are you sure you still wish to login? (y or n) :\c"
   read CONFIRM
   case $CONFIRM in
     y|Y|YES|yes|Yes) break ;;
     n|N|no|NO|No)
     echo Aborting - you entered $CONFIRM
     exit
     ;;
    *) echo Please enter only y or n
   esac
  done
fi

Create /usr/local/bin/restore_snapshot

Create file /usr/local/bin/restore_snapshot with execute permissions (500). This is the script that you will be executing to request snapshot restore.
#!/bin/sh

#Note: MASTER DISK hdisk0 is intentionally hard coded
# Make sure only root can run our script
if [ "$(id -u)" != "0" ]; then
   echo "This script must be run as root" 1>&2
   exit 1
fi

if [ `bootinfo -b` != "hdisk0" ]
then
        echo "System is booted to the secondary hdisk"
        echo "Changing boot device priority to boot to the master hdisk for snapshot restore..."
        set -x
        bootlist -m normal hdisk0     
        set +x
        echo "Rebooting..."
        /usr/sbin/shutdown now -r
else
        /etc/rc.resnapshot
fi

AIX 5.2 /etc/rc.resnapshot

Create /etc/rc.resnapshot (chmod 555). This script will be called during OS boot. It will check whether the snapshot restore must start after the reboot.
#!/bin/sh

#Note: MASTER DISK hdisk0 is intentionally hard coded
#Note: CLONE DISK: hdisk1 is intentionally hard coded

if [ `/usr/sbin/bootinfo -b` == "hdisk0" ];then
        echo "INFO: System is booted to the master image hdisk."
        echo "INFO: Placing banner."
        set -x
        cp /etc/banner_default /etc/banner
        set +x
        echo "INFO: Removing altinst_rootvg"
        set -x
        /usr/sbin/alt_disk_install -X altinst_rootvg
        set +x
        echo "INFO: Restoring snapshot. If successful, the system will reboot automatically."
        echo "INFO: Please wait..."
        set -x
        /usr/sbin/alt_disk_install -C hdisk1
        if [ $? -eq 0 ]; then
                echo "rootvg clone operation suceeded!"
                echo "Rebooting..."
                /usr/sbin/reboot
        else
                echo "rootvg clone operation failed!"
                echo "Aborting..."
        fi
        set +x
else
        set -x
        /usr/bin/rm -f /etc/banner
        set +x
fi

AIX 5.3, 6.1, 7.1 /etc/rc.resnapshot

Create /etc/rc.resnapshot (chmod 555)
#!/bin/sh

#Note: MASTER DISK hdisk0 is intentionally hard coded
#Note: CLONE DISK: hdisk1 is intentionally hard coded

if [ `/usr/sbin/bootinfo -b` == "hdisk0" ];then
        echo "INFO: System is booted to the master image hdisk."
        echo "INFO: Placing banner."
        set -x
        cp /etc/banner_default /etc/banner
        set +x

        echo "INFO: Removing altinst_rootvg"
        set -x
        /usr/sbin/alt_rootvg_op -X altinst_rootvg
        set +x
        echo "INFO: Restoring snapshot. If successful, the system will reboot automatically."
        echo "INFO: Please wait..."
        set -x
        /usr/sbin/alt_disk_copy -d hdisk1 -r
        set +x
else
        set -x
        /usr/bin/rm -f /etc/banner
        set +x
fi

Create first snapshot

Please note that you do not "create" the initial snapshot per se.
Your initial snapshot is your current OS hdisk0. What you are doing
is copying the good disk hdisk0 onto the second hdisk1 that after the
whole process is done will become the disk, where you will be testing
stuff. Once test is finished, you will simply execute restore_snapshot
script to request the restore of the snapshot. The restore process will
reboot the OS and will automatically start overwriting the test disk with
the clean OS image. Once that process finishes, the system reboots once
again and you are back on a clone disk.

To initiate the first cloning process, simply execute:

/usr/local/bin/restore_snapshot
And you are all set!

Important commands

#Display AIX disk information
#CURRENT BOOT DISK:      bootinfo –b
#ALL DISKS:              lspv
#DISK DETAILS:           lscfg -vl hdisk0

#MOUNT MASTER WHEN BOOTED ON TEST
#alt_rootvg_op –W –d hdisk0

#UNMOUNT MASTER WHEN BOOTED ON TEST
#alt_rootvg_op –S
#[alternatively specify –t to rebuild boot image.
Not recommended for minor changes.

Helpful links

No comments:

Post a Comment