Friday, April 10, 2009

HMC Tips IX - System Plan

How to make a system plan from a running machine ?
# mksysplan -f marc.sysplan -m Machine-Name -v
where
marc.sysplan is the file name.

How to list a system plan ?
# lssysplan

How to delete a particular system plan ?
# rmsysplan

How to reploy a system plan on a managed server ?
# deploysysplan

How to copy a system plan from/into the HMC ?
# cpsysplan

HACMP Tips III - C-SPOC Commands

CSPOC commands are located under /usr/es/sbin/cluster/sbin. If you need, please add this directory to your PATH.

1. To create a user in a cluster :

# cl_mkuser

2. To change/set passwd for a user in a cluster :

# cl_chpasswd

3. To change a user's attribute in a cluster :

# cl_chuser

4. To remove a user in a cluster :

# cl_rmuser

5. To list users in a cluster :

# cl_lsuser

6. To create a group in a cluster :

# cl_mkgroup

7. To change attributes of a group :

# cl_lsgroup

8. To remove a group in a cluster :

# cl_rmgroup

9. To create a shared VG in a cluster :

# cl_mkvg

10. To change the attributes of a shared VG :

# cl_chvg

11. To extend a VG (add a PV to a VG) :

# cl_extendvg

12. To reduce a VG (remove a PV from a VG) :

# cl_reducevg

13. To mirror a VG :

# cl_mirrorvg

14. To unmirror a VG :

# cl_unmirrorvg

15. To list VG's in a cluster :

# cl_lsvg

16. To sync a VG :

# cl_syncvg

17. To import a volume group :

# cl_importvg

18. To import a VG into a list of nodes :

# cl_updatevg

19. To activate/varyon a VG :

# cl_activate_vgs VG_name

20. To deactivate/varyoff a VG :

# cl_deactivate_vgs VG_name

21. To create a LV :

# cl_mklv

22. To change the attributes of a LV :

# cl_chlv

23. To list a LV :

# cl_lslv

24. To remove a LV :

# cl_rmlv

25. To make copies for a LV :

# cl_mklvcopy

26. To remove copies for a LV :

# cl_rmlvcopy

27. To extend a LV :

# cl_extendlv

28. To create a file system in a cluster :

# cl_crfs

29. To create a LV followed by a FS :

# cl_crlvfs

30. To change the attribute of a FS :

# cl_chfs

31. To lsit file systems :

# cl_lsfs

32. To remove a FS :

# cl_rmfs

33. To show JFS2 file systems with all attributes :

# cl_lsjfs2

34. To list JFS2 filesysems and their resource groups :

# cl_showfs2

35. To activate/mount a file system :

# cl_activate_fs /filesystem_mountpoint

36. To activate/mount a NFS file system :

# cl_activate_nfs retry NFS_Hostname /filesystem_mountpoint

37. To deactivate/unmount a file system :

# cl_deactivate_fs /filesystem_mountpoint

38. To deactivate/unmount a NFS file system :

# cl_deactivate_nfs /filesystem_mountpoint

39. To export(NFS) a file system :

# cl_export_fs hostname /filesystem_mountpoint

40. To list the process numbers using the NFS directory :

# cl_nfskill -u /nfs_mountpoint

41. To kill the processes using the NFS directory :

# cl_nfskill -k /nfs_mountpoint

This post is under construction.
Please check the updated version in future.

HACMP Tips II - Utility Commands

Following utility commands are available under /usr/es/sbin/cluster/utilities.
If you use it very frequently, its better to add it to your PATH variable.

To list cluster and node topology information :
# cltopinfo (or) cllscf

To show the config for the nodes :
# cltopinfo -n

To show all networks configured in the cluster :
# cltopinfo -w

To show resources defined for all groups :
# clshowres

To show resources defined to selected the group :
# clshowres -g

To list all resource groups :
# cllsgrp

To list all file systems :
# cllsfs

To list the service IPs configured for a node :
# cllsip nodename

To show the whole cluster configuration :
# cldump

To show adapter information :
# cllsif

To show network information :
# cllsnw

To show the status of resource groups :
# clfindres

To list all resources :
# cllsres

To list all tape resources :
# cllstape

To list all nodes in a cluster :
# cllsnode

To list all application servers alongwith their start and stop scripts :
# cllsserv

To list all logical volumes in a cluster :
# cllslv

To list all IP networks in a cluster :
# cllsipnw

To list all alive network interfaces :
# cllsaliveif

HACMP Tips I - Files and Scripts

1. Where is the rhosts file located for HACMP ?

Location: /usr/es/sbin/cluster/etc/rhosts
Used By: clcomd daemon to validate the addresses of the incoming connections
Updated By:
It is updated automatically by clcomd daemon during the first connection.
But we should update it manually incase of configuring the cluster on an unsecured network.

2. What happened to ~/.rhosts file in the current version of HACMP ?

~/.rhosts is only needed during the migration from pre-5.1 versions of hacmp.
Once migration is completed, we should remove the file if no other applications need rsh.
From HACMP V5.1, inter-node communication for cluster services is handled by clcomd daemon.

3. What is the entry added to /etc/inittab for to IP Address Takeover ?

harc:2:wait:/usr/es/sbin/cluster/etc/harc.net # HACMP network startup

4. What is the entry added to /etc/inittab file due auto-start of HACMP ?
hacmp:2:once:/usr/es/sbin/cluster/etc/rc.init

5. What is the script used to start cluster services ?

/usr/es/sbin/cluster/etc/rc.cluster

6. rc.cluster calls a script internally to start the cluster services. What is that ?

/usr/es/sbin/cluster/utilities/clstart

7. What is the equivalent script for clstart in CSPOC ?

/usr/es/sbin/cluster/sbin/cl_clstart

8. What is the script used to stop cluster services ?

/usr/es/sbin/cluster/utitilies/clstop

9. What is the equivalent script for clstop in CSPOC ?

/usr/es/sbin/cluster/sbin/cl_clstop

10. What happens when clstrmgr daemon terminates abmornally ?

/usr/es/sbin/ckuster/utilities/clexit.rs script ahlts the system.
You can change the default behavior of the clexit.rc script by configuring
/usr/es/sbin/cluster/etc/hacmp.term

11. What script is invoked by clinfo daemon incase of a network or node event ?

/usr/es/sbin/cluster/etc/clinfo.rc


This post is under construction.
Please check the updated version in future.

HMC Tips VIII - DLPAR Requirements

1. What is version requirment for DLPAR operations ?

a. A P4 processor based pSeries system or later
b. October 2002 or later system microcode update
c. A HMC at version R3V1.0 or later
4. AIX 5L Version 5.2 or later

2. What are the AIX filesets required for DLPAR ?

a. # lslpp -l rsct.core*
b. # lslpp -l csm.client

3. What are the daemons required for DLPAR ?

#lssrc -a | grep rsct
ctrmc rsct 21044 active
IBM.CSMAgentRM rsct_rm 21045 active
IBM.ServiceRM rsct_rm 11836 active
IBM.DRM rsct_rm 20011 active
IBM.HostRM rsct_rm 20012 active
IBM.DMSRM rsct_rm 906 active
IBM.LparCmdRM rsct_rm 901 active

4. On HMC, how to list partitions recognized by DLAPR ?

# lspartition -dlpar

If all active AIX 5.2 partitions are listed as Active<1>, ..., DCaps:<0xf> your system has been set up properly for DLPAR.
If you're missing some active partitions or some partitions are reported as Active<0>, your system probably still has a network/hostname set up problem.

5. How to resolve name resolution issues between LPARs and HMC ?

Step I :
# vi /etc/resolv.conf
1.Same DNS server for LPARs and HMC
2.Remove the duplicate entries.

Step II:
Please check to see the that ct_node_id is unique for each node in the environment:
"cat /var/ct/cfg/ct_node_id"\

If duplicate ct_node_id values are found issue a recfgct on the problem node(s) to have a new/unique ct_node_id generated.
# /usr/sbin/rsct/install/bin/recfgct

(This command will start/restart ctcas,ctrmc system and will generate a new id in the file /var/ct/cfg/ct_node_id )

Step III:
ping from aix.

Step IV:
Please also do the following steps on the LPAR(s) to refresh RMC subsystem
/usr/sbin/rsct/bin/rmcctrl -z ----> Stops the RMC subsystem and all resource managers.
/usr/sbin/rsct/bin/rmcctrl -A ----> Adds and starts the RMC subsystem
/usr/sbin/rsct/bin/rmcctrl -p ----> Enables remote client connections

Step V:
Ensure /var directory is not 100% full
After expanding the /var directory, execute the following command.
# /usr/sbin/rsct/bin/rmcctrl -z
# rm /var/ct/cfg/ct_has.thl
# rm /var/ct/cfg/ctrmc.acls
# /usr/sbin/rsct/bin/rmcctrl -A

Step VI:
If problem still persists, please run the below command to collect the DLPAR log in /tmp/ctsupt:
# /usr/sbin/rsct/bin/ctsnap


6. How to find the parent device of a device like cdrom in AIX ?

# lsdev -Cl cd0 -F parent

HMC Tips VII - General Terms

1. What is the maximum number of servers managed by HMC ?

> Maximum of 48 non-590-595 servers
> Maximum of 32 590/595 servers

2. What is the maximum number of LPARs supported by a HMC ?

> Maximum of 254 LPARs

3. How many HMCs can manage a server at one time ?

> You can have a maximum of 2 HMCs manage a server at one time

4. What are the different types of dynamic operations you can do with CPU, Memory and I/O Adapter on a LPAR ?

> Add
> Remove
> Move

5. How do we connect the HMC to power machines ?

For Power-4 machines, we connect the hmc using serial cables.

But for Power-5 machines, HMC connects to service processors via SSL-encrypted Ethernet, replacing the serial cables.

6. Do we have firewall configured in HMC ?

Yes. Each network card has an integrated firewall.

7. Do we need to configure DHCP in HMC ?

HMC may or may not act as a DHCP server for entry and mid-range servers.

But for high-end servers like P595, HMC must be a DHCP server

8. Can we have the same HMC to manage P4 and P5 machines ?

POWER5 HMCs cannot manage POWER4 servers, and vice versa.

9. Can we have the existing P4 HMCs upgraded to support P5 machines ?

Yes. We can. This involves a complete overwirte of teh disk and the loss of all previous configuration including user profiles.

10. What to do incase of disk failure in HMC ?

We can restore the HMC using recovery CD.
Then restore the latest Critical consule data backup which will restore the profiles, user ids, passwords, etc..

11. What is the default user id and password for the HMC ?

When the HMC is powered on the first time, login as hscroot and password as 'abc123'.

12. Can we manage a power machine without a HMC ?

Yes. We can run a server in manufacturing default mode, will all resources but no logical partitionings, CoD or Service Focal point,etc.. For entry level server, we can use the Integrated Virtualization Manager.

13. What is the network critetia for dual HMC connection ?

Dual HMCs require two different private networks.

14. What is the default service processor IP address in Power-5 Machines ?

Eth0 - HMC1 - 192.168.2.147 / 255.255.255.0
Eth1 - HMC2 - 192.168.3.147 / 255.255.255.0

15. What is the default user id and password for accessing service processor ?

User id - admin
Password - admin

16. Do we need a HMC for p5 model servers ?

One HMC is mandatory for 590, 595 or 575.
Dual HMC are recommended.

17. Do we need private network for HMc connectivity for p5-595 ?

One private network is mandatory for p5 590,595 or 575.

18. Can we have IVM support multiple servers ?

One IVM allowed per server and it only manages partitions on one server.

19. What does FSP (Flexible Service Processor) has ?

FSP has
a. Operating System
b. UserIds / Passwords
c. Filesystem
d. Networking
e. Firewall
f. Webserver
g. ASMI
h. Firmware

20. What to do if you forgot the admin password for FSP ?

If you do not know the admin password, place a hardware call with IBM to get 'celogin'.

21. What to do if you forgot the HMC hostname/ipaddress for a long running LPAR ?

You can always get the HMC IPaddress from a LPAR if we have performed "handshake" with the HMC.

Issue the below command to get the HMC IPAddress

# lsrsrc IBM.ManagementServer
Resource Persistent Attributes for IBM.ManagementServer
resource 1:
Name = "169.121.54.48"
Hostname = "169.121.54.48"
ManagerType = "HMC"
LocalHostname = "169.121.54.59"
ClusterTM = "9078-160"
ClusterSNum = ""
ActivePeerDomain = ""
NodeNameList = {"SAP-PRodServer"}

22. One HMC should be within 8metres of Managed Server

23. Each FSP Ethernet port should be connected to only one HMC