Thursday, March 17, 2016

Prerequisite check for Oracle12c RAC Cluster Installation

I am about to install Oracle12c Cluster on two node RAC. I am checking all the prerequisite before i start the cluster installation. I thought, i might be helpful for others if i post the things to be checked before we start installing the cluster software.

The RAC nodes are already built and ready to install the cluster.  My hostnames are ractest4 & racttest5.

Here is my ip info.












Step 1 :  Make sure you have either root password or Oracle account with sudo to root privileges to install cluster software. It ask to enter the password in the middle of the installation. I carry root password for my installation.

















Step 2 :  Make sure  you are able to ping public ip and private ip between two nodes.

Public IP 
[oracle@RACTEST4 grid]$ ping 192.168.56.212
PING 192.168.56.212 (192.168.56.212) 56(84) bytes of data.
64 bytes from 192.168.56.212: icmp_seq=1 ttl=64 time=1.49 ms
64 bytes from 192.168.56.212: icmp_seq=2 ttl=64 time=0.416 ms
64 bytes from 192.168.56.212: icmp_seq=3 ttl=64 time=0.411 ms
64 bytes from 192.168.56.212: icmp_seq=4 ttl=64 time=0.479 ms
^C
--- 192.168.56.212 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3362ms
rtt min/avg/max/mdev = 0.411/0.699/1.493/0.459 ms
[oracle@RACTEST4 grid]$

[root@RACTEST5 ~]#  ping 192.168.56.211
PING 192.168.56.211 (192.168.56.211) 56(84) bytes of data.
64 bytes from 192.168.56.211: icmp_seq=1 ttl=64 time=0.324 ms
64 bytes from 192.168.56.211: icmp_seq=2 ttl=64 time=0.425 ms
64 bytes from 192.168.56.211: icmp_seq=3 ttl=64 time=0.468 ms
64 bytes from 192.168.56.211: icmp_seq=4 ttl=64 time=0.436 ms
^C
--- 192.168.56.211 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3469ms
rtt min/avg/max/mdev = 0.324/0.413/0.468/0.055 ms
[root@RACTEST5 ~]#

Private IP 
[oracle@RACTEST4 grid]$ ping 192.168.1.212
PING 192.168.1.212 (192.168.1.212) 56(84) bytes of data.
64 bytes from 192.168.1.212: icmp_seq=1 ttl=64 time=0.387 ms
64 bytes from 192.168.1.212: icmp_seq=2 ttl=64 time=0.403 ms
64 bytes from 192.168.1.212: icmp_seq=3 ttl=64 time=0.384 ms
64 bytes from 192.168.1.212: icmp_seq=4 ttl=64 time=0.410 ms
^C
--- 192.168.1.212 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3245ms
rtt min/avg/max/mdev = 0.384/0.396/0.410/0.010 ms
[oracle@RACTEST4 grid]$

[root@RACTEST5 ~]# ping 192.168.1.211
PING 192.168.1.211 (192.168.1.211) 56(84) bytes of data.
64 bytes from 192.168.1.211: icmp_seq=1 ttl=64 time=1.76 ms
64 bytes from 192.168.1.211: icmp_seq=2 ttl=64 time=0.443 ms
64 bytes from 192.168.1.211: icmp_seq=3 ttl=64 time=0.348 ms
^C
--- 192.168.1.211 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2393ms
rtt min/avg/max/mdev = 0.348/0.853/1.768/0.648 ms
[root@RACTEST5 ~]#

 Step 3 :  Make sure you see the ASM disk on both the nodes.
[root@RACTEST4 ~]# oracleasm listdisks
DISK1
[root@RACTEST4 ~]#
[root@RACTEST5 ~]# oracleasm listdisks
DISK1
[root@RACTEST5 ~]#

 Step 4 :  Run the cluster verification utility. The runcluvfy.sh script found in the grid directory, the same directory where you find runinstaller. This utility helps resolve problems before running runInstaller script.

runcluvfy.sh stage -pre crsinst -n ractest4,ractest5

By default the CVU reports in non verbose mode by only reporting the summary of a test, we can obtain detailed output by using the -verbose argument. The -verbose argument produces detailed output of individual checks and where applicable shows results for each node in a tabular layout.

runcluvfy.sh stage -pre crsinst -n ractest4,ractest5 -verbose

[oracle@RACTEST4 grid]$ ls -l runcluvfy.sh
-rwxr-xr-x. 1 root root 5085 Dec 20  2013 runcluvfy.sh
[oracle@RACTEST4 grid]$ ./runcluvfy.sh stage -pre crsinst -n ractest4,ractest5

Performing pre-checks for cluster services setup

Checking node reachability...
Node reachability check passed from node "ractest4"


Checking user equivalence...
PRVG-2019 : Check for equivalence of user "oracle" from node "ractest4" to node "ractest5" failed

PRKC-1044 : Failed to check remote command execution setup for node ractest5 using shells /usr/bin/ssh and /usr/bin/rsh
File "/usr/bin/rsh" does not exist on node "ractest5"
No RSA host key is known for ractest5 and you have requested strict checking.Host key verification failed.
PRVG-2019 : Check for equivalence of user "oracle" from node "ractest4" to node "ractest4" failed

PRKC-1044 : Failed to check remote command execution setup for node ractest4 using shells /usr/bin/ssh and /usr/bin/rsh
File "/usr/bin/rsh" does not exist on node "ractest4"
No RSA host key is known for ractest4 and you have requested strict checking.Host key verification failed.

ERROR:
User equivalence unavailable on all the specified nodes
Verification cannot proceed


Pre-check for cluster services setup was unsuccessful on all the nodes.
[oracle@RACTEST4 grid]$


It seems ssh is not setup between two nodes. Let me setup the ssh. 

Step 5 :  Setup the ssh between two nodes.

Login to sshsetup directory and run the below commands. My sshsetup location is /backup/software/cluster/grid/sshsetup

./sshUserSetup.sh -user oracle -hosts "ractest4 ractest5" -noPromptPassphrase 

[oracle@RACTEST4 sshsetup]$ ./sshUserSetup.sh -user oracle -hosts "ractest4 ractest5" -noPromptPassphrase
The output of this script is also logged into /tmp/sshUserSetup_2016-03-17-09-19-03.log
Hosts are ractest4 ractest5
user is oracle
Platform:- Linux
Checking if the remote hosts are reachable
PING RACTEST4.localdomain (192.168.56.211) 56(84) bytes of data.
64 bytes from RACTEST4.localdomain (192.168.56.211): icmp_seq=1 ttl=64 time=0.012 ms
64 bytes from RACTEST4.localdomain (192.168.56.211): icmp_seq=2 ttl=64 time=0.032 ms
64 bytes from RACTEST4.localdomain (192.168.56.211): icmp_seq=3 ttl=64 time=0.029 ms
64 bytes from RACTEST4.localdomain (192.168.56.211): icmp_seq=4 ttl=64 time=0.030 ms
64 bytes from RACTEST4.localdomain (192.168.56.211): icmp_seq=5 ttl=64 time=0.029 ms

--- RACTEST4.localdomain ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 3999ms
rtt min/avg/max/mdev = 0.012/0.026/0.032/0.008 ms
PING RACTEST5.localdomain (192.168.56.212) 56(84) bytes of data.
64 bytes from RACTEST5.localdomain (192.168.56.212): icmp_seq=1 ttl=64 time=0.382 ms
64 bytes from RACTEST5.localdomain (192.168.56.212): icmp_seq=2 ttl=64 time=0.407 ms
64 bytes from RACTEST5.localdomain (192.168.56.212): icmp_seq=3 ttl=64 time=0.371 ms
64 bytes from RACTEST5.localdomain (192.168.56.212): icmp_seq=4 ttl=64 time=0.418 ms
64 bytes from RACTEST5.localdomain (192.168.56.212): icmp_seq=5 ttl=64 time=0.416 ms

--- RACTEST5.localdomain ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4001ms
rtt min/avg/max/mdev = 0.371/0.398/0.418/0.031 ms
Remote host reachability check succeeded.
The following hosts are reachable: ractest4 ractest5.
The following hosts are not reachable: .
All hosts are reachable. Proceeding further...
firsthost ractest4
numhosts 2
The script will setup SSH connectivity from the host RACTEST4.localdomain to all
the remote hosts. After the script is executed, the user can use SSH to run
commands on the remote hosts or copy files between this host RACTEST4.localdomain
and the remote hosts without being prompted for passwords or confirmations.

NOTE 1:
As part of the setup procedure, this script will use ssh and scp to copy
files between the local host and the remote hosts. Since the script does not
store passwords, you may be prompted for the passwords during the execution of
the script whenever ssh or scp is invoked.

NOTE 2:
AS PER SSH REQUIREMENTS, THIS SCRIPT WILL SECURE THE USER HOME DIRECTORY
AND THE .ssh DIRECTORY BY REVOKING GROUP AND WORLD WRITE PRIVILEDGES TO THESE
directories.

Do you want to continue and let the script make the above mentioned changes (yes/no)?
yes

The user chose yes
User chose to skip passphrase related questions.
Creating .ssh directory on local host, if not present already
Creating authorized_keys file on local host
Changing permissions on authorized_keys to 644 on local host
Creating known_hosts file on local host
Changing permissions on known_hosts to 644 on local host
Creating config file on local host
If a config file exists already at /home/oracle/.ssh/config, it would be backed up to /home/oracle/.ssh/config.backup.
Removing old private/public keys on local host
Running SSH keygen on local host with empty passphrase
Generating public/private rsa key pair.
Your identification has been saved in /home/oracle/.ssh/id_rsa.
Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.
The key fingerprint is:
94:de:17:5e:0e:08:fb:3f:eb:6f:f8:0f:3f:3f:6e:f2 oracle@RACTEST4.localdomain
The key's randomart image is:
+--[ RSA 1024]----+
|        .        |
|         + .     |
|        + . o .  |
|       o o . =   |
|        S o o .  |
|           o     |
|            o..  |
|            .+.* |
|           .o+BEB|
+-----------------+
Creating .ssh directory and setting permissions on remote host ractest4
THE SCRIPT WOULD ALSO BE REVOKING WRITE PERMISSIONS FOR group AND others ON THE HOME DIRECTORY FOR oracle. THIS IS AN SSH REQUIREMENT.
The script would create ~oracle/.ssh/config file on remote host ractest4. If a config file exists already at ~oracle/.ssh/config, it would be backed up to ~oracle/.ssh/config.backup.
The user may be prompted for a password here since the script would be running SSH on host ractest4.
Warning: Permanently added 'ractest4,192.168.56.211' (RSA) to the list of known hosts.
oracle@ractest4's password:
Done with creating .ssh directory and setting permissions on remote host ractest4.
Creating .ssh directory and setting permissions on remote host ractest5
THE SCRIPT WOULD ALSO BE REVOKING WRITE PERMISSIONS FOR group AND others ON THE HOME DIRECTORY FOR oracle. THIS IS AN SSH REQUIREMENT.
The script would create ~oracle/.ssh/config file on remote host ractest5. If a config file exists already at ~oracle/.ssh/config, it would be backed up to ~oracle/.ssh/config.backup.
The user may be prompted for a password here since the script would be running SSH on host ractest5.
Warning: Permanently added 'ractest5,192.168.56.212' (RSA) to the list of known hosts.
oracle@ractest5's password:
Done with creating .ssh directory and setting permissions on remote host ractest5.
Copying local host public key to the remote host ractest4
The user may be prompted for a password or passphrase here since the script would be using SCP for host ractest4.
oracle@ractest4's password:
Done copying local host public key to the remote host ractest4
Copying local host public key to the remote host ractest5
The user may be prompted for a password or passphrase here since the script would be using SCP for host ractest5.
oracle@ractest5's password:
Done copying local host public key to the remote host ractest5
cat: /home/oracle/.ssh/known_hosts.tmp: No such file or directory
cat: /home/oracle/.ssh/authorized_keys.tmp: No such file or directory
SSH setup is complete.

------------------------------------------------------------------------
Verifying SSH setup
===================
The script will now run the date command on the remote nodes using ssh
to verify if ssh is setup correctly. IF THE SETUP IS CORRECTLY SETUP,
THERE SHOULD BE NO OUTPUT OTHER THAN THE DATE AND SSH SHOULD NOT ASK FOR
PASSWORDS. If you see any output other than date or are prompted for the
password, ssh is not setup correctly and you will need to resolve the
issue and set up ssh again.
The possible causes for failure could be:
1. The server settings in /etc/ssh/sshd_config file do not allow ssh
for user oracle.
2. The server may have disabled public key based authentication.
3. The client public key on the server may be outdated.
4. ~oracle or ~oracle/.ssh on the remote host may not be owned by oracle.
5. User may not have passed -shared option for shared remote users or
may be passing the -shared option for non-shared remote users.
6. If there is output in addition to the date, but no password is asked,
it may be a security alert shown as part of company policy. Append the
additional text to the /sysman/prov/resources/ignoreMessages.txt file.
------------------------------------------------------------------------
--ractest4:--
Running /usr/bin/ssh -x -l oracle ractest4 date to verify SSH connectivity has been setup from local host to ractest4.
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL. Please note that being prompted for a passphrase may be OK but being prompted for a password is ERROR.
Thu Mar 17 09:21:15 EDT 2016
------------------------------------------------------------------------
--ractest5:--
Running /usr/bin/ssh -x -l oracle ractest5 date to verify SSH connectivity has been setup from local host to ractest5.
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL. Please note that being prompted for a passphrase may be OK but being prompted for a password is ERROR.
Thu Mar 17 09:21:15 EDT 2016
------------------------------------------------------------------------
SSH verification complete.
[oracle@RACTEST4 sshsetup]$

Step 6 :  Re run the runcluvfy.sh again and see any issues.
[oracle@RACTEST4 grid]$  ./runcluvfy.sh stage -pre crsinst -n ractest4,ractest5

Performing pre-checks for cluster services setup

Checking node reachability...
Node reachability check passed from node "ractest4"


Checking user equivalence...
User equivalence check passed for user "oracle"

Checking node connectivity...

Checking hosts config file...

Verification of the hosts config file successful

Node connectivity passed for subnet "192.168.56.0" with node(s) ractest5,ractest4
TCP connectivity check passed for subnet "192.168.56.0"

Node connectivity passed for subnet "192.168.1.0" with node(s) ractest5,ractest4
TCP connectivity check passed for subnet "192.168.1.0"


Interfaces found on subnet "192.168.56.0" that are likely candidates for VIP are:
ractest5 eth0:192.168.56.212
ractest4 eth0:192.168.56.211

Interfaces found on subnet "192.168.1.0" that are likely candidates for a private interconnect are:
ractest5 eth3:192.168.1.212
ractest4 eth1:192.168.1.211

WARNING:
Could not find a suitable set of interfaces with the same name for the private interconnect
Checking subnet mask consistency...
Subnet mask consistency check passed for subnet "192.168.56.0".
Subnet mask consistency check passed for subnet "192.168.1.0".
Subnet mask consistency check passed.

Node connectivity check passed

Checking multicast communication...

Checking subnet "192.168.56.0" for multicast communication with multicast group "224.0.0.251"...
Check of subnet "192.168.56.0" for multicast communication with multicast group "224.0.0.251" passed.

Check of multicast communication passed.

Checking ASMLib configuration.
Check for ASMLib configuration passed.
Total memory check failed
Check failed on nodes:
        ractest5,ractest4
Available memory check passed
Swap space check passed
Free disk space check passed for "ractest5:/usr,ractest5:/var,ractest5:/etc,ractest5:/sbin,ractest5:/tmp"
Free disk space check passed for "ractest4:/usr,ractest4:/var,ractest4:/etc,ractest4:/sbin,ractest4:/tmp"
Check for multiple users with UID value 60321 passed
User existence check passed for "oracle"
Group existence check passed for "oinstall"
Group existence check passed for "dba"
Membership check for user "oracle" in group "oinstall" [as Primary] passed
Membership check for user "oracle" in group "dba" passed
Run level check passed
Hard limits check passed for "maximum open file descriptors"
Soft limits check passed for "maximum open file descriptors"
Hard limits check passed for "maximum user processes"
Soft limits check passed for "maximum user processes"
System architecture check passed
Kernel version check passed
Kernel parameter check passed for "semmsl"
Kernel parameter check passed for "semmns"
Kernel parameter check passed for "semopm"
Kernel parameter check passed for "semmni"
Kernel parameter check passed for "shmmax"
Kernel parameter check passed for "shmmni"
Kernel parameter check passed for "shmall"
Kernel parameter check passed for "file-max"
Kernel parameter check passed for "ip_local_port_range"
Kernel parameter check passed for "rmem_default"
Kernel parameter check passed for "rmem_max"
Kernel parameter check passed for "wmem_default"
Kernel parameter check passed for "wmem_max"
Kernel parameter check passed for "aio-max-nr"

PRVG-1206 : Check cannot be performed for configured value of kernel parameter "panic_on_oops" on node "ractest5"
PRVG-1206 : Check cannot be performed for configured value of kernel parameter "panic_on_oops" on node "ractest4"

Kernel parameter check passed for "panic_on_oops"
Package existence check passed for "binutils"
Package existence check passed for "compat-libcap1"
Package existence check passed for "compat-libstdc++-33(x86_64)"
Package existence check passed for "libgcc(x86_64)"
Package existence check passed for "libstdc++(x86_64)"
Package existence check passed for "libstdc++-devel(x86_64)"
Package existence check passed for "sysstat"
Package existence check passed for "gcc"
Package existence check passed for "gcc-c++"
Package existence check passed for "ksh"
Package existence check passed for "make"
Package existence check passed for "glibc(x86_64)"
Package existence check passed for "glibc-devel(x86_64)"
Package existence check passed for "libaio(x86_64)"
Package existence check passed for "libaio-devel(x86_64)"
Package existence check passed for "nfs-utils"

Checking availability of ports "6200,6100" required for component "Oracle Notification Service (ONS)"
Port availability check passed for ports "6200,6100"

Checking availability of ports "42424" required for component "Oracle Cluster Synchronization Services (CSSD)"
Port availability check passed for ports "42424"
Check for multiple users with UID value 0 passed
Current group ID check passed

Starting check for consistency of primary group of root user

Check for consistency of root user's primary group passed

Starting Clock synchronization checks using Network Time Protocol(NTP)...
NTP configuration file "/etc/ntp.conf" existence check passed
No NTP Daemons or Services were found to be running
PRVF-5507 : NTP daemon or service is not running on any node but NTP configuration file exists on the following node(s):
ractest5,ractest4
Clock synchronization check using Network Time Protocol(NTP) failed

Core file name pattern consistency check passed.

User "oracle" is not part of "root" group. Check passed
Default user file creation mask check passed
Checking integrity of file "/etc/resolv.conf" across nodes

"domain" and "search" entries do not coexist in any "/etc/resolv.conf" file
All nodes have same "search" order defined in file "/etc/resolv.conf"
The DNS response time for an unreachable node is within acceptable limit on all nodes

Check for integrity of file "/etc/resolv.conf" passed

Time zone consistency check passed

Checking integrity of name service switch configuration file "/etc/nsswitch.conf" ...
All nodes have same "hosts" entry defined in file "/etc/nsswitch.conf"
Check for integrity of name service switch configuration file "/etc/nsswitch.conf" passed


Checking daemon "avahi-daemon" is not configured and running
Daemon not configured check passed for process "avahi-daemon"
Daemon not running check passed for process "avahi-daemon"

Starting check for /dev/shm mounted as temporary file system ...

Check for /dev/shm mounted as temporary file system passed

Starting check for /boot mount ...

Check for /boot mount passed

Starting check for zeroconf check ...

ERROR:

PRVE-10077 : NOZEROCONF parameter was not  specified or was not set to 'yes' in file "/etc/sysconfig/network" on node "RACTEST5.localdomain"
PRVE-10077 : NOZEROCONF parameter was not  specified or was not set to 'yes' in file "/etc/sysconfig/network" on node "RACTEST4.localdomain"

Check for zeroconf check failed

Pre-check for cluster services setup was unsuccessful on all the nodes.
[oracle@RACTEST4 grid]$

Now ssh issue resolved!  but still there are four more issues appeared.

Issue 1
The /etc/udev/rules.d/70-persistent-net.rules file has some incorrect naming convention. I fixed this file.

WARNING:
Could not find a suitable set of interfaces with the same name for the private interconnect

Issue 2
Added below entry in /etc/sysctl.conf file on both node 
kernel.panic_on_oops = 1

Run the sysctl -p on both nodes.

PRVG-1206 : Check cannot be performed for configured value of kernel parameter "panic_on_oops" on node "ractest5"
PRVG-1206 : Check cannot be performed for configured value of kernel parameter "panic_on_oops" on node "ractest4"

Issue 3The below issue  can be ignored. Since we are configuring  NTP protocol.  CSSD will take care of time synchronization. 

Clock synchronization check using Network Time Protocol(NTP) failed

Issue 4
Added below entry in /etc/sysconfig/network file on both nodes and restart the network service.
NOZEROCONF=yes


ERROR:
PRVE-10077 : NOZEROCONF parameter was not  specified or was not set to 'yes' in file "/etc/sysconfig/network" on node "RACTEST5.localdomain"
PRVE-10077 : NOZEROCONF parameter was not  specified or was not set to 'yes' in file "/etc/sysconfig/network" on node "RACTEST4.localdomain"

Check for zeroconf check failed

Step 7 :  Re run the runcluvfy.sh again and see any issues.
[oracle@RACTEST4 grid]$ ./runcluvfy.sh stage -pre crsinst -n ractest4,ractest5

Performing pre-checks for cluster services setup

Checking node reachability...
Node reachability check passed from node "ractest4"


Checking user equivalence...
User equivalence check passed for user "oracle"

Checking node connectivity...

Checking hosts config file...

Verification of the hosts config file successful

Node connectivity passed for subnet "192.168.56.0" with node(s) ractest5,ractest4
TCP connectivity check passed for subnet "192.168.56.0"

Node connectivity passed for subnet "192.168.1.0" with node(s) ractest5,ractest4
TCP connectivity check passed for subnet "192.168.1.0"


Interfaces found on subnet "192.168.56.0" that are likely candidates for VIP are:
ractest5 eth0:192.168.56.212
ractest4 eth0:192.168.56.211

Interfaces found on subnet "192.168.1.0" that are likely candidates for a private interconnect are:
ractest5 eth1:192.168.1.212
ractest4 eth1:192.168.1.211
Checking subnet mask consistency...
Subnet mask consistency check passed for subnet "192.168.56.0".
Subnet mask consistency check passed for subnet "192.168.1.0".
Subnet mask consistency check passed.

Node connectivity check passed

Checking multicast communication...

Checking subnet "192.168.56.0" for multicast communication with multicast group "224.0.0.251"...
Check of subnet "192.168.56.0" for multicast communication with multicast group "224.0.0.251" passed.

Check of multicast communication passed.

Checking ASMLib configuration.
Check for ASMLib configuration passed.
Total memory check failed
Check failed on nodes:
        ractest5,ractest4
Available memory check passed
Swap space check passed
Free disk space check passed for "ractest5:/usr,ractest5:/var,ractest5:/etc,ractest5:/sbin,ractest5:/tmp"
Free disk space check passed for "ractest4:/usr,ractest4:/var,ractest4:/etc,ractest4:/sbin,ractest4:/tmp"
Check for multiple users with UID value 60321 passed
User existence check passed for "oracle"
Group existence check passed for "oinstall"
Group existence check passed for "dba"
Membership check for user "oracle" in group "oinstall" [as Primary] passed
Membership check for user "oracle" in group "dba" passed
Run level check passed
Hard limits check passed for "maximum open file descriptors"
Soft limits check passed for "maximum open file descriptors"
Hard limits check passed for "maximum user processes"
Soft limits check passed for "maximum user processes"
System architecture check passed
Kernel version check passed
Kernel parameter check passed for "semmsl"
Kernel parameter check passed for "semmns"
Kernel parameter check passed for "semopm"
Kernel parameter check passed for "semmni"
Kernel parameter check passed for "shmmax"
Kernel parameter check passed for "shmmni"
Kernel parameter check passed for "shmall"
Kernel parameter check passed for "file-max"
Kernel parameter check passed for "ip_local_port_range"
Kernel parameter check passed for "rmem_default"
Kernel parameter check passed for "rmem_max"
Kernel parameter check passed for "wmem_default"
Kernel parameter check passed for "wmem_max"
Kernel parameter check passed for "aio-max-nr"
Kernel parameter check passed for "panic_on_oops"
Package existence check passed for "binutils"
Package existence check passed for "compat-libcap1"
Package existence check passed for "compat-libstdc++-33(x86_64)"
Package existence check passed for "libgcc(x86_64)"
Package existence check passed for "libstdc++(x86_64)"
Package existence check passed for "libstdc++-devel(x86_64)"
Package existence check passed for "sysstat"
Package existence check passed for "gcc"
Package existence check passed for "gcc-c++"
Package existence check passed for "ksh"
Package existence check passed for "make"
Package existence check passed for "glibc(x86_64)"
Package existence check passed for "glibc-devel(x86_64)"
Package existence check passed for "libaio(x86_64)"
Package existence check passed for "libaio-devel(x86_64)"
Package existence check passed for "nfs-utils"

Checking availability of ports "6200,6100" required for component "Oracle Notification Service (ONS)"
Port availability check passed for ports "6200,6100"

Checking availability of ports "42424" required for component "Oracle Cluster Synchronization Services (CSSD)"
Port availability check passed for ports "42424"
Check for multiple users with UID value 0 passed
Current group ID check passed

Starting check for consistency of primary group of root user

Check for consistency of root user's primary group passed

Starting Clock synchronization checks using Network Time Protocol(NTP)...
NTP configuration file "/etc/ntp.conf" existence check passed
No NTP Daemons or Services were found to be running
PRVF-5507 : NTP daemon or service is not running on any node but NTP configuration file exists on the following node(s):
ractest5,ractest4
Clock synchronization check using Network Time Protocol(NTP) failed

Core file name pattern consistency check passed.

User "oracle" is not part of "root" group. Check passed
Default user file creation mask check passed
Checking integrity of file "/etc/resolv.conf" across nodes

"domain" and "search" entries do not coexist in any "/etc/resolv.conf" file
All nodes have same "search" order defined in file "/etc/resolv.conf"
The DNS response time for an unreachable node is within acceptable limit on all nodes

Check for integrity of file "/etc/resolv.conf" passed

Time zone consistency check passed

Checking integrity of name service switch configuration file "/etc/nsswitch.conf" ...
All nodes have same "hosts" entry defined in file "/etc/nsswitch.conf"
Check for integrity of name service switch configuration file "/etc/nsswitch.conf" passed


Checking daemon "avahi-daemon" is not configured and running
Daemon not configured check passed for process "avahi-daemon"
Daemon not running check passed for process "avahi-daemon"

Starting check for /dev/shm mounted as temporary file system ...

Check for /dev/shm mounted as temporary file system passed

Starting check for /boot mount ...

Check for /boot mount passed

Starting check for zeroconf check ...

Check for zeroconf check passed

Pre-check for cluster services setup was unsuccessful on all the nodes.
[oracle@RACTEST4 grid]$


Now all set!  It is ready to start installing the cluster software!

During the cluster software installation,  i found cvuqdisk  package missing.












The package is existing in software location under grid/rpm directory. I installed this package on both nodes.

[root@RACTEST4 rpm]# pwd
/backup/software/cluster/grid/rpm
[root@RACTEST4 rpm]# rpm -ivh cvuqdisk-1.0.9-1.rpm
Preparing...                ########################################### [100%]
Using default group oinstall to install package
   1:cvuqdisk               ########################################### [100%]
[root@RACTEST4 rpm]#

[root@RACTEST5 cluster]# rpm -ivh cvuqdisk-1.0.9-1.rpm
Preparing...                ########################################### [100%]
Using default group oinstall to install package
   1:cvuqdisk               ########################################### [100%]
[root@RACTEST5 cluster]#

Again i clicked Check Again button and now the issue is disappeared.












I am not worried about physical memory now..  NTP protocol can be ignored. since CSSD will take care of time synchronization.

We are good now!!!

No comments: