Installation problems with 11.2.0.2 Oracle Clusterware/RAC on Solaris 10

If you’re having headaches with Solaris 10 and Oracle Clusterware 11.2.0.x (11.2.0.2.2 even), and get the following warning during root.sh:

CRS-2678: ‘ora.asm’ on ‘RAC1’ has experienced an unrecoverable failure
CRS-0267: Human intervention required to resume its availability.
CRS-5802: Unable to start the agent process
CRS-0215: Could not start resource ‘ora.CRSDATA.dg’.
CRS-4000: Command Create failed, or completed with errors.
create diskgroup CRSDATA … failed
ACFS-9200: Supported
/opt/oracrs/product/11.2.0.2/bin/srvctl start nodeapps -n rac1  … failed
FirstNode configuration failed at /opt/oracrs/product/11.2.0.2/crs/install/crsconfig_lib.pm line 8374.
/opt/oracrs/product/11.2.0.2/perl/bin/perl -I/opt/oracrs/product/11.2.0.2/perl/lib -I/opt/oracrs/product/11.2.0.2/crs/install /opt/oracrs/product/11.2.0.2/crs/install/rootcrs.pl execution failed

There’s a bug Bug 11834289: OHASD FAILED TO START TIMELY that causes OHASD to timeout due to issues in handling large number of filedescriptors. This also causes increased CPU usage!

From Oracle support:

The problem of bug:11834289 is that we take the rlimit and close all potential file handlers at process startup (even before logging in $GRID_HOME/log).
Hence, in case the stack is the same as before and the oraagent or orarootagent are still consuming cpu and the pfiles of those processes show
a rlimit as unlimited, then it is the problem. Those processes are furthermore special, i.e. they are agents started by the ora.crsd, that are failing (whereas all
other services are working, e.g. the ones started by the ohasd).

The workaround/fix for the problem I found is the following configuration on /etc/system:0::::process.max-file-descriptor=(basic,1024,deny),(privileged,1024,deny)
user.root:1::::process.max-file-descriptor=(basic,1024,deny),(privileged,1024,deny)
noproject:2::::process.max-file-descriptor=(basic,1024,deny)
default:3::::process.max-file-descriptor=(basic,1024,deny),(privileged,1024,deny);process.max-sem-nsems=(privileged,16384,deny);project.max-sem-ids=(privileged,65536,deny);project.max-shm-ids=(privileged,65536,deny);project.max-shm-memory=(privileged,68719476736,deny)
user.oracrs:100::oracrs::process.max-file-descriptor=(basic,65535,deny),(privileged,65535,deny);process.max-sem-nsems=(privileged,16384,deny);project.max-sem-ids=(privileged,65536,deny);project.max-shm-ids=(privileged,65536,deny);project.max-shm-memory=(privileged,68719476736,deny)
user.oracle11:101::oracle11::process.max-file-descriptor=(basic,65535,deny),(privileged,65535,deny);process.max-sem-nsems=(privileged,16384,deny);project.max-sem-ids=(privileged,65536,deny);project.max-shm-ids=(privileged,65536,deny);project.max-shm-memory=(privileged,68719476736,deny)

I hope this makes your life easier! Oracle support was able to help here. Thanks

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s