Friday, 9 September 2016

Applying July-2016 PSU on 2-Node RAC with Standby (Rolling)

Hey everyone!, I recently experimented on opatchauto tool for complete automation of patching. The environment I used comprised of a 2-node RAC with a 2-node Standby database in place.

Environment Details:


Nodes to be Patched:
Primary
Node 1 - RAC01
Node 2 - RAC02
 Standby
Node 3 - SBY01
Node 4 - SBY02

Database Version 12.1.0.2 64bit
Grid Version (Active/Release/Software) 12.1.0.2 64bit
Operating System RHEL v7.2 (Mapio) on all nodes.

Kindly note that there were no interim patches applied to either grid infrastructure or database.

Patches to be applied:
1. Patch 6880880: OPatch patch of version 12.2.0.1.7 for Oracle software releases 12.1.0.x (installer)
and 12.2.0.x (AUG 2016)
Since opatch version in my environment is 12.1.0.1.3, it needs to be upgraded to 12.2.0.1.7

2. Patch 23273629: GRID INFRASTRUCTURE PATCH SET UPDATE 12.1.0.2.160719 (JUL2016)

Note: Patch  23273629  includes database PSU and GI PSU, it does not includes OJVM PSU July-2016.

Upgrade OPATCH 

OPatch needs to be upgraded on GRID_HOME and RDBMS_HOME of all nodes

1. Download OPatch version 12.2.0.1.7














Downloaded patch on RAC01, RAC02, SBY01 and SBY02 as:
/u01/stage/opatch/p6880880_121010_Linux-x86-64.zip

2. Check current OPatch version
Run below commands against GRID_HOME or RDBMS_HOME as  grid owner user and oracle owner user respectively
$ export ORACLE_HOME=<GRID_HOME or RDBMS_HOME>
$ export PATH=$ORACLE_HOME/OPatch:$PATH
$ opatch version

This would show 12.1.0.1.3

3. Upgrade OPatch on Database software and verify
On all nodes as oracle user
$ export ORACLE_HOME=/oracle/app/product/12.1.0.2/db_1
$ mv $ORACLE_HOME/OPatch $ORACLE_HOME/OPatch_bkp_121013
$ unzip /u01/stage/opatch/p6880880_121010_Linux-x86-64.zip -d $ORACLE_HOME
$ opatch version


4. Upgrade OPatch on Grid software and verify
On all nodes as grid user
$ export ORACLE_HOME=/u01/app/product/12.1.0/grid
$ mv $ORACLE_HOME/OPatch $ORACLE_HOME/OPatch_bkp_121013
$ unzip /u01/stage/opatch/p6880880_121010_Linux-x86-64.zip -d $ORACLE_HOME
$ opatch version


The output of above commands will now show 12.2.0.1.7 opatch version.

Validate Oracle Inventory


It is important to ensure that Oracle Inventory is consistent on each GRID_HOME and RDBMS_HOME that are to be patched, follow below steps to confirm:

For Oracle Grid Infrastructure: as grid user on all nodes
$ export ORACLE_HOME=/u01/app/product/12.1.0/grid
$ $ORACLE_HOME/OPatch/opatch lsinventory -detail -oh $ORACLE_HOME

The successful execution of command will list Oracle components and patches, if any for Oracle Home.

For Oracle RDBMS: as oracle user on all nodes
$ export ORACLE_HOME=/u01/app/oracle/product/12.1.0/dbhome_1
$ $ORACLE_HOME/OPatch/opatch lsinventory -detail -oh $ORACLE_HOME


Apply PSU


1. Download Patch 23273629














Downloaded patch on RAC01, RAC02, SBY01 and SBY02 as:
/u01/stage/gipsu_jul16/p23273629_121020_Linux-x86-64.zip

The installation of patch will be carried-out as root user, however it is must that owner of the zip file is grid user. As root perform below steps to ensure this:

# cd /u01/stage/gipsu_jul16/
# unzip p23273629_121020_Linux-x86-64.zip
# chown grid:oinstall –R
/u01/stage/gipsu_jul16/23273629

Failing to give correct permissions would fail opatchauto with below error:

==Following patches FAILED in apply:

Patch: /u01/stage/gipsu_july16/23273629/23054327
Log: /u01/app/product/12.1.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2016-08-31_23-14-39PM_1.log 

Reason: Failed during Patching: oracle.opatch.opatchsdk.OPatchException:
Prerequisite check "CheckApplicable" failed.



2. Check for any conflicts
Although no interim patches are applied thus check would pass. As grid user run below command to check for conflicts in both  GRID_HOME and RDBMS_HOME on node RAC01 and SBY01,  run as root :

# export ORACLE_HOME=/u01/app/product/12.1.0/grid
# export PATH=$ORACLE_HOME/OPatch:$PATH
# opatchauto apply
/u01/stage/gipsu_jul16/23273629 -analyze

3. Stop logshipping from PRIMARY and stop Managed Recovery Process on STANDBY
Defer respective log_archive_dest_n parameter and cancel managed recovery.

4. Apply patch on STANDBY Nodes
Logon to SBY01 and SBY02 as root user, and follow below steps on one node at a time

Source environment for GRID HOME and Patch GRID_HOME on Node 1 and Node 2
NODE 1
# export ORACLE_HOME=/u01/app/product/12.1.0/grid
# export PATH=$ORACLE_HOME/OPatch:$PATH
[root@SBY01 23273629]# opatchauto apply /u01/stage/
gipsu_jul16/23273629 -oh /u01/app/product/12.1.0/grid

System initialization log file is /u01/app/product/12.1.0/grid/cfgtoollogs/opatchautodb/systemconfig2016-08-31_09-54-25PM.log.

Session log file is /u01/app/product/12.1.0/grid/cfgtoollogs/opatchauto/opatchauto2016-08-31_09-54-31PM.log
The id for this session is HW39
[init:init] Executing OPatchAutoBinaryAction action on home /u01/app/product/12.1.0/grid

Executing OPatch prereq operations to verify patch applicability on CRS Home........

[init:init] OPatchAutoBinaryAction action completed on home /u01/app/product/12.1.0/grid successfully
[init:init] Executing GIRACPrereqAction action on home /u01/app/product/12.1.0/grid

Executing prereq operations before applying on CRS Home........

[init:init] GIRACPrereqAction action completed on home /u01/app/product/12.1.0/grid successfully
[shutdown:shutdown] Executing GIShutDownAction action on home /u01/app/product/12.1.0/grid

Performing prepatch operations on CRS Home........

Prepatch operation log file location: /u01/app/product/12.1.0/grid/cfgtoollogs/crsconfig/crspatch_SBY01_2016-08-31_09-54-58PM.log

[shutdown:shutdown] GIShutDownAction action completed on home /u01/app/product/12.1.0/grid successfully
[offline:binary-patching] Executing OPatchAutoBinaryAction action on home /u01/app/product/12.1.0/grid

Start applying binary patches on CRS Home........

[offline:binary-patching] OPatchAutoBinaryAction action completed on home /u01/app/product/12.1.0/grid successfully
[startup:startup] Executing GIStartupAction action on home /u01/app/product/12.1.0/grid

Performing postpatch operations on CRS Home........

Postpatch operation log file location: /u01/app/product/12.1.0/grid/cfgtoollogs/crsconfig/crspatch_SBY01_2016-08-31_10-02-32PM.log

[startup:startup] GIStartupAction action completed on home /u01/app/product/12.1.0/grid successfully
[finalize:finalize] Executing OracleHomeLSInventoryGrepAction action on home /u01/app/product/12.1.0/grid

Verifying patches applied on CRS Home.

[finalize:finalize] OracleHomeLSInventoryGrepAction action completed on home /u01/app/product/12.1.0/grid successfully
OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:sby01
CRS Home:/u01/app/product/12.1.0/grid
Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /u01/stage/gipsu_jul16/23273629/21436941
Log: /u01/app/product/12.1.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2016-08-31_21-57-34PM_1.log

Patch: /u01/stage/gipsu_jul16/23273629/23054246
Log: /u01/app/product/12.1.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2016-08-31_21-57-34PM_1.log

Patch: /u01/stage/gipsu_jul16/23273629/23054327
Log: /u01/app/product/12.1.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2016-08-31_21-57-34PM_1.log

Patch: /u01/stage/gipsu_jul16/23273629/23054341
Log: /u01/app/product/12.1.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2016-08-31_21-57-34PM_1.log


NODE 2
# export ORACLE_HOME=/u01/app/product/12.1.0/grid
# export PATH=$ORACLE_HOME/OPatch:$PATH
[root@SBY02 23273629]# opatchauto apply /u01/stage/
gipsu_july16/23273629 -oh /u01/app/product/12.1.0/grid

System initialization log file is /u01/app/product/12.1.0/grid/cfgtoollogs/opatchautodb/systemconfig2016-08-31_10-13-29PM.log.

Session log file is /u01/app/product/12.1.0/grid/cfgtoollogs/opatchauto/opatchauto2016-08-31_10-13-37PM.log
The id for this session is 5AIS
[init:init] Executing OPatchAutoBinaryAction action on home /u01/app/product/12.1.0/grid

Executing OPatch prereq operations to verify patch applicability on CRS Home........

[init:init] OPatchAutoBinaryAction action completed on home /u01/app/product/12.1.0/grid successfully
[init:init] Executing GIRACPrereqAction action on home /u01/app/product/12.1.0/grid

Executing prereq operations before applying on CRS Home........

[init:init] GIRACPrereqAction action completed on home /u01/app/product/12.1.0/grid successfully
[shutdown:shutdown] Executing GIShutDownAction action on home /u01/app/product/12.1.0/grid

Performing prepatch operations on CRS Home........

Prepatch operation log file location: /u01/app/product/12.1.0/grid/cfgtoollogs/crsconfig/crspatch_SBY02_2016-08-31_10-14-04PM.log

[shutdown:shutdown] GIShutDownAction action completed on home /u01/app/product/12.1.0/grid successfully
[offline:binary-patching] Executing OPatchAutoBinaryAction action on home /u01/app/product/12.1.0/grid

Start applying binary patches on CRS Home........

[offline:binary-patching] OPatchAutoBinaryAction action completed on home /u01/app/product/12.1.0/grid successfully
[startup:startup] Executing GIStartupAction action on home /u01/app/product/12.1.0/grid

Performing postpatch operations on CRS Home........

Postpatch operation log file location: /u01/app/product/12.1.0/grid/cfgtoollogs/crsconfig/crspatch_SBY02_2016-08-31_10-21-21PM.log

[startup:startup] GIStartupAction action completed on home /u01/app/product/12.1.0/grid successfully
[finalize:finalize] Executing OracleHomeLSInventoryGrepAction action on home /u01/app/product/12.1.0/grid

Verifying patches applied on CRS Home.

[finalize:finalize] OracleHomeLSInventoryGrepAction action completed on home /u01/app/product/12.1.0/grid successfully
OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:SBY02
CRS Home:/u01/app/product/12.1.0/grid
Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /u01/stage/gipsu_jul16/23273629/21436941
Log: /u01/app/product/12.1.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2016-08-31_22-16-28PM_1.log

Patch: /u01/stage/gipsu_jul16/23273629/23054246
Log: /u01/app/product/12.1.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2016-08-31_22-16-28PM_1.log

Patch: /u01/stage/gipsu_jul16/23273629/23054327
Log: /u01/app/product/12.1.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2016-08-31_22-16-28PM_1.log

Patch: /u01/stage/gipsu_jul16/23273629/23054341
Log: /u01/app/product/12.1.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2016-08-31_22-16-28PM_1.log



Source environment for RDBMS HOME and Patch RDBMS_HOME on both nodes
I would recommend to shut down standby database before patching RDBMS_HOME, this is because opatchauto will attempt to load modified SQLs in database by running datapatch utility.

As oracle user stop database:
$ srvctl stop database -d prod

NODE 1
# export ORACLE_HOME=/u01/app/oracle/product/12.1.0/dbhome_1
# export PATH=$ORACLE_HOME/OPatch:$PATH
[root@SBY01 23273629]# opatchauto apply /u01/stage/gipsu_jul16/23273629 -oh /u01/app/oracle/product/12.1.0/dbhome_1


System initialization log file is /u01/app/oracle/product/12.1.0/dbhome_1/cfgtoollogs/opatchautodb/systemconfig2016-08-31_10-51-22PM.log.

Session log file is /u01/app/oracle/product/12.1.0/dbhome_1/cfgtoollogs/opatchauto/opatchauto2016-08-31_10-51-38PM.log
The id for this session is R9CQ
[init:init] Executing OPatchAutoBinaryAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Executing OPatch prereq operations to verify patch applicability on RAC Home........

[init:init] OPatchAutoBinaryAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[init:init] Executing GIRACPrereqAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Executing prereq operations before applying on RAC Home........

[init:init] GIRACPrereqAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[init:init] Executing RACDBPrereqAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Executing prereq operations before applying on RAC Home........

[init:init] RACDBPrereqAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[shutdown:prepare-shutdown] Executing RACPrepareShutDownAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Preparing RAC Home to bring down database service........

[shutdown:prepare-shutdown] RACPrepareShutDownAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[shutdown:shutdown] Executing RACShutDownAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Stopping the database service on RAC Home for patching........

[shutdown:shutdown] RACShutDownAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[offline:prepare] Executing RACPatchingAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Performing prepatch operation on RAC Home........

[offline:prepare] RACPatchingAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[offline:binary-patching] Executing OPatchAutoBinaryAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Start applying binary patches on RAC Home........

[offline:binary-patching] OPatchAutoBinaryAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[offline:finalize] Executing RACPatchingAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Performing postpatch operation on RAC Home........

[offline:finalize] RACPatchingAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[startup:startup] Executing RACStartupAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Starting the database service on RAC Home........

[startup:startup] RACStartupAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[startup:finalize] Executing RACFinalizeStartAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

No step execution required.........

[startup:finalize] RACFinalizeStartAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[online:product-patching] Executing RACOnlineAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Trying to apply SQL patches on RAC Home.

[WARNING] The database instance 'prod1' from '/u01/app/oracle/product/12.1.0/dbhome_1', in host'SBY01' is not running. SQL changes, if any,  will not be applied.
To apply. the SQL changes, bring up the database instance and run the command manually from any one node (run as oracle).
Refer to the readme to get the correct steps for applying the sql changes.

[online:product-patching] RACOnlineAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[finalize:finalize] Executing OracleHomeLSInventoryGrepAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Verifying patches applied on RAC Home.

[finalize:finalize] OracleHomeLSInventoryGrepAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:SBY01
RAC Home:/u01/app/oracle/product/12.1.0/dbhome_1
Summary:

==Following patches were SKIPPED:

Patch: /u01/stage/gipsu_jul16/23273629/21436941
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /u01/stage/gipsu_jul16/23273629/23054341
Reason: This patch is not applicable to this specified target type - "rac_database"


==Following patches were SUCCESSFULLY applied:

Patch: /u01/stage/gipsu_jul16/23273629/23054246
Log: /u01/app/oracle/product/12.1.0/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2016-08-31_22-52-06PM_1.log

Patch: /u01/stage/gipsu_jul16/23273629/23054327
Log: /u01/app/oracle/product/12.1.0/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2016-08-31_22-52-06PM_1.log


NODE 2
# export ORACLE_HOME=/u01/app/oracle/product/12.1.0/dbhome_1
# export PATH=$ORACLE_HOME/OPatch:$PATH

[root@SBY02 23273629]# opatchauto apply /u01/stage/gipsu_jul16/23273629 -oh /u01/app/oracle/product/12.1.0/dbhome_1


System initialization log file is /u01/app/oracle/product/12.1.0/dbhome_1/cfgtoollogs/opatchautodb/systemconfig2016-08-31_10-56-08PM.log.

Session log file is /u01/app/oracle/product/12.1.0/dbhome_1/cfgtoollogs/opatchauto/opatchauto2016-08-31_10-56-15PM.log
The id for this session is YI37
[init:init] Executing OPatchAutoBinaryAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Executing OPatch prereq operations to verify patch applicability on RAC Home........

[init:init] OPatchAutoBinaryAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[init:init] Executing GIRACPrereqAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Executing prereq operations before applying on RAC Home........

[init:init] GIRACPrereqAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[init:init] Executing RACDBPrereqAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Executing prereq operations before applying on RAC Home........

[init:init] RACDBPrereqAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[shutdown:prepare-shutdown] Executing RACPrepareShutDownAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Preparing RAC Home to bring down database service........

[shutdown:prepare-shutdown] RACPrepareShutDownAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[shutdown:shutdown] Executing RACShutDownAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Stopping the database service on RAC Home for patching........

[shutdown:shutdown] RACShutDownAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[offline:prepare] Executing RACPatchingAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Performing prepatch operation on RAC Home........

[offline:prepare] RACPatchingAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[offline:binary-patching] Executing OPatchAutoBinaryAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Start applying binary patches on RAC Home........

[offline:binary-patching] OPatchAutoBinaryAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[offline:finalize] Executing RACPatchingAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Performing postpatch operation on RAC Home........

[offline:finalize] RACPatchingAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[startup:startup] Executing RACStartupAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Starting the database service on RAC Home........

[startup:startup] RACStartupAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[startup:finalize] Executing RACFinalizeStartAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

No step execution required.........

[startup:finalize] RACFinalizeStartAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[online:product-patching] Executing RACOnlineAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Trying to apply SQL patches on RAC Home.

[WARNING] The database instance 'prod2' from '/u01/app/oracle/product/12.1.0/dbhome_1', in host'SBY02' is not running. SQL changes, if any,  will not be applied.
To apply. the SQL changes, bring up the database instance and run the command manually from any one node (run as oracle).
Refer to the readme to get the correct steps for applying the sql changes.

[online:product-patching] RACOnlineAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[finalize:finalize] Executing OracleHomeLSInventoryGrepAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Verifying patches applied on RAC Home.

[finalize:finalize] OracleHomeLSInventoryGrepAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:SBY02
RAC Home:/u01/app/oracle/product/12.1.0/dbhome_1
Summary:

==Following patches were SKIPPED:

Patch: /u01/stage/gipsu_jul16/23273629/21436941
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /u01/stage/gipsu_jul16/23273629/23054341
Reason: This patch is not applicable to this specified target type - "rac_database"


==Following patches were SUCCESSFULLY applied:

Patch: /u01/stage/gipsu_jul16/23273629/23054246
Log: /u01/app/oracle/product/12.1.0/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2016-08-31_22-56-43PM_1.log

Patch: /u01/stage/gipsu_jul16/23273629/23054327
Log: /u01/app/oracle/product/12.1.0/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2016-08-31_22-56-43PM_1.log


You may now start the standby database as oracle user:
$ srvctl start database -d prod -o mount


5. Apply Patch on PRIMARY Nodes
Apply on one node at a time:

NODE 1
# export ORACLE_HOME=/u01/app/product/12.1.0/grid
# export PATH=$ORACLE_HOME/OPatch:$PATH

[root@RAC01 23273629]# opatchauto apply /u01/stage/gipsu_jul16/23273629

System initialization log file is /u01/app/product/12.1.0/grid/cfgtoollogs/opatchautodb/systemconfig2016-08-31_11-07-14PM.log.

Session log file is /u01/app/product/12.1.0/grid/cfgtoollogs/opatchauto/opatchauto2016-08-31_11-07-25PM.log
The id for this session is 8UDP
[init:init] Executing OPatchAutoBinaryAction action on home /u01/app/product/12.1.0/grid
[init:init] Executing OPatchAutoBinaryAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Executing OPatch prereq operations to verify patch applicability on CRS Home........


Executing OPatch prereq operations to verify patch applicability on RAC Home........

[init:init] OPatchAutoBinaryAction action completed on home /u01/app/product/12.1.0/grid successfully
[init:init] OPatchAutoBinaryAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[init:init] Executing GIRACPrereqAction action on home /u01/app/oracle/product/12.1.0/dbhome_1
[init:init] Executing GIRACPrereqAction action on home /u01/app/product/12.1.0/grid

Executing prereq operations before applying on CRS Home........


Executing prereq operations before applying on RAC Home........

[init:init] GIRACPrereqAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[init:init] GIRACPrereqAction action completed on home /u01/app/product/12.1.0/grid successfully
[init:init] Executing RACDBPrereqAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Executing prereq operations before applying on RAC Home........

[init:init] RACDBPrereqAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[shutdown:prepare-shutdown] Executing RACPrepareShutDownAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Preparing RAC Home to bring down database service........

[shutdown:prepare-shutdown] RACPrepareShutDownAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[shutdown:shutdown] Executing GIShutDownAction action on home /u01/app/product/12.1.0/grid

Performing prepatch operations on CRS Home........

Prepatch operation log file location: /u01/app/product/12.1.0/grid/cfgtoollogs/crsconfig/crspatch_RAC01_2016-08-31_11-10-11PM.log

[shutdown:shutdown] GIShutDownAction action completed on home /u01/app/product/12.1.0/grid successfully
[offline:prepare] Executing RACPatchingAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Performing prepatch operation on RAC Home........

[offline:prepare] RACPatchingAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[offline:binary-patching] Executing OPatchAutoBinaryAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Start applying binary patches on RAC Home........

[offline:binary-patching] OPatchAutoBinaryAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[offline:finalize] Executing RACPatchingAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Performing postpatch operation on RAC Home........

[offline:finalize] RACPatchingAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[offline:binary-patching] Executing OPatchAutoBinaryAction action on home /u01/app/product/12.1.0/grid

Start applying binary patches on CRS Home........

[offline:binary-patching] Executing OPatchAutoBinaryAction action on home /u01/app/product/12.1.0/grid

Start applying binary patches on CRS Home........

[offline:binary-patching] OPatchAutoBinaryAction action completed on home /u01/app/product/12.1.0/grid successfully
[startup:startup] Executing GIStartupAction action on home /u01/app/product/12.1.0/grid

Performing postpatch operations on CRS Home........

Postpatch operation log file location: /u01/app/product/12.1.0/grid/cfgtoollogs/crsconfig/crspatch_RAC01_2016-08-31_11-26-30PM.log

[startup:startup] GIStartupAction action completed on home /u01/app/product/12.1.0/grid successfully
[startup:finalize] Executing RACFinalizeStartAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

No step execution required.........

[startup:finalize] RACFinalizeStartAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[online:product-patching] Executing RACOnlineAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Trying to apply SQL patches on RAC Home.

[online:product-patching] RACOnlineAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[finalize:finalize] Executing OracleHomeLSInventoryGrepAction action on home /u01/app/product/12.1.0/grid

Verifying patches applied on CRS Home.

[finalize:finalize] OracleHomeLSInventoryGrepAction action completed on home /u01/app/product/12.1.0/grid successfully
[finalize:finalize] Executing OracleHomeLSInventoryGrepAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Verifying patches applied on RAC Home.

[finalize:finalize] OracleHomeLSInventoryGrepAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:RAC01
RAC Home:/u01/app/oracle/product/12.1.0/dbhome_1
Summary:

==Following patches were SKIPPED:

Patch: /u01/stage/gipsu_jul16/23273629/21436941
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /u01/stage/gipsu_jul16/23273629/23054341
Reason: This patch is not applicable to this specified target type - "rac_database"


==Following patches were SUCCESSFULLY applied:

Patch: /u01/stage/gipsu_jul16/23273629/23054246
Log: /u01/app/oracle/product/12.1.0/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2016-08-31_23-12-54PM_1.log

Patch: /u01/stage/gipsu_jul16/23273629/23054327
Log: /u01/app/oracle/product/12.1.0/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2016-08-31_23-12-54PM_1.log


Host:RAC01
CRS Home:/u01/app/product/12.1.0/grid
Summary:

==Following patches were SKIPPED:

Patch: /u01/stage/gipsu_jul16/23273629/21436941
Reason: This patch is already been applied, so not going to apply again.

Patch: /u01/stage/gipsu_jul16/23273629/23054246
Reason: This patch is already been applied, so not going to apply again.


==Following patches were SUCCESSFULLY applied:

Patch: /u01/stage/gipsu_jul16/23273629/23054327
Log: /u01/app/product/12.1.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2016-08-31_23-23-59PM_1.log

Patch: /u01/stage/gipsu_jul16/23273629/23054341
Log: /u01/app/product/12.1.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2016-08-31_23-23-59PM_1.log


NODE 2
[root@RAC02 23273629]# opatchauto apply /u01/stage/gipsu_jul16/23273629

System initialization log file is /u01/app/product/12.1.0/grid/cfgtoollogs/opatchautodb/systemconfig2016-08-31_11-40-46PM.log.

Session log file is /u01/app/product/12.1.0/grid/cfgtoollogs/opatchauto/opatchauto2016-08-31_11-40-59PM.log
The id for this session is VYJ3
[init:init] Executing OPatchAutoBinaryAction action on home /u01/app/oracle/product/12.1.0/dbhome_1
[init:init] Executing OPatchAutoBinaryAction action on home /u01/app/product/12.1.0/grid

Executing OPatch prereq operations to verify patch applicability on CRS Home........


Executing OPatch prereq operations to verify patch applicability on RAC Home........

[init:init] OPatchAutoBinaryAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[init:init] OPatchAutoBinaryAction action completed on home /u01/app/product/12.1.0/grid successfully
[init:init] Executing GIRACPrereqAction action on home /u01/app/oracle/product/12.1.0/dbhome_1
[init:init] Executing GIRACPrereqAction action on home /u01/app/product/12.1.0/grid

Executing prereq operations before applying on RAC Home........


Executing prereq operations before applying on CRS Home........

[init:init] GIRACPrereqAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[init:init] GIRACPrereqAction action completed on home /u01/app/product/12.1.0/grid successfully
[init:init] Executing RACDBPrereqAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Executing prereq operations before applying on RAC Home........

[init:init] RACDBPrereqAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[shutdown:prepare-shutdown] Executing RACPrepareShutDownAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Preparing RAC Home to bring down database service........

[shutdown:prepare-shutdown] RACPrepareShutDownAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[shutdown:shutdown] Executing GIShutDownAction action on home /u01/app/product/12.1.0/grid

Performing prepatch operations on CRS Home........

Prepatch operation log file location: /u01/app/product/12.1.0/grid/cfgtoollogs/crsconfig/crspatch_RAC02_2016-08-31_11-44-51PM.log

[shutdown:shutdown] GIShutDownAction action completed on home /u01/app/product/12.1.0/grid successfully
[offline:prepare] Executing RACPatchingAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Performing prepatch operation on RAC Home........

[offline:prepare] RACPatchingAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[offline:binary-patching] Executing OPatchAutoBinaryAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Start applying binary patches on RAC Home........

[offline:binary-patching] OPatchAutoBinaryAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[offline:finalize] Executing RACPatchingAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Performing postpatch operation on RAC Home........

[offline:finalize] RACPatchingAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[offline:binary-patching] Executing OPatchAutoBinaryAction action on home /u01/app/product/12.1.0/grid

Start applying binary patches on CRS Home........

[offline:binary-patching] OPatchAutoBinaryAction action completed on home /u01/app/product/12.1.0/grid successfully
[startup:startup] Executing GIStartupAction action on home /u01/app/product/12.1.0/grid

Performing postpatch operations on CRS Home........

Postpatch operation log file location: /u01/app/product/12.1.0/grid/cfgtoollogs/crsconfig/crspatch_RAC02_2016-08-31_11-51-21PM.log

[startup:startup] GIStartupAction action completed on home /u01/app/product/12.1.0/grid successfully
[startup:finalize] Executing RACFinalizeStartAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

No step execution required.........

[startup:finalize] RACFinalizeStartAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[online:product-patching] Executing RACOnlineAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Trying to apply SQL patches on RAC Home.

[online:product-patching] RACOnlineAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
[finalize:finalize] Executing OracleHomeLSInventoryGrepAction action on home /u01/app/product/12.1.0/grid

Verifying patches applied on CRS Home.

[finalize:finalize] OracleHomeLSInventoryGrepAction action completed on home /u01/app/product/12.1.0/grid successfully
[finalize:finalize] Executing OracleHomeLSInventoryGrepAction action on home /u01/app/oracle/product/12.1.0/dbhome_1

Verifying patches applied on RAC Home.

[finalize:finalize] OracleHomeLSInventoryGrepAction action completed on home /u01/app/oracle/product/12.1.0/dbhome_1 successfully
OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:RAC02
RAC Home:/u01/app/oracle/product/12.1.0/dbhome_1
Summary:

==Following patches were SKIPPED:

Patch: /u01/stage/gipsu_jul16/23273629/21436941
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /u01/stage/gipsu_jul16/23273629/23054341
Reason: This patch is not applicable to this specified target type - "rac_database"


==Following patches were SUCCESSFULLY applied:

Patch: /u01/stage/gipsu_jul16/23273629/23054246
Log: /u01/app/oracle/product/12.1.0/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2016-08-31_23-45-45PM_1.log

Patch: /u01/stage/gipsu_jul16/23273629/23054327
Log: /u01/app/oracle/product/12.1.0/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2016-08-31_23-45-45PM_1.log


Host:RAC02
CRS Home:/u01/app/product/12.1.0/grid
Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /u01/stage/gipsu_jul16/23273629/21436941
Log: /u01/app/product/12.1.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2016-08-31_23-47-23PM_1.log

Patch: /u01/stage/gipsu_jul16/23273629/23054246
Log: /u01/app/product/12.1.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2016-08-31_23-47-23PM_1.log

Patch: /u01/stage/gipsu_jul16/23273629/23054327
Log: /u01/app/product/12.1.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2016-08-31_23-47-23PM_1.log

Patch: /u01/stage/gipsu_jul16/23273629/23054341
Log: /u01/app/product/12.1.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2016-08-31_23-47-23PM_1.log



6. Start Logshipping on PRIMARY and Managed Recovery Process on STANDBY
Enabled log_archive_dest_n parameter on primary and start MRP on standby.

Verify Patch

PROD
[oracle@RAC01 ~]$ opatch lspatches
23054327;OCW Patch Set Update : 12.1.0.2.160719 (23054327)
23054246;Database Patch Set Update : 12.1.0.2.160719 (23054246)

[grid@RAC01 ~]$ opatch lspatches
23054341;ACFS Patch Set Update : 12.1.0.2.160719 (23054341)
23054327;OCW Patch Set Update : 12.1.0.2.160719 (23054327)
23054246;Database Patch Set Update : 12.1.0.2.160719 (23054246)
21436941;WLM Patch Set Update: 12.1.0.2.5 (21436941)

[oracle@RAC02 ~]$ opatch lspatches
23054327;OCW Patch Set Update : 12.1.0.2.160719 (23054327)
23054246;Database Patch Set Update : 12.1.0.2.160719 (23054246)

[grid@RAC02 ~]$ opatch lspatches
23054341;ACFS Patch Set Update : 12.1.0.2.160719 (23054341)
23054327;OCW Patch Set Update : 12.1.0.2.160719 (23054327)
23054246;Database Patch Set Update : 12.1.0.2.160719 (23054246)
21436941;WLM Patch Set Update: 12.1.0.2.5 (21436941)


STANDBY
[oracle@SBY01 ~]$ opatch lspatches
23054327;OCW Patch Set Update : 12.1.0.2.160719 (23054327)
23054246;Database Patch Set Update : 12.1.0.2.160719 (23054246)

[grid@SBY01 ~]$ opatch lspatches
23054341;ACFS Patch Set Update : 12.1.0.2.160719 (23054341)
23054327;OCW Patch Set Update : 12.1.0.2.160719 (23054327)
23054246;Database Patch Set Update : 12.1.0.2.160719 (23054246)
21436941;WLM Patch Set Update: 12.1.0.2.5 (21436941)


[oracle@SBY02 ~]$ opatch lspatches
23054327;OCW Patch Set Update : 12.1.0.2.160719 (23054327)
23054246;Database Patch Set Update : 12.1.0.2.160719 (23054246)

[grid@SBY02 ~]$ opatch lspatches
23054341;ACFS Patch Set Update : 12.1.0.2.160719 (23054341)
23054327;OCW Patch Set Update : 12.1.0.2.160719 (23054327)
23054246;Database Patch Set Update : 12.1.0.2.160719 (23054246)
21436941;WLM Patch Set Update: 12.1.0.2.5 (21436941)

While logged-on to any of PRIMARY node as oracle user
SQL> col ACTION_TIME for a30
SQL> col DESCRIPTION for a60
SQL> set lines 150
SQL> select PATCH_ID, ACTION_TIME, DESCRIPTION from dba_registry_sqlpatch;

  PATCH_ID ACTION_TIME                    DESCRIPTION
---------- ------------------------------ ------------------------------------------------------------
  23054246 31-AUG-16 11.55.27.704930 PM   Database Patch Set Update : 12.1.0.2.160719 (23054246)



All done!

No comments:

Post a Comment