DB and RAC(11.2.0.3 ) Patch Set Update(11.2.0.3.6 )

现在是2013年05月28日，oracle最新的PSU为11.2.0.3.6,本次升级为RAC和DB分别进行升级。

系统:linux 5.7 x86_64

数据库版本：11.2.0.3.0

单机升级ORACLE

首先升级单机的DB到11.2.0.3.6：

下载11.2.0.3.6的补丁包，补丁编号为16056266。

1.1 检查补丁冲突

用oracle用户unzip补丁包到ORACLE_HOME目录下。

[oracle@bogon db_1]$ cd 16056266/

[oracle@localhost 16056266]$ ../OPatch/opatch prereq CheckConflictAgainstOHWithDetail -ph ./

Oracle Interim Patch Installer version 11.2.0.3.4
Copyright (c) 2012, Oracle Corporation. All rights reserved.

PREREQ session

Oracle Home : /u01/app/oracle/product/11g/db_1
Central Inventory : /u01/app/oraInventory
from : /u01/app/oracle/product/11g/db_1/oraInst.loc
OPatch version : 11.2.0.3.4
OUI version : 11.2.0.3.0
Log file location : /u01/app/oracle/product/11g/db_1/cfgtoollogs/opatch/opatch2013-05-27_11-00-06AM_1.log

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.

1.2 检查当前数据库补丁历史

SQL> select action,comments from registry$history;

ACTION
------------------------------
COMMENTS
--------------------------------------------------------------------------------
APPLY
Patchset 11.2.0.2.0

APPLY
Patchset 11.2.0.2.0

1.3 应用补丁进行升级

--升级前将数据库，监听及sqlplus都退出，否则在前置检查的时候会报错。

[oracle@bogon 16056266]$ ../OPatch/opatch apply
Oracle Interim Patch Installer version 11.2.0.3.4
Copyright (c) 2012, Oracle Corporation. All rights reserved.

Oracle Home : /u01/app/oracle/product/11g/db_1
Central Inventory : /u01/app/oraInventory
from : /u01/app/oracle/product/11g/db_1/oraInst.loc
OPatch version : 11.2.0.3.4
OUI version : 11.2.0.3.0
Log file location : /u01/app/oracle/product/11g/db_1/cfgtoollogs/opatch/opatch2013-05-28_11-00-14AM_1.log

Verifying environment and performing prerequisite checks...
OPatch continues with these patches: 13343438 13696216 13923374 14275605 14727310 16056266

Do you want to proceed? [y|n]
Y

User Responded with: Y
All checks passed.
Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name:

You have not provided an email address for notification of security issues.
Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]: y

Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = '/u01/app/oracle/product/11.2.0/dbhome_1')

Is the local system ready for patching? [y|n]
y
User Responded with: Y
Backing up files...
Applying sub-patch '13343438' to OH '/u01/app/oracle/product/11.2.0/dbhome_1'

Patching component oracle.rdbms.rsf, 11.2.0.3.0...

Patching component oracle.rdbms, 11.2.0.3.0...

......

......

--单机PSU可以不需要MOS账号

Composite patch 16056266 successfully applied.
UtilSession: N-Apply done.
--------------------------------------------------------------------------------
The following warnings have occurred during OPatch execution:
1) OUI-67215:
OPatch found the word "warning" in the stderr of the make command.
Please look at this stderr. You can re-run this make command.
Stderr output:
ins_precomp.mk:19: warning: overriding commands for target `pcscfg.cfg'
/u01/app/oracle/product/11g/db_1/precomp/lib/env_precomp.mk:2160: warning: ignoring old commands for target `pcscfg.cfg'
/u01/app/oracle/product/11g/db_1/precomp/lib/ins_precomp.mk:19: warning: overriding commands for target `pcscfg.cfg'
/u01/app/oracle/product/11g/db_1/precomp/lib/env_precomp.mk:2160: warning: ignoring old commands for target `pcscfg.cfg'
--------------------------------------------------------------------------------
OUI-67008:OPatch Session completed with warnings.
Finishing UtilSession at Tue May 28 11:10:43 CST 2013
Log file location: /u01/app/oracle/product/11g/db_1/cfgtoollogs/opatch/opatch2013-05-28_11-00-14AM_1.log

--虽然有告警，但依然安装成功

1.4 检查升级（PSU)后的版本

--查看当前补丁版本
[oracle@localhost OPatch]$ ./opatch lspatches
16056266;Database Patch Set Update : 11.2.0.3.6 (16056266)

--查看当前补丁详细信息
[oracle@localhost OPatch]$ ./opatch lsinventory
Oracle Interim Patch Installer version 11.2.0.3.4
Copyright (c) 2012, Oracle Corporation. All rights reserved.

Oracle Home : /u01/app/oracle/product/11g/db_1
Central Inventory : /u01/app/oraInventory
from : /u01/app/oracle/product/11g/db_1/oraInst.loc
OPatch version : 11.2.0.3.4
OUI version : 11.2.0.3.0
Log file location : /u01/app/oracle/product/11g/db_1/cfgtoollogs/opatch/opatch2013-05-28_11-33-02AM_1.log

Lsinventory Output file location : /u01/app/oracle/product/11g/db_1/cfgtoollogs/opatch/lsinv/lsinventory2013-05-28_11-33-02AM.txt

--------------------------------------------------------------------------------
Installed Top-level Products (1):

Oracle Database 11g 11.2.0.3.0
There are 1 products installed in this Oracle Home.

Interim patches (1) :

Patch 16056266 : applied on Tue May 28 11:09:50 CST 2013
Unique Patch ID: 15962803
Patch description: "Database Patch Set Update : 11.2.0.3.6 (16056266)"
Created on 12 Mar 2013, 02:14:47 hrs PST8PDT
Sub-patch 14727310; "Database Patch Set Update : 11.2.0.3.5 (14727310)"
Sub-patch 14275605; "Database Patch Set Update : 11.2.0.3.4 (14275605)"
Sub-patch 13923374; "Database Patch Set Update : 11.2.0.3.3 (13923374)"
Sub-patch 13696216; "Database Patch Set Update : 11.2.0.3.2 (13696216)"
Sub-patch 13343438; "Database Patch Set Update : 11.2.0.3.1 (13343438)"
Bugs fixed:
13566938, 13593999, 10350832, 14138130, 12919564, 13561951, 13624984
13588248, 13080778, 13914613, 13804294, 14258925, 12873183, 13645875
14472647, 12880299, 14664355, 14409183, 12998795, 14469008, 13719081
13492735, 13496884, 12857027, 14263036, 14263073, 13732226, 13742433
16368108, 16314469, 12905058, 13742434, 12849688, 12950644, 13742435
13464002, 13534412, 12879027, 13958038, 14613900, 12585543, 12535346
12588744, 11877623, 13786142, 12847466, 13649031, 13981051, 12582664
12797765, 14262913, 12923168, 13384182, 13612575, 13466801, 13484963
14207163, 11063191, 13772618, 13070939, 12797420, 13041324, 16314467
16314468, 12976376, 11708510, 13680405, 14589750, 13026410, 13742437
13737746, 14644185, 13742438, 13326736, 13596521, 13001379, 16344871
13099577, 9873405, 14275605, 13742436, 9858539, 14841812, 11715084
16231699, 14040433, 12662040, 9703627, 12617123, 12845115, 12764337
13354082, 14459552, 13397104, 13913630, 12964067, 12983611, 13550185
13810393, 12780983, 12583611, 14546575, 13476583, 15862016, 11840910
13903046, 15862017, 13572659, 16294378, 13718279, 14088346, 13657605
13448206, 16314466, 14480676, 13419660, 13632717, 14063281, 14110275
13430938, 13467683, 13420224, 13812031, 14548763, 16299830, 12646784
13616375, 14035825, 12861463, 12834027, 15862021, 13632809, 13377816
13036331, 14727310, 13685544, 15862018, 13499128, 16175381, 13584130
12829021, 15862019, 12794305, 14546673, 12791981, 13787482, 13503598
10133521, 12718090, 13399435, 14023636, 13860201, 12401111, 13257247
13362079, 14176879, 12917230, 13923374, 14220725, 14480675, 13524899
13559697, 9706792, 14480674, 13916709, 13098318, 13773133, 14076523
13340388, 13366202, 13528551, 12894807, 13454210, 13343438, 12748240
14205448, 13385346, 15853081, 14273397, 12971775, 13582702, 10242202
13035804, 13544396, 16382353, 8547978, 14226599, 14062795, 13035360
12693626, 13332439, 14038787, 14062796, 12913474, 14841409, 14390252
16314470, 13370330, 13059165, 14062797, 14062794, 12959852, 13358781
12345082, 12960925, 9659614, 13699124, 14546638, 13936424, 13338048
12938841, 12658411, 12620823, 12656535, 14062793, 12678920, 13038684
14062792, 13807411, 13250244, 12594032, 15862022, 9761357, 12612118
13742464, 14052474, 13911821, 13457582, 13527323, 15862020, 13910420
13502183, 12780098, 13705338, 13696216, 14841558, 10263668, 15862023
16056266, 15862024, 13554409, 13645917, 13103913, 13011409, 14063280

--------------------------------------------------------------------------------

OPatch succeeded.

./opatch lsinventory -detail --补丁详细信息查看

1.5 将升级信息载入数据库

cd $ORACLE_HOME/rdbms/admin
sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> STARTUP
SQL> @catbundle.sql psu apply
SQL> QUIT

RAC 的PSU升级

下载p16083653_112030_Linux-x86-64.zip，其中包含GI和DB的补丁。

2.1 升级准备

[root@oradb1 dbhome_1]# unzip p16083653_112030_Linux-x86-64.zip

[root@oradb1 dbhome_1]#chown 777 -R ./p16083653

[root@oradb1 dbhome_1]#cd p16083653
[root@oradb1 p16083653]# ls
16056266 16315641 atp_lfp bundle.xml README.html README.txt

[oracle@oradb1 ~]$ /u01/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch version
Invoking OPatch 11.2.0.1.7

OPatch Version: 11.2.0.1.7

OPatch succeeded.

--可以看到OPatch 是11.2.0.1.7 当时有点蒙，我安装的11.2.0.3.0啊！怎么会是这个文档版本，文档上说安装11.2.0.3.6需要OPatch的版本为
11.2.0.3.4以上的Opatch。
--下载p6880880_112000_Linux-x86-64.zip最新的Opatch，Patch 6880880总是会提供最新的Opatch的。

1、Opach升级，将ORACLE_HOME/OPatch 目录中的内容mv到一个备份文件夹中
2、将p6880880_112000_Linux-x86-64.zip 存放到ORACLE_HOME目录下，注意权限问题chown oracle：oinstall p6880880_112000_Linux-x86-64.zip
3、[oracle@oradb1 dbhome_1]$ unzip p6880880_112000_Linux-x86-64.zip
4、[oracle@oradb1 OPatch]$ /u01/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch version
OPatch Version: 11.2.0.3.4

OPatch succeeded.

2.2 ocm.rsp准备

--静默安装升级模式需要OCM的回应文件，GRID_HOME/OPatch/ocm.rsp
--这个OCM在安装11.2.0.3.0版本的时候，不管windows还是Linux从来没安装成。Oracle Configureation Manger（OCM）就是一个为oracle软件支持
提供监控信息的工具。软件支持总是被我忽略掉，因为不会支持外网的。
--还好有办法创建这个文件

% $ORACLE_HOME/OPatch/ocm/bin/emocmrsp -no_banner -output /u01/app/oracle/unconfig.rsp

Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name:

<<<< Press Enter/Return key and don't provide any input >>>>

You have not provided an email address for notification of security issues.
Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]: Y <<< type Y/Yes >>>
The OCM configuration response file (/u02/unconfig.rsp) was successfully created. <<

Note: we can use this response file for other similar ORACLE_HOME (Single OR GRID infra)
***************************************************************************************************************************************

% $ORACLE_HOME/OPatch/ocm/bin/emocmrsp -no_banner -output /u02/config.rsp

Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name: <Provide Oracle Metalink Email address Example: [e-mail] >

Password (optional): <Password will be verified if provided / Press return/Enter Key>
The OCM configuration response file (/u02/config.rsp) was successfully created. <<<

Note: we can use this response file for other similar ORACLE_HOME (Single OR GRID infra)

--此时需要连接网络输入MOS账号

While applying CPU
% opatch napply -silent -ocmrf /u02/config.rsp
While applying PSU
% opatch apply -silent -ocmrf /u02/unconfig.rsp
While using opatch auto (for Grid infrastruture)
% opatch auto -silent -ocmrf /u02/config.rsp

2.3 安装前的检查

--确认安装项目
[grid@oradb1 OPatch]$ sh opatch lsinventory -detail -oh /u01/app/11.2.0/grid/
如果有OPatch succeeded. 最后可以安装补丁

--冲突检查
--Grid用户检查
[grid@oradb1 OPatch]$ sh opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir ../p16083653/16315641/ -oh /u01/app/11.2.0/grid/
Oracle Interim Patch Installer version 11.2.0.3.4
Copyright (c) 2012, Oracle Corporation. All rights reserved.

PREREQ session

Oracle Home : /u01/app/11.2.0/grid
Central Inventory : /u01/app/oraInventory
from : /u01/app/11.2.0/grid//oraInst.loc
OPatch version : 11.2.0.3.4
OUI version : 11.2.0.3.0
Log file location : /u01/app/11.2.0/grid/cfgtoollogs/opatch/opatch2013-05-24_13-25-23PM_1.log

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
--oracle 用户检查
[oracle@oradb1 OPatch]$ sh opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/app/11.2.0/grid/p16083653/16056266/ -oh /u01/app/oracle/product/11.2.0/dbhome_1/
Oracle Interim Patch Installer version 11.2.0.3.4
Copyright (c) 2012, Oracle Corporation. All rights reserved.

PREREQ session

Oracle Home : /u01/app/oracle/product/11.2.0/dbhome_1
Central Inventory : /u01/app/oraInventory
from : /u01/app/oracle/product/11.2.0/dbhome_1//oraInst.loc
OPatch version : 11.2.0.3.4
OUI version : 11.2.0.3.0
Log file location : /u01/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch2013-05-24_14-22-14PM_1.log

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.

--两个节点分别做

--如果存在EM Agent的情况停止服务
<ORACLE_HOME>/bin/emctl stop dbconsole

2.4 补丁应用

--为Oracle RAC Database Homes 和GI 一起安装PSU
--如果数据库家目录没有被共享并且ACFS没有被配置过
--root用户执行
# opatch auto <UNZIPPED_PATCH_LOCATION> -ocmrf <ocm response file>

或者分别执行
--更新GI,然后在另一节点上同样执行
# opatch auto <UNZIPPED_PATCH_LOCATION> -oh <GI_HOME> -ocmrf <ocm response file>
--更行DB，同样在另一节点上执行

# opatch auto <UNZIPPED_PATCH_LOCATION> -oh <GI_HOME> -ocmrf <ocm response file>

--更新之后执行
SQL> CONNECT / AS SYSDBA
SQL> STARTUP
SQL> @catbundle.sql psu apply
SQL> QUIT

2.5 更新后查看

SQL> select action,comments from registry$history;
ACTION COMMENTS
--------------- --------------------
APPLY Patchset 11.2.0.2.0
APPLY Patchset 11.2.0.2.0
APPLY PSU 11.2.0.3.6
3 rows selected.
SQL> select * from v$version;
BANNER
--------------------------------------------------------------------------------
Oracle Database11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
PL/SQL Release 11.2.0.3.0 - Production
CORE 11.2.0.3.0 Production
TNS for Linux: Version 11.2.0.3.0 - Production
NLSRTL Version 11.2.0.3.0 - Production
5 rows selected.

SQL>select comp_name,version,status from dba_registry;
COMP_NAME VERSION STATUS
------------------------------ ---------- ---------------
OWB 11.2.0.3.0 VALID
Oracle Application Express 3.2.1.00.12 VALID
Oracle Enterprise Manager 11.2.0.3.0 VALID
OLAP Catalog 11.2.0.3.0 VALID
Spatial 11.2.0.3.0 VALID
Oracle Multimedia 11.2.0.3.0 VALID
Oracle XML Database 11.2.0.3.0 VALID
Oracle Text 11.2.0.3.0 VALID
Oracle Expression Filter 11.2.0.3.0 VALID
Oracle Rules Manager 11.2.0.3.0 VALID
Oracle Workspace Manager 11.2.0.3.0 VALID
Oracle Database Catalog Views11.2.0.3.0 VALID
Oracle Database Packages and T 11.2.0.3.0 VALID
ypes
JServer JAVA Virtual Machine 11.2.0.3.0 VALID
Oracle XDK 11.2.0.3.0 VALID
Oracle Database Java Packages11.2.0.3.0 VALID
OLAP Analytic Workspace 11.2.0.3.0 VALID
Oracle OLAP API 11.2.0.3.0 VALID
Oracle Real Application Clusters 11.2.0.3.0 VALID

--结果并不在V$version和dba_registry视图中体现

[oracle@oradb1 OPatch]$ ./opatch lspatches

16056266;Database Patch Set Update : 11.2.0.3.6 (16056266)
16315641;Grid Infrastructure Patch Set Update : 11.2.0.3.6 (16083653)

[oracle@oradb1 OPatch]$ ./opatch lsinventory
Oracle Interim Patch Installer version 11.2.0.3.4
Copyright (c) 2012, Oracle Corporation. All rights reserved.

Oracle Home : /u01/app/oracle/product/11.2.0/dbhome_1
Central Inventory : /u01/app/oraInventory
from : /u01/app/oracle/product/11.2.0/dbhome_1/oraInst.loc
OPatch version : 11.2.0.3.4
OUI version : 11.2.0.3.0
Log file location : /u01/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch2013-05-28_11-26-33AM_1.log

Lsinventory Output file location : /u01/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs/opatch/lsinv/lsinventory2013-05-28_11-26-33AM.txt

--------------------------------------------------------------------------------
Installed Top-level Products (1):

Oracle Database 11g 11.2.0.3.0
There are 1 products installed in this Oracle Home.

Interim patches (2) :

Patch 16056266 : applied on Fri May 24 17:11:46 CST 2013
Unique Patch ID: 15962803
Patch description: "Database Patch Set Update : 11.2.0.3.6 (16056266)"
Created on 12 Mar 2013, 02:14:47 hrs PST8PDT
Sub-patch 14727310; "Database Patch Set Update : 11.2.0.3.5 (14727310)"
Sub-patch 14275605; "Database Patch Set Update : 11.2.0.3.4 (14275605)"
Sub-patch 13923374; "Database Patch Set Update : 11.2.0.3.3 (13923374)"
Sub-patch 13696216; "Database Patch Set Update : 11.2.0.3.2 (13696216)"
Sub-patch 13343438; "Database Patch Set Update : 11.2.0.3.1 (13343438)"
Bugs fixed:
13566938, 13593999, 10350832, 14138130, 12919564, 13561951, 13624984
13588248, 13080778, 13914613, 13804294, 14258925, 12873183, 13645875
14472647, 12880299, 14664355, 14409183, 12998795, 14469008, 13719081
13492735, 13496884, 12857027, 14263036, 14263073, 13732226, 13742433
16368108, 16314469, 12905058, 13742434, 12849688, 12950644, 13742435
13464002, 13534412, 12879027, 13958038, 14613900, 12585543, 12535346
12588744, 11877623, 13786142, 12847466, 13649031, 13981051, 12582664
12797765, 14262913, 12923168, 13384182, 13612575, 13466801, 13484963
14207163, 11063191, 13772618, 13070939, 12797420, 13041324, 16314467
16314468, 12976376, 11708510, 13680405, 14589750, 13026410, 13742437
13737746, 14644185, 13742438, 13326736, 13596521, 13001379, 16344871
13099577, 9873405, 14275605, 13742436, 9858539, 14841812, 11715084
16231699, 14040433, 12662040, 9703627, 12617123, 12845115, 12764337
13354082, 14459552, 13397104, 13913630, 12964067, 12983611, 13550185
13810393, 12780983, 12583611, 14546575, 13476583, 15862016, 11840910
13903046, 15862017, 13572659, 16294378, 13718279, 14088346, 13657605
13448206, 16314466, 14480676, 13419660, 13632717, 14063281, 14110275
13430938, 13467683, 13420224, 13812031, 14548763, 16299830, 12646784
13616375, 14035825, 12861463, 12834027, 15862021, 13632809, 13377816
13036331, 14727310, 13685544, 15862018, 13499128, 16175381, 13584130
12829021, 15862019, 12794305, 14546673, 12791981, 13787482, 13503598
10133521, 12718090, 13399435, 14023636, 13860201, 12401111, 13257247
13362079, 14176879, 12917230, 13923374, 14220725, 14480675, 13524899
13559697, 9706792, 14480674, 13916709, 13098318, 13773133, 14076523
13340388, 13366202, 13528551, 12894807, 13454210, 13343438, 12748240
14205448, 13385346, 15853081, 14273397, 12971775, 13582702, 10242202
13035804, 13544396, 16382353, 8547978, 14226599, 14062795, 13035360
12693626, 13332439, 14038787, 14062796, 12913474, 14841409, 14390252
16314470, 13370330, 13059165, 14062797, 14062794, 12959852, 13358781
12345082, 12960925, 9659614, 13699124, 14546638, 13936424, 13338048
12938841, 12658411, 12620823, 12656535, 14062793, 12678920, 13038684
14062792, 13807411, 13250244, 12594032, 15862022, 9761357, 12612118
13742464, 14052474, 13911821, 13457582, 13527323, 15862020, 13910420
13502183, 12780098, 13705338, 13696216, 14841558, 10263668, 15862023
16056266, 15862024, 13554409, 13645917, 13103913, 13011409, 14063280

Patch 16315641 : applied on Fri May 24 17:08:24 CST 2013
Unique Patch ID: 15966967
Patch description: "Grid Infrastructure Patch Set Update : 11.2.0.3.6 (16083653)"
Created on 1 Apr 2013, 03:41:20 hrs PST8PDT
Bugs fixed:
16315641, 15876003, 14275572, 13919095, 13696251, 13348650, 12659561
14305980, 14277586, 13987807, 14625969, 13825231, 12794268, 13000491
13498267, 11675721, 14082976, 12771830, 14515980, 14085018, 13943175
14102704, 14171552, 12594616, 13879428, 12897902, 12726222, 12829429
13079948, 13090686, 12995950, 13251796, 13582411, 12990582, 13857364
13082238, 12947871, 13256955, 13037709, 14535011, 12878750, 14048512
11772838, 13058611, 13001955, 13440962, 13727853, 13425727, 12885323
12870400, 14212634, 14407395, 13332363, 13430626, 13811209, 12709476
14168708, 14096821, 14626717, 13460353, 13694885, 12857064, 12899169
13111013, 12558569, 13323698, 10260842, 13085732, 10317921, 13869978
12914824, 13789135, 12730342, 12950823, 13355963, 13531373, 14268365
13776758, 12720728, 13620816, 13023609, 13024624, 13039908, 13036424
13938166, 13011520, 13569812, 12758736, 13001901, 13077654, 13430715
13550689, 13806545, 13634583, 14271305, 12538907, 13947200, 12996428
13066371, 13483672, 12897651, 13540563, 12896850, 13241779, 12728585
12876314, 12925041, 12650672, 12398492, 12848480, 13652088, 16307750
12917897, 12975811, 13653178, 13371153, 14800989, 10114953, 14001941
11836951, 14179376, 12965049, 14773530, 12765467, 13339443, 13965075
16210540, 14307855, 12784559, 14242977, 13955385, 12704789, 13745317
13074261, 12971251, 13993634, 13523527, 13719731, 13396284, 12639013
12867511, 12959140, 14748254, 12829917, 12349553, 12849377, 12934171
13843080, 14496536, 13924431, 12680491, 13334158, 10418841, 12832204
13838047, 13002015, 12791719

Rac system comprising of multiple nodes
Local node = oradb1
Remote node = oradb2

--------------------------------------------------------------------------------

OPatch succeeded.

自此，oracleRAC和单机的数据库全部升级完毕