Creating standby database 10.1.0.4.0 using OMF

While creating a standby database on 10.1.0.4.0 , i ran in some trouble while performing the following RMAN statements :

RMAN> run{
ALLOCATE AUXILIARY CHANNEL dup1 DEVICE TYPE DISK
DUPLICATE TARGET DATABASE FOR STANDBY DORECOVER;
}

First a channel will be allocated on which the duplication starts. The datafiles are places in the right location. But when it comes to the point where the duplicate action executes :

contents of Memory Script:
{
switch clone datafile all;
}
executing Memory Script

The following error is thrown :

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of Duplicate Db command at 10/28/2009 19:08:41
RMAN-03015: error occurred in stored script Memory Script
RMAN-06136: ORACLE error from auxiliary database: ORA-19625: error identifying file /u02/oradata/COREF/datafile/o1_mf_system_5fyfx677_.dbf
ORA-27037: unable to obtain file status
Linux Error: 2: No such file or directory
Additional information: 3

RMAN>

Here i could see that the filename which is called for , is indeed the correct one.

After some researching i found that there is a difference in the database version in which the performed actions take place.
I did set the DB_FILE_NAME_CONVERT and LOG_FILE_NAME_CONVERT , which actually should NOT be done in rdbms version 10.1.4.0. ( other versions you should , if appropriate )
Also , when performing a duplicate when the primairy database is using OMF , you should set the standby_file_management=AUTO as this will restore datafile using %u which will be converted in the correct name to be picked up by the script.

After setting the following parameters in the auxiliary pfile, the RMAN duplicate finished successfully.

db_name = COREP (- must be set to a name of the primary db_name)
db_unique_name = COREF (- must be set to a name different from primary db_unique_name)
standby_file_management = AUTO (- must set to AUTO)
db_create_file_dest = ‘/u02/oradata’ (- must be set)

Restore Datafile using RMAN backup

When for some reason a datafile is lost and RMAN is in place. You might want to recover the file using the RMAN restore option for restoring datafiles.

Here is a brief overview on how to proceed.

oracle@xxx: $ ls -ltr
total 1656572
-rw-r—– 1 oracle oinstall 20979712 Jul 11 06:00 temp01.dbf
-rw-r—– 1 oracle oinstall 52429312 Jul 12 22:01 redo01.log
-rw-r—– 1 oracle oinstall 262152192 Jul 12 22:02 users01.dbf
-rw-r—– 1 oracle oinstall 52429312 Jul 12 22:02 redo03.log
-rw-r—– 1 oracle oinstall 199237632 Jul 13 08:29 sysaux01.dbf
-rw-r—– 1 oracle oinstall 209723392 Jul 13 08:50 undotbs01.dbf
-rw-r—– 1 oracle oinstall 314580992 Jul 13 08:50 system01.dbf
-rw-r—– 1 oracle oinstall 536879104 Jul 13 08:54 xwiki01.dbf
-rw-r—– 1 oracle oinstall 52429312 Jul 13 08:54 redo02.log
-rw-r—– 1 oracle oinstall 7094272 Jul 13 08:54 control03.ctl
-rw-r—– 1 oracle oinstall 7094272 Jul 13 08:54 control01.ctl

oracle@xxx: $ rm xwiki01.dbf

Here we are 1 datafile short !

oracle@xxx: $ ls -ltr
total 1656572
-rw-r—– 1 oracle oinstall 20979712 Jul 11 06:00 temp01.dbf
-rw-r—– 1 oracle oinstall 52429312 Jul 12 22:01 redo01.log
-rw-r—– 1 oracle oinstall 262152192 Jul 12 22:02 users01.dbf
-rw-r—– 1 oracle oinstall 52429312 Jul 12 22:02 redo03.log
-rw-r—– 1 oracle oinstall 199237632 Jul 13 08:29 sysaux01.dbf
-rw-r—– 1 oracle oinstall 209723392 Jul 13 08:50 undotbs01.dbf
-rw-r—– 1 oracle oinstall 314580992 Jul 13 08:50 system01.dbf
-rw-r—– 1 oracle oinstall 52429312 Jul 13 08:54 redo02.log
-rw-r—– 1 oracle oinstall 7094272 Jul 13 08:54 control03.ctl
-rw-r—– 1 oracle oinstall 7094272 Jul 13 08:54 control01.ctl

First connect to your RMAN “catalog” and list backup to find the datafile number to restore.

oracle@xxx: $ $ORACLE_HOME/bin/rman

Recovery Manager: Release 10.2.0.1.0 – Production on Mon Jul 13 08:54:59 2009

Copyright (c) 1982, 2005, Oracle. All rights reserved.

RMAN> connect target;

connected to target database: XWIKI (DBID=675473154)

RMAN> list backup;

using target database control file instead of recovery catalog

List of Backup Sets
===================

BS Key Type LV Size Device Type Elapsed Time Completion Time
——- —- — ———- ———– ———— —————
82 Full 137.80M DISK 00:00:55 06-JUL-09
BP Key: 82 Status: AVAILABLE Compressed: NO Tag: TAG20090706T220341
Piece Name: /u04/backup/XWIKI/rman/backup_df_88_691538622.bak
List of Datafiles in backup set 82
File LV Type Ckp SCN Ckp Time Name
—- — —- ———- ——— —-
1 Full 2127473 06-JUL-09 /u02/oradata/xwiki/system01.dbf

BS Key Type LV Size Device Type Elapsed Time Completion Time
——- —- — ———- ———– ———— —————
84 Full 76.31M DISK 00:00:59 06-JUL-09
BP Key: 84 Status: AVAILABLE Compressed: NO Tag: TAG20090706T220341
Piece Name: /u04/backup/XWIKI/rman/backup_df_86_691538622.bak
List of Datafiles in backup set 84
File LV Type Ckp SCN Ckp Time Name
—- — —- ———- ——— —-
5 Full 2127471 06-JUL-09 /u02/oradata/xwiki/xwiki01.dbf

..
…
..

BS Key Type LV Size Device Type Elapsed Time Completion Time
——- —- — ———- ———– ———— —————
141 Full 6.83M DISK 00:00:00 12-JUL-09
BP Key: 141 Status: AVAILABLE Compressed: NO Tag: TAG20090712T220211
Piece Name: /u04/backup/XWIKI/rman/c-675473154-20090712-00
Control File Included: Ckp SCN: 2330680 Ckp time: 12-JUL-09
SPFILE Included: Modification time: 08-JUL-09

RMAN>

We know now that datafile 5 is the one to restore. Before we can proceed we need to put it offline.

RMAN> SQL ‘ALTER DATABASE DATAFILE 5 OFFLINE’;

sql statement: ALTER DATABASE DATAFILE 5 OFFLINE

RMAN>

Now that the datafile is offline , we are ready for the RMAN datafile restore.

RMAN> RESTORE DATAFILE 5;

Starting restore at 13-JUL-09
allocated channel: ORA_DISK_1
channel ORA_DISK_1: sid=141 devtype=DISK
allocated channel: ORA_DISK_2
channel ORA_DISK_2: sid=140 devtype=DISK
allocated channel: ORA_DISK_3
channel ORA_DISK_3: sid=149 devtype=DISK
allocated channel: ORA_DISK_4
channel ORA_DISK_4: sid=139 devtype=DISK

channel ORA_DISK_1: starting datafile backupset restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
restoring datafile 00005 to /u02/oradata/xwiki/xwiki01.dbf
channel ORA_DISK_1: reading from backup piece /u04/backup/XWIKI/rman/backup_df_142_692056911.bak
channel ORA_DISK_1: restored backup piece 1
piece handle=/u04/backup/XWIKI/rman/backup_df_142_692056911.bak tag=TAG20090712T220151
channel ORA_DISK_1: restore complete, elapsed time: 00:00:16
Finished restore at 13-JUL-09

RMAN>

At this point we are not yet ready to put the datafile back online

RMAN> SQL ‘ALTER DATABASE DATAFILE 5 ONLINE’;

sql statement: ALTER DATABASE DATAFILE 5 ONLINE
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of sql command on default channel at 07/13/2009 08:53:45
RMAN-11003: failure during parse/execution of SQL statement: ALTER DATABASE DATAFILE 5 ONLINE
ORA-01113: file 5 needs media recovery
ORA-01110: data file 5: ‘/u02/oradata/xwiki/xwiki01.dbf’

As this is an “old” datafile and not the up to date one, we need to recover the datafile to match the others. This can easily be done by executing the following.

RMAN> RECOVER DATAFILE 5;

Starting recover at 13-JUL-09
using channel ORA_DISK_1
using channel ORA_DISK_2
using channel ORA_DISK_3
using channel ORA_DISK_4

starting media recovery

archive log thread 1 sequence 6 is already on disk as file /u03/oradata/xwiki/archiving/1_6_691685756.arc
archive log thread 1 sequence 7 is already on disk as file /u03/oradata/xwiki/archiving/1_7_691685756.arc
archive log thread 1 sequence 8 is already on disk as file /u03/oradata/xwiki/archiving/1_8_691685756.arc
archive log thread 1 sequence 9 is already on disk as file /u03/oradata/xwiki/archiving/1_9_691685756.arc
archive log thread 1 sequence 10 is already on disk as file /u03/oradata/xwiki/archiving/1_10_691685756.arc
archive log thread 1 sequence 11 is already on disk as file /u03/oradata/xwiki/archiving/1_11_691685756.arc
archive log thread 1 sequence 12 is already on disk as file /u03/oradata/xwiki/archiving/1_12_691685756.arc
archive log filename=/u03/oradata/xwiki/archiving/1_6_691685756.arc thread=1 sequence=6
archive log filename=/u03/oradata/xwiki/archiving/1_7_691685756.arc thread=1 sequence=7
archive log filename=/u03/oradata/xwiki/archiving/1_8_691685756.arc thread=1 sequence=8
media recovery complete, elapsed time: 00:00:03
Finished recover at 13-JUL-09

RMAN>

Now that the datafile is brought up to speed, we can put our datafile online again.

RMAN> SQL ‘ALTER DATABASE DATAFILE 5 ONLINE’;

sql statement: ALTER DATABASE DATAFILE 5 ONLINE

RMAN>

error while loading shared libraries: libaio.so.1: cannot open shared object file

While wanting to preform a disaster recovery using RMAN on a spare server which has an ORACLE_HOME 10.2.0.1 64bits installed on a Red Hat Enterprise Linux AS release 4 Operating System.
I tried the following to read back my controlfile. But before getting to this point i faced this error :

oracle@xxx: $ $ORACLE_HOME/bin/rman target /

Recovery Manager: Release 10.2.0.1.0 – Production on Wed Jul 8 10:32:56 2009

Copyright (c) 1982, 2005, Oracle. All rights reserved.

oraclewiki: error while loading shared libraries: libaio.so.1: cannot open shared object file: No such file or directory
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-00554: initialization of internal recovery manager package failed
RMAN-04005: error from target database:
ORA-12547: TNS:lost contact

Assuming the libaio library was installed i searched and tried to clear this up. As the server is a x86_64 server there should be a 64 bit library of libaio installed :

oracle@xxx: # find / -name libaio.so.1
/local/app/oracle/product/db10g/lib/stubs/libaio.so.1
/local/app/oracle/product/agent10g/lib/stubs/libaio.so.1
/local/app/oracle/product/db10gtest/lib/stubs/libaio.so.1
/local/app/oracle/product/10.2.0/db/lib/stubs/libaio.so.1
/local/app/oracle/product/10.2.0/db/lib32/stubs/libaio.so.1
/usr/lib/libaio.so.1

oracle@xxx: $ ldd $ORACLE_HOME/bin/oracle | grep libaio
libaio.so.1 => not found

As i could see here, this was not the case. Only the 32 bit is availible and therefor not used by the Oracle 64 bits Database software.
The output of the ldd command supports this as there is a dependency problem for the oracle process on libaio.so.1.

After adding the following rpm’s i was able to connect without problems.

root@xxx: # up2date –showall | grep libaio | grep 64
libaio-0.3.105-2.x86_64
libaio-devel-0.3.105-2.x86_64

As you can see now the correct library is found and the dependency problem is solved.

oracle@xxx: $ ldd $ORACLE_HOME/bin/oracle | grep libaio
libaio.so.1 => /usr/lib64/libaio.so.1 (0×0000002a96f40000)