Issues with RMAN on AIX using TDP for Oracle

So our group has thankfully decided to go with RMAN for database backups, and you know i'm first in line to take this task on.

Here's my setup at a high level:

• 10gR2 database (rmandb - snappy name huh? :)
• IBM's TDP 5.3 for Oracle
  
• AIX 5.2

Steps:

1. My internal IT department setup my TSM domain, etc.
2. Sysadmin installed the TDP fileset and then generated the password file for me
   
3. I used the TSM Quick Start Guide to get all of my config files and directories setup
4. I used OEM GC R3 to set the RMAN preferences, and to setup the Tape parameters: ENV=(TDPO_OPTFILE=/usr/tivoli/tsm/client/oracle/bin64/tdpo.opt)
   

Once all of this was done and "$ tdpoconf SHOWENVironment" looked good, i went for my first test drive.

I logged into RMAN, connected to target, and then ran a connection test:

run {
allocate channel oem_sbt_backup type 'sbt_tape' format '%U' parms 'ENV=(TDPO_OPTFILE=/usr/tivoli/tsm/client/oracle/bin64/tdpo.opt)';
release channel oem_sbt_backup;
}

And it worked fine. Woot!

Next, I used the "Test Backup" button on the Backup Setting OEM page. It's a simple test consisting of:

run {
allocate channel oem_sbt_backup type 'sbt_tape' format '%U' parms 'ENV=(TDPO_OPTFILE=/usr/tivoli/tsm/client/oracle/bin64/tdpo.opt)';
backup as BACKUPSET current controlfile tag '05252007020711';
restore controlfile validate from tag '05252007020711';
release channel oem_sbt_backup;
}

And... it fails with this lovely error output:

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of backup command on ORA_SBT_TAPE_1 channel at 05/22/2007 16:28:09
ORA-19506: failed to create sequential file, name="0aiiceoi_1_2", parms=""
ORA-27028: skgfqcre: sbtbackup returned error
ORA-19511: Error received from media manager layer, error text:
ANS0231E (RC2025) The management class name was not found.

Nice huh?? What the heck is "ANS0231E (RC2025) The management class name was not found."?

If you think that's depressing, try entering "ANS0231E" into Google. There is less than 30 results worldwide! I hate when that happens...

After working with IS and IBM for 3 weeks (3 weeks!!!), we finally worked our way up to a SEV-1 analyst on a conference call and he figures it out.

Wait for it...

My tdpo.opt file had no stanza for "TDPO_MGMT_CLASS_2" (or _3, or _4) because I was only allocating one channel (these values are only used when more than one channel is opened to TSM, otherwise, it used the channel appropriately named "DEFAULT").

Here's the gotcha: I setup RMAN to do 'duplex' copies, e.g.: create 2 copies of all datafiles and 2 copies of all control files.

The problem is RMAN opens 2 channels when duplex is set!!!

Bottom Line: (I had IBM add this to their KB)
If you receive the dreaded "ANS0231E (RC2025) The management class name was not found." error, the first question you need to ask yourself is, "Am i using duplex copies", if so, i'll bet you don't have a value for "TDPO_MGMT_CLASS_2" in your tdpo.opt file.

This was a hard lesson learned... :-)

Upgrading EM Grid Control from 10.2.0.1 to 10.2.0.3

Okay, so I finally bit the bullet and upgraded EM Grid Control from the initial install version of 10.2.0.1 to 10.2.0.3 and upgraded the EM repository (cleverly named EMREP) from 10.1.0.5 to 10.2.0.1 .

Here's a few observations about the upgrade:

1. The EMREP page in GC still showed the old 10.1 home instead of the new 10.2 home.
   
1. This stems from the fact that I had 2 different ORACLE_HOME installs: /orabase/product/ora10g and /orabase/OracleHomes/db10g.
2. Upgrading meant moving from one home to another and GC did not pick up on this change
   
3. Solution: Remove EMREP from monitoring and re-add it (BTW, refreshing the host configuration did not work!)
2. It took much longer than I had planned.
1. As part of the 10.2.0.3 patch, a repository upgrade was necessary - this step alone took about 45 minutes - yikes!
   
3. There are about 8 new security policies that come with GC 10.2.0.3 ("Password Complexity Verification Function Usage", etc.)
1. Once you've upgraded and logged in to GC, you'll see a whole bunch of new security policy warnings.
2. Sadly, these have to be fixed by going to your Monitoring Templates (you do use templates right??) and using the "Add Policies To Template" button to add the offending policies and then set them appropriately.

One last thing:

• They finally fixed that god-forsaken bug that kept the Database->Administration->Tablespaces page from rendering on 9i databases in under an hour! Jeez, that was annoying as hell! :-)