What’s that? You thought Oracle already supported their flagship software product on the current version of their flagship operating system and kernel? Well that is what most of Blue Gecko’s Remote DBA customers thought too. Too often we have had to break the news to a customer who has already dutifully built out their hosts with Oracle’s latest Linux that they will need to reinstall with OEL 5.x in order to be running a supported combination.

Oracle’s certification site adds a few more details. Although the UEK v.2 has been available since March 13, this certification only applies to OEL6 with UEK v.1. So don’t go upgrading to the latest and greatest UEK yet. Also, this certification appears to be valid only for x86-64 architectures, not i386. My Oracle Support’s Certify tool also only shows the certification as valid only for Oracle databases running patchset 11.2.0.3.

Finally, certain things you would want to have when running Oracle, such as the oracle-validated RPM bundle, are still missing from the Unbreakable Linux Network’s OEL6 Yum repositories.

Our advice is to not go rushing to upgrade yet. Let’s give it a few months for the early adopters to flush out the usual SNAFUs, and then take another look.

Related posts:

1. Oracle Certification vs Oracle Support
2. Blue Gecko Announces Free Oracle Database Performance Reviews
3. Oracle Database Hosting on AWS

Did I ever tell you the makers of RAC
had seven features and named each flashback?

Well they did, and it wasn’t a smart thing to do.
You see, when the customers wanted a clue
as to how to keep data from getting deleted
the RAC folks said “flashback” and customers heeded.

They turned on all seven of those flashback features
Each one was a slightly dissimilar creature.
Some used the UNDO, some used flashback files
Some just renamed tables to bin$ styles.

One was a place you keep things for recovering
Another was just for forensic discovering
With so many features called by the same name
when thinks broke no one knew just which one they should blame.

On a Friday three minutes past seventeen hundred,
users ask, “What’s the deal? Our data’s been plundered.”
“It looks like all names in the customer table,
are now ‘John Q Public,’ a certain mislabel.”

It was Nimrod, an intern fresh from his instruction.
What he thought was just test was really production.
“No Problem,” says Morton, the wise DBA,
“Flashback is on. I’ll restore it today.”

First Morton asked Nimrod what time he committed
his update that had any where clause omitted.
“I ran it at just past noon yesterday lunch,
It ran for so long that I went for a munch.”

By now it had been almost thirty one hours
but Morton knew that he could call on his powers,
and on the mere fact that that the undo_retention
was set to two days, a quite lengthy extension.

But woe! When he tried to engage Flashback Query,
he got “snapshot too old” and it ruined his theory
that undo_retention makes Oracle hold
all undo data no matter how old.

It turns out that there was no undo_guarantee
or autoextend which would also be key
to use all Flashback features reliant on Undo
A realization he would slowly come to.

Now Flashback Transaction and Versions Between
were out of the picture, although unforseen.
But he still had four more kinds of Flashback to try
So he thought them each through, to see if they might fly.

Undropping the table would use flashback syntax,
But that wouldn’t help, since the table was intact.
With database flashback all could be reverted
But subsequent changes would then be subverted

Do you know who was governor of California
When Flashback Data Archive came out and I’ll warn ya,
it will not help poor Morton, though it’s called Total Recall.
He’d have had to enable it for tables to see all.

The last of the flashbacks was the old FRA,
the Flashback Recovery Area they say.
And it’s just a directory where things are kept
Like logs and old backups made while Morton slept.

So after all that there was nary a way
to use Flashback of any sort to save the day.
Old-fashioned LogMiner was what Morton used
To restore all the rows that poor Nimrod abused.

With so many flashbacks and so much confusion
I bet that Oracle regrets the profusion
of so many things that they call the same name.
But now its too late and there’s no one to blame.

Related posts:

1. In-Memory Undo
2. Emergency Database Repair
3. Recovering a Schema From InnoDB .frm Files

One day in 2009, a company in Florida called my cell phone at 2AM. They described their problem as follows:

We have a 32-bit Intel server running Red Hat Enterprise Linux 4 and Oracle Database Enterprise Edition 9.2.0.1. There are four databases ranging in size from 20G to 100G. The storage is EXT3 filesystems on partitions of an Apple Xserv RAID5 array.

We had a power outage yesterday, and the database server powered down and booted back up. Prior to yesterday, it has not rebooted for about one year. We have been running trouble-free for the previous year. Upon reboot, Oracle started automatically, but all of the databases appeared as they did about one year ago. It is like the database hasn’t been saving the changes we have been making for the past year. None of the inserts, updates or deletes made in the past year are present in the databases. We are absolutely flummoxed. Please help!

I logged into the server and it was just as they described. Even the alert log and messages files ended suddenly about one year prior, and picked up again on the day of the most recent reboot. There was no trace of the intervening 12 months of work. The customer was ready to resort to their backups, but wanted to understand the problem before they proceeded. In addition, restoring backups would mean losing the last 24 hours of transactions, since archivelogs had not gone to tape for that long, and they were missing just like everything else from before the most recent reboot.

They weren’t the only ones who were flummoxed. I just sat there thinking, “where do I start?” After some poking around, though, I solved the problem. Any guesses what went wrong here? I’ll post the solution in about a week. No fair posting the solution if I’ve told you this story before!

Related posts:

1. Oracle (finally) announces support for Oracle Database 11gR2 on OEL6
2. Oracle Database Hosting on AWS
3. Oracle Backups to S3