The Lazy Tech's Blog

Designing a secured Landing Zone in AWS

2023-04-13T03:44:00.002+01:00

When adopting cloud computing, securing your cloud infrastructure should be a top priority. Starting with a landing zone provides the foundational infrastructure for all workloads and applications. This should also include designing for security in your landing zone.

In this post, we'll discuss how to design a secure landing zone in AWS and the best practices to follow.

1. Create a multi-account structure: Creating a multi-account structure is a best practice for securing your landing zone in AWS. This allows you to separate workloads, limit blast radius, and apply specific security controls to each account, or set of accounts. You can use AWS Organizations to create and manage multiple accounts in your AWS environment.

2. Define your security requirements: Before designing your landing zone, you should first define your security requirements. This will help you determine what security controls you need to put in place. Identify the type of data you will be storing (i.e. data classification), who will have access to it (i.e. data access), and the compliance regulations you must comply with (i.e. data compliance). The security controls are applied in what is referred to as the Security Baseline or Layer.

3. Use AWS Identity and Access Management (IAM): IAM is a service that enables you to manage user access and permissions to AWS resources. You should use IAM to enforce the principle of least privilege and ensure that users only have access to the resources they need. You should also enable multi-factor authentication (MFA) for added security. These service resources are applied in what is referred to as the Identity Baseline or Layer.

4. Implement encryption: Encryption is the process of encoding data so that only authorized parties can access it. You should encrypt all sensitive data at rest and in transit. AWS offers a variety of encryption options, including Amazon S3 encryption, AWS Key Management Service (KMS), and AWS Certificate Manager. The encryption key service is applied in the Data Protection Baseline or Layer.

5. Use AWS Config and AWS CloudTrail: AWS Config and AWS CloudTrail are services that provide visibility and auditing capabilities for your AWS environment. AWS Config helps you monitor resource configuration changes, while AWS CloudTrail provides a detailed record of all API activity in your AWS account. These services are applied in the Logging Baseline or Layer, and are dependent upon the Data Protection Baseline to provide the encryption keys necessary to encrypt log data.

6. Implement network segmentation: Network segmentation is the process of dividing your network into smaller, more secure segments. This helps to prevent lateral movement and limit the impact of a security breach. You can use Virtual Private Cloud (VPC) to create network segments in AWS. This capability is applied in the Network Baseline or Layer, and is dependent upon the previous deployed Logging Baseline and Data Protection Baseline to provide logging and encryption keys respectively.

7. Implement automated security checks: You should implement automated security checks to ensure that your landing zone remains secure over time. AWS provides a range of automated security tools, including AWS Security Hub, AWS Config Rules, and Amazon Inspector. This capability is implemented in the Compliance Baseline or Layer.

In conclusion, designing a secure landing zone in AWS requires careful planning and attention to detail. By following the best practices outlined above, you can create a landing zone that is secure, scalable, and easy to manage. Remember to regularly review your security controls and update them as necessary to keep up with changing security threats.

Understanding Cloud Foundations

2023-03-13T01:39:00.000+00:00

“Cloud foundations" generally refers to the fundamental building blocks or components of cloud computing infrastructure. There are a few key aspects typically included in cloud foundations, such as:

Virtualization: Cloud computing relies heavily on virtualization technology to provide a layer of abstraction between physical resources (like servers, storage devices, and networks) and the software applications that use those resources. Virtualization allows for more efficient use of resources, greater flexibility, and easier management of cloud environments.
Infrastructure-as-a-Service (IaaS): IaaS is a cloud service model that provides virtualized computing resources over the internet, including servers, storage, and networking. Cloud providers like Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP) offer IaaS services that customers can use to build and deploy their own applications and services.
Platform-as-a-Service (PaaS): PaaS is another cloud service model that provides a platform for developing, testing, and deploying applications without having to worry about the underlying infrastructure. PaaS providers like Heroku, Google App Engine, and AWS Elastic Beanstalk offer preconfigured platforms that include operating systems, programming languages, databases, and other tools.
Software-as-a-Service (SaaS): SaaS is a cloud service model that allows users to access software applications over the internet, typically through a web browser or mobile app. Examples of SaaS applications include Google Workspace, Salesforce, and Microsoft Office 365.
Cloud security: As with any computing environment, security is a critical consideration in the cloud. Cloud providers offer various security features and services to help customers protect their data and applications, including encryption, access controls, firewalls, and threat detection.

Overall, cloud foundations are the underlying technologies, services, and practices that enable cloud computing to work effectively and efficiently.

If you want to learn more about cloud foundations and what a practical implementation entails, stay tuned here as I will be detailing the components that would go into a real-world cloud deployment.

What is a landing zone and why you should be using one

2023-03-08T23:35:00.000+00:00

Cloud computing has become so common place that the question is no longer “should?”, but “why not?”. Now as you, like so many others, begin your journey to the cloud, trying to decipher the many technical terms and jargon, and ensuring you follow best practices for security, cost efficiency and operations, you will likely come across the term, “landing zone”. So what is a landing zone, and why would I need one? A landing zone is a best practice in cloud computing for establishing a secure and well-architected foundation that can help you scale and manage your cloud environment effectively. You can definitely get started without one, get your services up and running and be productive. But unless you are a one-person shop, you are more typically going to run into challenges at some point, and start asking questions like:

How do I better isolate my environments for improved security and protection from mistakes?
How do I provide the appropriate access to persons, making sure they have the right access level, to the right things?

There are several reasons why a landing zone is important when starting out in cloud computing:

Security: A landing zone provides a secure foundation for your cloud environment by establishing security controls and best practices from the outset. This helps to reduce the risk of security breaches, data leaks, and other security incidents.
Compliance: A landing zone can help you meet compliance requirements for your industry or region, by establishing policies and controls that are specific to your compliance needs.
Scalability: A landing zone provides a scalable foundation that can grow with your cloud environment. By establishing a set of repeatable patterns and configurations (or blueprints), you can reduce the time and effort required to deploy new workloads and applications in your cloud environment.
Cost optimization: A landing zone can help you optimize costs by establishing cost controls and best practices from the outset. By implementing cost optimization strategies early on, you can avoid common cost pitfalls and ensure that your cloud environment is cost-effective over the long term.
Management and governance: A landing zone can help you establish management and governance policies that are specific to your business needs. By creating a set of standardized practices for deploying and managing resources in your cloud environment, you can ensure that your environment is consistent, well-organized, and easy to manage.

Overall, a landing zone provides a foundation for a secure, compliant, scalable, and cost-effective cloud environment. It can help you get started with cloud computing on the right foot and avoid common pitfalls and challenges that can arise in cloud environments.

If you found this useful, please follow along as I will provide future posts on the details of a landing zone, such as AWS Control Tower, custom, third-party, and provide some lessons learned.

Shifting IT to scale

2015-01-05T21:28:00.001+00:00

While some IT groups are fine tuning their practices, many others are still on the journey to achieve greater efficiency at scale. Not all can make this shift (typically due to size, and budget) and that may be fine for them, but there are the unfortunate few who have not yet realized the need to shift their IT practices. Integration, deployment, and operations have been going through a huge transition, in large part due to business needs to better support more customers globally, provide better availability and response times; all at low costs (how else can you maximize profits?). A simple phrase for this is “cloud enablement”. Yeah, that catch all phrase that hopefully you’ve come to realize you need to be a part of, or risk failing behind. To achieve this, a fully automated deployment pipeline is a necessary component which requires a few things be put in place, namely:

Thorough application monitoring
A Collaborative culture
Developer virtual machines
One-click integrations
Continuous integration

To support many daily deployments, the development process should revolve around making many small, continuous changes, while keeping risk to a minimum. To be comfortable with at all times, you'll need to adopt a range of tools and practices.

Making Developers Comfortable

One of the best ways to ensure a developer can be comfortable with making any deployment is to ensure each developer has their own full production stack. Every developer should have their own virtual machine (there are free options like Virtual Box, my preferred, KVM and Xen), configured with a configuration management tool such as Puppet, Chef, Ansible, Salt or whatever you use internally, with the same configuration used in production. Ensuring the whole provisioning process is automated is another necessity since this not only optimizes productivity by minimizing creation times, but eliminates human error in typing in multiple commands which may vary according to environment and platforms.

On the continuous integration front, having a tool which allows developers to test their changes without having to commit to the production code repository helps to keep production clean and thus deployable while allowing quick and reliable testing. With the rising increase in popularity of container technologies (looking at you Docker), one can spin up on-demand, isolated and parallelized containers to conduct separate tests. The deployment process then becomes a simple one-click process between environments. A/B or other such zero-downtime type production testing further adds to the comfort level, not just for developers but the business in general.

Just as important to doing continuous delivery is monitoring. KPIs (key performance indicators) should be well known and graphed (I like graphing everything that can graphed as being a visual person I find it tells the story much quicker and is more effective). Most monitoring solutions now provide anomaly pattern detection which can be quite useful vs. eyeballing some numbers or even a graph. Your typically better off with a hybrid log approach where each application/service sends logs to two locations, i.e. it’s own log aggregator service which provides short term storage, and insight into it’s local activities without any external depedencies; and a centralized log aggregator service providing longer term storage with the end-to-end insight of all services and clients.

Achieving a Collaborative Culture

Any highly ambitious or for that matter successful endeavor requires a high degree of collaboration, ongoing collaboration to be exact. Most high performers by their very nature are social and want to talk, to share, and be collaborative. One only needs to look at the success of Twitter, Facebook and other social media platforms to see this truth. Enabling that collaboration with the appropriate solution and practices are the only seeds required to make this desire grow and succeed. A highly collaborative communication style, which I like, is based on IRC with chat rooms or channels for various specific purposes. For example, each team can have it’s own room/channel for private communication, another room/channel for a specific service/application, and yet another for general discussion or perhaps “war room” (such as #warroom for outage related conversations to coordinate an investigation, discuss counter measures and resolution monitoring). Many such solutions are available in the market offering a full breadth of features such as email and ticket integration, video conferencing, white boarding and so on.

Part of a collaborative culture also involves doing a post-mortem, lessons learned or root cause analysis following an incident. I really like the idea of making this blameless as, I think, this gets things done more effectively. Typically everyone already knows whose at fault (or the team) and assigning blame in a public manner only serves to decrease moral, job satisfaction and actually learning more about what and how something happened. Finger pointing is never productive in my experience.

A final word on on-call

I don’t like being on call, as in I don’t like being called at 3am out of my sleep or having to sit by the phone, I’m sure no one actually does, but it is a necessary process for operations, support and developers. Being on-call not only makes you want to have things working so you don’t get called, but also ensures you stay in touch of the day-to-day issues that are faced. This is especially important when introducing new features or improving existing processes. I like going with a rotation schedule of one-week every four weeks. I think this quite typically and agreeable to most.

Installing Tomcat 8.0.x on OS X

2014-09-28T22:49:00.000+01:00

Prerequisite: Java

On OS X 10.9.x (Mavericks) Java is not installed by default anymore, at least not initially.The easiest way to get Java on your Mac is to open the Terminal app and type ‘java’. You will be asked if you want to install Java and OS X will take care of the rest - you just need to follow the instructions and you’ll end up with Java 7. This involves sending you to the Oracle’s Java SE web page where you will need to select the appropriate JDK (JDK 7u67 as of this writing) for download and installation.

The JDK installation package comes in a dmg and installs easily on a Mac. In the same or different Terminal app window entering:

java -version

Will now show something like this:

java version "1.7.0_67"
Java(TM) SE Runtime Environment (build 1.7.0_67-b01)
Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)

Confirming you have successfully installed Java 7.

Installing Tomcat

Now comes the Apache Tomcat installation, which is actually quite easy.

1. Download a binary distribution of the core module (apache-tomcat-8.0.12.tar.gz).

2. Using any available unarchive tool, unarchive the file from the Downloads folder to ‘/usr/local’. You likely will need Administrative privileges for this step. You can unarchive the file contents to another location of our choosing, I chose ‘/usr/local’ since it’s my standard and will centralize Tomcat.

3. To make things easier in future for transparent release updates, create a symbolic link for your location (‘/usr/local/apache-tomcat-8.0.12’) to the library location:

sudo ln -s /usr/local/apache-tomcat-8.0.12 /Library/Tomcat

4. Permissions and mode should be okay (it was for me), but to make sure you can run the below:

sudo chown -R /Library/Tomcat
sudo chmod +x /Library/Tomcat/bin/*.sh

5. Startup your Tomcat instance:

/Library/Tomcat/bin/startup.sh

6. Verify things are working by opening a browser window/tab using the default URL (http://localhost:8080) and take a look at the default page.

Everything should be functional and now ready for your customizations and/or deployments.

How to resolve "vagrant up" failing with "VBoxManage.exe: error: Code CO_E_SERVER_EXEC_FAILURE"

2014-05-28T16:12:00.000+01:00

I'm trying to use a pre-built vagrant box, one built by Mathew Baldwin for WLS12c on CentOS 6.4, and ran into a problem. The vagrant up command failed:

A few things:

Running the VirtualBox command "vboxmanage list hostonlyifs" separately is fine.
I'm not running the command or session as an Administrator, in fact that fails with a completely different error given that's a separate account and does not have the required vagrant box.
I'm using the awesome Console2 (by Marko Bozikovic) but that matters not since the same error occurs in plain old cmd.exe
The command does complete successfully in MobaXterm 7.1 (another awesome tool!), though later down during the processing of the box (during Puppet configuration I believe) my machine does a hard shutdown/crash
VirtualBox version is 4.3.12
Vagrant version is 1.6.2
Windows 7 Pro SP1, 64-bit

A Google search revealed others have had this problem, but not too many actual solutions, or at least one that worked for me. Here's how I got past this problem outside of using my MobaXterm (given that had shutdown my machine each time previously).

The fix was to start VirtualBox before running vagrant up. Even still, there remains some instability due to what seems to me like timing issues with vagrant sending commands to VirtualBox as on occasion the process fails (i.e. times out) waiting for the VM to start up and show the login prompt.

Oracle OpenWorld 2012: Monday

2012-10-02T08:13:00.001+01:00

Database 12c Features

Following yesterday's big announcements and new rumors I made a few schedule changes ensuring I allotted more time in the demo grounds to talk to various Oracle specialists concerning Database 12c. The demo grounds are truly amazing with a wealth of contacts to be made and things to be learnt from talking to the various "informed" vendors (including Oracle). Even more amazing are the large number of people that come solely for the purpose of winning free stuff - not that I'd complain if I won one of the numerous iPads on offer or even better the $10,000 offered by EMC (good to know where my companies money is going). I'm curious if any of these vendors do any analysis on the "real" contacts made vs. just those looking for stuff and follow-up sales made.

There are many new and usable features with 12c, and I would argue this will be the biggest release Oracle will have done to date (when it ships) in terms of changes and features. The upgrade process via DBUA has been given some attention with parallelism during the upgrade itself, fix-it scripts, resumption from some failures (instead of starting from scratch for everything) and a post-upgrade health check. Transportable Tablespaces (TTS) via Data Pump will be more efficient by automatically running all pre-requisite checks, and doing the full export and import. Meaning it figures out all the metadata dependencies, creates all the users, objects, grants, etc. from the source unto the target then copies across all the data and viola!

Some questions (but not all) surrounding Pluggable Databases (PDB), which I've mentioned yesterday, were answered today as well. It will support pre-12.1 databases (only 11.2.0.3 is my guess and based on the slides used) which will plug into a 12.1 container database (the housing or hypervisor database if you will). All databases can be backed up as one and recovered separately including point-in-time (PIT). PDB can also be run in standby setups though I'm still left wondering how PDB works exactly in a RAC environment. Migration into this architecture appears to be done via Data Pump (I'm guessing TTS since otherwise it would be long migration). Resource utilization is handled by Resource Manager (DBRM) so processor (and other settings?) usage can be allocated to each database. Patching/upgrades can be done separately to each database though I'd imagine the container DB must always be at the highest release (similar to Grid Infrastructure). A question, out of many I have, is how does affect Oracle VPD and will it be a paid feature? (we all know the answer is yes)

Another very interesting and immediately usable feature is "Automatic Usage Based Compression". Essentially a heat map of table partitions is used to compress various partitions based on usage/activity (INSERT, UPDATE and DELETE statements) using some user defined policies. Compression is done online, in the background. Does this mean HCC is open to all now? Is this using DBMS_REDEFINITION under the covers for the online compression change? What about compressing table blocks and not just partitions? Will the threshold for hot, warm and cold be adjustable (there is always some hidden parameter)? Is this part of the compression package/option and how much will it cost?

Redaction of Sensitive Data is another big feature. This moves the masking of sensitive data from the application level to the database level where the DBA does this change online and immediately (no logoff/logon required) based on set policies. I'm left wondering how this affects the Data Masking Pack and Label Security? (and again, how much?)

A new feature (which is actually available now supported for Exadata) is RMAN Cross Platform Incremental Backups. This is using RMAN to do a platform conversion from a big endian platform such as Solaris SPARC, IBM AIX and HP-UX to Linux using backups/restores where the incrementals can be applied to the target until ready to switch platforms at which point the actual switch over is considerably less time (and effort). Note 1389592.1 explains this with greater details.

The Jimmy Cliff Experience

Whomever thought up the Oracle Music Festival deserves a raise, and even better, the party responsible for bringing in Jimmy Cliff deserves a promotion! The man, Jimmy Cliff, must be in his 60's at least but has more energy than a 20 year old! He completely rocked the house with his energy, song arrangements and charisma. The crowd (including myself) was completely involved and so much so enjoyed his performance that an encore was demanded, and graciously accepted.

I had another big day ahead of me and so left during his encore. I can already feel the pain in various body parts following my "dancing" (I use the term loosely). I good end to another great day at OpenWorld...

Oracle OpenWorld 2012: Sunday (continued...)

2012-10-01T07:58:00.001+01:00

Sessions and Keynote

I attended a few sessions which for the most part were informative in some way. One take-away was that Oracle Enterprise Manager 12c Cloud Control is very popular as it is on a lot of minds (mine including since I'm trying to get my company to bypass our implementation of 11g and go straight to 12c but I digress). The few sessions I attended had quite a lot of questions. I was also able to meet a few of my Twitter contacts (@dbakevlar @aakela and @fuadar) which was awesome!

An interesting nugget in the session "Will it blend? Verifying Capacity in Server and Database Consolidations" was that of 'Consolidated DB Replay'. This is feature introduced in a patch (13947480) for 11.2.0.2+ which (as the name suggestions) allows for the concurrent running or replay of multiple captured workloads. The use case for this, of course, is capturing workloads from multiple source database systems and replaying on a single target system which is intended as a consolidation database. Ideally the capture time and periods across the multiple source databases should be the same to get the best picture of what the consolidated workload would look like on a single consolidate database. This feature would replace (or minimize) manual efforts involving visually analyzing workload graphs using OEM (as an example) for each database or looking at consolidated/merged AWR information for the multiple source systems.

Keynote

There have been many rumors surrounding a new Exadata 1/8 rack configuration, Exadata hardware upgrades (would there be an X3-2) and database 12c (if it would be announced or not). I did not attend Enkitec Extreme Exadata Expo (E4) 2012 but did read some of the Tweets and postings concerning the sessions which did point to such announcements. As it turned out, the Larry's Keynote did not disappoint and confirmed the rumors with the announcements of Exadata X3-2 (including an Exadata X3-2 1/8 rack), Oracle Database 12c release sometime in 2013 (my guess is some features will be cut for a January/February released in 2013), Oracle Private Cloud and IaaS.

Oracle Private Cloud is an offering for companies needing their own private infrastructure which can either run externally at Oracle facilities or inside the companies own data center but managed completely by Oracle. Having experienced various Oracle Support Services (OCS, OCMS or Oracle On-Demand) I can say the success for this offering will depend heavy on improvement for these support offerings and clear understanding between all involved parties as to what is meant by "managed". Oracle Cloud is Oracle's Infrastructure as a Service or IaaS offering composed of Exadata, Exalogic, Oracle Linux, Oracle VM, Oracle Storage, and InfiniBand (IB) components.

Oracle Database 12c which will be released sometime in 2013 (January/February is the whisper) will have some interesting features (not sure if I disclose at this time other than what has just been announced) such as Pluggable Databases. This is essentially multiple databases sharing the same server using containerization at the database level therefore being more efficient (so not complete separation in terms of processes and memory) and not requiring any application changes. For those familiar with SQL Server, PostgreSQL (including Netezza) and other such database platforms this is not anything knew. It is however, in the context of Oracle databases and has several benefits in the area of consolidation and hosting.

Exadata X3-2 was announced as the hardware refresh for the previous generation X2-2 along with a new 1/8 rack deployment option (starting price for negotiations is $200,000, nice!). Strangely enough, there is also an Exadata X3-8 as the refresh for Exadata X2-8, but this got no recognition (perhaps these do not sell as well and are a niche offering?). A few quick overview specifications are below:

Database Nodes

up to 8 x Oracle/Sun X3-2 servers
up to 2 TB RAM or 256 GB/node
up to 128 cores using 2x8-core Intel E5-2690 (2.9 GHz) per node

Storage Nodes

Up to 14 x Sun X3-2L
Up to 168 cores using 2x6-core Intel E5-2600 series per node
Up to 22 TB Flash memory
Up to 168 x 600 GB 15K rpm HP or 168 x 3 TB 7.2K rpm HC HDD

In terms of performance a full rack X3-2 should scream with:

~50K IOPS using 8K IO requests (most vendors use 2K so be careful doing comparisons)
100 GB/s bandwidth taking into consideration HP HDD and Flash
16 TB/hour data loads (from past experience this is within the same array so again, be careful and ask specific questions).

A software upgrade to the platform (this means available now w/o upgrading to X3-2) brings Cached Writes along with the previous Cached Reads. So maximum IOPS for 8K IO requests involving Flash is ~1,500,000 for read and ~1,000,000 for write. With compression (your mileage will vary according to your data) numbers should be better but again, test, test and test again. Your workload was not used when obtaining these benchmark figures. Usable disk capacity is ~45 TB for HP and ~224 TB for HC HDD. Also, Oracle Cloud and Private Cloud will start with Exadata X3-2 systems.

For me, here is what is missing from Exadata or what I'd like to see:

A more appliance-centric approach where even the ASM and DB configuration is standard and factory setup (sorry, but OCS involvement/engagement would be minimized)
More work being pushed down to the storage level (more analytics, more parallel processing, more "transparent" indexing so I don't have to create and maintain)
Automatic data compression (can still provide better/advanced compression levels at cost)
Built-in Hadoop integration (storage nodes as data nodes and a dynamic compute as the named node?)
Integrated monitoring via included OEM appliance as either an included 2x1U server in the rack or external servers. You can argue you can just use existing or build your own but wouldn't it be nice to have this option? Quite frankly I'm puzzled as to why Oracle has not come out with an OEM appliance yet and have suggested as such to some powers in the OEM team (also wondering about a MySQL engineered system, ExaSQL :-))

Fujitsu Keynote
Moving on (or back) to the Fujitsu portion of the keynote, I found it most interesting since they are doing very similar work with their "Fujitsu Agricultural Cloud Service" to that of my company (though not quite as full-featured a service if I do so say). The services gathers data collected by farmers via various devices and runs analysis which will aid in providing information to improve yields. This is being done very cost effectively and near real time. Then there is project "Athena" which is the merging of hardware and software (OS and database if I understood correctly) to bring forth a new processing model which will far surpass anything currently available. Leveraging knowledge and technology from the K supercomputer, Liquid Loop Cooling (LLC), 512 GB per socket (32 TB per system), 4 CPUs w/2TB each scaling/connecting in a building block fashion (up to 16 blocks?) and software on chip (database software also in silicon) the SPARC64 X was/will be born in 2013. Testing has shown a 2x increase in performance over IBM Power7 though no specifics were given (it was just a keynote). I do love how they showed real world type scenarios and business usage instead of just pure tech.

So far that has been my OpenWorld 2012 experience to date. Sunday down, next up Monday to Thursday.

Oracle OpenWorld 2012: Sunday

2012-10-01T05:41:00.000+01:00

MySQL Connect

First up today was a visit to the MySQL Keynote featuring some speakers from Twitter, Paypal and Verizon Wireless. I've been interested in MySQL for sometime but never really played with it for myself. My company is seeking to investigate it further since we've got over 50 instances of it running, but also to look into open source alternatives (cost reduction mainly).

Very interesting what these large companies are doing with MySQL and how they are using the technology. Twitter uses it extensively because:

It is fast, even compared with NoSQL alternatives (depending on what you are doing with it of course)
It has very low latency
It scales well
It has a large ecosystem
The safety of InnoDB (i.e. it does not lose data)

It is not all roses however, as there were some words of caution::

It is not yet optimized for SSD (which they are working on)
There is room for a configuration management tool
It is not a complete solution and should be used as a building block
It is not a purpose-built key-value (KV) store (so try other NoSQL if that is your true requirement)
It is not schema-less (both pro and con)
There is a need for better performance/response time metrics
There is a need for better monitoring

What they could share of there environment is also very interesting:

25 traditional master-slaves
3-100 machines with 6 DBAs and 1 developer
> 6 million queries per second (qps) w/400 million tweets per day (+ metadata)

The talk from Verizon Wireless was also interesting as they apparently use MySQL for their intranet with a customized landing page for each user (based on location, function, etc.) which the user can further customize. It is all highly performant (since that was a main criteria) and scalably. Their "Verizon Infrastructure as a Service (IaaS) Group" was inspirational! An internal group which provides IaaS to the rest of the company.

Oracle Internet Directory (OID) 11g: Part IV - OID Installation

2012-05-01T20:18:00.002+01:00

This is the final post in my series on OID11g. I'll try and follow-up with a few other posts but essentially from here on out you would be ready to go with OID11g. If you are interested in making your OID highly available using LDAP multi-master replication then stay tuned for that follow-up post.

So OID11g (11.1.1.5.0) installation actually consists of three phases, namely installation, patching and configuration. That is how I've broken up this post which as a side effect I think, makes it easier to follow. To provide some further clarity, some Fusion Middleware 11.1.1.5.0 components are offered as full installers, but not all. You can get the distribution details for the components on MOS, or via the documentation on on OTN. Unfortunately, OID falls into the case requiring a software installation of 11.1.1.2.0, followed by patching to 11.1.1.5.0 and subsequent configuration to complete the "installation". Hopefully Oracle will move towards full installers for all products much like they've done for the database (and other products such as GoldenGate and so on).

Installation of 11.1.1.2.0

1. Edit your response file for silent installation. The items of interest are highlighted as shown below:

[ENGINE]
#DO NOT CHANGE THIS.
Response File Version=1.0.0.0.0

[GENERIC]

#Set this to true if installation and configuration need to be done, all other required variables need to be provided. Variable "INSTALL AND CONFIGURE LATER TYPE" must be set to false if this is set to true as the variables are mutually exclusive
INSTALL AND CONFIGURE TYPE=false

#Set this to true if only Software only installation need to be done. If this is set to true then variable "INSTALL AND CONFIGURE TYPE" must be set to false, since the variables are mutually exclusive.
INSTALL AND CONFIGURE LATER TYPE=true

#Write the name of the Oracle Home directory. The Oracle Home directory name may only contain alphanumeric , hyphen (-) , dot (.) and underscore (_) characters, and it must begin with an alphanumeric character.
ORACLE_HOME=/oracle/app/fmw/Oracle_IDM1

#Write the complete path to a valid Middleware Home.
AS_HOME_LOCATION=/oracle/app/fmw

#Provide the My Oracle Support Username. If you wish to ignore Oracle Configuration Manager configuration provide empty string for user name.
MYORACLESUPPORT_USERNAME=

#Provide the My Oracle Support Password
MYORACLESUPPORT_PASSWORD=

#Set this to true if you wish to decline the security updates. Setting this to true and providing empty string for My Oracle Support username will ignore the Oracle Configuration Manager configuration
DECLINE_SECURITY_UPDATES=true

#Set this to true if My Oracle Support Password is specified
SECURITY_UPDATES_VIA_MYORACLESUPPORT=false

#Provide the Proxy Host
PROXY_HOST=

#Provide the Proxy Port
PROXY_PORT=

#Provide the Proxy Username
PROXY_USER=

#Provide the Proxy Password
PROXY_PWD=

[SYSTEM]

[APPLICATIONS]

[RELATIONSHIPS]

2. Run the installation using OUI for OID 11.1.1.2.0, as the oracle user:

./runInstaller -silent -response /oracle/stage/rsp/oid11g-inst.rsp

Below is a sample execution run:

Starting Oracle Universal Installer...

Checking Temp space: must be greater than 80 MB. Actual 18983 MB Passed
Checking swap space: must be greater than 500 MB. Actual 7724 MB Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2012-01-24_04-05-56PM. Please wait ...[oracle@orads02 Disk1]$ Log: /u01/app/oraInventory/logs/install2012-01-24_04-05-56PM.log
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
Reading response file..
Expected result: One of enterprise-5.4,enterprise-4,enterprise-5,redhat-5.4,redhat-4,redhat-5,SuSE-10
Actual Result: redhat-5
Check complete. The overall result of this check is: Passed
CertifiedVersions Check: Success.
Checking for binutils-2.17.50.0.6; found binutils-2.17.50.0.6-14.el5-x86_64. Passed
Checking for compat-libstdc++-33-3.2.3-x86_64; found compat-libstdc++-33-3.2.3-61-x86_64. Passed
Checking for compat-libstdc++-33-3.2.3-i386; found compat-libstdc++-33-3.2.3-61-i386. Passed
Checking for elfutils-libelf-0.125; found elfutils-libelf-0.137-3.el5-i386. Passed
Checking for elfutils-libelf-devel-0.125; found elfutils-libelf-devel-0.137-3.el5-x86_64. Passed
Checking for gcc-4.1.1; found gcc-4.1.2-50.el5-x86_64. Passed
Checking for gcc-c++-4.1.1; found gcc-c++-4.1.2-50.el5-x86_64. Passed
Checking for glibc-2.5-12-x86_64; found glibc-2.5-58.el5_6.3-x86_64. Passed
Checking for glibc-2.5-12-i686; found glibc-2.5-58.el5_6.3-i686. Passed
Checking for glibc-common-2.5; found glibc-common-2.5-58.el5_6.3-x86_64. Passed
Checking for glibc-devel-2.5-x86_64; found glibc-devel-2.5-58.el5_6.3-x86_64. Passed
Checking for glibc-devel-2.5-12-i386; found glibc-devel-2.5-58.el5_6.3-i386. Passed
Checking for libaio-0.3.106-x86_64; found libaio-0.3.106-5-x86_64. Passed
Checking for libaio-0.3.106-i386; found libaio-0.3.106-5-i386. Passed
Checking for libaio-devel-0.3.106; found libaio-devel-0.3.106-5-i386. Passed
Checking for libgcc-4.1.1-x86_64; found libgcc-4.1.2-50.el5-x86_64. Passed
Checking for libgcc-4.1.1-i386; found libgcc-4.1.2-50.el5-i386. Passed
Checking for libstdc++-4.1.1-x86_64; found libstdc++-4.1.2-50.el5-x86_64. Passed
Checking for libstdc++-4.1.1-i386; found libstdc++-4.1.2-50.el5-i386. Passed
Checking for libstdc++-devel-4.1.1; found libstdc++-devel-4.1.2-50.el5-x86_64. Passed
Checking for make-3.81; found make-1:3.81-3.el5-x86_64. Passed
Checking for sysstat-7.0.0; found sysstat-7.0.2-3.el5_5.1-x86_64. Passed
Check complete. The overall result of this check is: Passed
Packages Check: Success.
Checking for VERSION=2.6.18; found VERSION=2.6.18-238.12.1.el5. Passed
Checking for hardnofiles=4096; found hardnofiles=131072. Passed
Checking for softnofiles=4096; found softnofiles=131072. Passed
Check complete. The overall result of this check is: Passed
Kernel Check: Success.
Expected result: ATLEAST=2.5-12
Actual Result: 2.5-58.el5_6.3
Check complete. The overall result of this check is: Passed
GLIBC Check: Success.
Expected result: 1024MB
Actual Result: 3948MB
Check complete. The overall result of this check is: Passed
TotalMemory Check: Success.
Expected result: LD_ASSUME_KERNEL environment variable should not be set in the environment.
Actual Result: Variable Not set.
Check complete. The overall result of this check is: Passed
Check Env Variable Check: Success.
Verifying data......
Copying Files...
-----------20%----------40%----------60%----------80%--------100%

Applying Oneoff Patch...
The installation of Oracle AS Common Toplevel Component, Oracle Identity Management 11g completed successfully.

Patching 11.1.1.2.0 to 11.1.1.5.0

1. Edit your response file for silent patching. It's not much different from the installation, the items of interest are highlighted as shown below:

[ENGINE]

#DO NOT CHANGE THIS.
Response File Version=1.0.0.0.0

[GENERIC]

#Provide the Oracle Home location. The location has to be the immediate child under the specified Middleware Home location. The Oracle Home directory name may only contain alphanumeric , hyphen (-) , dot (.) and underscore (_) characters, and it must begin with an alphanumeric character. The total length has to be less than or equal to 128 characters. The location has to be an empty directory or a valid IDM Oracle Home.
ORACLE_HOME=/oracle/app/fmw/Oracle_IDM1

#Provide existing Middleware Home location.
MIDDLEWARE_HOME=/oracle/app/fmw

#Provide the My Oracle Support Username. If you wish to ignore Oracle Configuration Manager configuration provide empty string for user name.
MYORACLESUPPORT_USERNAME=

#Provide the My Oracle Support Password
MYORACLESUPPORT_PASSWORD=

#Set this to true if you wish to decline the security updates. Setting this to true and providing empty string for My Oracle Support username will ignore the Oracle Configuration Manager configuration
DECLINE_SECURITY_UPDATES=true

#Set this to true if My Oracle Support Password is specified
SECURITY_UPDATES_VIA_MYORACLESUPPORT=false

#Provide the Proxy Host
PROXY_HOST=

#Provide the Proxy Port
PROXY_PORT=

#Provide the Proxy Username
PROXY_USER=

#Provide the Proxy Password
PROXY_PWD=

#Type String (URL format) Indicates the OCM Repeater URL which should be of the format [scheme[Http/Https]]://[repeater host]:[repeater port]
COLLECTOR_SUPPORTHUB_URL=

#
CONFIG_WIZARD_RESPONSE_FILE_LOCATION=0

[SYSTEM]

[APPLICATIONS]

[RELATIONSHIPS]

2. Run the patch application using OUI for OID 11.1.1.5.0, as the oracle user:

./runInstaller -silent -response /oracle/stage/rsp/oid11g-patch.rsp

Below is a sample execution run:

Starting Oracle Universal Installer...

Checking Temp space: must be greater than 80 MB. Actual 18983 MB Passed
Checking swap space: must be greater than 512 MB. Actual 7406 MB Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2012-01-24_04-27-11PM. Please wait ...[oracle@orads02 Disk1]$ Log: /u01/app/oraInventory/logs/install2012-01-24_04-27-11PM.log
Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved.
Reading response file..
Verifying data......
Copying Files...
-----------20%----------40%----------60%----------80%--------100%

Applying Oneoff Patch...
The installation of Oracle AS Common Toplevel Component on Oracle AS Common Toplevel Component home ,Oracle Identity Management 11g Patchset on Oracle Identity Management 11g home completed successfully.

Configuring OID with ODIP, ODSM and Fusion Middleware Control in a new WebLogic Domain

At this point you now need to configure your installation of OID11g. I went with the option of configuring OID with ODIP, ODSM and Fusion Middleware Control in a new WebLogic Domain. I wanted ODIP as an option to connect and synchronize to AD, ODSM and Fusion Middleware Control (FMC) for the GUI management and monitoring, and a new WebLogic Domain (for ODSM and FMC) since I don't have one that I would like to use currently. Please check the documentation for configuration using other options.

The steps to conduct the configuration are below. Note that I've not had any success doing a silent command line installation and as such the GUI method is what is shown. I suspect this is the only option thus far unless I am missing something (not unlikely) though I have attempted many options.

1. Start the configuration as the oracle user by running '$ORACLE_HOME/bin/config.sh':

Click 'Next' to continue to the next screen...

2. Enter the credentials for the new domain's user, along with the domain name. Click on 'Next' to continue.

3. Confirm and/or correct the locations for the WebLogic Server and Oracle Instance directories as well as specify an Oracle Instance Name. When completed click 'Next' to continue.

4. The next screen concerns the usual security notifications. I do not care for security updates so I simply continued.

5. Select Oracle Internet Directory and Oracle Directory Integration Platform. The Oracle Directory Services Manager and Fusion Middleware Control management components are automatically selected for this installation. Ensure no other components are selected and click 'Next' when completed to continue.

6. Select Auto Port Configuration to allow the installer to configure ports from a predetermined range. Click 'Next' when completed to continue.

7. We already used RCU to create and configure the OID schema so here we just need to select 'Use Existing Schema', enter the connection details to the repository database in the form '::' and enter the ODS schema password. Click 'Next' when completed to continue.

8. Next up is the OID information, i.e. the realm and administrator ('orcladmin') credentials. Click 'Next' to continue to the installation summary when completed.

9. Following the installation summary you will see the configuration progress screen.

10. If all goes well you will see the Installation Completion screen

Installation Verification

To verify a successful installation you should run the following commands:

1. Execute '$ORACLE_INSTANCE/bin/opmnctl status -l'

Processes in Instance: asinst_1

---------------------------------+--------------------+---------+----------+------------+----------+-----------+------

---------------------------------+--------------------+---------+----------+------------+----------+-----------+------

oid1 | oidldapd | 8245 | Alive | 1068702846 | 375296 | 67:57:58 | N/A

oid1 | oidldapd | 8229 | Alive | 1068702845 | 95868 | 67:57:58 | N/A

oid1 | oidmon | 8214 | Alive | 1068702844 | 83744 | 67:57:58 | LDAPS:3131,LDAP:3060

EMAGENT | EMAGENT | 7402 | Alive | 1068702843 | 63908 | 68:01:31 | N/A

2. Execute the '$ORACLE_HOME/bin/ldapbind' command on the Oracle Internet Directory for non-SSL and SSL ports. Note that ORACLE_HOME must be set correctly (i.e. not the DB_HOME).

On Non-SSL ports:

$ORACLE_HOME/bin/ldapbind -h -p -D cn=orcladmin -w

On SSL ports:

$ORACLE_HOME/bin/ldapbind -h -p -D cn=orcladmin -w -U 1

Enabling WebLogic Startup

Every time an Administrator wants to run the WebLogic startup script, he/she is prompted with username and password. If the administrator wants to be configure weblogic to startup on bootup or reboot, then they will need the username and password to be automatically recognized. To enable WLS startup without password prompting create $DOMAIN_HOME/servers/AdminServer/security/boot.properties and $DOMAIN_HOME/servers/wls_ods1/security/boot.properties files with entries:

username=weblogic
password=wlsP#ssw0rd

After the initial startup, the password will be encrypted.

Summary

So now you have your first OID instance up and functional. All that is left is some configuration and tuning after some period of being operational. I will end the series on OID11g here but will try and follow-up with some further entries on setting up LDAP multi-master replication (MMR), backup/recovery and migration from 10g. I would like to point out that you should enable anonymous binds which are disabled by default. Otherwise, you will receive the error:

"Configuration exception: Could not check for the Oracle Schema:

oracle.net.config.ConfigException: TNS-04409: Directory Service Error"

When attempting to use DBCA to add your database to OID. This can be done in two ways:

Using OEM11g Fusion Middleware Control

a. Navigate to "Identity and Access' -> oid1

b. Click on 'Oracle Internet Directory' and select 'Administration' -> 'Server Properties'

c. Switch 'Anonymous Bind' from 'Disallow except for Read Access on the root DSE' to 'Allows'

d. Click 'Apply'

Using Command-line

ldapmodify -D cn=orcladmin -q -p 3060 -h orads01.na.ds.g240.lab -f [ldifFile]

LDIF File:

dn: cn=oid1,cn=osdldapd,cn=subconfigsubentry

changetype: modify

replace: orclAnonymousBindsFlag

orclAnonymousBindsFlag: 1

WebLogic Server (WLS) Installation

2012-04-13T19:32:00.001+01:00

I was going to this blog entry under my OID series but it occured to me that it made more sense to do a generic installation blog post instead given WLS is used by OID, OEM and many other components with which I've used and the only difference would be the WLS version used. The installation process is the same for 10.3.4 to 10.3.6. I can't vouch for other versions since I've not used prior versions and 12c is a bit different. So without further ado here is the install process, first the usual command line silent installation and then the GUI method.

Common Step - JDK Installation

1. Install a certified version of JDK to use. In the case of WLS 10.3.5 this would be JDK 1.6.0_24

For Oracle JRocket (required a GUI):

chmod +x jrockit-jdk1.6.0_24-R28.1.3-4.0.1-linux-x64.bin
./jrockit-jdk1.6.0_24-R28.1.3-4.0.1-linux-x64.bin

For Sun JDK:
chmod +x jdk-6u24-linux-x64.bin
./jdk-6u24-linux-x64.bin

2. I like to setup the environment for the installation process, specfiically calling out the location of the Java version to use. This would be required if you have multiple Java installations and you want to ensure the correct location/version is used.

export JAVA_HOME=/u01/app/oracle/jdk
export PATH=$JAVA_HOME/bin:$PATH

Method 1: Command Line Silent Installation

1. Build or edit the "silent.xml" file as below:

<?xml version="1.0" encoding="UTF-8"?>


<bea-installer>
<input-fields>
<data-value name="BEAHOME" value="/u01/app/oracle/fmw" />
<data-value name="WLS_INSTALL_DIR" value="/u01/app/oracle/fmw/wlserver_10.3" />
<data-value name="COMPONENT_PATHS" value="WebLogic Server/Core Application Server|WebLogic Server/Administration Console|WebLogic Server/Configuration Wizard and Upgrade Framework|WebLogic Server/Web 2.0 HTTP Pub-Sub Server|WebLogic Server/WebLogic SCA|WebLogic Server/WebLogic JDBC Drivers|WebLogic Server/Third Party JDBC Drivers|WebLogic Server/WebLogic Server Clients|WebLogic Server/WebLogic Web Server Plugins|WebLogic Server/UDDI and Xquery Support|Oracle Coherence/Coherence Product Files" />
<data-value name="LOCAL_JVMS" value="/u01/app/oracle/jdk"/>
</input-fields>
</bea-installer>

Save the file in the same location as the installation program ("/u01/app/stage" in my case).

2. Run the installation as the oracle user:

cd /u01/app/stage/
java -d64 -Xms512m -Xmx1024m -jar wls103x_generic.jar -mode=silent -silent_xml=/u01/app/stage/silent.xml

Extracting 0%....................................................................................................100%
Jan 24, 2012 3:10:29 PM java.util.prefs.FileSystemPreferences$2 run
INFO: Created user preferences directory.

Method 2: GUI Installation

1. Run the installation as the oracle user:

For Oracle JRocket:
java -Xms512m -Xmx1024m -jar wls103x_generic.jar

For Hybrid Oracle/Sun JDK:
java -d64 -Xms512m -Xmx1024m -jar wls103x_generic.jar

2. Choose the location of middleware home:

3. Optionally register for security updates. I tend to decline since this is not production, but feel free.

It requires internet access so if you don't have, you should decline otherwise you will be greeted with the message below (you can also setup your proxy access here as well):

4. Select "Typical" installation. If you want to be a bit more specific in the components select "Custom" but ensure all the expected components are installed as expected by the application.

5. Next up is the JDK select screen. The installation will detect the available JDK versions on your server if you have multiple versions and allow you to select the appropriate version. Select the correct version.

6. Specify the product installation directories:

7. You are presented with an installation summary where you can review what products will be installed:

8. Following is the actual installation process where you can view the progress:

9. Following a successful installation the completion screen is shown. Deselect the option to run the Quickstart process and you have completed your WLS installation:

Oracle Internet Directory (OID) 11g: Part III - Schema Installation

2012-04-11T18:17:00.001+01:00

This is part II in the series I started on installating and setting up OID11g (11.1.1.5.0) which is part of Oracle Fusion Middleware 11g and in this case licensed under Oracle Directory Services Plus. Please check my previous posts on licensing and requirements for those details. For this posting I'm assuming you've already created a certified repository database and are now ready to install and setup the OID schemas. In my case I created an 11.2.0.3 EE single instance database repository using ASM for storage management. Check my blog entries for details on installing and setting up Grid Infrastructure, and installing and creating a database. Yes, I know these are shameless plugs for additional page views and I would also appreciate you clicking on a few advertiser links while there as well, thanks!

The OID 11g installation will no longer create a repository so this must be done upfront. You do have the option of setting up the schemas but using the Repository Creation Utility or RCU upfront is recommended. RCU is a tool to create the repositories scheams for various Fusion Middleware tools, which in this case we will be using it to create schemas required for OID and DIP (Directory Integration Platform). What is RCU exactly? It is effectively the replacement in FMW 11g for the Oracle Application Server 10g Metadata Repository Creation Assistant (also known as MRCA or RepCA). The documentation will tell you that RCU can be downloaded from MOS, OTN and Oracle e-Delivery Cloud but for me it was a bit difficult to locate the actual download except for the latter, i.e. Oracle e-Delivery Cloud, so this is where I'd recommend searching to save time and effort. The RCU version must match the version of FMW or rather OID that you intend to use, which in my case is 11.1.1.5.0. I prefer command line installations since they can be scripted and run without user interaction so I'll start out with that method and show the GUI interactive method following.

Setup
1. Extract the download to a staging directory and export the location to your environment as RCU_HOME:

$> export RCU_HOME=/tmp/stage/rcu

2. RCU is apparently a 32-bit installation and requires a 32-bit OS to install but it is possible since 11.1.1.4.0 to install from a 64-bit OS:

$> set linux32 bash

Method 1 - Using Command Line in Silent Mode

3. Create a text file which contains the various passwords in sequence of being prompted:

$> vim rcupwd.txt
sysP#ssw0rd
odsP#ssw0rd
odsP#ssw0rd

The first password is for SYS, the second and third are for the Oracle Directory Services or ODS schema. It is repeated since there are two prompts for this user's password.

4. I prefer to use ASSM for my tablespaces but by default RCU does not create all the required tablespaces for the schemas as such. A bit curious given Oracle's constant recommendations on this front but not completely unexpected if you've done repository installations before. Modify file $RCU_HOME/integration/oid/oid_Storage.xml replacing as shown below:

<TablespaceAttributes NAME="OLTS_BATTRSTORE" >
<AutoSegmentSpaceManagement>false</AutoSegmentSpaceManagement>
...
<TablespaceAttributes NAME="OLTS_SVRMGSTORE" >
<AutoSegmentSpaceManagement>false</AutoSegmentSpaceManagement>
TO
<TablespaceAttributes NAME="OLTS_BATTRSTORE" >
<AutoSegmentSpaceManagement>true</AutoSegmentSpaceManagement>
...
<TablespaceAttributes NAME="OLTS_SVRMGSTORE" >
<AutoSegmentSpaceManagement>true</AutoSegmentSpaceManagement>

This ensures all TBS are created with ASSM as the database default of local extent management will be picked up. You may need to modify the extend sizes for the DBF manually after installation to ensure all extents are spread across all ASM disks for your AU size.

Installation

5. Run the RCU installation:

$> cd $RCU_HOME/bin

$> ./rcu -silent -createRepository -databaseType ORACLE -connectString [orads01.mydomain.lab:1521:dsrep.mydomain.lab] -dbUser sys -dbRole SYSDBA -component OID -variables RCU_LOG_LOCATION=/tmp -schemaPrefix LAB -f < rcupwd.txt

Processing command line ....

Repository Creation Utility - Checking Prerequisites

Checking Global Prerequisites

Repository Creation Utility - Checking Prerequisites

Checking Component Prerequisites

Repository Creation Utility - Creating Tablespaces

Validating and Creating Tablespaces

Repository Creation Utility - Create

Repository Create in progress.

Percent Complete: 0

Percent Complete: 35

Percent Complete: 29

Percent Complete: 91

Repository Creation Utility: Create - Completion Summary

Database details:

Host Name : orads02.mydomain.lab

Port : 1521

Service Name : DSREP.MYDOMAIN.LAB

Connected As : sys

Prefix for (non-prefixable) Schema Owners : DEFAULT_PREFIX

RCU Logfile : /u01/app/rcuHome/rcu/log/logdir.2012-01-24_13-37/rcu.log

Component schemas created:

Component Status Logfile

Oracle Internet Directory Success /u01/app/rcuHome/rcu/log/logdir.2012-01-24_13-37/oid.log

Repository Creation Utility - Create : Operation Completed

You can get the meaning of the various arguments I've used from the RCU documentation but they are pretty self explanatory.

Method 2 - Using the GUI

1. Issue the command:

$> $RCU_HOME/bin/rcu

On the welcome screen, click "Next".

2. Select "Create" for creation of the schemas:

3. Input the connection details into the appropriate fields, as shown in the screen shot:

RCU will verify the information by making a test connection while also verifying that the database requirements are met to act as a repository:

4. Select the components to install, which is "Oracle Internet Directory". I also chose the "Directory Integration Platform" in my case though this is not shown in the screen shots.

RCU checks that the component prequisites are okay.

5. The tablespace mapping is shown. I suggest leaving the defaults with the possible exception of the temp changed if needed (I changed mine to TEMP). You can manage the settings (such as ASSM and so on) via clicking on the "Manage Tablespaces" button.

A confirmation box is displayed informing any non-existent tablespaces are about to be created, which is followed by another completion confirmation box:

6. The summary screen is displayed which indicates you are about to begin creation of the schemas:

7. Once completed summary completion screen is shown:

At this point setup of the OID repository is complete and you can move on to the installation of WebLogic or OID software which I will demonstrate in another blog entry. The sequence does not matter since only the software for OID will be installed (and patched), however, the WLS piece and all other pre-requisites (DB repository) should be in place prior to starting OID configuration.

Using udev to configure disks for ASM in Linux

2012-02-16T21:14:00.000+00:00

I've never been the biggest fan of Oracle ASMLib. While it is far easier to configure disks for ASM using ASMLib than udev or multipath, it also added an additional requirement for rpms, an additional layer in the stack, and dependency on the kernel version. An internet search will reveal numerious discussions on this very topic, with many having a preference to not use ASMLib. However, due to the increased ease of configuration, strong recommendations from Oracle and believed performance improvements I had always used ASMLib. For what it's worth, there does seem to be some truth behind ASMLib having a performance benefit as though I've not personally tested it, I have read blog postings where its removal resulted in greater CPU usage. If you have enough resources this is not a major concern though over time it may end of being the case.

My stance, and I would think a lot of others, have since changed with the recent developments in Oracle's competition with Red Hat. To be more specific, MOS note "Oracle ASMLib Software Update Policy for Red Hat Enterprise Linux Supportd by Red Hat [ID 1089399.1]", the quote of interest being:

"For RHEL6, Oracle will only provide ASMLib software and updates when configured with a kernel distributed by Oracle. Oracle will not provide ASMLib packages for kernels distributed by Red Hat as part of RHEL6. ASMLib updates will be delivered via Unbreakable Linux Network(ULN) which is available to customers with Oracle Linux support. ULN works with both Oracle Linux or Red Hat Linux installations, but ASMlib usage will require replacing any Red Hat kernel with a kernel provided by Oracle."

With this note, I've since been devoting time to updating my Oracle database standards for Linux, i.e. SLES and RHEL since we are not an Oracle Linux shop and have no plans to pursue such a course (nothing personal Oracle, it's just business). So I began work on what I though was simply making a few modifications to use udev instead of ASMLib following which I'd do due diligence to test it out on our standard versions of SLES and RHEL. I've used it before on RHEL and it should prove simple enough, right?

The Case of the Missing SCSI ID

I was so wrong. When I tried to obtain the SCSI ID for the disks during my testing nothing was returned. I spent some time researching, but came up empty until I realized I was not searching correctly. You see we now run most of our Linux environments on VMware vSphere and I was not using this in my search terms. Correct input and bingo! Seems there is a known issue where by default VMware does not expose SCSI IDs for the disks. To correct this you will need to:

Shutdown your VM guest
In vCenter, right click your VM guest in the LHS pane and select 'Edit Settings...' (can also be obtained directly via 'Summary' tab)
Click the 'Options' tab
Select the 'Advanced -> General' navigation item on the left side and click the 'Configuration Parameters...' button displayed on the right
Click the 'Add Row' button
Add the name 'disk.EnabledUUID' with a value of 'TRUE' (no quotes of any kind) and click 'OK' all the way through the screens to save
Restart your VM guest and enjoy receiving SCSI IDs

An alternate method is to use a text editor and add the entry disk.EnabledUUID="TRUE" to your VM guest's VMX file and restart your VM guest.

UDEV Steps
Now onwards with the steps for udev configuration!

1. Add the "options=-g" line to the /etc/scsi_id.config file as the root user.

2. Obtain the SCSI ID for your disks as the root user:

/sbin/scsi_id -g -s /block/{sd_device}

3. Create a udev rules file in /etc/udev/rules.d directory as the root user with entries similar to the below:

vi /etc/udev/rules.d/99-udev-oracle.rules
# ####################################################
# FILE: 99-udev-oracle.rules
# DESC: UDEV rules file for Oracle ASM functionality.
# Should be placed under /etc/udev/rules.d
# ####################################################
# DATA disks
KERNEL=="sd*",BUS=="scsi",ENV{ID_SERIAL}=="{scsi_id}", NAME="asmdisk1", OWNER="oracle", GROUP="oinstall", MODE="660"
# FRA disks
KERNEL=="sd*",BUS=="scsi",ENV{ID_SERIAL}=="{scsi_id}", NAME="asmdisk2", OWNER="oracle", GROUP="oinstall", MODE="660"

4. Stop and start udev as the root user:

/etc/init.d/boot.udev stop
/etc/init.d/boot.udev start

The SCSI devices can now be accessed by ASM, and you can set your ASM_DISKSTRING parameter to be "/dev/asmdisk*".

Oracle Internet Directory (OID) 11g: Part II - Requirements and Components

2012-02-13T19:54:00.000+00:00

The first part of this series on OID 11g focused on purchasing and licensing OID. Now we are ready to focus on what you is needed for a functional OID implementation. Before we get into the details, it is useful to explain what is OID and some of its components.

What is OID?
OID, a component of Fusion Middleware, is a general purpose LDAPv3 compliant directory store. Based on configuration it can scale, be highly available and secure. Most would initially use it for Directory Naming, as a central store for database connectivity details. This would alleviate clients from requiring a local TNSNAMES.ORA file (reduces management and improves efficiency), though instead they would typically require an LDAP.ORA file. This file stores the LDAP connection details, i.e. the OID hostname/IP, connection ports, and realm.

OID can also serve as the central repository for user identification and role information which again reduces management and improves efficiencies. You may think this sounds like Microsoft Active Directory (MSAD) and you would be right! They are both LDAP directories, however, Oracle does not speak with MSAD (or any other LDAP directory) out-of-box directly and this is where OID comes into play. There is also Oracle Virtual Directory (OVD) which acts as a virtual directory service, being a front-end to any LDAP directory and is a valid alternative if you don't have OID or you have multiple directory servers and are okay with extending the directories schema to support the Oracle meta-data. However, you may run into organization constraints extending your MSAD schema which in my case is what occured.

How does OID integrate with MSAD?
There are a few options when integrating OID to your existing LDAP directory store and in this post I'll focus on MSAD though the concepts are similar to other LDAP directory stores with only the details being different. Reviewing the options in the arcticle here (this is rather dated but seems still relevant and its not like Oracle has anything more recent) lead to me to several thoughts. Originally I thought to use the Directory Integration Platform (DIP) tool to synchronize passwords between MSAD and OID but that required installing the Active Directory Password Filter on each domain controller. This of course is simply not practical given the number of AD controllers and corporate policies and was immediately rejected by the Windows Admins. Another option I explored was Server Chaining but that would only work with one AD controller and required a plug-in for password change notification. Even with some creative workarounds such as using a round-robin or LB front end for some of the MSAD controllers, the plug-in had to exist on each domain controller. Again, not practical and rejected.

To take a step back, MSAD and Oracle do not use compatible hashing algorithyms for their passwords so when a user changes their AD password Oracle needs a way to capture that password, and hash it in its own format for transmission to OID. This is why the Password Filter and even the plug-in are required, and on each domain controller since the change may occur on any controller. To be fair we are currently trying to implement an IDM product from another vendor as well and it has run into the same problems so this is not anything specific to OID and MSAD.

At this point it seems my only option is using a combination of Kerberos and Server Chaining. We already use Kerberos via QAS (Quest Authentication Services) for our UNIX/Linux integration to MSAD so their is some experience. Kerberos is rather difficult I've been told and I'll just have to see how things go when I start the detailed implementation in the lab. If anyone has thoughts, experiences or details around how they successfully integrated OID and MSAD please do contact me, I would love to discuss. Such details seem to be not exact in the blogosphere, forums and definitely documentation. My vision is below:

OID Components and requirements
The basic components are:

Database Repository - This database is where OID stores all its information and can be Standard Edition (SE) or Enterprise Edition (EE). Directory Services Plus comes with a restricted use license for SE (as seen here), iAS comes with restricted use of SE and EE (or so it would seem here), though complicating all this is the statement mentiond in my Part I of this series (so I've no idea but am going with what Oracle has stated). You can visit MOS for the current listing of certified versions, I chose 11.2.0.3 EE (single instance). The basic requirements are listed in MOS note 872885.1 and documentation here:

Character Set = AL32UTF8
aq_tm_processes=1
db_block_size=8192
db_cache_size>=144M
dml_locks>=200
java_pool_size>=120M
job_queue_processes>=10
open_cursors>=500 (800 for RAC)
shared_pool_size>=175M
session_max_open_files>=50
sessions>=500
processes>=500 (2500 for RAC)
sga_target>=4G
pga_aggregate_target>=2G
session_cached_cursors>=500
_b_tree_bitmap_plans=FALSE

WebLogic Server (WLS) - I am using OID 11.1.1.5.0 so the certified WLS version is 10.3.5. Note that this is not required if you don't need a nice interface such as ODSM and Fusion Middleware Control to OID. I would recommend though that you install WLS as you lose nothing since you still have command-line and gain a nice interface for those times when you need a pretty face.

Following the installaton of the above required components, you can proceed with the installation of OID itself which, for my own installation consisted of:

Oracle Directory Services Manager (ODSM) - This is a web-based interface for managing instances of OID and OVD and replaces Oracle Directory Manager (ODM) which is now deprecated.

Oracle Enterprise Manager Fusion Middleware Control - This is a graphical interface that provides comprehensive systems management for Oracle Fusion Middleware including OID (target ype). Based on the name you would think you could just use regular OEM but thus far I've not had time to investigate this thoroughly.

Oracle Directory Integration Platform (DIP) - Provides directory synchronization capabilities which includes connectors for out-of-the-box synchronization with Novel eDirectory, and MSAD. This of course is only required if you are seeking such functionality.

Oracle Internet Directory (OID) - The internet directory component itself. Note that I am using version 11.1.1.5.0 which is actually a patchset and requires installation of 11.1.1.2.0 and then the application of the patch ontop. This is a bit strange since there are components within Fusion Middleware 11.1.1.5.0 which are self-contained installations and not patches, much like the Oracle Database 11.2.0.x software. I'm hoping Oracle will give this, along with every Oracle product, similar treatment to be installed as self-contained units instead of patches. I find this leads to clearner, more streamlined and less error-prone installations along with better out-of-place patching.

Note about High Availability
There are many options for HA with OID. The one I've chosen, as shown in my diagram, uses two OID servers each running separate OID and database instances; synchronized via LDAP multi-master replication (MMR), and optional fan-out replicas. Unless you are planning a single OID instance, I do not believe RAC makes a lot of business sense given you get the same features (availability, scalability and performance) along with others (rolling upgrades) from MMR. Using LDAP instead of database MMR provides better granular control over the replication process as an option (security, and performance) though if using SSO you will need to use database MMR. The fan-out replication is essentially just LDAP MMR with the synch process being down stream read only (and possibly filtered).

Oracle Internet Directory (OID) 11g: Part I - Which License?

2012-02-13T16:22:00.000+00:00

Well, it has been quite some time since I last posted but I've been kept busy doing a lot of different things at work and of course family life. One such thing that has kept me occupied at work is getting up to speed on Oracle Internet Directory (OID) 11g. My previous experiences with OID were to do with merely using it much like most other people. Yes, I knew how it worked at the high level and of course what it was for, but not the internals such as how to do an installation, configuration, migrations, upgrades, patching, maintenance, backup/recovery, and other fine grained details. To be honest I still don't know a LOT of this stuff as the more I get to know OID the more I realize how much I did not know about directories and their internals and how much I appreciate the need for a separate Identity Management (IDM) Administrator. There is simple a lot to know and do, much like a normal DBA.

Anyways, the point of this series of blogs is to try and help others by exposing a lot of the simple things which I now know. Things such as installation and requirements, backup/recovery, configuration, a few notes on designs and usage, and some license assistance. By far the trickiest thing for me was the licensing which is the focus of this first blog.

How do I get OID?
At first glance this might seem like a simple topic and is exactly what I though. However, OID is bundled, and can be purchased as well, with a variety of software suites such Internet Application Server or iAS, Identity Manager, and Directory Services Plus as examples. You will need to purchase the correct suite, which depends on your actual need or usage, otherwise you may end up buying software and licenses you don't need and spend excessive capital funds as the cost depends on the suite and can be quite a large difference. Case in point:

Directory Services Plus: $50,000 per processor (+$11,000 for maintenance)
Internet Application Server (EE): $35,000 per processor (+$7,700 for maintenance)
Internet Application Server (SEO): $11,500 per processor (+$2,530 for maintenance)
Internet Application Server (SE): $5,800 per processor (+$1,276 for maintenance)

This listing does not include the other suites in which you can obtain OID, they are just examples as to the varying prices (and options). Of course, each suite also has different pieces, and restricted use licenses for varying included components. The latest pricing information can be obtained here, with FMW11g license information here. I know what some may be thinking that this is just Fusion Middleware. Yes, but Fusion Middleware is the broad software suite name, out of which you purchase individual application suites (such as those I've mentioned). Going through the various options and such is too much for a simple blog posting so I'll just say my current employ licensed OID via iAS as at the time that was the best option. Now it seems the best option, based on our current and future usage, is Directory Services Plus. This means new licenses, or does it?

OID for free!
Apparently, and I say this because regardless of going through 2 months of discussions via email and phone with Oracle sales representatives and product specialists I'm still a little confused, if you are only using OID for Directory Naming, then there is no license to be purchased. Below is the key statements taken from the "Oracle Database License Information 11g Release 2 (11.2)" manual which brought about this conclusion:

"The following restricted-use licenses are included with Oracle Database 11g in the editions indicated:

• A restricted-use license for Oracle Internet Directory (OID) is included with all editions (except for Oracle Database Express Edition) if users use the Directory Naming feature to configure Oracle Net Services. OID may not be used or deployed for other uses. Please contact your Oracle sales representative for additional information on Oracle Internet Directory (OID)."

A colleague of mine was of the opinion this meant either you can license the Oracle DB EE as the repository and use the OID mid-tier piece without cost based on this statement, or license the OID mid-tier and not the repository database (since OID also comes with a restricted use license for using the Oracle database). I though (or more hoped) this meant a license similar to OEM where the OID mid-tier does not need a license (and neither does its restricted use database repository) so long as each remote client database is only using it for Directory Naming. Turns out Oracle was of the same thinking as myself. But the problem is the license seems like such an interpretation. In any case, until we expand usage from Directory Naming into identity management we can save on some capital expenditure (CAPEX) for this financial year.

Conclusion
OID is bundled with many different software suites which all fall under the Fusion Middleware umbrella. Choosing the correct suite is important in getting the best deal, but if you only need the Directory Naming functionality it is already included in your database license purchase (so there is no cost). I would strongly recommend you check this out with your own Oracle sales representatives and if you are given a different response please do let me know! It has been my experience that licenses are different based on who you ask within Oracle and what type of relationship you have with Oracle (i.e. how big or important of a customer you are). You may disagree with this statement but that has been my experience.

In my next piece I will explore the requirements and installation of some required components for OID11g.

Enterprise Manager 12c Cloud Control Installation

2011-10-21T21:32:00.002+01:00

This is a long overdue posting on my experience with installing the recently released Oracle Enterprise Manager 12c (12.1.0.1). But before a few items of interest that I think should be pointed out:

Product Rename - Oracle has renamed the product from "Enterprise Manager Grid Control" to "Enterprise Manager Base Platform". A side effect of this is easier knowledge searching and support for this specific release/product.

Architecture redesign - The architecture has been redesigned (a good thing) to utilize plug-ins for everything. So now the OMS can be patched and/or upgraded independent of any capabilities in monitored targets, and even more importantly, so plug-ins can now be patched and/or upgraded to take advantage of new features and/or functionalities of their respective targets. For example, when Database 12c is released a simple upgrade to the "Enterprise Manager for Oracle Database" plugin should be all that is required to allow EM12c to monitor and leverage all the new features DB12c would offer. Previously, the Agent and the OMS would require patches and/or upgrades to fully support such a new database release. At least this is the plan, we will see when DB12c is released if this is indeed the case.

Repository is master - The repository is now the master for data with the Agents being even more stateless. This is an improvement which allows agents to simple pass monitoring data to the OMS, and hence can be upgraded or re-installed without any loss of target data since they can be fully re-synchronized from the repository.

On now with my installation...

Installation Setup

I wont go into the full detailed list of prerequisites since you can get that from the documentation here. It is pretty much the usual, you can simple reuse your settings for database (11.2 since that is what I use) installations. Lets just say it was all accurate as at this writing. So I created a VM guest environment with the following specifications:

OS: RHEL 5.6 x86_64
RAM: 8 GB
CPU: 2 x 2.67 GHz Intel Xeon
VM: Yes (VMware 4.1u1)

While many may shy aware from using VMs, I think EM is one area where this is ideail since there are no license implications (except for the OMR, more on that later) and the requirements are optimal for a VM. For production deployment I would have at least two OMS, spread across separate cluster hosts with node affinity setup so both OMS never share the same host (unless temporarily necessary when doing some form of VM host maintenance). There are multiple ways to do the configuration, if you need ideas based on your unique situation please feel free to contact me.

On the issue if license, Enterprise Manager does come with a restricted use license for all its components (the basic tool is free, you pay for the plugins, connectors, and packs). This means however, that if you chose to go outside those boundaries, you will need to license the component as appropriate. For example, the repository database is licensed as an Enterprise Edition Single Instance, choosing to use a RAC setup would mean you need to pay for those RAC licenses, as well as the second instance (or each additional RAC instance as the case may be). This is all documented in the license information here.

As most should be aware since EM 11.1, the OMR is no longer installed as part of the installation. So you will need to install and setup an existing database to use as your OMR prior to installation of EM. For my installation I chose 11.2.0.3. Yes, as of this writing it is not yet on the certification listing, but this minor and I fully expect this version to be included shortly though for production I will be adhering to the certification matrix. I chose to let the installation install WebLogic (10.3.5). I prefer this method since it is simply cleaner and avoids any unnecessary patch, or upgrade issues which would result from using a shared WebLogic farm.

Step 1: Meet pre-requisite requirements
Check the documentation here for the requirements.

Step 2: Setup the repository database

This is included in the documentation here but I chose to include it since a few things have changed, or perhaps have not yet been documented, and otherwise stand out. In prior versions EM required setting '_b_tree_bitmap_plans=false', and 'aq_tm_processes=1' but there is no mention of these settings being required for EM12c (so I did not include them). Also, the documentation states "The instance parameters db_cache_size, large_pool_size, java_pool_size, and streams_pool_size must not be set for Enterprise Manager 12c installations.". I definitily do not agree with this note as my experience has been not setting minimum values for these parameters in the real world causes headaches with pool starvation and the reallocation not keeping up with demand (i.e. poor performance). AMM is recommended but since I'm installing on Linux and prefer to enable huge pages (in my humble experience this provides better performance), I instead use ASMM with huge pages (huge pages and AMM are mutually exclusive). I will leave my thoughts of ASMM and AMM for another blog. So in any case, the installation will complain (give a warning) but will go through otherwise with no issues. My usage thus far has also not encountered any issues. A final thing to note, that is not mentioned, is that the automatic stats gathering job should be disabled in the repository database. This will be detected during installation and can be disabled at that point so it is not critical.

So after you download, met all the pre-requisits and built your repository you are ready for the actual installation.

Step 3: Start installation
./runInstaller

This is my lab environment which has no internet access so I disable the email option and clicked 'Next'. This leads to a follow-up screen confirming your intent to remain uninformed:

Step 4: Software updates

I selected to skip the updates (not that any are available just yet) since I have no internet access.

Step 5: Prerequisite Checks

Ensure that all the checks are successful here otherwise you may run into problems later on. I've been a good boy and everything is passed successfully.

Step 6: Select your installation type

Even though this a lab environment I prefer to choose the 'Advanced' option to 'Create a new Enterprise Manager System' since this is what I will be doing when testing for my company. So I then will need to specify a location for the middleware software location.

You can also choose to upgrade, or install just the software. I'll let you know how things work out with the other options once I've had a chance to test.

Step 7: Select Plug-Ins

Here is where things get interesting. This is the new architecture where everything is a plug-in. Those you see in the screen shot are selected by default and mandatory (i.e. you can't unselect). What is not so obvious is that the 'Oracle Fusion Middleware' plug-in is dependent on 'Exalogic Elastic Cloud Infrastructure', which is of course not selected. Seems silly to not just select and install the two together since there is a dependency, even if it is a soft one since you can move on by selecting to install it later. I chose to install it now and avoid any potential issues (admittedly I doubt there would be since I would not be using the feature).

Step 8: Weblogic Server Configuration

Default domain is GCDomain and default node manager is 'nodemanager'. This could be changed when invoking the runInstaller but I don't see any reason. Enter the passwords, and the OMS instance base location.

Step 9: Repository Database Configuration Details

Clicking 'Next' makes a test connection, and if successful verification of the repository commences. I did not know of the requirement to disable the automatic stats job in the repository database, but the installion handles disabling it (if I so choose which I did):

After fixing the stats collection job, a further warning is given for failing to exactly meet requirements:

My settings are higher than the requirement so most of these are silly and should not be displayed as they could cause unnecessary changes. For example the minimum for 'processes' is 300, and mine is set to 1024. Annoying me about the minimum which I've exceeded is a bug IMHO, after all it is not a maximum recommendation.

Next up is the SYSMAN credentials and tablespace setup. Enter the appropriate file system location for the database files. Here the system has detected (partially) that I'm using ASM, so I oblige by entering my disk group information to store the tablespace data files.

Step 9: Port Configuration

I don't have any need to modify anything here and leave the defaults. Your security requirements may be different and require the usage of non-standard ports, which is fine.

Step 10: Review

Almost done!

Step 11: Install

Watch (or get some refreshments) as the installation should breeze through! Note that EM12c uses the same database 11g process of packing already installed software, so the installation is essentially compressed into copying over those files, relinking for your system, and making some configuration changes.

Run the 'allroot.sh' script when prompted:

Step 12: Installation Complete!

All should have gone well so you need only review the results of your successfully installation, and make some notes, if you don't want to check files later, as to URLs and ports. I highly recommend following the note about exporting the configuration which includes the encryption key. This should actually be a scheduled job and become part of your backup/recovery process for EM.

Step 13: Access the New UI

Redesigned (dear I say gorgeous?) login prompt.

Your first access greets you with an updated license acceptance screen. Following which you will need to select your future welcome screen:

This can be changed later, but the point is EM12c allows your to easily customize your welcome screen. Very cool!

So that is it! It is a very clean and easy installation process, compared with previous versions, and I am left impressed. Having used it for a little while things do take a bit of getting used to and finding your way around (as with any redesign), but it is a definte improvement. I'll be following up with usage posts in future and letting you know what I find.

Oracle Enterprise Manager 12c Grid Control

2011-10-03T21:45:00.003+01:00

I've always used Enterprise Manager to some extent. I can remember when it just started out as a Java console program. Clunky, unrefined and not very friendly. Yet, I could see the potential and where Oracle likely wanted to take the product, but wondered as to if they were up to the task. I often thought they would just buy Quest who had an impressive portfolio and was probably (I thought) used most by DBAs, developers, and architects alike. Maybe even Embarcadero, whose DBArtisan tool I actually preferred and found to be a better tool than Quest Toad back then. It is now several years later and how Enterprise Manager has grown up. It is now offered as "DB Console" for single database administration, and as "Grid Control" for multiple database administration and your enterprise-wide monitoring and administration tool.

Grid Control 11g (11.1.0.1) is a web-based tool leveraging WebLogic Server (WLS) 10.3.2 for UI presentation, a database back-end for its storage repository, and agents for data gathering and carrying out monitoring and administration. I'm sure I've been told on many an occasion why each Grid Control will require a specific WLS version but the answer escapes me for now. This however should not be an issue since the WLS should be self-contained (i.e. only for Grid Control) and IMHO installed on the same server as the OMS. If you have multiple OMS for HA, then install a WLS instance on each OMS as well for HA.

Now released is Enterprise Manager 12c (12.1.0.1) Grid Control which will require WLS 10.3.5. This brings a new and improved interface, which also includes cloud computing management features such as charge back and metering.

My interest in this case are the available upgrade options as shown below:

Two-System Upgrade - Creating a new EM 12.1 infrastructure and switching the new 12.1 agents to point to the new EM 12.1 infrastructure. Requires minimal downtime but new EM infrastructure and you will also have to merge data that was accruing while the upgrade was taking place.

One-System Upgrade - This option requires more downtime but no additional infrastructure. Here you are installing the new 12.1 OMS (and 10.3.5 WLS) on the same servers, and then upgrading your OMS to 12.1 (which includes upgrading the repository). Prior to this you would have shutdown your pre-12.1 agents and started up your new 12.1 agents (which were already installed).

I prefer the Two-System approach since it has the advantage of minimizing downtime, while providing a fallback option. It also seems a bit less complicated. The previous hardware can be recycled so I don't find that to be an issue (also serves to provide a hardware refresh opportunity). I think most would agree with this being the better of the two options given. To provide a bit more information see the comparison below:

Exalytics Business Intelligence Machine Released at Oracle OpenWorld 2011

2011-10-03T21:10:00.002+01:00

Yes, this is yet another blog entry about Oracle OpenWorld 2011, and the just announced Oracle Exalytics Business Intelligence Machine. What makes this blog different is that I'm not just re-stating everything that has already been said and announced (at least not entirely), but will provide a bit of compare and contrast.

I've been speculating, along with others, that Oracle was working on some appliance (and of course had to call it 'Exa-something', sigh) that did in-memory computing. Whether this was a response to SAP's HANA in-memory computing solution, or SAP just beat them to the punch can be argued, but that is not the point. The point is that both are here and it's interesting to see how they stack up. Quite frankly, I was expecting TimesTen to be integrated with Exadata to make it more general purpose, but integrating EssBase, BI Suite and creating a separate front-end analytics machine is very welcome and possibly a better solution.

Until recently, via several colleagues attending SAP's annual TechEd conference, I was not able to track down details on HANA. I've managed to learn the certified configurations and more as to what it is meant to accomplish, what is missing (and planned for future), as well as some other helpful information. Even still details are lacking, with SAP preferring you contact a sales agent for full details and an engagement. I much prefer Oracle's policy of providing as much public domain information as possible for all it's products and solutions, as soon as possible. Still several months after introduction, HANA public domain information is limited, while already Oracle has a solution brief available here.

Hardware
Exalytics leverages commodity, off the shelf Oracle/Sun hardware components (model appears to be Sun Fire X4470-M2). The pertinent details are below:

CPU: 4 x 10-core Intel E7-4800 (40 processing cores)
RAM: 1 TB
HDD: direct attached storage (DAS) with 3.6 TB raw capacity (RAID layout)
Network: 2 x 40 Gb/s (QDR) Infiniband (IB) ports (for Exadata connection); 2 x 10 GbE ports (for client access); 4 x 1 GbE ports

It comes in single units of 3RU rack-mountable servers which can be clustered for expanded memory and availability. NAS storage is supported for storing shared configuration data and metadata. Based on my understanding, this is meant to connect to Exadata machines (not sure if exclusively) which is where the IB connections come into play. In a multi-Exadata setup, my guess would be that the IB connections are made to a single IB spine switch, similar to how connections to the ZFS appliance are made. This has always been a point of concern as we've suffered an entire environment outage due to a bug in the IB switch taking down the entire IB network, and hence all the Exadata machines. You can read about it here.

Software
Unlike the hardware, the software is not all available outside the Exalytics environment. It consists of:

Oracle BI Foundation - provides relational and multi-dimensional OLAP analysis, scorecards, predictive analysis, visualizations, alerts and notifications, search and mobile access.
Oracle Essbase - provides multi-dimensional OLAP cubes with parallel processing enhancements (including others)
Oracle TimesTen - provides in-memory caching and SQL access, enhanced for in-memory analytics. Also provides columnar compression support which effectively can turn the 1 TB RAM into 5 TB using Oracle's 5x compression ratio estimate.

The Oracle BI Foundation and Oracle TimesTen have apparently been enhanced to work together and provide features exclusive to the Exalytics platform. Essbase has also been memory optimized. The columnar compression support is interesting from the standpoint of fully supporting Exadata, and turning 1 TB into 5 TB effectively. Clustering is available with multiple units, to not only provide high availability but also to pool the memory resources and accommodate larger data sets. Automated management was mentioned in Larry Ellison's keynote address and is also highlighted in the solution brief document. I've heard this all before, as I'm sure most have, I will reserve judgement until I get a chance to see it in action, but am skeptical when I hear terms such as "intelligent" and "automated". The combination of enhanced versions of TimesTen and EssBase provide:

In-memory data replication (TimesTen) for data sets which can fit at least 2x entirely in memory.
In-memory adaptive data mart by EssBase using TimesTen's caching of recent or 'hot' data to update data marts.
In-memory intelligent result cache (TimesTen), which I'm pretty sure the intelligent part is a bit of marketing from past experiences.
In-memory cubes (EssBase) which are writable.

Administration

Similar to Exadata this is an intergrated, pre-built system which touches several areas of IT, i.e. networking, server, applications (EssBase and BI), and database (TimesTen). Of course, Oracle will prefer, and highly recommend, that companies choose their managed services offering to setup and administer the system. However, this is simply not going to happen in every case and for those who choose to do setup and administration in-house, the same questions will be asked for Exalytics as has been for Exadata and Exalogic. That is, who or what teams will be responsible for the maintenance? Arup Nanda has put forward the idea of an Exadata DBA, but I think this should be taken a bit further. I believe with the introduction of these various appliances (not just Oracle), there needs to be a new job description for "Appliance Administrators" with the specific appliance called out. So there would be an Exalytic Administrator with the necessary skills overlapping each section. If needs be the administrator can always fall back on resources with skill sets for a specific area (such as the networking component).

Closing
I think the Oracle Exalytic BI Machine is a good salvo by Oracle and I look forward to testing it out (hopefully soon) and making public my findings. Let the analytic appliance wars now begin in full force.

Oracle 11gR2 (11.2.0.2) Installation - Database software

2011-07-19T15:28:00.002+01:00

This is the 2nd in my series of Oracle11gR2 installations, focusing now on the installation of the database or RDBMS software for 11.2.0.2 having already installed the Grid Infrastructure component per my previous post here. If you don't want to install the Grid Infrastructure and use ASM (and Oracle Restart) that is fine, go ahead and skip that first article. In such a case I assume you would be using file systems (perhaps with dNFS which is a post for another time).

Here again, I will be using the command line approach since this is an easy way to have everything scripted and automated (and not require a GUI). I'll show the parameters that need to be adjusted, but if you are not comfortable then I'd suggest doing an initial GUI installation, saving that response file when prompted and then using the saved response file as your gold image for further scripted installations. Note that in the below I use RDBMS_HOME instead of ORACLE_HOME to make the distinction between the actual database home and the grid infrastructure home.

Requirements

See my previous post here

Installation

1. If this is your first installation, then you will want to create the '/etc/oraInst.loc' file, as the root user:

Note: This is optional since you will be prompted at the end of the installation to run $ORACLE_BASE/../oraInventory/orainstRoot.sh which does this for you if this files does not exist.

echo "inventory_loc=/oracle/app/oraInventory

inst_group=dba"

> /etc/oraInst.loc

chown oracle:dba /etc/oraInst.loc

chmod 664 oraInst.loc

2. Edit the DB response file, 'db_inst.rsp' for the values as show below:

ORACLE_HOSTNAME=oradb01

ORACLE_BASE=/oracle/app/oracle

ORACLE_HOME=/oracle/app/product/11.2.0/db_1

3. Run the DB installation (using responsefile and silent installation) as the oracle user:

Note: Ensure you set your DISPLAY environment variable, or you are at run level 5, otherwise you will encounter an error.

./runInstaller -silent -noconfig -responseFile /home/oracle/rsp/db_inst.rsp

4. As the root user run '$RDBMS_HOME/root.sh' when the above completes as prompted.

5. As the oracle user, create an OCM response file. This saves a lot of time later down when you are prompted for those values. Simple run the following and follow the instructions to create and save the response file:

$RDBMS_HOME/OPatch/ocm/bin/emocmrsp

6. Apply the latest OPatch patch (MOS patch ID 6880880), then the latest PSU to this installation as the oracle user. Simple unzip the OPatch patch to the $RDBMS_HOME. For the PSU, unzip to a temporary location, navigate to the patch location and run:

$RDBMS_HOME/OPatch/opatch apply -ocmrf /home/oracle/rsp/ocm.rsp 

7. Apply patch 12431716 (as required by PSU 2) by unzipping to a temporary location, change to the patch directory and as oracle user running:

$RDBMS_HOME/OPatch/opatch apply -ocmrf /home/oracle/rsp/ocm.rsp

At this point you have fully prepared GI and RDBMS software installations with a ready listener and two disk groups. Now you can create a new database, or migrate an exiting database. I'll leave the new installation to another post in which I'll show how to use DBCA and a template to do it silently, without a GUI.

Oracle Exadata Storage Expansion Rack

2011-07-12T17:51:00.000+01:00

Today Oracle announced the availability of the Exadata Storage Expansion Rack and I can't say I didn't see this one coming. I don't think my current company was alone in forecasting running low on space within the confines of a single Exadata machine, but not wanting to pay for another full rack of Exadata since the compute nodes were not needed. Sure, you can buy just the additional storage, but it's a pain as from what I've seen that is a negotiation with Oracle again.

There does not seem to be anything different between previously getting just the additional Storage Servers vs. now buying a Storage Expansion Rack. I see this announcement as just Oracle making official or public the fact that you can get the Storage Servers without the compute nodes. As can be expected, the racks are available in the same configurations as the database machine itself, i.e. quarter, half, and full rack, and include IB switches and cables. Since these are the standard configurations, it is a bit curious as to why Oracle choose to nonetheless license the Storage Software separately. Everyone knows the Storage Software costs, and the configuration is fixed so why bother? Sure, you can negotiate discounts for the software costs (as everyone does), but why not just give the full starting price without asking me to go calculate it?

Full rack: 18 Oracle Exadata Storage Servers with up to 6.75 TB of smart flash cache, 432 TB of raw disk storage and 216 CPU cores, will run you $750,000 a box.
Half rack: 9 Exadata Storage Servers has 3.4 TB of smart flash cache, 216 TB of raw disk storage and 108 CPUs will run you $425,000.
Quarter rack: system has 1.5 TB of flash cache, 96 TB of disk storage and 48 CPU cores costs $225,000.

Another point of note is that this is really for backup, storage archival, or low bandwidth processing as only the high capacity or HC disks (7.2K rpm, 2TB) are used in the current expansion racks. Sorry, no high performance (HP) or 15K rpm SAS disks (600GB) yet. Considering this is an expansion rack to augment storage and not necessarily for primary usage this is not to be totally unexpected though it would be nice to have that option also available in case you are in the situation of requiring additional HP storage. Yes, you still get the TB of Smart Flash Cache and the Storage Software but it's still not as fast when you need 15K rpm and true SAS. Maybe this might be a future standard offering as well with enough customer demand.

If you want to read more you can head over to the Oracle Exadata website which has the full details, a direct link to the PDF data sheet is here as well. But happy times are ahead. My prediction is that we will also see some sort of Exadata announcement at OpenWorld 2011 (where's my blogger approval Oracle?) perhaps newer Intel CPUs and faster/more storage and/or a MySQL machine.

Oracle 11gR2 (11.2.0.2) Installation - Grid Infrastructure

2011-07-07T16:13:00.001+01:00

I quite often find myself doing Oracle installations and referencing back to the manuals, read me, other blogs, and various other information sources in an attempt to ensure I get everything just perfect. This is regardless of having done it many times and having documented it myself. These days however, I find this a time consuming exercise, especially given my increasing workload so I thought to simply blog about my method which follows below.

The installation below utilizes a command line only method as I often need to be do multiple installations in a standard, and scripted way. The best method I've found for reaching this point is to initially start with a GUI approach, from which the response file is saved which can then be used for further silent, command line driven installations. I'll start with the installation of the Grid Infrastructure component which is used for ASM and Oracle Restart, and continue with a follow-up posting with the RDBMS software installation. In the method that follows I use a simplified setup with only a single group assigned to the oracle user, i.e. 'dba'. Of course you will need to modify the below for your environment.

Requirements

OS: SUSE Linux Enterprise Service 10.3 (64-bit)

Packages (standard installation):

make-3.80-202.2
binutils-2.16.91.0.5-23.34.33
libstdc++33-3.3.3-7.8.1
gcc-4.1.2_20070115-0.29.6
gcc-c++-4.1.2_20070115-0.29.6
glibc-2.4-31.74.1
glibc-devel-2.4-31.74.1
glibc-devel-32bit-2.4-31.74.1
ksh-93s-59.11.35
libaio-0.3.104-14.2
libaio-32bit-0.3.104-14.2
libaio-devel-0.3.104-14.2
libaio-devel-32bit-0.3.104-14.2
libelf-0.8.5-47.2
libgcc-4.1.2_20070115-0.29.6
libstdc++-4.1.2_20070115-0.29.6
libstdc++-devel-4.1.2_20070115-0.29.6
numactl-0.9.6-3.25.28
sysstat-8.0.4-1.7.27
unixODBC-2.2.12
unixODBC-devel-2.2.12
unixODBC-32bit-2.2.12
oracleasmlib-2.0.4-1.SLE10
oracleasm-2.0.5-7.4.50
oracleasm-support-2.1.4-1.SLE10

Configuration files & settings:

/etc/sysctl.conf

# Oracle settings
kernel.core_uses_pid = 1
kernel.suid_dumpable = 1
kernel.msgmni = 2878
# physical RAM size (8 GB) / pagesize (4096)
kernel.shmall = 2097152
# 1/2 of physical RAM typically, 5 GB here
kernel.shmmax = 5368709120
kernel.shmmni = 4096
#kernel.sem = 250 32000 100 128
kernel.sem = 250 32000 100 142
# 512 x processes (for example 6815744 for 13312 processes)
fs.file-max = 6815744
fs.aio-max-nr = 1048576
net.ipv4.ip_local_port_range = 9000 65500
net.ipv4.conf.default.rp_filter = 1
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
# For RDS use larger value below
#net.core.wmem_max = 2097152
net.core.wmem_max = 1048576
# group ID of dba group
vm.hugetlb_shm_group = 7001
# number of hugepages (2 MB size) to be used for ASMM
vm.nr_hugepages=2560

/etc/security/limits.conf

# Oracle settings
oracle soft core unlimited
oracle hard core unlimited
oracle soft nproc 131072
oracle hard nproc 131072
oracle soft nofile 131072
oracle hard nofile 131072
oracle soft stack 32768
oracle hard stack 32768
oracle soft memlock 5242880
oracle hard memlock 5242880
root soft core unlimited
root hard core unlimited
root soft nproc 131072
root hard nproc 131072
root soft nofile 131072
root hard nofile 131072
root soft memlock 5242880
root hard memlock 5242880

/etc/pam.d/su

# Oracle setting
session required pam_limits.so

/etc/pam.d/xdm

# Oracle setting
session required pam_limits.so

/etc/sysconfig/ntp
NTPD_OPTIONS="-x -u ntp" # add '-x' to existing string

~/.ssh/config
Host *
ForwardX11 no # Set in the oracle user's file

~/.bashrc
if [ -t 0 ]; then
stty intr ^C
fi # Disable terminal output commands (assumes bash shell)

Links

ln -s /bin/fuser /sbin/fuser
ln -s /usr/bin/ssh /usr/local/bin/ssh
ln -s /usr/bin/perl /usr/local/bin/perl
ln -s /usr/bin/scp /usr/local/bin/scp

Installation

1. (optional) Create '/etc/oraInst.loc' file as the root user.

Note: This step is optional as it will be done automatically for first time installations when the oraInst.sh script is run.

echo "inventory_loc=/oracle/app/oraInventory

inst_group=dba"

> /etc/oraInst.loc

chown oracle:dba /etc/oraInst.loc

chmod 664 oraInst.loc

2. Edit GI response file, 'grid.rsp' for the below values:

ORACLE_HOSTNAME=oradb01

ORACLE_BASE=/oracle/app/oracle

ORACLE_HOME=/oracle/app/11.2.0/grid_1

3. As the oracle user, run GI installation (using responsefile 'grid.rsp' and silent installation).

Notes:
- Ensure your DISPLAY environment variable is set, or you are at run level 5. Otherwise you will not be able to run the command successfully.
- You may see some warnings for the installation pertaining to a missing library package compat-libstdc++ which can safely be ignored as for SLES 10.3 and greater libstdc++33 is used instead (reference MOS: 'Requirements for Installing Oracle 11gR2 64-bit (AMD64/EM64T) on SLES 10 (Doc ID 884435.1)').

./runInstaller -silent -noconfig -responseFile /home/oracle/rsp/grid.rsp

4. As the root user, run '/oracle/app/oraInventory/oraInst.sh', '$GI_HOME/root.sh' and then '$GI_HOME/crs/install/roothas.pl' (per log file contents) after the above completes. See below for example run of '$GI_HOME/crs/install/roothas.pl':

/oracle/app/11.2.0/grid_1/perl/bin/perl -I/oracle/app/11.2.0/grid_1/perl/lib -I/oracle/app/11.2.0/grid_1/crs/install /oracle/app/11.2.0/grid_1/crs/install/roothas.pl

Using configuration parameter file: /oracle/app/11.2.0/grid_1/crs/install/crsconfig_params

Creating trace directory

LOCAL ADD MODE

Creating OCR keys for user 'oracle', privgrp 'dba'..

Operation successful.

LOCAL ONLY MODE

Successfully accumulated necessary OCR keys.

Creating OCR keys for user 'root', privgrp 'root'..

Operation successful.

CRS-4664: Node oemgc02 successfully pinned.

Adding daemon to inittab

ACFS-9300: ADVM/ACFS distribution files found.

ACFS-9307: Installing requested ADVM/ACFS software.

ACFS-9308: Loading installed ADVM/ACFS drivers.

ACFS-9321: Creating udev for ADVM/ACFS.

ACFS-9323: Creating module dependencies - this may take some time.

FATAL: Module oracleoks not found.

FATAL: Module oracleadvm not found.

FATAL: Module oracleacfs not found.

ACFS-9327: Verifying ADVM/ACFS devices.

ACFS-9121: Failed to detect control device '/dev/asm/.asm_ctl_spec'.

ACFS-9310: ADVM/ACFS installation failed.

ACFS-9311: not all components were detected after the installation.

oemgc02     2011/06/23 10:02:19     /oracle/app/11.2.0/grid_1/cdata/oemgc02/backup_20110623_100219.olr

Successfully configured Oracle Grid Infrastructure for a Standalone Server

4b. To address the failure of the modules as seen above, reference MOS article "ACFS-9327 ACFS-9121 ACFS-9310: ADVM/ACFS installation failed [ID 1265276.1]" and run the following as the root user:

mkdir -p /lib/modules/`uname -r`/extra/usm

cp -p /lib/modules/2.6.16.60-0.54.5*/extra/usm/* /lib/modules/`uname -r`/extra/usm

/sbin/depmod

4c. OHASD will not starting after reboot on SLES (reference MOS: "OHASD not Starting After Reboot on SLES [ID 1325718.1]"). Run the following to fix this issue:

oradb01:~ # chkconfig --list|grep ohasd

init.ohasd                0:off  1:off  2:off  3:off  4:off  5:off  6:off

ohasd                     0:off  1:off  2:off  3:off  4:off  5:off  6:off

oradb01:~ # chkconfig --list|grep raw

raw                       0:off  1:off  2:off  3:off  4:off  5:off  6:off

oradb01:~ # chkconfig raw on

oradb01:~ # chkconfig ohasd on

oradb01:~ # chkconfig --list|grep raw

raw                       0:off  1:off  2:on   3:on   4:off  5:on   6:off

oradb01:~ # chkconfig --list|grep ohasd

init.ohasd                0:off  1:off  2:off  3:off  4:off  5:off  6:off

ohasd                     0:off  1:off  2:off  3:on   4:off  5:on   6:off

Note: Just turning on OHASD will result in error:

oradb01:~ # chkconfig ohasd on

insserv: Service raw has to be enabled for service oracle_has

insserv: exiting now!

/sbin/insserv failed, exit code 1

5. Apply latest OPatch patch, then latest PSU to GI software installation (Oracle Restart Home). As root user run '$GI_HOME/OPatch/opatch auto /oracle/stage/patches_grid -oh $GI_HOME -ocmrf /home/oracle/rsp/ocm.rsp'. An example below:

oemgc02:/oracle/stage/patches_grid # $ORACLE_HOME/OPatch/opatch auto /oracle/stage/patches_grid -oh $ORACLE_HOME -ocmrf /home/oracle/rsp/ocm.rsp

Executing /usr/bin/perl /oracle/app/11.2.0/grid_1/OPatch/crs/patch112.pl -patchdir /oracle/stage -patchn patches_grid -oh /oracle/app/11.2.0/grid_1 -ocmrf /home/oracle/rsp/ocm.rsp -paramfile /oracle/app/11.2.0/grid_1/crs/install/crsconfig_params

opatch auto log file location is /oracle/app/11.2.0/grid_1/OPatch/crs/../../cfgtoollogs/opatchauto2011-06-23_10-32-00.log

Detected Oracle Restart install

Using configuration parameter file: /oracle/app/11.2.0/grid_1/crs/install/crsconfig_params

Successfully unlock /oracle/app/11.2.0/grid_1

patch /oracle/stage/patches_grid/12311357  apply successful for home  /oracle/app/11.2.0/grid_1

patch /oracle/stage/patches_grid/11724916  apply successful for home  /oracle/app/11.2.0/grid_1

ACFS-9300: ADVM/ACFS distribution files found.

ACFS-9312: Existing ADVM/ACFS installation detected.

ACFS-9314: Removing previous ADVM/ACFS installation.

ACFS-9315: Previous ADVM/ACFS components successfully removed.

ACFS-9307: Installing requested ADVM/ACFS software.

ACFS-9308: Loading installed ADVM/ACFS drivers.

ACFS-9321: Creating udev for ADVM/ACFS.

ACFS-9323: Creating module dependencies - this may take some time.

ACFS-9327: Verifying ADVM/ACFS devices.

ACFS-9309: ADVM/ACFS installation correctness verified.

CRS-4123: Oracle High Availability Services has been started.

5b. Apply patch 12431716, as root user run '$GI_HOME/crs/install/roothas.pl -unlock'. An example below:

oemgc02:/oracle/stage/12431716 # $ORACLE_HOME/crs/install/roothas.pl -unlock

Using configuration parameter file: /oracle/app/11.2.0/grid_1/crs/install/crsconfig_params

Successfully unlock /oracle/app/11.2.0/grid_1

Next, as GI owner run '$GI_HOME/OPatch/opatch apply -oh $GI_HOME -local -ocmrf /home/oracle/rsp/ocm.rsp'.

Next, as root user run '$GI_HOME/crs/install/roothas.pl -patch'. An example below:

oemgc02:/oracle/stage/12431716 # $ORACLE_HOME/crs/install/roothas.pl -patch

Using configuration parameter file: /oracle/app/11.2.0/grid_1/crs/install/crsconfig_params

ACFS-9300: ADVM/ACFS distribution files found.

ACFS-9312: Existing ADVM/ACFS installation detected.

ACFS-9314: Removing previous ADVM/ACFS installation.

ACFS-9315: Previous ADVM/ACFS components successfully removed.

ACFS-9307: Installing requested ADVM/ACFS software.

ACFS-9308: Loading installed ADVM/ACFS drivers.

ACFS-9321: Creating udev for ADVM/ACFS.

ACFS-9323: Creating module dependencies - this may take some time.

ACFS-9327: Verifying ADVM/ACFS devices.

ACFS-9309: ADVM/ACFS installation correctness verified.

CRS-4123: Oracle High Availability Services has been started.

6. Create listener from GI home (it will automatically get added to CRS by NETCA). By creating the listener before the ASM or database instances, Clusterware will automatically make the dependency relation. As oracle user edit the netca.rsp file and run:

Note: Ensure you have set the DISPLAY environment variable, or you are at run level 5. Otherwise the command will fail.

$GI_HOME/bin/netca -silent -responsefile /home/oracle/resp/netca.rsp

7. Create an ASM instance in silent mode by using ASMCA. The parameters for ASM in 11.2 require a disk group. I like to create a separate disk group (using a 2 GB disk) for this purpose which also serves the purpose of storing the OCR and voting files should the instance be migrated to RAC (since I also like to store those files using a separate disk group). To create this initial disk group, called SYSTEMDG run ASMCA as the oracle (or GI owner) user, then to create the additional disk groups for DATA and FRA, run similar commands using '-createDiskGroup' instead of '-configureASM'. See the examples below:

oracle@oradb01:~/bin> $GI_HOME/bin/asmca -silent -configureASM -sysAsmPassword sysPassw0rd -asmsnmpPassword asmsnmpPassw0rd -diskGroupName SYSTEMDG -diskList 'ORCL:SYSTEMD' -redundancy EXTERNAL -au_size 4 -compatible.asm '11.2.0.2.0' -compatible.rdbms '11.2.0.2.0'

ASM created and started successfully.

Disk Group SYSTEMDG created successfully.

oracle@oradb01:~/bin> $GI_HOME/bin/asmca -silent -createDiskGroup -sysAsmPassword sysPassw0rd -diskGroupName D1_T2_A4 -diskList 'ORCL:D_T2_*' -redundancy EXTERNAL -au_size 4 -compatible.asm '11.2.0.2.0' -compatible.rdbms '11.2.0.2.0'

Disk Group D1_T2_A4 created successfully.

oracle@oradb01:~/bin> $GI_HOME/bin/asmca -silent -createDiskGroup -sysAsmPassword sysPassw0rd -diskGroupName F1_T2_A4 -diskList 'ORCL:F_T2_*' -redundancy EXTERNAL -au_size 4 -compatible.asm '11.2.0.2.0' -compatible.rdbms '11.2.0.2.0'

Disk Group F1_T2_A4 created successfully.

You have now successfully installed Grid Infrastructure 11.2.0.2, applied a PSU and recommended patch, and created an ASM instance (along with a corresponding listener). At this point you are ready to commence the database installation process which I will include in a follow-up blog.

Ensure CELL_OFFLOAD_PROCESSING is FALSE in Non-Exadata

2011-06-29T14:44:00.000+01:00

A colleague of mine recently brought to my attention a curious wait event that they were experiencing for 'ASM file metadata operation' which occurs when doing operations such as a DROP TABLESPACE, or in this case when using Data Pump. Though the server was basically not doing anything the CPU usage was 100% and there was the strange wait event on 'ASM file metadata operation' with high waits on 'ksv master wait'. So to Oracle Support they went via opening an SR.

It seems that upon applying a PSU to 11.2.0.2 (as of this writing PSU 2 is that latest so likely this will not occur in PSU 3 but that is only my guess) the parameter cell_offload_processing is set to TRUE. In an Exadata environment this is the appropriate setting, however, in a non-Exadata environment, which it was in this case, this causes performance issues to arise as processes on the RDBMS side await on a reply from the ASM side which is trying to delivery smart-scan results.

The quick fix is of course to simply reset the parameter to FALSE, i.e. 'ALTER SYSTEM SET cell_offload_processing = FALSE'. If you prefer you can instead apply patch 11800170. Per the MOS note, "High 'ksv master wait' And 'ASM File Metadata Operation' Waits In Non-Exadata 11g [ID 1308282.1]" this issue is fixed in 11.2.0.3 and 12.1.

Of course 11.2.0.3 is the likely next patch set which is not yet released but is likely in testing. I've been seeing this 12.1 designation on MOS recently which is a good indication that this is the next version for the database, which is likely in early testing. I think that is an entry for a good blog, namely what features I'd like to see in the next database release, and what is likely to be included by Oracle. Attending OpenWorld 2011 would certainly help with that posting, come on Oracle, where is my Blogger approval ;-).

Oracle OpenWorld 2011

2011-06-27T15:30:00.000+01:00

I recently signed up for Oracle OpenWorld 2011 (Blogger registration) just last week. So now I am awaiting to be approved (or not as the case may be). I found out about this additional registration process last year when a friend of mine got through.The requirements are pretty basic, i.e. a blog about Oracle products and/or solutions and updates within the last month. Your application will then get sent through to Oracle Corporate Communications team for approval. My friend signed up again, so we are both hoping to get approved and attend together though with a new baby on the way he is more likely to not attend.

I've been to two OpenWorld events (both in San Francisco), and in all honesty it should be a must visit event for anyone in the industry (at least once). The full attendance fee might be a bit prohibitive for some at over $2,000 (for regular folk) but is well worth the investment, which is exactly how it should be seen. But, in case you don't have the budget for the full attendance there are a few options for attending such as the Blogger registration, or a Discover pass ($50 - $125 depending on registration date) to name a few. I'm not too keen on the Discover pass since you don't get to attend any sessions (or the parties!), but it is an entry way to the other invaluable opportunities such a networking.You can find a comparison of the different packages on the Oracle OpenWorld site here. If you do get approved, there is still of course, the travel and accommodation considerations which many book a full year in advance to avoid the rise in prices, and also availability. My first attendance was on a pass from our Oracle solution provider in 2006, with hotel and flight covered by the company. I was lucky enough to find a surprisingly good motel just 3 blocks away which is walking distance and where you want to be unless you like commuting. Last year I stayed in the Oakland area with a friend and had to commute via a mile walk, train, and then 2 blocks. Lots of good exercise but with all the walking you will do at OpenWorld, your feet will not be thanking you regardless of good shoes!

If you think selling your attendance is difficult (so you can get the company to pay for something to help you out, even if it is just for your time), think again. Oracle is very thoughtful in helping you to write a return on investment letter to your managers or whomever. You can find a link here to that helpful page. I have similar points:

- great educational opportunity to learn about Oracle products and solutions from experts, actual customers with experience, and Oracle themselves. You do need to plan your sessions based on your level of expertise and what you are trying to get out of your trip in order to get the full benefit. From my experience too many of the sessions are not appropriately labeled for the correct expertise level and their contents, so I often find myself moving from room to room within the same time period to catch something of interest based on my schedule (which is why I register for a few concurrent sessions).

- networking with peers, experts, and Oracle staff from all areas including development, support, consulting, managerial, and 'C' levels. This is a big sell since you can get really good contacts within Oracle to aid in addressing immediate or future issues such as the developers that work on GoldenGate, Enterprise Manager, ASM, or the Oracle kernel.

- contact existing and new vendors. While you do get the usual song and dance, there are really good contacts to be made as some vendors (mostly the larger companies) bring out some of their hidden internal talent to answer questions. Seeing the various offerings, especially for something you need but didn't know a company existed that offered a solution is golden.

Of course, the parties are incredible, and the overall experience overwhelming. I do think that for anyone that has ever attended the feedback is the same. You need to know what you want from the event, plan accordingly, and send enough people to get the job done. At least 3 people should be sent, as only one will simply not work unless your needs are really narrow. There is simple too much ground to cover for one person.

The event is from Oct 2 - 6 at the Moscone Center in San Francisco. Do yourself a favor and register now to attend. You will not regret the experience.

RMAN is not removing my logs, why?

2011-06-13T18:19:00.002+01:00

I'm not going to go into the details and various scenarios of RMAN and archive logs not being removed. The purpose of this entry is (other than a filler) to provide a head's up, if you will, as to what you are likely to encounter now in an 11g environment using RMAN and GoldenGate (or Streams). I've made mention in my prior post on the relationship between GoldenGate and Streams here.

So it seems that again, due to our usage of GoldenGate, that Streams is also being enabled under the covers. It seems also that there is a 6 hour delay in archive logs being removed via RMAN even if they are not required! Once again, doing the requisite research, we found that this is actually expected behavior. RMAN is looking at the MIN_REQUIRED_CAPTURE_CHANGE# in v$database and not at DBA_CAPTURE which only gets updated every 6 hours. There is a workaround which involves forcefully deleting the archive logs using an RMAN statement similar to:

delete noprompt force archivelog all completed before 'sysdate-10/1440';

I'm not sure why the change in behavior which now leads to this situation but I can't say were too thrilled. Seems like a bug or perhaps not yet fully thought out or completed perhaps? In any case, we've got the backup scripts and everything thought out so no harm, no foul. Moving on with the project...

Oh, you can reference the MOS article: "RMAN-08137 When deleting archivelogs even when Streams CAPTURE does not require them [ID 1079953.1]"

Oracle GoldenGate and Streams Integration?

2011-06-13T18:05:00.001+01:00

We are still doing our Exadata migration, apologies for not posting more on this as it progresses but I find it hard to determine what is, and is not acceptable to blog (confidentially, NDA and all that stuff). I'm also quite busy and hardly find the time to blog generally. In any case, we are now on the 3rd environment, that being production support or QA, and have noticed that even though we do not have any Streams configured, we are seeing miscellaneous Streams events as one of the top wait events. It turns out that is not only expected but can be regarded as an idle wait. This happens because we are running GoldenGate Extract with the THREADOPTIONS DBLOGREADER parameter which enables the use of the OCIPOGG module. The wait (OTHER) is being reported while GoldenGate waits for additional logfile data to be made available.

So it would seem with the latest GoldenGate (11.1.1.1) that there is some integration with Streams. This makes sense since they are both Oracle products and per Oracle's Goldengate Statement of Direction GoldenGate is the strategic direction where the best of Streams will be integrated into GoldenGate.

For further information please reference MOS article: "Why do I see Streams Miscellaneous Event in AWR as a top event when Goldengate Extract is running [ID 1317122.1]".