Memos of a fellow IT guy

Infrastructure automation with Ansible and Docker: part 1

2016-01-05T02:52:00.000-08:00

Up until now, I've been using Packer along with a bunch of shell provisioning scripts to set up development and production environments, but as the complexity of environments (no. of packages/tools/dependencies, configurations and data) increased along with the need for rapid creation/deletion of these environments I decided to give some automation tools out there a try.
I'm sure you've already heard enough about Docker and it's offerings. I have been using it for a few specific cases where it really came in handy for deploying and experimenting with different apps in isolated containers while keeping the host machine clean and tidy. I also used the Gradle Docker plugin to package my .war artifacts into Docker Tomcat images for deployment into the Docker host machine. However, moving on I feel that should be job for Jenkins.
The interesting bit is all about Ansible, which you've also probably heard of. Now, although it might seem quite young as compared to it's heavyweight competitors like Puppet and Chef, it caught my attention early on for one simple yet very important reason: simplicity.
Ansible, at it's core uses SSH, which comes natural to most of us when it comes to managing infrastructure. Don't be fooled by the simplicity factor, as when used in conjunction with the composability of "playbooks", can make your life a lot easier!
With the help of forks, it can be quite handy, when it comes to running otherwise cumbersome tasks such as upgrading software/packages on tens or even thousands of machines (see how twitter uses it for managing thousands of machines).
Now, as for the using Ansible and Docker together, I came to the conclusion that building Docker images should be left out of Ansible, which is what Michael Dehaan, one of the creators of Ansible recommends. So as I mentioned before, leave that job to your build automation system.
To get started, here is a simple Ansible playbook for installing docker on a local VM.
The installation steps for Docker have been taken from Docker documentation. You can find the steps for installing Ansible here.

---
# file: ubuntu_docker.yml
# Playbook for setting up docker

-
  hosts: all

  sudo: True

  user: vagrant

  tasks:
    - name: Check that the server is live
      action: ping

    - name: Add docker gpg key
      apt_key:
        keyserver: "hkp://p80.pool.sks-keyservers.net:80"
        id: "58118E89F3A912897C070ADBF76221572C52609D"
        state: present

    - name: Add docker repo
      apt_repository:
        repo: "deb https://apt.dockerproject.org/repo ubuntu-trusty main"
        state: present

    - name: Install the required packages
      apt: pkg={{ item }} state=present update_cache=yes
      with_items:
        - linux-image-extra-{{ ansible_kernel }}
        - docker-engine

    - name: Update Ubuntu firewall to allow forwarding
      lineinfile:
        dest: /etc/default/ufw
        regexp: DEFAULT_FORWARD_POLICY=
        line: DEFAULT_FORWARD_POLICY="ACCEPT"
        state: present

    - name: Reload ufw
      ufw:
        state: reloaded

    - name: Check docker daemon running
      service:
        name: docker
        state: started

For running this playbook:

ansible-playbook -i local --ask-pass ubuntu_docker.yml

Note that you either need a file in the same directory with your target machines or an ANSIBLE_INVENTORY environment variable pointing to a file where you have stored list of your target machines.
For this example I used a file called "local" with the following contents:

dev-vm ansible_ssh_host=192.168.100.100

Also, the --ask-pass flag would ask for the target machine password when the playbook is run.

For further information on Ansible playbook best practices and discovering playbook syntax read here and here.

Changing the maximum size for uploaded files in Oracle WebCenter

2012-02-17T00:35:00.000-08:00

Those of you who have worked with Oracle WebCenter, have probably a content repository configured for your collaboration space. If you have tried uploading files in the document explorer task flow, you would notice that the maximum allowed file size is 2 MB, which is not what most of us are used to. As you can see the message says that the max. file size allowed is 2 GB, which is clearly a lie. So here is how to change it.

1. Export your WebCenter Spaces metadata file using the following command(s)

cd <WC HOME>\common\bin\
wlst.bat

2. Wait for the script to finish initializing, then type

connect('weblogic', 'welcome1', 't3://<host>:7001')

3. Replace the values in quotes with yours of course, then

exportMetadata(application='webcenter', server='WC_Spaces', toLocation='c:/', docs='/oracle/webcenter/webcenterapp/metadata/webcenter-config.xml')

4. Then navigate to c:/oracle/webcenter/webcenterapp/metadata and edit the webcenter-config.xml file. Change the <webcenter:uploadedFileMaxDiskSpace> value, the default is 2097152 which is around 2 MB, so change it to 2147483648 (2 GB).

5. Then import the modified file back, if you have closed the wlst session, repeat step 1 and 2 then type

importMetadata(application='webcenter',server='WC_Spaces',fromLocation='c:/',docs='/oracle/webcenter/webcenterapp/metadata/webcenter-config.xml')

6. Exit wlst

exit()

7. Restart the WC_Spaces managed server

Using P6 R8 Web Services in Web Applications (Weblogic)

2012-02-03T01:43:00.000-08:00

If you are familiar with Primavera P6 development, you would know about the "P6 Web Services", which in practice, is a web application exposing the functionality of the Integration API through web services. Recently I had to consume a bunch of these web services in a Web Application (to be deployed on Weblogic AS). If you take a look at the Demo provided by the Web Services Installation (a Java swing application that creates a sample project), you would notice the libraries required to consume such services, the most important of which are cxf-bundle, p6ws-jaxws-client and osdt-*. After developing a pretty simple application and deploying it on Weblogic AS, I hit the following error:

weblogic.wsee.jaxws.spi.ClientInstanceInvocationHandler cannot be cast to org.apache.cxf.frontend.ClientProxy

After lots of googling, I finally found the problem. The CXF libraries used in the web service stubs are being overridden by Weblogic AS which causes the problem. The solution is to add the package preference setting in weblogic.xml file in your application (if you don't have one create it using File->New->Weblogic Deployment Descriptor). So your weblogic.xml should look something like:

<?xml version = '1.0' encoding = 'windows-1252'?>
<weblogic-web-app xmlns="http://xmlns.oracle.com/weblogic/weblogic-web-app" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd http://xmlns.oracle.com/weblogic/weblogic-web-app http://xmlns.oracle.com/weblogic/weblogic-web-app/1.2/weblogic-web-app.xsd">
<context-root>yourapp</context-root>
<container-descriptor>
<prefer-application-packages>
<package-name>org.apache.cxf.binding.soap.saaj.*</package-name>
<package-name>org.apache.cxf.endpoint.*</package-name>
<package-name>org.apache.cxf.frontend.*</package-name>
<package-name>org.apache.cxf.transport.http.*</package-name>
<package-name>org.apache.cxf.interceptor.*</package-name>
<package-name>org.apache.cxf.transports.http.configuration.*</package-name>
</prefer-application-packages>
</container-descriptor>
</weblogic-web-app>

This should resolve the library conflict. If you get an error like this:

This class does not support SAAJ 1.1 weblogic

The Reason for the problem is Weblogic’s default SAAJ implementation, so you need to override the properties to use a better implementation weblogic.xml.saaj package. You can do that by adding the following parameters to your Weblogic startup script:

JAVA_OPTIONS= %JAVA_OPTIONS% -Djavax.xml.soap.MessageFactory=com.sun.xml.messaging.saaj.soap.ver1_1.SOAPMessageFactory1_1Impl -Djavax.xml.soap.SOAPConnectionFactory=weblogic.wsee.saaj.SOAPConnectionFactoryImpl

Integrating Primavera P6v7 Web Access with Oracle UCM 11g

2011-12-09T00:06:00.000-08:00

I have been dealing with quite a number of Oracle products, and integrating them in the context of project management infrastructure. I found some of them quite hard and complicated and the documentation doesn't always help you, but it just over complicates things. Here is the first of many I would like to share with you.

So, assuming you have installed your P6 Web Access (meaning getting the DB installed and created the P6 schemas, installed your application server i.e. Weblogic and deployed P6 war file on it) and also gotten your Oracle UCM installed (the DB schemas, Weblogic and UCM), here is how you would do it: (I would post the installation for UCM in future)

1- Add the P6 Web Server's IP address to the list of security filter in the UCM configuration file. You can find this file at
<middleware_home>\user_projects\domains\<your_ucm_domain>\ucm\cs\config\config.cfg
Append the IP address to the SocketHostAddressSecurityFilter, something like this
SocketHostAddressSecurityFilter=127.0.0.1|0:0:0:0:0:0:0:1|192.168.0.*
You can use the asterisk to allow all values for the segment (you can do something like *.*.*.*)

2- Now log into your UCM home page (usually at http://<server_address>:16200) with the administrator credentials (usually weblogic + password)

3- Expand the administration section on the left panel and click on "Admin Applets", and then click on "Configuration Manager" on the right panel.

4- On the information fields tab, add the following fields using the "Add" button:

PrmUserId
PrmProjectId
PrmWorkgroupId
PrmSecurityPolicy
PrmTemplate (uncheck “Enable Search Index” attributes)
PrmCheckedOutUserId
PrmCheckedOutDate
PrmLocalFilePath (Type = Long Text)
PrmAuthorId

5- Once done, click on "Update Database Design".

6- From the "Options" menu on the top select "Content Types" and add a new type as

Name: P6Docs
Description: Oracle Primavera Web Access Docs

7- Close the applet and expand the "Browse Content" section from the left panel. Click on "Contribution Folders". If you don't see this you have probably not enabled "Folders_g" feature in the administration console.

8- On the top right of the screen, click "New Item" and choose "New Folder" and create a folder with the following details:

Virtual Folder Name: Oracle Primavera
Owner: sysadmin
Title: Oracle Primavera Web Access Document Home
Type: P6Docs
Security Group: Public
Author: sysadmin

We're pretty much done on the UCM side of things, now lets move to P6 Web Access settings

9- Run the P6 Administration Application from the start menu on the server where you have installed P6 Web (Oracle Primavera P6->Primavera P6 Web Access Utilities->Administration Application)

10- Enter your admin user name and password.

11- Expand the "Custom" node under the configurations tree on the "Tree View" tab. Then expand your P6 database configuration profile->Database-><your_database_instance>->Content Repository.

12- Configure the content repository as follows:

Type: Oracle

Under the Oracle Universal Content Management

Host: <IP_address_of_ucm_server>
Port: <idc_port> usually 4444 (you have set this during UCM setup)
Oracle Home: \\Contribution Folders\Oracle Primavera\
Oracle Security Group: Public
Oracle Security Account: leave blank
Oracle Document Type: P6Docs
Metadata Prefic: Prm
Admin User: sysadmin
Authentication Mode: Single User (this is how I prefer it, you can choose to use Multiple User but keep in mind that you have to keep both user directories (P6 and UCM) in sync.

13- Restart P6 Web Server.

14- You can now open any project in P6 Web Access and browse/upload/modify documents under the "Documents" tab.

Implementing SOA Governance in Service Driven Enerprise

2011-06-24T20:08:00.000-07:00

I conducted a research on how to implement SOA Governance in companies with existing SOA. If you are familiar with SOA, you are aware of the organizational and business challenges (besides technical ones) it imposes to adopters. Most of the content and a case study I developed myself was based on my experience in companies driven by SOA initiative but with little or no Governance.

By definition,

SOA governance is the ability to ensure that all of the independent (SOA) efforts (whether in the design, development, deployment, or operations of a service) come together to meet enterprise requirements.

The topic is quite complex and broad in nature and its comprehension might seem a challenge for those new to it. However, for companies looking into serious SOA implementations, Governance is the way to go. Here is a link to an article by the authors of an excellent reference book, which I used in my research. I would highly recommend this for those who want to get started with SOA Governance.

http://www.infoq.com/articles/Governing-Service-Factory