BytesForge

Symfony2 design patterns @ symfonyday

2015-06-21T23:36:00.002-07:00

Design pattern in Symfony2 - Nanos gigantium humeris insidentes from Giulio De Donato

Symfony2 Framework Complete flow:

Android: Trusting SSL certificates

2012-11-20T07:36:00.000-08:00

We use a self-signed SSL certificate for the test version of our backend web service. Since our certificate isn't signed by a CA that Android trusts by default, we need to add our server's public certificate to our Android app's trusted store.

These same instructions apply to trusting a custom CA, except you'd get the public certificate directly from the CA instead of from a server.

Required tools:

OpenSSL's command line client
Java SE 6 (for keytool)
Bouncy Castle's provider jar

1. Grab the public certificate from the server you want to trust. Replace ${MY_SERVER} with your server's address.

echo | openssl s_client -connect ${MY_SERVER}:443 2>&1 | \
 sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > mycert.pem

For example, here's the PEM-encoded public certificate from google.com:

2. Android has built-in support for the Bouncy Castle keystore format (BKS). Put Bouncy Castle's jar in your classpath, and create a keystore containing only your trusted key.

export CLASSPATH=bcprov-jdk16-145.jar
CERTSTORE=res/raw/mystore.bks
if [ -a $CERTSTORE ]; then
    rm $CERTSTORE || exit 1
fi
keytool \
      -import \
      -v \
      -trustcacerts \
      -alias 0 \
      -file <(openssl x509 -in mycert.pem) \
      -keystore $CERTSTORE \
      -storetype BKS \
      -provider org.bouncycastle.jce.provider.BouncyCastleProvider \
      -providerpath /usr/share/java/bcprov.jar \
      -storepass ez24get

3. Create a custom Apache HttpClient that uses your custom store for HTTPS connections.

import android.content.Context;
import org.apache.http.conn.ClientConnectionManager;
import org.apache.http.conn.scheme.PlainSocketFactory;
import org.apache.http.conn.scheme.Scheme;
import org.apache.http.conn.scheme.SchemeRegistry;
import org.apache.http.conn.ssl.SSLSocketFactory;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.impl.conn.SingleClientConnManager;

import java.io.InputStream;
import java.security.KeyStore;

public class MyHttpClient extends DefaultHttpClient {

  final Context context;

  public MyHttpClient(Context context) {
    this.context = context;
  }

  @Override protected ClientConnectionManager createClientConnectionManager() {
    SchemeRegistry registry = new SchemeRegistry();
    registry.register(
        new Scheme("http", PlainSocketFactory.getSocketFactory(), 80));
    registry.register(new Scheme("https", newSslSocketFactory(), 443));
    return new SingleClientConnManager(getParams(), registry);
  }

  private SSLSocketFactory newSslSocketFactory() {
    try {
      KeyStore trusted = KeyStore.getInstance("BKS");
      InputStream in = context.getResources().openRawResource(R.raw.mystore);
      try {
        trusted.load(in, "ez24get".toCharArray());
      } finally {
        in.close();
      }
      return new SSLSocketFactory(trusted);
    } catch (Exception e) {
      throw new AssertionError(e);
    }
  }
}

Originally Posted in  crazybob blog

Using Hadoop And PHP

2012-11-14T18:52:00.001-08:00

Using Hadoop And PHP

Getting Started

So first things first. If you haven’t used Hadoop before you’ll first need to download a Hadoop release and make sure you have Java and PHP installed. To download Hadoop head over to:
http://hadoop.apache.org/common/releases.html
Click on download a release and choose a mirror. I suggest choosing the most recent stable release. Once you’ve downloaded Hadoop, unzip it.

user@computer:$ tar xpf hadoop-0.20.2.tar.gz

I like to create a symlink to the hadoop-<release> directory to make things easier to manage.

user@computer:$ link -s hadoop-0.20.2 hadoop

Now you should have everything you need to start creating a Hadoop PHP job.

Creating The Job

For this example I’m going to create a simple Map/Reduce job for Hadoop. Let’s start by understanding what we want to happen.

We want to read from an input system – this is our mapper
We want to do something with what we’ve mapped – this is our reducer

At the root of your development directory, let’s create another directory called script. This is where we’ll store our PHP mapper and reducer files.

user@computer:$ ls

.
..
hadoop-0.20.2
hadoop-0.20.2.tar.gz
hadoop

user@computer:$ mkdir script

Now let’s being creating our mapper script in PHP. Go ahead and create a PHP file called mapper.php under the script directory.

user@computer:$ touch script/mapper.php

Now let’s look at the basic structure of a PHP mapper.

#!/usr/bin/php
<?php
//this can be anything from reading input from files, to retrieving database content, soap calls, etc.
//for this example I'm going to create a simple php associative array.
$a = array(
'first_name' => 'Hello',
'last_name' => 'World'
);
//it's important to note that anything you send to STDOUT will be written to the output specified by the mapper.
//it's also important to note, do not forget to end all output to STDOUT with a PHP_EOL, this will save you a lot of pain.
echo serialize($a), PHP_EOL;
?>

So this example is extremely simple. Create a simple associative array and serialize it. Now onto the reducer. Create a PHP file in the script directory called reducer.php.

user@computer:$ touch script/reducer.php

Now let’s take a look at the layout of a reducer.

#!/usr/bin/php
 
<?php
 
//Remember when I said anything put out through STDOUT in our mapper would go to the reducer.
//Well, now we read from the STDIN to get the result of our mapper.
//iterate all lines of output from our mapper
while (($line = fgets(STDIN)) !== false) {
    //remove leading and trailing whitespace, just in case  
    $line = trim($line);
    //now recreate the array we serialized in our mapper
    $a = unserialize($line);
    //Now, we do whatever we need to with the data.  Write it out again so another process can pick it up,
    //send it to the database, soap call, whatever.  In this example, just change it a little and
    //write it back out.
    $a['middle_name'] = 'Jason';
    //do not forget the PHP_EOL
    echo serialize($a), PHP_EOL;
}//end while
?>

So now we have a very simple mapper and reducer ready to go.

Execution

So now let’s run it and see what happens. But first, a little prep work. We need to specify the input directory that will be used when the job runs.

user@computer:$ mkdir input

user@computer:$ touch input/conf

Ok, that was difficult. We have an input directory and we’ve created an empty conf file. The empty conf file is just something that the mapper will use to get started. For now, don’t worry about it. Now let’s run this bad boy. Make sure you have your JAVA_HOME set, this is usually in the /usr directory. You can set this by running #export JAVA_HOME=/usr.

user@computer:$ hadoop/bin/hadoop jar hadoop/contrib/streaming/hadoop-0.20.2-streaming.jar -mapper script/mapper.php -reducer script/reducer.php -input input/* -output output

So here’s what the command does. The first part executes the hadoop execute script. The “jar” argument tells hadoop to use a jar, in this case it tells it to use “hadoop/contrib/streaming/hadoop-0.20.2-streaming.jar”. Next we pass the mapper and reducer arguments to the job and specify input and output directories. If we wanted to, we could pass configuration information to the mapper, or files, etc. We would just use the same line read structure that we used in the reducer to get the information. That’s what would go in the input directory if we needed it to. But for this example, we’ll just pass nothing. Next the output directory will contain the output of our reducer. In this case if everything works out correct, it will contain the PHP serialized form of our modified $a array. If all goes well you should see something like this:

user@computer:$ hadoop/bin/hadoop jar hadoop/contrib/streaming/hadoop-0.20.2-streaming.jar -mapper script/mapper.php -reducer script/reducer.php -input input/* -output output

10/12/10 12:53:56 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
10/12/10 12:53:56 WARN mapred.JobClient: No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
10/12/10 12:53:56 INFO mapred.FileInputFormat: Total input paths to process : 1
10/12/10 12:53:56 INFO streaming.StreamJob: getLocalDirs(): [/tmp/hadoop-root/mapred/local]
10/12/10 12:53:56 INFO streaming.StreamJob: Running job: job_local_0001
10/12/10 12:53:56 INFO streaming.StreamJob: Job running in-process (local Hadoop)
10/12/10 12:53:56 INFO mapred.FileInputFormat: Total input paths to process : 1
10/12/10 12:53:56 INFO mapred.MapTask: numReduceTasks: 1
10/12/10 12:53:56 INFO mapred.MapTask: io.sort.mb = 100
10/12/10 12:53:57 INFO mapred.MapTask: data buffer = 79691776/99614720
10/12/10 12:53:57 INFO mapred.MapTask: record buffer = 262144/327680
10/12/10 12:53:57 INFO streaming.PipeMapRed: PipeMapRed exec [/root/./script/mapper.php]
10/12/10 12:53:57 INFO streaming.PipeMapRed: MRErrorThread done
10/12/10 12:53:57 INFO streaming.PipeMapRed: Records R/W=0/1
10/12/10 12:53:57 INFO streaming.PipeMapRed: MROutputThread done
10/12/10 12:53:57 INFO streaming.PipeMapRed: mapRedFinished
10/12/10 12:53:57 INFO mapred.MapTask: Starting flush of map output
10/12/10 12:53:57 INFO mapred.MapTask: Finished spill 0
10/12/10 12:53:57 INFO mapred.TaskRunner: Task:attempt_local_0001_m_000000_0 is done. And is in the process of commiting
10/12/10 12:53:57 INFO mapred.LocalJobRunner: Records R/W=0/1
10/12/10 12:53:57 INFO mapred.TaskRunner: Task 'attempt_local_0001_m_000000_0' done.
10/12/10 12:53:57 INFO mapred.LocalJobRunner:
10/12/10 12:53:57 INFO mapred.Merger: Merging 1 sorted segments
10/12/10 12:53:57 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 70 bytes
10/12/10 12:53:57 INFO mapred.LocalJobRunner:
10/12/10 12:53:57 INFO streaming.PipeMapRed: PipeMapRed exec [/root/./script/reducer.php]
10/12/10 12:53:57 INFO streaming.PipeMapRed: R/W/S=1/0/0 in:NA [rec/s] out:NA [rec/s]
10/12/10 12:53:57 INFO streaming.PipeMapRed: Records R/W=1/1
10/12/10 12:53:57 INFO streaming.PipeMapRed: MROutputThread done
10/12/10 12:53:57 INFO streaming.PipeMapRed: MRErrorThread done
10/12/10 12:53:57 INFO streaming.PipeMapRed: mapRedFinished
10/12/10 12:53:57 INFO mapred.TaskRunner: Task:attempt_local_0001_r_000000_0 is done. And is in the process of commiting
10/12/10 12:53:57 INFO mapred.LocalJobRunner:
10/12/10 12:53:57 INFO mapred.TaskRunner: Task attempt_local_0001_r_000000_0 is allowed to commit now
10/12/10 12:53:57 INFO mapred.FileOutputCommitter: Saved output of task 'attempt_local_0001_r_000000_0' to file:/root/output
10/12/10 12:53:57 INFO mapred.LocalJobRunner: Records R/W=1/1 > reduce
10/12/10 12:53:57 INFO mapred.TaskRunner: Task 'attempt_local_0001_r_000000_0' done.
10/12/10 12:53:57 INFO streaming.StreamJob: map 100% reduce 100%
10/12/10 12:53:57 INFO streaming.StreamJob: Job complete: job_local_0001
10/12/10 12:53:57 INFO streaming.StreamJob: Output: output

If you get errors where it’s complaining about the output directory, just remove the output directory and try again.

Result

Once you’ve got something similar to the above and no errors, we can check out the result.

user@computer:$ cat output/*

a:3:{s:10:"first_name";s:5:"Hello";s:9:"last_name";s:5:"World";s:11:"middle_name";s:5:"Jason";}

There we go, a serialized form of our modified PHP array $a. That’s all there is to it. Now, go forth and Hadoop.

Downloadable books

2012-11-04T21:45:00.003-08:00

Chris J started a page of links to various books available for download in PDF format. The page contains direct links to the books, without going through a data-harvesting site, as is so often posted here.

http://cfajohnson.com/computers/pdflinks.shtml

"Self-Service Linux®: Mastering the Art of Problem Determination - Free 456 Page eBook"

Get the book directly, without going through a data-harvesting site:http://www.informit.com/content/images/013147751X/downloads/013147751X_book.pdf

HOWTO Set KeepAlive in Apache HTTPd

2012-06-19T20:35:00.001-07:00

To set keep=alive parameters in Apache2 Httpd, you need to pay attention to 3 directives:

KeepAlive

KeepAliveTimeout

MaxKeepAliveRequests

KeepAlive Directive

Description:	Enables HTTP persistent connections
Syntax:	`KeepAlive On\|Off`
Default:	`KeepAlive On`
Context:	server config, virtual host
Status:	Core
Module:	core

The Keep-Alive extension to HTTP/1.0 and the persistent connection feature of HTTP/1.1 provide long-lived HTTP sessions which allow multiple requests to be sent over the same TCP connection. In some cases this has been shown to result in an almost 50% speedup in latency times for HTML documents with many images. To enable Keep-Alive connections, set KeepAlive On.

For HTTP/1.0 clients, Keep-Alive connections will only be used if they are specifically requested by a client. In addition, a Keep-Alive connection with an HTTP/1.0 client can only be used when the length of the content is known in advance. This implies that dynamic content such as CGI output, SSI pages, and server-generated directory listings will generally not use Keep-Alive connections to HTTP/1.0 clients. For HTTP/1.1 clients, persistent connections are the default unless otherwise specified. If the client requests it, chunked encoding will be used in order to send content of unknown length over persistent connections.

When a client uses a Keep-Alive connection it will be counted as a single "request" for the MaxRequestsPerChild directive, regardless of how many requests are sent using the connection.

KeepAliveTimeout Directive

Description:	Amount of time the server will wait for subsequent requests on a persistent connection
Syntax:	`KeepAliveTimeout seconds`
Default:	`KeepAliveTimeout 5`
Context:	server config, virtual host
Status:	Core
Module:	core

The number of seconds Apache will wait for a subsequent request before closing the connection. Once a request has been received, the timeout value specified by the Timeout directive applies.

Setting KeepAliveTimeout to a high value may cause performance problems in heavily loaded servers. The higher the timeout, the more server processes will be kept occupied waiting on connections with idle clients.

In a name-based virtual host context, the value of the first defined virtual host (the default host) in a set of NameVirtualHost will be used. The other values will be ignored.

MaxKeepAliveRequests Directive

Description:	Number of requests allowed on a persistent connection
Syntax:	`MaxKeepAliveRequests number`
Default:	`MaxKeepAliveRequests 100`
Context:	server config, virtual host
Status:	Core
Module:	core

The MaxKeepAliveRequests directive limits the number of requests allowed per connection whenKeepAlive is on. If it is set to 0, unlimited requests will be allowed. We recommend that this setting be kept to a high value for maximum server performance.

For example:

MaxKeepAliveRequests 500

How to Install Apache CouchDB on CentOS 6 (from Source and EPEL)

2012-06-17T22:07:00.000-07:00

CouchDB is an Apache project.

Just like the name suggest it is a database. CouchDB is a NoSQL database. NoSQL databases doesn’t have any schema, tables, etc, that you’ll typically see in a traditional databases like Oracle or MySQL. The data in CouchDB are stored as JSON document, which you can access from a web browser using HTTP.

This article explains how to install CouchDB on RHEL based systems. For example, CentOS, Red Hat, Oracle Enterprise Linux, etc.

There are two methods to install CouchDB. You can install it from EPEL repository, or install from CouchDB source code.

Method 1: Install from EPEL

First, enable EPEL repository as we explained earlier.

Verify that the couchdb is available for yum install.

# yum info couchdb
Name        : couchdb
Arch        : x86_64
Version     : 1.0.3
Release     : 2.el6
Size        : 1.7 M
Repo        : epel
Summary     : A document database server, accessible via a RESTful JSON API

Install couchdb. Depending on your system, this might install lot of dependent packages. On my system, it installed 36 packages in total.

# yum install couchdb

Modify the local.ini file and add a line for bind_address and give the ip-address the system where couchdb is installed.

# vi /etc/couchdb/local.ini
[httpd]
;port = 5984
;bind_address = 127.0.0.1
bind_address = 192.168.101.38

Start the couchdb services

# service couchdb start
Starting couchdb:                [  OK  ]

# service couchdb status
couchdb (pid  29915) is running...

Verify that the couchdb works by going to the URL: http://{your-ip-address}:5984 , it should display a web-page that has the following message.

{“couchdb”:”Welcome”,”version”:”1.0.3″}
Go to: http://{your-ip-address}:5984/_utils/ from where you can create and manage the couchdb database.

Method 2: Install from CouchDB Source Code

If you like to install it yourself from the source code, first you should satisfy all the dependencies.
Install the following standard packages from the CentOS repository.

# yum info gcc libtool xulrunner-devel libicu-devel openssl-devel

Install Erlang

Download the latest version of Erland from here. Or, you can use the wget as shown below to directly download it.

cd /usr/src
wget http://www.erlang.org/download/otp_src_R15B01.tar.gz
tar xvfz otp_src_R15B01.tar.gz

We’ll be installing couchdb and all its dependencies under /opt/couchdb directory. So, while installing erlang, give the prefix as /opt/couchdb/erlang as shown below.

cd otp_src_R15B01
./configure --prefix=/opt/couchdb/erlang --without-termcap --without-javac --enable-smp-support --disable-hipe
make
make install

Install Curl

Download the latest version of Curl from here. Or, you can use the wget as shown below to directly download it.

cd /usr/src
wget http://curl.haxx.se/download/curl-7.26.0.tar.gz
tar xvfz curl-7.25.0.tar.gz

Just like Erlang, we’ll be installing Curl also under /opt/couchdb directory. So, while installing curl, give the prefix as /opt/couchdb/curl as shown below.

cd curl-7.25.0
./configure --prefix=/opt/couchdb/curl
make
make install

SpiderMonkey JS Engine

Download the latest version of SpiderMonkey JS from here. Or, you can use the wget as shown below to directly download it.

cd /usr/src
wget http://ftp.mozilla.org/pub/mozilla.org/js/js185-1.0.0.tar.gz
tar xvfz js185-1.0.0.tar.gz

Please note that you have to cd to “js/src” subdirectory under js-1.8.5 to do the ./configure and make as shown below to install spidermonkey js engine.

cd js-1.8.5/js/src
./configure
make
make install

You’ll see the libmozjs185.so.1.0.0 and libmozjs185-1.0.a installed under /usr/local/lib

# ls -ltr /usr/local/lib
-rwxr-xr-x. 1 root root 3671764 May 30 09:39 libmozjs185.so.1.0.0
-rwxr-xr-x. 1 root root 5523616 May 30 09:39 libmozjs185-1.0.a
lrwxrwxrwx. 1 root root      35 May 30 09:40 libmozjs185.so.1.0 -> /usr/local/lib/libmozjs185.so.1.0.0
lrwxrwxrwx. 1 root root      33 May 30 09:40 libmozjs185.so -> /usr/local/lib/libmozjs185.so.1.0

Note: If spidermonkey JS library is not installed, you’ll get the following error mesage while trying to do the ./configure mentioned in the next step.

checking for JS_NewObject in -lmozjs185... no

configure: error: Could not find the js library.

Is the Mozilla SpiderMonkey library installed?

Install CouchDB

Download the latest version of Couchdb from here. Or, you can use the wget as shown below to directly download it.

cd /usr/src
wget http://apache.mirrors.pair.com/couchdb/releases/1.2.0/apache-couchdb-1.2.0.tar.gz
tar xvfz apache-couchdb-1.2.0.tar.gz

While installing couchdb, you should set ERL, ERLC, CURL_CONFIG environment variables as shown below. These are required during the ./config of couchdb.

Just like the prereqs, we’ll be installing couchdb under /opt/couchdb directory. So, give the prefix as /opt/couchdb/couchdb as shown below.

cd apache-couchdb-1.2.0
export ERL=/opt/couchdb/erlang/bin/erl
export ERLC=/opt/couchdb/erlang/bin/erlc
export CURL_CONFIG=/opt/couchdb/curl/bin/curl-config
export LDFLAGS=-L/opt/couchdb/curl/lib
./configure --prefix=/opt/couchdb/couchdb --with-erlang=/opt/couchdb/erlang/lib/erlang/usr/include/ --enable-js-trunk
make
make install

Note: You’ll use the –enable-js-trunk only if you’ve installed the latest version (anything newer than js185-1.0.0) of the SpiderMonkey JS engine. I recommend that you use –enable-js-trunk option.

If you are not using the latest version of spidermonekey JS engine, you may want to use the flags –with-js-include and –with-js-lib and point it to the appropriate location as shown below.

./configure --prefix=/opt/couchdb/couchdb --with-erlang=/opt/couchdb/erlang/lib/erlang/usr/include/ --with-js-include=/usr/include/xulrunner-2/ --with-js-lib=/usr/lib64/xulrunner-devel-2/lib/

Note: If you’ve installed the latest version of spidermonkey js, and if you are not using –enable-js-trunk, you’ll get the following error message during “make” of couchdb:

cc1: warnings being treated as errors
In file included from couch_js/main.c:20:
couch_js/sm170.c: In function req_status:
couch_js/sm170.c:105: error: implicit declaration of function INT_FITS_IN_JSVAL
couch_js/sm170.c: In function evalcx:
couch_js/sm170.c:138: error: implicit declaration of function JS_GetStringChars
couch_js/sm170.c:138: error: assignment makes pointer from integer without a cast
couch_js/sm170.c: In function seal:
couch_js/sm170.c:220: error: implicit declaration of function JS_SealObject
couch_js/sm170.c: At top level:
couch_js/sm170.c:236: error: initialization from incompatible pointer type

Setup CouchDB Startup Services

Create a couchdb user that is required by the couchdb startup program.

# adduser couchdb

Change the ownership of the var directory, where couchdb will write logs and some other information.

# chown -R couchdb /opt/couchdb/couchdb/var/

Create a link under /etc/init.d for couchdb service

# ln -s /opt/couchdb/couchdb/etc/rc.d/couchdb /etc/init.d/couchdb

Finally start the couchdb service.

# service couchdb start
Starting database server couchdb

Verify that the couchdb works by going to the URL: http://{your-ip-address}:5984 , it should display a web-page that has the following message.

{“couchdb”:”Welcome”,”version”:”1.0.3″}

Go to: http://{your-ip-address}:5984/_utils/ from where you can create and manage the coucbdb database.

Compile Apache (with SSL), PHP 5 and MySQL on Linux

2010-03-28T22:29:00.001-07:00

This article comes with a full reference on how to compile from source latest Apache 2.0.53 server, including support for SSL, PHP 5.0.3 as a module, and MySQL 4.1.10 database on a Linux. It was fully tested under SUSE Linux 9.1, SUSE Linux 9.2 and Fedora Core 3, but shall work with any Linux distribution (only on Debian you will have to change RPMs for proper deb packages).

Today, when almost every Linux distribution comes with a binary form of Apache 2.0.x, PHP 4.3.x and MySQL 4.0.x, it may seem a bit unnecessary to compile these, but, if you want some special configuration, latest components, or simply tune performance of your Apache, PHP and MySQL, compilation from source is the only possibility.
Basic system description:

PHP 5.0.3 will be compiled with support for: bz2, cpdflib, ctype, curllib, dom, ftp, gd2, freetype2, gettext, libiconv, libxml, mbstring, mysql, openssl, pcre, posix, session, SimpleXML, SPL, SQLite, tokenizer, xml, and zlib.

Apache 2.0.53 will be compiled with support for mod_access, mod_auth, mod_auth_digest, mod_deflate, mod_env, mod_headers, mod_setenvif, mod_ssl, mod_mime, mod_imap, mod_alias and mod_rewrite.
Compilation options:

Compilation can be customized by passing several parameters to gcc at runtime, for my Pentium-IV/HT/3.2GHz, this is a good starting set of parameters:

export CFLAGS="-march=pentium4 -mfpmath=sse -msse2 -O2 -pipe -s -fomit-frame-pointer"

You may get a list of gcc compilation options for your CPU at gcc.gnu.org.

All these scripts were fully tested under SESE Linux 9.1 with custom-built kernel 2.6.8.1 and Fedora Core 3 with custom-built kernel 2.6.9.1, and gcc version 3.3.3 / 3.4.2, but they shall work with any Linux distro (only on Debian you may need to change rpm packages for deb ones).

This manual assumes that all source files are located (downloaded to) /usr/local/src directory, SSL keys are placed into /home/ssl directory, and web root is located at /home/www directory.
Compile from source (Open) SSL:

* We will need OpenSSL library: openssl-0.9.7e.tar.gz

Compilation of OpenSSL:

su
cd /usr/local/src
tar -zxvf openssl-0.9.7e.tar.gz
cd openssl-0.9.7e
./config --prefix=/usr/local
make
make install

Create a private key and place it into directory /home/ssl:

mkdir /home/ssl
cd /home/ssl
/usr/local/bin/openssl genrsa -des3 -rand \
some_big_file_1:some_big_file_2 \
-out localhost.key 1024

Next, we will create a private key without a pass-phrase, this is less secure, but it allows us to bootup the server without manually entering the pass-phrase every time…

/usr/local/bin/openssl rsa \
-in localhost.key \
-out localhost.key.unsecure

We will also create a request file that will be emailed to proper certification authority for getting a trusted SSL certificate (if needed) under file localhost.key.csr:

/usr/local/bin/openssl req -new \
-key localhost.key \
-out localhost.key.csr

While waiting for the certification authority, we can create a temporary self-signed certificate, good for 30 days:

/usr/local/bin/openssl x509 -req \
-days 30 \
-in localhost.key.csr \
-signkey localhost.key \
-out localhost.cert
chmod 400 localhost.cert
chmod 400 localhost.key
chmod 400 localhost.key.unsecure

Compile MySQL 4.1.10 database from source:

* MySQL 4.1.10 source codes - mysql-4.1.10.tar.gz
* libtermcap library (most distros have it already) - libtermcap-2.0.8-35.i386.rpm
* libtermcap-devel library (most distros have it already) - libtermcap-devel-2.0.8-35.i386.rpm

MySQL 4.1.10 has a completely different communication protocol and associated PHP mysqli functions. If your scripts were not designed for MySQL 4.1, you shall rather get MySQL release 4.0.23, to stay 100% compatible! Compilation options for MYSQL 4.0.23 will be the same, just remove one line with mysqli from PHP ./configure script.

However for any new development, MySQL 4.1.10 is recommended.

Compiling MySQL from source, and creating user / group called mysql:

cd /usr/local/src
tar -zxvf mysql-4.1.10.tar.gz
cd mysql-4.1.10
./configure \
--prefix=/usr/local/mysql \
--with-unix-sock-path=/tmp/mysql.sock \
--with-charset=utf8
make
make install

groupadd mysql
useradd -g mysql mysql
cp support-files/my-medium.cnf /etc/my.cnf
cd /usr/local/mysql
bin/mysql_install_db --user=mysql
chown -R root .
chown -R mysql var
chgrp -R mysql .

MySQL configuration file /etc/my.cnf can (for our local testing) look like this:

[client]
port = 3306
socket = /tmp/mysql.sock
[mysqld]
port = 3306
socket = /tmp/mysql.sock
skip-locking
key_buffer = 16K
max_allowed_packet = 1M
table_cache = 4
sort_buffer_size = 64K
net_buffer_length = 2K
thread_stack = 64K
skip-networking
server-id = 1
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
no-auto-rehash
[isamchk]
key_buffer = 8M
sort_buffer_size = 8M
[myisamchk]
key_buffer = 8M
sort_buffer_size = 8M
[mysqlhotcopy]
interactive-timeout

Compile from source Apache 2.0.53 web server:

Quite a few web-hosting companies still use Apache 1.3.x, but time of Apache 2.0 incompatibilities and problems is long gone, so 2.0 is a better choice now.

* We will need Apache 2.0.53 source codes - httpd-2.0.53.tar.gz
* RPM: libxml2 library (most distros have it already) - libxml2-devel-2.6.7-28.i586.rpm
* RPM: zlib library (most distros have it already) - zlib-devel-1.2.1-70.i586.rpm
* RPM: readline-devel library (most distros have it already) - readline-devel-4.3-306.i586.rpm

And compile it:

cd /usr/local/src
tar -zxvf httpd-2.0.53.tar.gz
cd httpd-2.0.53
./configure \
--prefix=/usr/local/apache2 \
--enable-so \
--enable-auth-digest \
--enable-rewrite \
--enable-setenvif \
--enable-mime \
--enable-deflate \
--enable-ssl \
--with-ssl=/usr/local \
--enable-headers
make
make install

Next we have to modify (alter) main Apache config file located at /usr/local/apache2/conf/httpd.conf (this also assumes your web root is located at /home/www):

DocumentRoot "/home/www"

And we well add support for PHP 5 (as a module):

LoadModule php5_module modules/libphp5.so
DirectoryIndex index.html index.htm index.php
AddType application/x-httpd-php .php
AddType application/x-httpd-php-source .phps

We also have to allow / create basic mod_rewrite rules:

Options Indexes Includes FollowSymLinks MultiViews
AllowOverride All
Order allow,deny
Allow from all

And dissalow clients to access .htaccess:

Order allow,deny
Deny from all

Next, if using SSL (on standard port 443), we will have to modify file /usr/local/apache2/conf/ssl.conf as follows (just replace the file content with this):

Listen 443

ServerName localhost
SSLEngine on
SSLCertificateFile /home/ssl/localhost.cert
SSLCertificateKeyFile /home/ssl/localhost.key.unsecure

Compile from source PHP 5.0.3:

For compiling PHP, we will need quite a few external libraries, like libcurl, libiconv, libjpeg, libpng, and few others, which we have to download and compile first:

* PHP 5.0.3 itself - php-5.0.3.tar.bz2
* CURL library - curl-7.12.1.tar.gz
* libiconv library - libiconv-1.9.2.tar.gz
* JPEG library: jpegsrc.v6b.tar.gz
* PNG library: libpng-1.2.8.tar.gz
* cpdflib library: clibpdf202r1.tar.gz
* Freetype 2 library: freetype-2.1.9.tar.bz2

Compile libiconv from source:

cd /usr/local/src
tar -zxvf libiconv-1.9.2.tar.gz
cd libiconv-1.9.2
./configure --prefix=/usr/local
make
make install

Compile libjpeg from source:

cd /usr/local/src
tar -zxvf jpegsrc.v6b.tar.gz
cd jpeg-6b
./configure --prefix=/usr/local
make
make install
make install-lib

Compile libpng from source:

cd /usr/local/src
tar -zxvf libpng-1.2.8.tar.gz
cd libpng-1.2.8
cp scripts/makefile.linux makefile
make
make install

Compile cpdflib from source:

cd /usr/local/src
tar -zxvf clibpdf202r1.tar.gz
cd ClibPDF/source
cp Makefile.Linux makefile
make
make install

Compile curl from source:

cd /usr/local/src
tar -zxvf curl-7.12.1.tar.gz
cd curl-7.12.1
./configure --prefix=/usr/local
make
make install

Compile freetype 2 from source:

cd /usr/local/src
tar -jxvf freetype-2.1.9.tar.bz2
cd freetype-2.1.9
./configure --prefix=/usr/local
make
make install

Next, we will compile PHP, with support for MySQL, iconv, curl, zlib, gd2, mbstring, SSL and many other modules:

cd /usr/local/src
tar -jxvf php-5.0.3.tar.bz2
cd php-5.0.3
./configure \
--with-apxs2=/usr/local/apache2/bin/apxs \
--with-mysql=/usr/local/mysql \
--with-mysqli=/usr/local/mysql/bin/mysql_config \
--with-mysql-sock=/tmp/mysql.sock \
--with-sqlite \
--enable-sqlite-utf8 \
--with-zlib \
--with-zlib-dir \
--with-bz2 \
--with-gd \
--enable-gd \
--enable-gd-native-ttf \
--with-jpeg-dir=/usr/local \
--with-png-dir=/usr/local \
--with-ttf \
--with-freetype-dir=/usr/local \
--with-iconv=/usr/local \
--with-curl=/usr/local \
--enable-track-vars \
--with-gettext \
--with-config-file-path=/usr/local/apache2/conf \
--enable-trans-id \
--enable-ftp \
--with-cpdflib=/usr/local \
--enable-mbstring \
--with-openssl=/usr/local
make
make install
cp php.ini-dist /usr/local/apache2/conf/php.ini

Next, we have to modify PHP configuration in file /usr/local/apache2/conf/php.ini, including basic PHP security settings:

mysql.default_socket = /tmp/mysql.sock
short_open_tag = Off
register_globals = Off
allow_url_fopen = Off

How to start Apache and MySQL at bootup?

The last thing left is to create a startup script, whitch will allow to run Apache and MySQL at bootup, automatically, so that we don’t have to do it manually. We will create a new file (for SuSE Linux 9.1, other ditros may vary here) /etc/init.d/web and set “executable” flag to it.

#! /bin/sh
#
# /etc/init.d/web
#
# (c) Radek HULAN
# http://hulan.info/
#
### BEGIN INIT INFO
# Provides: apache-mysql
# Default-Start: 5
# Default-Stop: 5
# Description: Starts Apache2 and MySQL 4
### END INIT INFO

case "$1" in
start)
/usr/local/apache2/bin/apachectl start
/usr/local/mysql/share/mysql/mysql.server start
;;
stop)
/usr/local/apache2/bin/apachectl stop
/usr/local/mysql/share/mysql/mysql.server stop
;;
restart)
/usr/local/apache2/bin/apachectl restart
/usr/local/mysql/share/mysql/mysql.server restart
;;
esac

Next we will run YaST, section “System”, sub-section “Run level editor”, where we will enable service web for runlevel 3 and 5.
Testing the system?

First, start Apache and MySQL servers by entering:

/etc/init.d/web start

Next, create file /home/www/index.php with the following content:

In your browser, type URL http://localhost/ and https://localhost/, and if everything is installed fine, you will see a lot of information about your new Apache/PHP/MySQL installation.
phpMyAdmin:

We will also need phpMyAdmin to manage MySQL database, by entering http://localhost/db/ into our browser:

* phpMyAdmin 2.6.1 - phpMyAdmin-2.6.1.tar.bz2

Installation of phpMyAdmin into /home/www/db:

mkdir /home/www
cd /home/www
tar -jxvf /usr/local/src/phpMyAdmin-2.6.1.tar.bz2
ln -s phpMyAdmin-2.6.1 db

Next, we will configure phpMyAdmin’s advanced feaures, by modifying file /home/www/db/config.inc.php:

// URL to phpMyAdmin
$cfg['PmaAbsoluteUri'] = 'http://localhost/db/';

//connection settings
$cfg['Servers'][$i]['connect_type'] = 'socket';
$cfg['Servers'][$i]['extension'] = 'mysqli';

// user na password
$cfg['Servers'][$i]['auth_type'] = 'config';
$cfg['Servers'][$i]['user'] = 'root';
$cfg['Servers'][$i]['password'] = '';

// PMA settings
$cfg['Servers'][$i]['pmadb'] = 'phpmyadmin';
$cfg['Servers'][$i]['bookmarktable'] = 'pma_bookmark';
$cfg['Servers'][$i]['relation'] = 'pma_relation';
$cfg['Servers'][$i]['table_info'] = 'pma_table_info';
$cfg['Servers'][$i]['table_coords'] = 'pma_table_coords';
$cfg['Servers'][$i]['pdf_pages'] = 'pma_pdf_pages';
$cfg['Servers'][$i]['column_info'] = 'pma_column_info';
$cfg['Servers'][$i]['history'] = 'pma_history';
$cfg['Servers'][$i]['verbose_check'] = FALSE;

// persistent connections
$cfg['PersistentConnections'] = TRUE;

// do not display logo on the left
$cfg['LeftDisplayLogo'] = FALSE;

// show MySQL and PHP info
$cfg['ShowMysqlInfo'] = TRUE;
$cfg['ShowMysqlVars'] = TRUE;
$cfg['ShowPhpInfo'] = TRUE;

// show BLOBs
$cfg['ShowBlob'] = TRUE;

After everything is installed, use phpMyAdmin SQL window to run script /home/www/db/scripts/create_tables_mysql_4_1_2+.sql to create PMA tables, needed by phpMyAdmin.
Debugging PHP:

There are several tools, like PHPeclipse, which allow to debug PHP, in a full-featured IDE. In order to use PHPeclipse, you need to install PHP debugger first.

* Get Nusphere DBG: dbg-2.11.32-src.tar.gz.

Installation:

cd /usr/local/src
tar -zxvf dbg-2.11.32-src.tar.gz
cd dbg-2.11.32
./deferphpize
mkdir /usr/local/modules
cp modules/dbg.so /usr/local/modules
cp modules/dbg.la /usr/local/modules

Next, you will have to modify PHP configuration file located at /usr/local/apache2/conf/php.ini, add here:

; load debugger
extension_dir = "/usr/local/modules"
extension=dbg.so

; debugger configuration
[debugger]
debugger.enabled = true
debugger.profiler_enabled = true
debugger.JIT_host = localhost
debugger.JIT_port = 10001
debugger.JIT_enabled = on

; implicint flush - use only when debugging
implicit_flush = On

Do you need mod_perl as well?

* Get mod_perl-2.0-current.tar.gz.

Installation and compilation:

cd /usr/local/src
tar zxvf mod_perl-2.0-current.tar.gz
cd mod_perl-1.99_16
perl Makefile.PL MP_APXS=/usr/local/apache2/bin/apxs
make
make install

Next, you have to modify Apache configuration file located at /usr/local/apache2/conf/httpd.conf to load mod_perl, and set to use perl at directory /home/www/perl:

LoadModule perl_module modules/mod_perl.so
PerlModule Apache2
Alias /perl/ /home/www/perl/

SetHandler perl-script
PerlResponseHandler ModPerl::Registry
PerlOptions +ParseHeaders
Options +ExecCGI

Testing? Create file /home/www/perl/test.pl, issue chmod 755 test.pl on it, and type http://localhost/perl/test.pl in your browser to test your mod_perl installation.

#!/usr/bin/perl
print "Content-type: text/plain\n\n";
print "mod_perl 2.0 rocks!\n";

Compile Apache (with SSL), PHP 5 and MySQL on Linux

2010-03-28T22:29:00.000-07:00

PHP4 Vs PHP5

2010-03-25T23:11:00.000-07:00

You probably know that PHP 5 is destined for OOP, but it appears that habitual programming can be used too. Moreover, OOP is used in PHP4 as well, with the difference that in PHP5 things are a little more evaluated. This means that in PHP4 safety modes for classes (public, private) are not accepted. In PHP4 the objects are a kind of structures which accept objects and functions as well, according to OOP. In PHP4 they are useful as well.

If you are used to working with PHP4 you can use PHP5 with no problems because the differences are not significant and the changes were made so that programmers would not be confused. An example would be class builders which, in PHP4 were functions within the classes bearing the same name as the class. In PHP5 it is firstly checked if there is a function (method) __construct (). If it does not exist, check if there is a function (method) which has the same name as the class. This means that even if you are not aware of the latest news in the domain of PHP5, your scripts will function without any problem.

If you want to install PHP4 and PHP5 in parallel, you can set USE indicators, which are different for each version and you can find out a lot of other important pieces of information asking us.

PHP5 Vs PHP6

2010-03-25T23:09:00.001-07:00

PHP is actually a set of extensions that are all gathered together in order to make what we currently have. Nevertheless, these extensions can be changed and the same happens to the collection. Let’s imagine, for example the XML Writer extension. It is a very good extension for writing XML files. A similar extension, XML Reader, has already been attached and allowed in the essential distribution in PHP 5.1, and currently XML Writer will use its model in PHP 6, making an excellent couple in order to work easily with XML files.

Another modification in the essential distribution is the exclusion of the ereg frequent expressions library that is going to become an extension. ereg is now used as a competitor of PCRE (preg_match, etc.), but it seems to cause some problems. Consequently, the experts have made the decision to take it out of the core and change it into an extension.

Now, media type detection is not quite good in PHP. There is mime_magic extension, which is not quite useful. Therefore in PHP 6, the Fileinfo extension will replace mime_magic and it will be integrated in the core while mime_magic will be taken out of the core and changed into extension.

These changes seem to be really interesting. You can conclude that PHP 6 is not really a huge characteristics upgrade, but it is more a great cleanup based on optimizing a lot of functions which already exist, on the move. There are professionals working with PHP almost every day and they can notice improvements every moment.

Symmetric vs. asymmetric algorithms

2010-01-11T04:15:00.001-08:00

When using symmetric algorithms, both parties share the same key for en- and decryption. To provide privacy, this key needs to be kept secret. Once somebody else gets to know the key, it is not safe any more. Symmetric algorithms have the advantage of not consuming too much computing power. A few well-known examples are: DES, Triple-DES (3DES), IDEA, CAST5, BLOWFISH, TWOFISH.

Asymmetric algorithms use pairs of keys. One is used for encryption and the other one for decryption. The decryption key is typically kept secretly, therefore called ``private key'' or ``secret key'', while the encryption key is spread to all who might want to send encrypted messages, therefore called ``public key''. Everybody having the public key is able to send encrypted messages to the owner of the secret key. The secret key can't be reconstructed from the public key. The idea of asymmetric algorithms was first published 1976 by Diffie and Hellmann.

Asymmetric algorithms seem to be ideally suited for real-world use: As the secret key does not have to be shared, the risk of getting known is much smaller. Every user only needs to keep one secret key in secrecy and a collection of public keys, that only need to be protected against being changed. With symmetric keys, every pair of users would need to have an own shared secret key. Well-known asymmetric algorithms are RSA, DSA, ELGAMAL.

However, asymmetric algorithms are much slower than symmetric ones. Therefore, in many applications, a combination of both is being used. The asymmetric keys are used for authentication and after this has been successfully done, one or more symmetric keys are generated and exchanged using the asymmetric encryption. This way the advantages of both algorithms can be used. Typical examples of this procedure are the RSA/IDEA combination of PGP2 or the DSA/BLOWFISH used by GnuPG.

in reference to:

"Symmetric vs. asymmetric algorithms When using symmetric algorithms, both parties share the same key for en- and decryption. To provide privacy, this key needs to be kept secret. Once somebody else gets to know the key, it is not safe any more. Symmetric algorithms have the advantage of not consuming too much computing power. A few well-known examples are: DES, Triple-DES (3DES), IDEA, CAST5, BLOWFISH, TWOFISH. Asymmetric algorithms use pairs of keys. One is used for encryption and the other one for decryption. The decryption key is typically kept secretly, therefore called ``private key'' or ``secret key'', while the encryption key is spread to all who might want to send encrypted messages, therefore called ``public key''. Everybody having the public key is able to send encrypted messages to the owner of the secret key. The secret key can't be reconstructed from the public key. The idea of asymmetric algorithms was first published 1976 by Diffie and Hellmann. Asymmetric algorithms seem to be ideally suited for real-world use: As the secret key does not have to be shared, the risk of getting known is much smaller. Every user only needs to keep one secret key in secrecy and a collection of public keys, that only need to be protected against being changed. With symmetric keys, every pair of users would need to have an own shared secret key. Well-known asymmetric algorithms are RSA, DSA, ELGAMAL. However, asymmetric algorithms are much slower than symmetric ones. Therefore, in many applications, a combination of both is being used. The asymmetric keys are used for authentication and after this has been successfully done, one or more symmetric keys are generated and exchanged using the asymmetric encryption. This way the advantages of both algorithms can be used. Typical examples of this procedure are the RSA/IDEA combination of PGP2 or the DSA/BLOWFISH used by GnuPG."
- Symmetric vs. asymmetric algorithms (view on Google Sidewiki)

2008-03-13T01:03:00.000-07:00

An Introduction to Microsoft SharePoint Portal Server

2007-07-11T08:24:00.000-07:00

As the volume of intranet content grows, administrators and users are increasingly looking for ways to manage the volume of information. Just as Netscape aggregated the then-growing internet content into a manageable framework (called a portal), application vendors have created many packages that allow their customers to aggregate their intranet content into corporate portals.

This series of articles will discuss Microsoft's portal product — SharePoint Portal Server. I hope to provide some value to the readers of Intranet Journal regarding the capabilities (and limitations) of SharePoint. Your feedback is encouraged, and will help me to cover topics that you find relevant.

Introduction to SharePoint

Like most portal products, SharePoint allows visitors to create custom views of the Web site. This customization can be very simple or very complex, depending on the type and version of the product that is installed. In addition, SharePoint has a powerful collaboration model that is tightly integrated with the Office 2003 suite.

The current version of SharePoint is its third (despite its name). Like most Microsoft software, (Windows in particular) version 3 is gaining rapid adoption. A recent article in The Register indicated that SharePoint Portal Server is Microsoft's fastest growing product, with 30 million licenses.

History of SharePoint

Microsoft's first portal application was called Digital Dashboard. This product introduced the concept of Web parts — boxes of information on a page that represented a summary or overview of information. (Other vendors referred to these items as "portlets.") By assembling multiple parts on a page, each user could customize his view of the portal to contain the information that pertained to them. In theory, every visitor of the site could have different content at the same URL. However, the technology behind the Digital Dashboard was not up to the task, and it never made it out of the beta stage.

At the same time, Microsoft's Office group was working toward a collaboration solution. The need for many people to contribute to a single document or worksheet was growing. And, these people were not necessarily working at the same location. The result was SharePoint Team Services (STS), a Web-based solution that allowed shared access to information and documents. STS also allowed end-users to make changes to the site via a Web browser instead of requiring a development-oriented application.

The merging of the collaboration and aggregation functions lead to SharePoint Portal Server 2001. Portal Server has been upgraded to run on the .Net framework and is now referred to as SharePoint Products and Technologies. The "Product" is SharePoint Portal Server 2003 (SPS) and the "Technologies" are Windows SharePoint Services (WSS). A significant point about these two is that WSS is included with the Windows Server 2003 license. Any organization that is licensed for Windows Server 2003 can also host Websites that are based on WSS.

Features of SharePoint

In general, SharePoint contains all of the features you would expect from a portal or collaboration tool:

Browser-based customization of page

Browser-based content administration

Aggregation capabilities

Document Repository

Message board

Ad-hoc data storage

E-mail notifications

Announcements, event calendar and contact list.

A complete feature matrix that also indicates whether the feature is part of WSS or SPS can be found in the whitepaper Implementing Rich Collaboration Infrastructure Using Windows SharePoint Services and SharePoint Portal Server 2003 on Microsoft's Web site (http://www.microsoft.com/sharepoint/evaluationoverview.asp).

Uses for SharePoint

In addition to being the default intranet portal, there are many additional uses for SharePoint. Microsoft has included pre-defined templates for Web sites to facilitate meetings, manage projects and create documents. Combined with the "self-service" site creating feature, teams can create and use a Web site with minimal assistance from the Information Technology department.

The Document Workspace template will allow a group to work on a document. The template combines a document repository with a task list and a links list. While using Word 2003, a user can have the document open and at the same time view the task or links list. Changes made to the task list are immediately visible to site visitors. And the document library allows versioning, so edits are not lost.

The Meeting Workspace template combines the agenda, attendee list and Outlook's calendar function. When sending a meeting request in Outlook 2003, a user can create a workspace on the server. The URL of the workspace is automatically included in the message and the attendees are added to the site. The materials required for the meeting can be centrally located, which is preferable for attendees who travel frequently.

The browser-based customization feature, combined with the ability to create ad-hoc lists allows an advanced user to create a site for almost any purpose. A user group could host its meeting schedule, complete handouts. A youth sport organization can post its schedule and roster. An individual can host a blog. The possibilities are endless.

About this series
This series of articles on SharePoint is intended to help you understand the capabilities of the product, as well as provide tips and tricks, development ideas, information from Microsoft, information from the community, and perhaps some samples. Like many other series on IntranetJournal.com, I plan to include how-to articles that can help you with your deployments — ways to customize a page; deployment scenarios; content management; etc. With such a diverse product, there is no lack of topics for this series of articles. What would you like to read?

About the author
Paul Schaeflein is a developer with over twenty years experience. Paul has been developing dynamic and interactive Web sites since 1996. Paul has worked on all of the versions of SharePoint and has worked with the .Net framework since its debut. You can reach Paul through his blog at http://www.schaeflein.net/blog/.