Worse still for the PHP programmer is that every instance of queries posted to forums and support websites are from non-PHP programmers – .Net seems to be the most common, and there are also questions from Java and Delphi programmers.

One thing all of these developers have in common though is the need to extract information from these troublesome OLE fields in order to migrate from a legacy database. But not one article specifically targets for PHP programmers. Nor do any of them address “packages”, only specific data types such as JPEG, PNG, GIF, etc.

In this article we’ll see how PHP can be used to extract objects from two OLE types: packages and Acrobat PDF documents. This is the first part of a two-part series in which we’ll look at OLE packages, which, as we’ll see, can be identified as either “Package” or “Packager Shell Object”. Values expressed in this article are based on an Access 2000 database.

OLE Container

The storage of a blob (binary large object) in a database is never a simple matter, and Microsoft’s Access database is no exception. Let’s summarize some key aspects of the OLE container used when an object is inserted to an OLE field in Access:

• No external file would have been inserted to an Access OLE field without first being wrapped in a container. For our purposes, this container can be considered to be a header in front of the object we are interested in and a trailer after it.
• The header length is not fixed. Even for the OLE package type the header length can be different depending on the object that is embedded in the container.
• The data is not in a human-readable form, making it harder to visually find crucial items such as the original name of the embedded object, or its name, or the start of the object component.
• The trailer can be ignored. Our focus here will be on the header and object components.

One other point: it’s not just the object component that’s encoded, but the header is too. Before we can make any sense of any of the data, the header needs to be decoded first.

Encoded Header and Data

The test database is defined as simply as possible, with the embedded files contained in the image field of Table1:

Throughout this worked example, we’ll be using an Apache logo GIF that I stored in the OLE field. Extracting the field contents without decoding, then inspecting the output in a hexadecimal viewer reveals, well, gibberish:

After converting the extracted header from hexadecimal to decimal encoding using PHP’s hex2dec() function we have something more useful:

It looks uninviting, but it’s clear that we can now make some sense of the data. To help simplify matters the first 20 bytes can be ignored.

The next block of characters tells us that the OLE object is of type Packager Shell Object. An OLE package is a general purpose container for those file types not recognized by the database when the files were inserted.

In the words of Professor Farnsworth, “Good news, everyone!” If we look closer at the ASCII column of the hex dump we can see the name of the original file that is embedded in the container – apache_02.gif. This will be useful later when we try to locate the end of the header and the start of the data that we need to extract. Even better, the filename always starts at byte 84 of a Packager Shell Object header and at byte 70 of a Package header.

Before moving on to how we extract the GIF from the package, here’s a little teaser: In both of these two hex dumps, there’s the sequence 0A0600 underlined in yellow. For now, accept that 0A0600 is important. We’ll see why in a moment.

Extracting the Object

Before we can extract the embedded object, we need to know where it starts and ends in the data. Careful inspection of the hexadecimal representation reveals that the character sequence for the filename occurs three times – once at byte 84 (or 70), then again twice as part of the full path.

At least, that how it seems in this example. Just to complicate matters, it’s possible that the full path may appear only once. Also, Access may have changed the case of the file path, or changed each level of the file path to old-fashioned 8:3 format (meaning that names longer than 8 character will be truncated, and characters 7 and 8 replaced by ~1).Fortunately there will always be an instance of the full path somewhere around the location of the second full path in this example. “Somewhere around the location?” Just as the header length is not fixed, so too are the locations of the full path. We need to work around these niggles.

“More good news, everyone!” The final instance of the filename is the second last important item of interest in the header. It is null-terminated (00), after which is the hexadecimal sequence 0A0600 we saw earlier. This the original size in bytes of the embedded object. It’s stored in little-endian format, so this needs to be reversed to big-endian before we can use it: 0A0600 becomes 00060A, which means 1,546 bytes. This group of three bytes gives a maximum allowable embedded file size of 16 MB.

After this sequence is the object we need. It starts with the character sequence 474946383961. Converting this to ASCII yields “GIF89a”. It doesn’t show in the ASCII column because Access does not store data in the OLE header in consistent two-character blocks. That’s why the first instance of the full path displays nicely up to “\Documents\Wri” then goes awry. In this example, the character after “Wri” ought to be “t” as in “Writing” but Access inserted a null character into the sequence. Why? Yeah, Microsoft, why! Note that “ting” (74646E67) immediately follows the null. Another niggle to code around.

So let’s recap. What do we have now? We have the original name of the embedded file, the size in bytes of the original file, and we have the start location of the data we need to extract. We thus have all we need to extract the embedded file from a legacy Access database, and then save it to disc using its original name and extension.

Putting Theory into Practice

What follows is the PHP I used to extract from my test database the object details we discussed above. It’s no-frills, procedural PHP which should be easy for anyone to understand the steps involved in extracting an object from an OLE package.

You’ll notice that some of the logic has multiplied certain numbers by a factor of two. For example, the offset for the embedded file name is defined as 168 rather than the 84 bytes mentioned above. That’s because some parts of the code work on the raw data from the Access database’s OLE field, which is encoded in hexadecimal format. That is, each single byte is stored as two hexadecimal characters.

<?php
if (!function_exists("hex2bin")) {
   function hex2bin($hexStr) {
      $hexStrLen = strlen($hexStr);
      $binStr = "";
      $i = 0;
      while ($i < $hexStrLen) {
         $a = substr($hexStr, $i, 2);
         $c = pack("H*", $a);
         $binStr .= $c;
         $i += 2;
      }
      return $binStr;
   }
}

$dbName = "db1.mdb";
$db = new PDO("odbc:DRIVER={Microsoft Access Driver (*.mdb)}; DBQ=$dbName; Uid=; Pwd=;");

$sql = "SELECT * FROM Table1 WHERE id = 2";
foreach ($db->query($sql) as $row) {
   $objName = "";

   switch (getOLEType($row["image"])) {
      case "Packager Shell Object":
          list($objName, $objData) = extractPackagerShellObject($row["image"]);
          break;
      case "Package":
          list($objName, $objData) = extractPackage($row["image"]);
          break;
      default:
          throw new Exception("Unknown OLE type");
   }
   if ($objName != "") {
      file_put_contents($objName, $objData);
   }
}

function flipEndian($data, $size) {
   $str = "";
   for ($i = $size - 2; $i >= 0; $i -= 2) {
      $str .= substr($data, $i, 2);
   }
   return $str;
}

function findNullPos($str) {
   // must start on a two-character boundary
   return floor((strpos($str, "00") + 1) / 2) * 2;
}

function getOLEType($data) {
   // fixed position of OLE type
   $offset = 40;

   $tmp = substr($data, $offset, 255);
   $nullPos = findNullPos($tmp);
   $tmp = substr($tmp, 0, $nullPos);
   $type = hex2bin($tmp);

   return $type;
}

function extractPackagerShellObject($data) {
   $headerBlock = 500; // usable header size
   $offset = 168; // location of name

   // find name
   $tmp = substr($data, $offset, 255);
   $nullPos = findNullPos($tmp);
   $name = substr($tmp, 0, $nullPos);
   $pos = $offset + strlen($name);

   // find data
   $path1 = strpos($data, $name, $pos); // 1st full path
   $pos = $path1 + strlen($name);
   $path2 = strpos($data, $name, $pos); // 2nd full path
   // check if only one full path
   if ($path2 > $pos) {   
      $pos = $path2 + strlen($name);
   }
   $oleSizePos = $pos + 2;
   $oleObjSize = flipEndian(substr($data, $oleSizePos, 8), 8);
   $oleHeaderEnd = $oleSizePos + 8;
   $objName = hex2bin(substr($tmp, 0, $nullPos));

   // extract object
   $data = substr($data, $oleHeaderEnd, hexdec($oleObjSize) * 2);
   $objData = hex2bin($data);

   return array($objName, $objData);
}

function extractPackage($data) {
   $headerBlock = 500; // usable header size
   $offset = 140; // location of name

   // find name
   $tmp = substr($data, $offset, 255);
   $nullPos = findNullPos($tmp);
   $name = substr($tmp, 0, $nullPos);
   $pos = $offset + strlen($name);

   // find data
   $path1 = strpos($data, $name, $pos); // 1st full path
   $pos = $path1 + strlen($name);
   $path2 = strpos($data, $name, $pos); // 2nd full path
   // check if only one full path
   if ($path2 > $pos) {   
      $pos = $path2 + strlen($name);
   }
   $oleSizePos = $pos + 2;
   $oleObjSize = flipEndian(substr($data, $oleSizePos, 8), 8);
   $oleHeaderEnd = $oleSizePos + 8;
   $objName = hex2bin(substr($tmp, 0, $nullPos));

   // extract object
   $data = substr($data, $oleHeaderEnd, hexdec($oleObjSize) * 2);
   $objData = hex2bin($data);

   return array($objName, $objData);
}

Note that the bespoke hex2bin() function is only required if your PHP installation does not have its own (it was added in PHP 5.4). Also, the switch statement makes the logic extensible, as we’ll discover in the next part of this article.

The two package functions look very similar. These will be revisited in the next part of this article, when the code will be refactored because of the similar nature of the logic required for these and other object types.

I have tested this PHP with the following file types: BMP, GIF, JPEG, PNG, DOC, XLS, and PPT, all of which were stored as packages. The logic presented above should allow any file stored in an OLE package to be rescued from a legacy database, not just the GIF used in this example.

Summary

In this article we have covered the essential elements of extracting package objects from OLE fields in a Microsoft Access database using PHP. Having learned the basic structure of an OLE object, and how to extract the file contained therein, the method presented above can be applied to other OLE types to ensure that as much data as possible can be retrieved from a legacy Access database.

In the second part of this series we’ll look at the more complex issue of extracting known object types, focusing in particular on extracting Acrobat PDF documents from a legacy Access database.

Image via Fotolia

Build the First Module

Let’s get to the exciting part – creating a module; we’ll start with a simple customer editing page. On this page users can create, update, and delete customers.

First let’s make a database table customer. The SQL to create the table is:

CREATE TABLE customer (
  id INTEGER NOT NULL AUTO_INCREMENT,
  name VARCHAR(64) NOT NULL,
  description VARCHAR(255) DEFAULT NULL,
  address VARCHAR(255) DEFAULT NULL,
  phone VARCHAR(20) DEFAULT NULL,
  fax VARCHAR(20) DEFAULT NULL,
  status INTEGER DEFAULT NULL,
  create_by INTEGER DEFAULT '1',
  create_time DATETIME DEFAULT NULL,
  update_by INTEGER DEFAULT '1',
  update_time DATETIME DEFAULT NULL,
  PRIMARY KEY (id)
)
ENGINE=InnoDB

The next step to creating a module is to create the XML metadata files for the customer editing page. Here we use gen_meta, the metadata generation command under the cubi/bin/tools directory.

/dev/cubi/bin/tools$ php gen_meta.php Default customer

You can simply press the Enter key to complete the wizard. After the command executes, the following files are generated:

Module configuration file

• modules/customer/mod.xml

Module DO file

• modules/customer/do/CustomerDO.xml

Module Form file

• modules/customer/form/CustomerListForm.xml
• modules/customer/form/CustomerDetailForm.xml
• modules/customer/form/CustomerEditForm.xml
• modules/customer/form/CustomerNewForm.xml
• modules/customer/form/CustomerCopyForm.xml

Module View file

• modules/customer/view/CustomerListView.xml

Module Widget files

• modules/customer/widget/DashboardForm.xml
• modules/customer/widget/LeftMenu.xml

Module Template files

• modules/customer/template/detail.tpl
• modules/customer/template/detail_elementset.tpl
• modules/customer/template/grid.tpl
• modules/customer/template/view.tpl

Next we load the newly created module with the load_module command.

/dev/cubi/bin/tools/php$ load_module.php customer
Start loading customer module ...
--------------------------------------------------------
[2013-01-26T17:57:16+08:00] Loading module customer
[2013-01-26T17:57:16+08:00] Install Module customer
[2013-01-26T17:57:16+08:00] Install Module ACL.
[2013-01-26T17:57:16+08:00] Install Module Menu.
[2013-01-26T17:57:16+08:00] Install Module Widget.
[2013-01-26T17:57:16+08:00] Install Module Resource.
[2013-01-26T17:57:16+08:00] Install Module Change Logs.
[2013-01-26T17:57:16+08:00] Copy resource files to /cubi/resources folder.
[2013-01-26T17:57:16+08:00] customer is loaded.

Give admin to access all actions of module 'customer'
--------------------------------------------------------
End loading customer module

Let’s test the module. If you are already logged in to Cubi, log out and log back in again. You should see a new tab named “Customer” appearing in the header section. Click the tab to enter the Customer Dashboard page, then click the Customer Manage link to enter the customer management page. Now we’re able to create, update, search, and delete customers.

Under the Hood

You may wonder without any programming, how does Openbiz Cubi make everything work? In the previous steps, gen_meta created XML files under the customer module folder. Cubi then knows how to interpret these files. Let’s visit the module description file mod.xml first.

Module Description File

We created the customer module with mod.xml under the modules/customer directory.

<?xml version="1.0" standalone="no"?>
<Module Name="customer" Description="customer module" Version="0.1.0" OpenbizVersion="3.0">
 <ACL>
  <Resource Name="customer">
   <Action Name="Access" Description="Access Customer Module Dashboard"/>
  </Resource>
  <Resource Name="customer">
   <Action Name="Access" Description="Access Customer"/>
   <Action Name="Manage" Description="Manage Customer"/>
  </Resource>
 </ACL>
 <Menu>
  <MenuItem Name="CustomerTop" Title="Customer" Description="Customer Description" URL="/customer/dashboard" Parent="" Order="10">
   <MenuItem Name="Customer" Title="Customer" Description="Customer description" URL="" Order="10">
    <MenuItem Name="Customer.List" Title="Customer Manage" Description=""  URL="/customer/customer_list" Order="10"/>
   </MenuItem>  
  </MenuItem>  
 </Menu>
 <Dependency>
  <Module Name="system"/>
 </Dependency>
</Module>

mod.xml contains 3 sections:

• Access control – the ACL section defines the resources and their actions. These definitions are used to control permissions for given roles.
• Menu – the Menu section defines the page links in the navigation system (application tabs, breadcrumb, and menus).
• Dependencies – the Dependency section defines modules that the current module depends on. The depended modules should be installed first.

Usually each module includes its own data models and presentation XML files. The framework has a metadata engine that can then understand the XML and load the data and UI objects on the fly. The framework mainly work with two types of metadata objects:

• Data objects – Cubi maps physical data stores (such as a database table) to logic objects.
• Form and view objects – Form objects describe how to present data objects’ data on a block in a page, while view objects define a container of Form objects. In a browser, a view is the same as a web page.

Data Object

Now let’s take a look at the data object XML in modules/customer/do/CustomerDO.xml:

<?xml version="1.0" standalone="no"?>
<BizDataObj Name="CustomerDO" Description="" Class="BizDataObj" DBName="Default" Table="customer" SearchRule="" SortRule="" OtherSQLRule="" Uniqueness="" Stateless="N" IdGeneration="Identity" CacheLifeTime="0" CreateCondition="customer.Manage" UpdateCondition="customer.Manage" DeleteCondition="customer.Manage">
 <BizFieldList>
  <BizField Name="Id" Column="id" Type="Number"/>
  <BizField Name="name" Column="name" Length="64" Required="Y" Type="Text"/>
  <BizField Name="description" Column="description" Length="255" Required="N" Type="Text"/>
  <BizField Name="address" Column="address" Length="255" Required="N" Type="Text"/>
  <BizField Name="phone" Column="phone" Length="20" Required="N" Type="Text"/>
  <BizField Name="fax" Column="fax" Length="20"   Required="N" Type="Text"/>
  <BizField Name="status" Column="status"    Required="N" Type="Number"/>
  <BizField Name="create_by" Column="create_by" Type="Number" ValueOnCreate="{@profile:Id}"/>
  <BizField Name="create_time" Column="create_time"  Type="Datetime" ValueOnCreate="{date('Y-m-d H:i:s')}"/>
  <BizField Name="update_by" Column="update_by" Type="Number" ValueOnCreate="{@profile:Id}" ValueOnUpdate="{@profile:Id}"/>		
  <BizField Name="update_time" Column="update_time" Type="Datetime" ValueOnCreate="{date('Y-m-d H:i:s')}" ValueOnUpdate="{date('Y-m-d H:i:s')}"/>
 </BizFieldList>
 <TableJoins>
 </TableJoins>
 <ObjReferences>
 </ObjReferences>
</BizDataObj>

The XML defines a mapping between the customer table to the CustomerDO object. It also defines certain rules on the objects as well as validation, type, and value of each field.

Form Object

Let’s also take a look at the form object XML in modules/customer/form/CustomerListForm.xml.

<?xml version="1.0" encoding="UTF-8"?>
<EasyForm Name="CustomerListForm" Class="EasyForm" FormType="List" jsClass="jbForm" Title="Customer Management" Description="" BizDataObj="customer.do.CustomerDO" PageSize="10" DefaultForm="Y" TemplateEngine="Smarty" TemplateFile="grid.tpl" Access="customer.Access">
 <DataPanel>
  <Element Name="row_selections" Class="RowCheckbox" Label="" FieldName="Id"/>
  <Element Name="fld_Id" Class="ColumnText" FieldName="Id" Label="Id" Sortable="Y"/>
  <Element Name="fld_name" Class="ColumnText" FieldName="name" Label="Name" DefaultValue="New Customer" Sortable="Y" Link="javascript:">
  <EventHandler Name="fld_name_onclick" Event="onclick" Function="SwitchForm(customer.form.CustomerDetailForm,{@:Elem[fld_Id].Value})"   />
 </Element>
 <Element Name="fld_description" Class="ColumnText" FieldName="description" Label="Description"  Sortable="Y"/>
  <Element Name="fld_address" Class="ColumnText" FieldName="address" Label="Address"  Sortable="Y"/>
  <Element Name="fld_phone" Class="ColumnText" FieldName="phone" Label="Phone" Sortable="Y"/>
  <Element Name="fld_fax" Class="ColumnText" FieldName="fax" Label="Fax" Sortable="Y"/>
  <Element Name="fld_status" Class="ColumnText" FieldName="status" Label="Status" Sortable="Y"/>
 </DataPanel>
 <ActionPanel>
...
  <Element Name="btn_delete" Class="Button" Text="Delete" CssClass="button_gray_m" Access="customer.Manage">
   <EventHandler Name="del_onclick" Event="onclick" EventLogMsg="" Function="DeleteRecord()" ShortcutKey="Ctrl+Delete" ContextMenu="Delete"/>
  </Element>
...
 </ActionPanel>
 <NavPane>
...
 </NavPanel>
 <SearchPane>
...
 </SearchPanel>
</EasyForm>

The XML maps fields from data objects to UI elements. It also defines panels for the logical layout of elements. For each UI element, interaction behavior can be described within its EventHandler section.

Customize Objects

Each Metadata element comes with a Class attribute. In the metadata generated by gen_meta, the attribute is set to a core framework class (BizDataObj for data objects, EasyForm for form objects). Developers can simply replace this to implement special business logic. For example, to add special logic for deleting a customer record, you can create the file /modules/customer/form/CustomerForm.php, and set the Metadata element’s Class attribute to it. The PHP class might look like:

<?php
class CustomerForm extends EasyForm
{
    public function deleteRecord($id = null) {
        // add your logic here
        Parent::deleteRecord($id); // call parent deleteRecord method
    }
}

The Cubi Execution Flow

So what happens when the http://host/cubi/index.php/customer/customer_list is called in the browser?

1. index.php calls _forward.php to parse the URL. It understands that /customer/customer_list points to cubi/modules/customer/view/CustomerListView.xml based on Cubi’s URL naming convention.
2. The render method of the view is invoked.
3. The render method calls the render method of its included form (cubi/modules/customer/form/CustomerListForm.xml).
4. The render method of the form calls its data object’s data query methods.
5. The query methods prepare the SQL and executes it against the database.
6. The form uses the returned dataset from the DO to generate HTML with its template.
7. The view uses the output of the forms to generate HTML with its template.
8. The server sends the HTML output to browser.

Once a view is loaded in the browser, most user interactions happen on forms through Ajax (no page reload). The diagram below shows how the Ajax call works:

Conclusion

In the series we’ve learned how to install Cubi, set up the system, and create a module. You may also find that the XML metadata file can be easier to learn and maintain than programming code. I hope you’ve enjoyed the series and wish you luck with your very own Openbiz Cubi powered business applications.

Image via Fotolia

Digest Access was originally defined in RFC 2069, and optional security enhancements were later added in RFC 2617 which should be considered the current standard if you wish to implement this method yourself. We’ll have a look at both in this article.

The Problem with Basic Access Authentication

First, lets look a Basic Access Authentication. With your browser, you request some sort of object that requires you to authenticate yourself. If the browser hasn’t cached credentials you’ve already provided, you’ll be met with a “HTTP 401 Not Authorized” response which includes the following header:

WWW-Authenticate: Basic realm="example.com"

The client/browser is then expected to respond with the appropriate credentials in order to be allowed access.

The username and password are concatenated together, like “admin:p@ssw0rd”, This string is then Base64 encoded. The encoded string (YWRtaW46cEBzc3cwcmQ=) is used in an Authorization header which is sent back to the server.

Authorization: Basic YWRtaW46cEBzc3cwcmQ=

When the server receives this request, it notices the Authorization header and applies the steps in reverse (decode and split the resulting string) to get the original username and password. The process can then do whatever lookup/verification is necessary with the credentials.

The major problem here is that Base64 is a data transfer scheme; it’s not a hashing or encryption scheme. It may as well be clear text. Digest Access Authentication tries to improve upon this by sending the password as a hashed value, which is much harder (not impossible) to reverse engineer a clear text value from.

Working with Digest Access Authentication

When working with Digest Access Authentication, it’s the server who must make the first move when it discovers a client is trying to access a restricted area. There are a few values the server needs to provide as part of the WWW-Authenticate header that the client needs.

The server generates a value, referred to as nonce, which should be unique for each request.
It’s recommended this value be either Base64 or hexadecimal, specifically because it will be enclosed in double quotes and we want to ensure there are no double quotes in the string. The nonce will be used by the client to generate a hash to send back to the server.

In PHP, applying md5() to a unique string is generally sufficient.

<?php
$nonce = md5(uniqid());

The next value needed by the client is opaque. This is another unique string generated by the server and is expected to be sent and returned by the client unaltered.

<?php
$opaque = md5(uniqid());

The last value, realm, is just a string to display to users so they know which username and password they should provide. It’s also used by the client to generate a hash to send back to the server.

<?php
$realm = 'Authorized users of example.com';

All of these values are used to compose the WWW-Authenticate directive and send as a response to the client.

<?php
if (empty($_SERVER['PHP_AUTH_DIGEST']) {
    header('HTTP/1.1 401 Unauthorized');
    header(sprintf('WWW-Authenticate: Digest realm="%s", nonce="%s", opaque="%s"', $realm, $nonce, $opaque));
    header('Content-Type: text/html');
    echo '<p>You need to authenticate.</p>';
    exit;
}

When the client receives this response, it has to compute a return hash. It does so by concatenating the username, realm, and password and hashing the result with MD5 as follows:

1. Compute A1 as MD5("username:realm:password").
2. Compute A2 as MD5("requestMethod:requestURI").
3. Compute the final hash, know as “response”, as MD5("A1:nonce:A2").

The client sends the response back to the server in an Authorization header and includes the username, realm, nonce, opaque, uri, and the computed response.

Authorization: Digest username="%s", realm="%s", nonce="%s", opaque="%s", uri="%s", response="%s"'

Note that realm, nonce, and opaque are all returned to the server unchanged.

When the server receives the response, the same steps are taken to compute the server’s version of the hash. If the computed hash and the received response hash values match, then the request is considered authorized. It looks something like this:

<?php
$A1 = md5("$username:$realm:$password");
$A2 = md5($_SERVER['REQUEST_METHOD'] . ":$uri");
$response = md5("$A1:$nonce:$A2");

Of course, the user’s password is never passed to the server. To perform authentication look ups from a database then, the value of A1 can be stored. Keep in mind however the stored hash becomes invalid if you ever change your realm.

Improving on the Original Digest Access Spec

Now that you’re familiar with the workings of Digest Access Authentication from RFC 2069, let’s turn our attention to some of the enhancements that were added in 2617: qop, nc, and cnonce.

qop, or quality of protection, is specified in the WWW-Authenticate header and can have a value of “auth” or “auth-int”. When no qop directive is found or when it is set to “auth”, Digest Access is used for client authentication only – the default mode which you’ve seen so far. When set to “auth-int”, an attempt is made to provide some level of integrity protection of the response as well and the client must also include the request body as part of the message digest. This allows the server to determine whether the request has been adulterated in transfer between the indented client and intended server.

The client nonce, or cnonce, is similar to nonce but is generated by the client. The cnonce figures into the response digest computed by the client and its original value is passed along to the server so that it can be used there to compare digests. This provides some response integrity and mutual authentication, in that both the client and server have a way to prove they know a shared secret. When the qop directive is sent by the server, the client must include a cnonce value.

The nonce count, nc, is a hexadecimal count of the number of requests that the client has sent with a given nonce value. This way, the server can guard against replay attacks.

With the enhancements, the computation of A2 and the response goes something like this:

<?php
if ($qop == 'auth-int') {
    $A2 = md5($_SERVER['REQUEST_METHOD'] . ":$uri:" . md5($respBody));
}
else {
    $A2 = md5($_SERVER['REQUEST_METHOD'] . ":$uri");
}
$response = md5("$A1:$nonce:$nc:$cnonce:$qop:$A2");

Strengths and Weaknesses of Digest Access Authentication

Digest Access has some advantages over Basic Authentication, since Basic Authentication uses a clear-text exchange of username and passwords, which is almost the same as telling the world what your password is. Digest Access passes a hashed value and not the password itself, so it’s much more secure than Basic Auth. The server nonce, which should be unique per request, will drastically change the computed hash on each new request, and the nc value provided by RFC 2617 is helpful at preventing replay attacks, whereby a malicious individual is able to intercept your request data and “replay” or repeat it as his own request.

There are a few weaknesses with Digest Authentication as well. When RFC 2617 replaced the original specification, the enhancements that were added to provide extra security measures between the client and server were made purely optional, and Digest Authentication will proceed in its original RFC 2069 form when not implemented.

Another problem is MD5 is not a strong hashing algorithm. All it takes is time and CPU to brute force the original value back to life. Bcrypt is preferable as it is more resilient against brute force attacks. There’s also no way for a server to verify the identity of the requesting client when using Digest Access Authentication. This opens the possibility for man in the middle attacks, where a client can be led to believe a given server is really who he thinks it is, but ends up sending his login credentials to an unknown entity.

Your best bet when dealing with authentication is to use SSL and encrypt passwords using Bcrypt. You can use Basic Authentication or a home brewed authentication mechanism over SSL, but for situations where SSL is not possible for whatever reason, Digest Access is better than simple Basic Authentication and sending passwords in plain text over the public Internet.

There’s a simple example of RFC-2069 HTTP Digest Access available on the PHPMaster GitHub page to accompany this article. If you’re the type of person who likes to explore and tinker, feel free to clone it. A good place to start your exercise is to make the necessary modifications to work with accounts stored in a database instead of the hard-coded credentials in the script (remember, you’ll need to calculate and store A1 as the user’s password hash for later lookup at login), and then enhance it with RFC-2617 features. Afterwards, you’ll have a pretty solid understanding of HTTP Digest Access Authentication.

Summary

By now you should have an idea of what your available HTTP authentication methods are.

Basic Authentication is the easiest to implement and also the least secure. Usernames and passwords are encoded in Base64 but effectively sent to the server in plain text.

Digest Authentication improves on the Basic method by sending authentication data through MD5. While MD5 is still not a strong password encryption algorithm, it is much harder to decode than the plain text/Base64 method of Basic Authentication. In its original form, there were still problems of man in the middle attacks and not being able to confirm server identity, but these weaknesses have been improved in later revisions to the RFC.

SSL is the most modern and secure method of sending user authentication data over the public Internet. But when SSL is not available, please use Digest over Basic authentication.

Image via Fotolia

Openbiz Cubi is a robust PHP application framework giving developers the ability to create business applications with minimal effort. In this two-part series I’ll explain the concepts and steps necessary to create your own business web applications with Cubi. We’ll look first at the challenges web developers face and how Openbiz Cubi can help, and then how to install Cubi. In part 2 we’ll see how to create our own modules.

Openbiz Cubi Features

Even with many choices for a web development framework, application development is still a very challenging job.

A good framework can help developers code with good programming practices such MVC and ORM, although to build a real world application we sometimes have to spend time writing code beyond the capability of the framework.

• Learn the framework and code with it. After creating a “Hello World” app with the framework, developers still have a steep learning curve to build the first prototyped application.
• Implement common features like user registration, login, password reset, etc.
• Provide permission control for users. Because of the complexity of generalizing access control, permission logic is often hardcoded in the software.
• Make a professional UI. Fine-tuning HTML, CSS, and JavaScript is time-consuming, especially to please all major browsers.

All the above tasks are necessary but not the highlighted features of the application. Spending too much time on them brings frustration to both clients and the development team.

Openbiz Cubi is a mature platform, mainly for the fast development of business applications. It was designed to relieve the pain of such application development by providing:

• An XML-based coding scheme. Developers use intuitive XML to describe the data objects, pages and forms, as well as user interaction.
• A modular platform that has many common component built-in. Developers make their own modules and load them in the platform.
• A default professional-looking UI with multi-theme support.
• Flexible permission control options, from simple to sophisticated.

To learn more about Cubi, be sure to visit the project site code.google.com/openbiz-cubi and official website www.openbiz.me.

Installing Openbiz Cubi

To install Openbiz Cubi, you need to download either the source code or Windows installer from code.google.com/p/openbiz-cubi/downloads/list, or you can get the latest source code from the Openbiz Cubi SVN server.

When you choose to download the source ZIP archive or get the source from SVN, you can follow the steps below to install it:

1. Prepare the LAMP stack. Openbiz Cubi can be run on Unix, Windows, and Mac servers. The runtime environment should include:
• Web server – Apache, IIS, etc.
• Database server – MySQL, MSSQL, Oracle, PgSQL, and databases supported by Zend_DB
• PHP 5.2 and above with mysql, PDO, and mcrypt extensions
2. In your web server’s web directory, create a folder named cubi.
3. Unzip the Cubi ZIP file to the directory (or check out the source from SVN under this directory).

If you use Windows as your development environment, you can install Cubi with its Windows Installer. The installer:

• Installs Apache 2.4, PHP 5.4 and MySQL 5.3. After installation is completed, you can find Apache and MySQL in the System Services list.
• Installs the Openbiz Cubi platform and business applications. You can choose to unselect the options of installing business applications which are not released under open source.
• Adds desktop icons and start menu items.

After the code is installed to the web server’s directory, you can launch Cubi’s web installation wizard in a browser to set up the database and load modules. Run the installation wizard by launching http://host/cubi/install in your browser.

Click the Start Now button and follow the steps until you see Installation Completed page. Then you’ll be ready to test drive Cubi.

A Quick Tour of Openbiz Cubi

After you log in to Cubi as admin, you’ll see the Administration dashboard. You may see other tabs as well, like “Contacts” and “Calendar”.

Cubi is made of modules. All modules are under the cubi/modules directory. Of the many built-in modules, the following are the most important; they are core modules that are often used by other modules:

• System module – provides the ability for system administrators to manage users, roles, modules, groups and permissions.
• Menu module – provides support for page navigation by menus, tabs, and breadcumbs.
• User module – provides functions for users to register, sign in, and reset passwords.
• MyAccount module – provides My Account pages where a user can manage his own profile, preferences, activities, and password.

Cubi comes with other modules such as Contact, Email, Event Log, Security, Theme, Translation, Attachment, Picture, Chart, Payment, OAuth, Web Service, and more.

A typical Cubi page on the front-end is composed of four sections:

• Header – this section contains the logo, My Account link, application tabs, and breadcrumb navigation.
• Left Menu – this section contains navigation menus and other widgets.
• Content – this is the main area users will work in with their data and business logic.
• Footer – the footer may have links about the application provider, copyright, etc.

Manage Users and Roles

For the application administrator, one of the most important tasks is to manage users and their permissions to access certain resources. Cubi supports several widely used access control models including Role-based access control (RBAC) and Group-based access control (similar to Unix file permissions). I’ll briefly discuss how to use RBAC.

A Cubi user account is simply called “user”. A role usually means a type of user. Different roles are permitted to do different things. Cubi comes with three roles: administrator, member, and visitor. A user can be assigned to one or many roles.

An administrator needs to use the Role Management page to allow or deny resources for a given role. When he wants to grant a user certain permissions, he associates the user to a new role that has those permissions.

Conclusion

This is where I’ll end the first part of the series. So far we’ve talked about the challenges web developers face and how Openbiz Cubi can help, how to install Cubi, and undertook a brief overview of how Cubi is organized. In the next part I’ll dig deeper and show you how to create your own module. Stay tuned!

Image via Fotolia

Deprecation can happen for various reasons – perhaps an API is no longer useful and has reached its end-of-life, or the refactoring of code to improve its reusability and testability obsoletes particular methods.

In this article I’ll share with you some key points that you should follow when deprecating APIs so you can continue to grow your code and provide fair warning to those who depend on it.

Prepare for Refactoring

Even though it is tedious, make sure you first document your code. This will assist you in better understanding what code is responsible for what functionality and how it functions as a whole. Use a standardized document format so it can be used by your IDE and easily compiled into readable documentation.

Then, to ensure your service or library continues to work after you deprecate parts of its API, test the original code with your unit tests. If the tests fail, you must fix the problems so you can distinguish between failures that were brought by deprecating the API and failures that were already there.

Employ the Single Responsibility Principle

Logically organize your code based on the purpose and function of various components, and employ the Single Responsibility Principle (SRP). The principle ensures the methods and classes that perform more than one job are refactored so each has a single task. This increases the reusability and testability of your code. Don’t worry if there are many classes and methods in your API as a result of applying the SRP; reusable and testable code allows deprecating changes to be targeted to a minimal amount of code.

Communicate with your Users

Depending on the size of your user base, establishing good communication can be difficult. A good solution is to establish a mailing list to which they can subscribe. The period of time that you can allow users to continue using old methods once you’ve announced the intended changes on the list depends on factors like the importance and why the change is being made.
Depending on your API and the resources available to your project, it might also be beneficial to host a project wiki where developers can share alternative strategies and workarounds to avoid deprecated methods calls.

For libraries, the methods and classes that provide your API should be updated to issue descriptive warning messages that can be logged when they’re used. If you’re providing a web service, define headers that do not affect the current implementation but which allow you to include metadata with the service call. These headers can allow you communicate with your users about service changes.

Remove the Old Code

After making the necessary changes, be sure to read through your code carefully to make sure the unneeded, unused, or old code, is safe to remove. Code analysis tools can be helpful as well.

When a suitable deprecation period has passed, you can go ahead and prune the old code. After removing code that is no longer necessary, it is important you perform tests to make sure the system is still functioning properly so as to avoid failures.

Summary

Deprecation can improve the internal consistency of a web service or library in the software code, its structure or the user interface. This makes it easier for users to understand how the system works reducing the cost of coming up with a new system.
Refactoring helps to improve the user experience of your API and makes future changes to your code easier to implement. Code becomes more reusable and testable. But the process of refactoring also means some code because unnecessary and should be removed to keep the project clean.

Deprecating features before they are done away or transformed gives users who depend on your code time to update their own code. By following the advice in this article, you will find it easier to keep your code fresh and your users won’t have any unhappy surprises.

Image via Fotolia

Of course, this part assumes that you know how to create an index on a and use the explain() method to analyze it. If you don’t already know how, I suggest you go back and read part one before continuing here.

We used a collection named posts in the last article. For our work here, let’s add a new field location to it to store the location from which the post was made. The field is a sub-document, and stores the city, state, and country of the user as shown below (a sub-document is a field having a document structure):

{
    "_id": ObjectId("5146bb52d852470060001f4"),
    "comments": {
        "0": "This is the first comment",
        "1": "This is the second comment"
    },
    "post_likes": 40,
    "post_tags": {
        "0": "MongoDB",
        "1": "Tutorial",
        "2": "Indexing"
    },
    "post_text": "Hello Readers!! This is my post text",
    "post_type": "private",
    "user_name": "Mark Anthony",
    "location": {
        "city": "Los Angeles",
        "state": "California",
        "country": "USA"
    }
}

Indexing on Sub-documents

Suppose we want to search posts based on where the user lives. For this, we need to create an index on the sub-document location field, which in turn indexes the sub-fields. Then we’ll be able to use the index for the following kinds of queries:

<?php
// query to find posts from the city of Los Angeles
$cursor = $collection->find(
    array(
        "location" => "Los Angeles"
    ),
    array()
);

// query to find posts from the state of California
$cursor = $collection->find(
    array(
        "location" => "California"
    ),
    array()
);

// query to find posts from the United States
$cursor = $collection->find(
    array(
        "location" => "USA"
    ),
    array()
);

We’re able to search all of the sub-fields (city, state, and country) in the sub-document using only location as the key. The query looks to see if any of the sub-fields of location meet our search criteria.

It should be noted that, similar to indexing on arrays, separate indexes are created for all the of the sub-fields internally. In this case, three indexes are created as location.city, location.state and location.country, hence such indexes should be used with care since each index occupies space in memory.

Indexing on Embedded Fields

It will happen sometimes that we won’t need indexes on all of the fields of a sub-document. If in our application we only want to find posts based on city but not state or country, we can create the index on the embedded field city.

We can now use this index in queries to find posts based on city:

<?php
// query to find posts from the city of Los Angeles
$cursor = $collection->find(
    array(
        "location.city" => "Los Angeles"
    ),
    array()
);

Index Direction (Ascending/Descending)

We’ve always provide an index direction (1 or -1) to keys when creating our indexes. I touched on this briefly in part 1, but this is actually an important discussion point I’d like like to pick up again. If we have one key in the index, direction 1 or -1 doesn’t really matter, but it comes into play when we do sorting or ranged queries with compound indexes.

Suppose we have a compound index with key field1 ascending and key field2 descending. In this case, the indexing table may look like this:

A query with sorting on field1 ascending and field2 ascending will travel rows in this order: 1, 2, 3, 4, 5, 6, 7, 8, 9. A query with field1 ascending and field2 descending will travel: 3, 2, 1, 6, 5, 4, 9, 8, 7. Such out-of-order jumps in the search tree can end up being costly to query performance.

Of course index structure above is represented as a table just for the purposes of understanding. Remember, MongoDB uses tree structures internally; each element is stored as a node of a tree. The elements closer to each other would be under the same branches, and hence easily approachable. If a query has to retrieve multiple records in a sorted manner, it would be logically correct to place the elements near each other in the tree for faster retrieval in comparison to the case where the query has to jump from one node to another far node to grab the elements.

If you are looking to sort on field1:1,field2:1, then index {field1:1, field2:1} would be faster than {field1:1, field2:-1} or {field1:-1, field2:1}.

Covered Queries

As per MongoDB’s documentation, a covered query is the one in which:

• all fields used in the query are part of an index used in the query, and
• all the fields returned in the results are in the same index

Since all the fields are covered in the index itself, MongoDB can match the query condition as well as return the result fields using the same index without looking inside the documents. Since indexes are stored in RAM or sequentially located on disk, such access is a lot faster.

Consider we have a compound index defined on the post_type and user_name fields. This index covers the following query:

<?php
// query to find posts with type public and get only user_name in result
$cursor = $collection->find(
    array(
        "post_type" => "public",
    ),
    array(
        "user_name" => 1,
        "_id" => 0
    )
);

We’ve explicitly excluded the _id field from the result to take advantage of the covered query. As you may already know, all queries return the _id field by default. As per the second condition for covered queries, all the fields returned in the result should be included in the index. We don’t have _id in our compound index on post_type and user_name, so we have to exclude this field from the result.

To check if the query is covered, we can look to the indexOnly field in the result of the explain() method. A true value of indicates that ours was a covered query.

It’s important to know that an index can’t cover a query if:

• any of the indexed fields is an array (e.g. post_tags), or
• any of the indexed fields are fields in sub-documents (e.g. location.city)

Thus, it’s always a good practice to check your query index usage with explain().

Removing Indexes

To check the current index size for a database, we can use the totalIndexSize() method which returns the index size in bytes.

We just have to ensure that we have enough RAM available to accommodate indexes as well as the data that MongoDB manages and uses regularly.

To delete an existing index, and thus free up resources, we use the dropIndex() method.

Conclusion

That’s all for this part and also the series. We’ve touched on a lot of important topics to get you up to speed with indexing in MongoDB.

Analyzing your indexes to make sure they are doing well is an on-going process as your application grows and your data changes, so if you have any kind questions or comments about the article, feel free to post in the comments below.

Image via Fotolia

Through this article you will gain familiarity with Maven for PHP, and how to install and use the PHP-Maven plugin from the command line and in Eclipse.

Install Maven

PHP-Maven uses the power of Maven for building, reporting, and creating documentation of your PHP projects. It adapts the Maven build life cycle to the PHP world while fully supporting PHP 5. PHP-Maven uses PHPUnit for unit testing and phpDocumentor for creating the project documentation.

To install Maven:

1. Download Maven from http://maven.apache.org/download.cgi. For this article, the version is 3.0.4.
2. Unpack the archive wherever you would like to store the binaries, and a folder named apache-maven-<version> will be created.
3. Add its bin folder to your PATH.
4. Make sure JAVA_HOME is set to the location of your JDK.

After you’ve completed the above steps, to test whether Maven is installed correctly or not you should run mvn --version at a command prompt.

Once Maven is successfully installed, go to the settings.xml file (found in ~/.m2 on Unix/Mac OS X and in C:\Documents and Settings\username\.m2 on Windows) and add the PHP for Maven repository. If there is no settings.xml file, you must create it.

Below is a sample settings.xml file:

<settings>
 <profiles>
  <profile>
   <id>profile-php-maven</id>
   <pluginRepositories>
    <pluginRepository>
     <id>release-repo1.php-maven.org</id>
     <name>PHP-Maven 2 Release Repository</name>
     <url>http://repos.php-maven.org/releases</url>
     <releases>
      <enabled>true</enabled>
     </releases>
    </pluginRepository>

    <pluginRepository>
     <id>snapshot-repo1.php-maven.org</id>
     <name>PHP-Maven 2 Snapshot Repository</name>
     <url>http://repos.php-maven.org/snapshots</url>
     <releases>
      <enabled>false</enabled>
     </releases>
     <snapshots>
      <enabled>true</enabled>
     </snapshots>
    </pluginRepository>
   </pluginRepositories>

   <repositories>
    <repository>
     <id>release-repo1.php-maven.org</id>
     <name>PHP-Maven 2 Release Repository</name>
     <url>http://repos.php-maven.org/releases</url>
     <releases>
      <enabled>true</enabled>
     </releases>
    </repository>

    <repository>
     <id>snapshot-repo1.php-maven.org</id>
     <name>PHP-Maven 2 Snapshot Repository</name>
     <url>http://repos.php-maven.org/snapshots</url>
     <releases>
      <enabled>false</enabled>
     </releases>
     <snapshots>
      <enabled>true</enabled>
     </snapshots>
    </repository>
   </repositories>
  </profile>
 </profiles>

 <activeProfiles>
  <activeProfile>profile-php-maven</activeProfile>
 </activeProfiles>
</settings>

Create Your First Project

To create a simple project from the command line I’ll use the Maven Archetype Plugin. The Archetype Plugin allows a user to create a Maven project from an existing template called an archetype. You can run the Maven Archetype plugin with the mvn archetype:generate command to generate a new project from an archetype, in a folder corresponding to its artifact ID.

Maven will start downloading all the dependencies needed to your computer. At some point, Maven will ask you to define a value for groupId, artifactId, version and package, as you see here:

Notice that at the official Maven Archetype Plugin page you will find all the available parameters along with their description.

After successfully creating your first project, you should find the following in the corresponding folder, in my case Octavia_project:

• src/main/php - contains the project’s source code.
• src/test/php - contains the project’s test code.
• src/site – contains the project’s site descriptor.
• pom.xml - contains the project’s POM description.

The contents of pom.xml looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" 
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

 <parent>
  <groupId>org.phpmaven</groupId>
  <artifactId>php-parent-pom</artifactId>
  <version>2.0.2</version>
 </parent>

 <modelVersion>4.0.0</modelVersion>
 <groupId>org.phpsample</groupId>
 <artifactId>Octavia_project</artifactId>
 <packaging>php</packaging>
 <version>1.0-SNAPSHOT</version>
 <build>
  <plugins>
   <plugin>
    <groupId>org.phpmaven</groupId>
    <artifactId>maven-php-plugin</artifactId>
    <extensions>true</extensions>
   </plugin>

   <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-site-plugin</artifactId>
    <version>3.0</version>
    <configuration>
     <reportPlugins>
      <plugin>
       <groupId>org.phpmaven</groupId>
       <artifactId>maven-php-plugin</artifactId>
       <reportSets>
        <reportSet>
         <reports>
          <report>phpdocumentor</report>
          <report>phpunit-coverage</report>
          <report>phpunit</report>  
         </reports>
        </reportSet>
       </reportSets>
      </plugin>
     </reportPlugins>
    </configuration>
   </plugin>
  </plugins>
 </build>

 <dependencies>
  <!--  phpUnit for PHP 5 -->
  <dependency>
   <groupId>de.phpunit</groupId>
   <artifactId>PHPUnit</artifactId>
   <version>3.6.10</version>
   <type>phar</type>
  </dependency>
 </dependencies>
</project>

The test phase comes after creating your application, and for that you need the PHPUnit dependency. If it’s missing, be sure to add it to your pom.xml as shown above.

The PHPUnit tests should be place into the src/test/php folder and a test should be named SomthingTest.php, that is, the suffix “Test.php” is a must. My test, MyTest.php, is listed below:

<?php
class MyTest extends PHPUnit_Framework_TestCase
{
    public function testBar() {
        include "org/sample/app.php";
        $this->fail("we will fail");
    }	
}

To execute a test you use the command mvn test. In the official documentation has a section How to Ignore Failing Tests, and you can find there some commands that may help you run your test. (The only command that actually worked for me was the one that executed a single test.)

To build your newly created Maven project, you run the command mvn:package. You will notice again that Maven automatically starts downloading any dependencies needed for your project.

After downloading all needed dependencies and performing the build actions, you should get the following success message:

To create documentation from the project, you’ll need the phpDocumentor 2 PEAR package. In the src/site folder, create the site.xml file with the following contents:

<?xml version="1.0" encoding="ISO-8859-1"?>
<project name="Maven">
 <version position="left" />

 <skin>
  <groupId>org.apache.maven.skins</groupId>
  <artifactId>maven-stylus-skin</artifactId>
  <version>1.0</version>
 </skin>
 <body>
  <links>
   <item name="PHP-Maven" href="http://www.php-maven.org/" />
  </links>

  <menu name="Main">
   <item name="Welcome" href="index.html" />
  </menu>

  <menu ref="reports" />
 </body>
</project>

Next run the mvn site command and you’ll find the results in the target/site folder.

Eclipse Integration

The php-maven plugin supports integration with the Eclipse IDE, but it doesn’t contain the php-maven plugin by default. You need to integrate it manually. To do that, follow these steps:

1. From the Help menu, choose Install New Software option and then press the “Add…” button.
2. In the Add Repository window enter the name: “PHPMaven update site” and URL http://www.php-maven.org/eclipse/update.
3. The PHPMaven update site is listed and you can choose the PHP-Maven option.
4. Hit Next/Finish to install the plugins.

After the installation you will find the PHP-Maven project option when you go to create a new project. Select it, and press Next in order to install the plugin.

To create a new project within Eclipse, select File > New > Other (or press the CTRL + N combination) and you should see something like in the image below:

After pressing the Next button you will receive a list of different archetypes.

Choose your artifact and you will be prompted for the project information, just like earlier on the command prompt.

The new project will be added to the Project Explorer tab. The folder structure is mostly similar to the one for the project created from command line.

After you create the project, right click on it and you’ll see the PHP-Maven option at the bottom of the menu, and its children list the most important phases of a PHP-Maven project.

Summary

In this article you’ve learned how to install and use the PHP-Maven plugin from the command line and in Eclipse. Maven for PHP is a capable build automation tool for the PHP platform. Using Maven, the user only needs to provide the configuration for the project, while the configurable plugins do the work of compiling the project, running unit tests, generating API documentation and so on. Maven for PHP will quickly become a necessary tool in the PHP developer’s toolbox.

Image via Fotolia

In the beginning of my career I stumble upon CodeIgniter and I love it for its simplicity, small footprint, and good documentation. It worked great for me when I developed a small to medium sized applications (usually Facebook apps), and the learning curve for new developers who worked with me was shallow enough that they could easily be integrated in the development process.

But last year, because of the Twitter buzz from some in the PHP community, blog posts, and the suggestions of some friends, I give Laravel 3 a try – and since that time I’ve never looked back. So, in this article I’d like to present a comparison of the two frameworks from my point of view.

Requirements

CodeIgniter 2.1.3 needs PHP 5.1.6, but versions before 2 still worked with PHP 4. This was a drag since everybody moved on to use PHP 5. Laravel 3 needs PHP 5.3 and also the Mcrypt extension.

CodeIgniter 1 didn’t need Mcrypt, since if you didn’t have the extension installed then it would use a custom functions to encode/decode strings. Of course this slows down the application, and we were amazed at some of the production servers that didn’t have it installed.

<?php
function encode($string, $key = '') {
    $key = $this->get_key($key);
    if ($this->_mcrypt_exists === true) {
        $enc = $this->mcrypt_encode($string, $key);
    }
    else {
        $enc = $this>_xor_encode($string, $key);
    }
    return base64_encode($enc);
}

REST

It’s easy to build a REST API with Laravel using RESTful Controllers where you simply have the $restful property set true inside your controller. For example, if you’re building an API for a task manager, for updating a task it would use something like this:

<?php
class Tasks_Controller extends Base_Controller {
    public $restful = true;
    public function get_index()
    {
    }
    public function put_update()
    {
    }
...
}

To handle PUT in CodeIgniter you would need this workaround:

<?php
if ($_SERVER["REQUEST_METHOD"] == "PUT") {
    parse_str(file_get_contents("php://input"), $post_vars);
    $arData = json_decode(array_pop(array_keys($post_vars)));
}

As you can see, this is error prone and not very clean.

Code Organization

Most of the time, the project you are working on will have a frontend and backend. In order to better organize the code in CodeIgniter, the best approach is to use HMVC. In this way you would have separate directories for backend code and frontend code so that different developers can work independently. To achieve something similar with Laravel, you can use nested controllers.

The View

CodeIgniter doesn’t have a default template engine, but you can easily integrate one like Smarty. On the other hand, Laravel uses Blade as its template engine which is very simple but powerful. With Blade, it’s very easy to set up a two-step view.

You have a master template where usually you put the basic HTML structure with a header, navigation, footer, etc. Most of the time this template is called layout.blade.php or master.blade.php. The portion of code that will be injected from different different controllers is declared in the master template by @yield('content').

<!DOCTYPE HTML>
<html>
 <head>
  ...
 </head>
 <body>
  <div class="header">
   ...
  </div>
  @yield('content')
 </body>
</html>

In custom templates declared per each method of a controller, for example a CMS page like “About Us”, you have a template called page.blade.php which looks something like this:

@layout('master')
...
@section('content')
About page section
@endsection
...

You can have as many section as you like to declare and reuse.

A more detailed explanation about this can be found in the section Blade Layouts in the online open source book called Code Happy by Dayle Rees.

To achieve this in CodeIgniter, at least as far as I’ve been able to figure out, is to create a master controller which all other controllers extend and call a render method. A CodeIgniter start project for implementing two step views can be found under my GitHub account.

The Model

Codeigniter uses Active Record for database manipulation which is quite easy to master, but you can also use raw queries if you have complicated requirements. Laravel uses Eloquent as a ORM which is simple and works with major database servers. For writing queries, you can also use Fluent Query Builder, which looks like a lot like the Active Record approach. And of course, you can also write raw queries too.

Command Line (Crons)

For Codeigniter to call a cron script, the best approach is to create a Cron controller. You also need to duplicate index.php and create a cron.php file right in the root directory, overriding the following the following $_SERVER values:

<?php
$_SERVER['SERVER_NAME'] = "xxxxxx.org";
$_SERVER['REQUEST_URI'] = "cron/index";
$_SERVER['QUERY_STRING'] = "lang=en";
$_SERVER['SERVER_PORT'] = "0";
$_SERVER['REQUEST_METHOD'] = "GET"

Then you can call it from the command line with php cron.php cron index.

In Laravel it’s easier. You have Tasks using the command line tool that Laravel comes with called Artisan.

Miscellaneous

• Unit Testing – Codeigniter uses a very rudimentary testing class for writing unit tests. It is limited, but you can integrate PHPUnit. Laravel comes with PHPUnit out of the box, the most popular PHP unit testing framework.
• Authentication – Codeigniter doesn’t have anything out of the box for authentication, but you can always use a third-party library, like Ion Auth. Larvel on the other hand has a basic auth library that you only need to configure before using.
• Caching – With CodeIgniter, the driver supports APC, Memcache, and the filesystem. Laravel on the other hand supports those and also database tables and Redis.
• Hooks/Filters – If you want to step in and run certain code at different points of execution in CodeIgniter, you should do so using Hooks. Laravel offers Filters.
• Routing – Routing is done pretty much the same way in both Laravel and CodeIgniter, although I’ve found Laravel to be more flexible.
• Configuration – Configuration is done via predefined arrays stored in config files which support multiple environments in both frameworks.
• Extensibility – Extending the core functionality of CodeIgniter is done with Libraries, and you can find a lot of them on Sparks (a package management system for CodeIgniter). Laravel uses Bundles, which might seem familiar to you if you’ve worked with Symfony 2. You can find check already-created bundles at bundles.laravel.com.
• Migrations – Database migrations are specific for Laravel, a concept borrow from Ruby on Rails which is versioning at the database level.

Community

CodeIgniter used to have a bigger community, but many moved to different frameworks after EllisLab, the company behind it, dropped support and no new features were added. Some of those people joined the Laravel community which is now very active with a lot of tutorials, books, and even its first international conference.

The future seems to be very bright for Laravel since Laravel 4 is in beta 2 and its first conference has just ended. On the other side for CodeIgniter, there are few member contributors who so commit/merge patches into CodeIgniter 2 but there is no release date for CodeIgniter 3.

Conclusion

With Laravel 4 on the horizon, the framework definitely has the momentum now and the community behind it is very active. I’m kind of nostalgic when I talk about CodeIgniter, but it served me very well at the time. It taught me how to use MVC, was easy to learn, earned me my first money, and I even did my first book review about it, but as time went on it hardly changed. Maybe it’s just because it was very good at what it did? But web development changes rapidly, and new technologies come around, so if you want to survive you have to keep up.

Just like in politics, everyone chooses a side. And for most of the time you can stick with your choice, but it’s always good to explore new things. For me, it meant goodbye CodeIgniter, hello Laravel.

Image via Fotolia

In this article we’ll explore the following five types of indexes:

1. Default _id Index
2. Secondary Index
3. Compound Index
4. Multikey Index
5. Multikey Compound Index

There are some other types too to discuss, but I’ve logically kept them for part 2 to provide a clear understanding and avoid any confusion.

Although more than one index can be defined on a collection, a query can only use one index during its execution. The decision of choosing the best index out of the available options is made at runtime by MongoDB’s query-optimizer.

This article assumes that you have a basic understanding of MongoDB concepts (like Collections, Documents, etc.) and performing basic queries using PHP (like find and insert). If not, I suggest you to read our beginner articles: Introduction to MongoDB and MongoDB Revisited.

For the series we’ll assume we have a collection named posts populated with 500 documents having the following structure:

{
    "_id": ObjectId("5146bb52d852470060001f4"),
    "comments": {
        "0": "This is the first comment",
        "1": "This is the second comment"
    },
    "post_likes": 40,
    "post_tags": {
        "0": "MongoDB",
        "1": "Tutorial",
        "2": "Indexing"
    },
    "post_text": "Hello Readers!! This is my post text",
    "post_type": "private",
    "user_name": "Mark Anthony"
}

Now, let’s explore various types of indexing in detail.

Default _id Index

By default, MongoDB creates a default index on the _id field for each collection. Each document has a unique _id field as a primary key, a 12-byte ObjectID. When there are no other any indexes available, this is used by default for all kinds of queries.

To view the indexes for a collection, open the MongoDB shell and do the following:

The getIndexes() method returns all of the indexes for our collection. As you can see, we have the default index with name _id_. The key field indicates that the index is on the _id field, and the value of 1 indicates an ascending order. We’ll learn about ordering in next section.

Secondary Index

For cases where we want to use indexing on fields other than _id field, we have to define custom indexes. Suppose we want to search for posts based on the user_name field. In this case, we’ll define a custom index on the user_name field of the collection. Such custom indexes, other than the default index, are called secondary indexes.

To demonstrate the effect of indexing on database, let’s briefly analyze query performance without indexing first. For this, we’ll execute a query to find all posts having a user_name with “Jim Alexandar”.

<?php
// query to find posts with user_name "Jim Alexandar"
$cursor = $collection->find(
    array("user_name" => "Jim Alexandar")
);
//  use explain() to get explanation of query indexes
var_dump($cursor->explain());

An important method often used with indexing is explain() which returns information relevant to indexing. The output of the above explain() is as shown below:

Some of the important keys worth looking at are:

1. cursor – indicates the index used in the query. BasicCursor indicates that the default _id index was used and MongoDB had to search the entire collection. Going ahead, we’ll see that when we apply indexing, BtreeCursor will be used instead of BasicCursor.
2. n – indicates the number of documents the query returned (one document in this case).
3. nscannedObjects – indicates the number of documents searched by the query (in this case, all 500 documents of the collection were searched). This can be an operation with large overhead if the number of documents in collection is very large.
4. nscanned – indicates the number of documents scanned during the database operation.

Ideally, n should be equal to or near to nscanned, which means a minimum number of documents were searched.

Now, let’s execute the same query but using a secondary index. To create the index, execute the following in the MongoDB shell:

We created an index on the user_name field in the posts collection using the ensureIndex() method. I’m sure you’ve niced the value of the order argument to the method which indicates either an ascending (1) or descending (-1) order for the search. To better understand this, note that each document has a timestamp field. If we want the most recent posts first, we would use descending order. For the oldest posts first, we would choose ascending order.

After creating the index, the same find() and explain() methods are used to execute and analyze the query as before. The output of is:

The output shows that the query used a BtreeCursor named user_name_1 (which we defined earlier) and scanned only one document as opposed to the 500 documents searched in the previous query without indexing.

For now, understand that all MongoDB indexes uses a BTree data structure in its algorithm, and BtreeCursor is the default cursor for it. A detailed discussion of BTreeCursor is out of scope for this article, but this doesn’t affect any further understanding.

The above comparison indicates how indexes can dramatically improve the the query performance.

Compound Index

There will be cases when a query uses more than one field. In such cases, we can use compound indexes. Consider the following query which uses both the post_type and post_likes fields:

<?php
// query to find posts with type public and 100 likes
$cursor = $collection->find(
    array(
        "post_type" => "public",
        "post_likes" => 100
    ),
    array()
);

Analyzing this query with explain(), gives the following result, which shows that the query uses BasicCursor and all 500 documents are scanned to retrieve one document.

This is highly inefficient, so let’s apply some indexes. We can define a compound index on fields post_type and post_likes as follows:

Analyzing the query now gives the follow result:

A very important point of note here is that compound indexes defined on multiple fields can be used to query a subset of these fields. For example, suppose there is a compound index {field1,field2,field3}. This index can be used to query on:

• field1
• field1, field2
• field1, field2, field3

So, if we’ve defined the index {field1,field2,field3}, we don’t need to define separate {field1} and {field1,field2} indexes. However, if we need this compound index while querying field2 and field2,field3, we can use hint() if the optimizer doesn’t select the desired index.

The hint() method can be used to force MongoDB to use an index we specify and override the default selection and query optimization process. You can specify the field names used in the index as a argument as shown below:

<?php
// query to find posts with type public and 100 likes
// use hint() to force MongoDB to use the index we created
$cursor = $collection
    ->find(
        array(
            "post_type" => "public",
            "post_likes" => 100
        )
    )
    ->hint(
        array(
            "post_type" => 1,
            "post_likes" => 1
        )
    );

This ensures the query uses the compound index defined on the post_type and post_likes fields.

Multikey Index

When indexing is done on an array field, it is called a multikey index. Consider our post document again; we can apply a multikey index on post_tags. The multikey index would index each element of the array, so in this case separate indexes would be created for the post_tags values: MongoDB, Tutorial, Indexing, and so on.

Indexes on array fields must be used very selectively, though, as they consume a lot of memory because of the indexing of each value.

Multikey Compound Index

We can create a multikey compound index, but with the limitation that at most one field in the index can be an array. So, if we have field1 as a string, and [field2, field3] as an array, we can’t define the index {field2,field3} since both fields are arrays.

In the example below, we create an index on the post_tags and user_name fields:

Indexing Limitations and Considerations

It is important to know that indexing can’t be used in queries which use regular expressions, negation operators (i.e. $ne, $not, etc.), arithmetic operators (i.e. $mod, etc.), JavaScript expressions in the $where clause, and in some other cases.

Indexing operations also come with their own cost. Each index occupies space as well as causes extra overhead on each insert, update, and delete operation on the collection. You need to consider the read:write ratio for each collection; indexing is beneficial to read-heavy collections, but may not be for write-heavy collections.

MongoDB keeps indexes in RAM. Make sure that the total index size does not exceed the RAM limit. If it does, some indexes will be removed from RAM and hence queries will slow down. Also, a collection can have a maximum of 64 indexes.

Summary

That’s all for this part. To summarize, indexes are highly beneficial for an application if a proper indexing approach is chosen. In the next part, we’ll look at using indexes on embedded documents, sub-documents, and ordering. Stay tuned!

Image via Fotolia

In this article I’ll show you how to create an image cropping tool with the help of the ImageMagick PHP extension. This tutorial assumes that you already have the extension installed, so if not then be sure to read the manual.

Getting Familiar

The ImageMagick extension performs image processing using the ImageMagick library. ImageMagick provides a lot of API methods through which you can manipulate an image. It offers a simple object-oriented interface to use the API; you just need to create an instance of the Imagick class and then call the appropriate methods to start manipulating the images.

Since we’re going to create an image cropper, we’ll mostly be using the two methods: cropImage() and thumbnailimage().

cropImage

The cropImage() method accepts four arguments. The first two arguments indicate the height and width of the cropped region, and the last two indicate the X and Y coordinates of the top-left corner of the cropped area.

For example:

<?php
$inFile = "test.jpg";
$outFile = "test-cropped.jpg";
$image = new Imagick($inFile);
$image->cropImage(400,400, 30,10);
$image->writeImage($outFile);

We create an Imagick object first, passing to its constructor the filename of our image. Then we call cropImage() with appropriate arguments. In this case the code will produce a cropped image of size 400×400px starting at 30px from the top and 10px in from the left of the original image. Finally, writeImage() saves the result back to disk for us.

thumbnailImage

The thumbnailImage() method simply accepts the height and width of the resized image and can be used as follows:

<?php
$inFile = "test.jpg";
$outFile = "test-thumbnail.jpg";
$image = new Imagick($inFile);
$image->thumbnailImage(200, 200);
$image->writeImage($outFile);

The above code produces a 200×200px version of image. If either the width or height argument to thumbnailImage() is set as 0, the aspect ratio is maintained.

We can also pass a third argument known as bestfit; If this is set true, the image will be resized in such a way that the new dimensions can be contained within the height and width specified. For example, a call to thumbnailImage(400, 400, true) on a 1200×768px image will produce a 400×200px version. The new dimensions are less than or equal to the specified dimensions.

Now that you’re familiar with the two methods, we’re good to go.

Upload and Cropping

Of course we’ll need to create an HTML form with which users can upload photos:

<form action="upload.php" method="post" enctype="multipart/form-data">
 <label for="file">Image:</label>
 <input type="file" name="file" id="file"><br>
 <input type="submit" name="submit" value="Upload and Crop">
</form>

As soon as the user hits the upload button a POST request will be sent to a script which will handle the file upload process and show the uploaded image to the user for cropping. Remember, when uploading files you need to set the form’s enctype attribute to “multipart/form-data”.

An upload script handles the image upload and resizing the image if required.

<?php
if (isset($_FILES["file"])) {
    $tmpFile = $_FILES["file"]["tmp_name"];
    $fileName = ... // determine secure name for uploaded file

    list($width, $height) = getimagesize($tmpFile);
    // check if the file is really an image
    if ($width == null && $height == null) {
        header("Location: index.php");
        return;
    }
    // resize if necessary
    if ($width >= 400 && $height >= 400) {
        $image = new Imagick($tmpFile);
        $image->thumbnailImage(400, 400);
        $image->writeImage($fileName);
    }
    else {
        move_uploaded_file($tmpFile, $fileName);
    }
}

You should be very careful while accepting files from users as someone can upload a malicious file to your server if you don’t create a secure upload system. For more information, read the article File Uploads with PHP, paying special attention to the “Security Considerations” section.

The getimagesize() function returns a null height and width if the file is not an image, so if the code detects the file is not an image, it simply redirects the user elsewhere. Additionally, you can also check the image type and size of the file using getimagesize(), but in this case we just check if the file is an image based on the null values.

Since the script already knows the width and height of the uploaded image at this point, it can determine whether the image should be resized down. If dimensions are more than 400×400px, it’s resized to create a 400×400px thumbnail by calling thumbnailImage(). As I explained earlier, the method takes two parameters: width and height. If you want to maintain the image’s aspect ratio, it’s recommended to pass 0 as one of the arguments.

Finally, the resized image is saved by calling writeImage(), or moved with move_uploaded_file() if it was already a suitable size.

The saved image is outputted to the browser so that the user gets a chance to crop it. To allow users to select specific portion of the image, I use a jQuery plugin called ImageAreaSelect. To use the plugin, the jQuery library needs to be available, as well as the plugin’s CSS and JavaScript files.

<script type="text/javascript" src="scripts/jquery.min.js"></script>
<script type="text/javascript" src="scripts/jquery.imgareaselect.pack.js"></script>
<link rel="stylesheet" type="text/css" href="css/imgareaselect-default.css">

To initialize the plugin you can use the following code:

selection = $('#photo').imgAreaSelect({
    handles: true,
    instance: true
});

#photo is the id of the image shown to the user. By setting the handles property true you can show resize handles on the selection area, and instance true gets an instance and save it to the variable selection. It’s through selection that you can easily retrieve the selection area later when the user finalizes his cropping.

getSelection() gives you everything you need. For example, getSelection().width gives you the width of the selection area. Similarly, you can use getSelection().x1 and getSelection().y1 to find the coordinates of the top left-corner of the selection area.

Putting this all together in a page, you’ll most likely register an onclick callback to a link or button, which the user can click to finalize his selection. When the button is clicked, the selection area width, height, and top-left corner coordinates is retrieved.

$("#crop").click(function(){
    var s = selection.getSelection();
    var width = s.width;
    var height = s.height;
    var x = s.x1;
    var y = s.y1;
    ...
});

An Ajax request is sent to a cropping script, passing these values as request parameters since they are needed to crop the image through ImageMagick. Additionally, you’ll want to send the image name since the script will need to know the name of the image that it will be cropping.

var request = $.ajax({
    url: "crop.php",
    type: "GET",
    data: {
        x: x,
        y: y,
        height: height,
        width: width,
        image: $("#photo").attr("src")
    }
});

When the request finishes, the image is reloaded so that the new cropped version will be shown in place of the old image.

request.done(function(msg) {
    $("#photo").attr("src", msg);
    selection.cancelSelection();
});

When cropping is done, the request returns the name of the cropped image, and the new image is loaded by changing the src attribute.

So what does the cropping script look like?

<?php
$file = basename($_GET["image"]);
$cropped = "cropped_" . $file;
$image = new Imagick($file);
$image->cropImage($_GET["width"], $_GET["height"], $_GET["x"], $_GET["y"]);
$image->writeImage($cropped);
echo $cropped;

The script first extracts the name of the image that needs to be cropped; the image name will be a full URL in the form of http://example.com/path/image.jpg, but only the image.jpg part is needed. Next, the string “cropped” is prefixed to the original name so that it will look like cropped_image.jpg. This allows the cropped image to be saved without overwriting the original.

The cropImage() method is used to crop the original image using the four parameters related to the selection area that were sent to the script, and then the cropped image is saved to the new file. The name of the cropped image is then echoed back to the Ajax request for display in the browser.

Conclusion

In this article I created a simple cropping tool to show you the power and easy of use of the ImageMagick extension. You can learn more about it, and be creative and make something more useful using its powerful API. For instance, if you wanted to extend what I’ve presented here, you could give users the option to download their images in multiple sizes. Sample code to accompany this article is available on the PHPMaster GitHub page to get you started.

Image via Fotolia