Chanaka Blog

Stream Encrypted Video with Windows Azure Media Services

2016-01-21T10:23:00.000+05:30

After a massive break I thought to write on the topic since there are lack of resources. I'm streaming my AES encrypted video with JWT authentication and generating the auth token with Java. If you have Azure Media Service enabled account you are welcome to try this out :).

Following are couple of useful resources in the web for testing this process.

JWT official site and token verification tool - https://jwt.io/
Azure media player - http://amsplayer.azurewebsites.net/azuremediaplayer.html
Base64URL encoder - http://kjur.github.io/jsjws/tool_b64uenc.html
HMAC generator - http://www.freeformatter.com/hmac-generator.html

At first we need to have encrypted videos uploaded to our Azure account. Following are the steps to doing that.
1. Login to manage.windowsazure.com
2. Go inside your media service account
3. Click on Upload a video file link under Managemant Tasks section (or else click on Content tab and then click on Upload Content link) and upload a video file.
4. Click on Process icon at the bottom, and click OK with default values.
5. Click on Sync Metadata icon at the bottom (Probably this will take some time)
6. Click on Encryption > Enable AES Encryption (If the Encryption icon not yet enable, logout and login again and try. It may take some time to enable that)
7. Click on Publish icon (SS for 4-7 https://www.diigo.com/item/image/5j6wi/wdb1)

Now you can copy the encrypted video publish URL which will be like : http://chanaka.streaming.mediaservices.windows.net/481ea237-7f1f-48b8-a4c1-096f34c10cd6/Best%20Sweet%20Minions%20Video%20-%20Let%20The%20UFO%20Go%20Home.ism/Manifest

Then go to Content Protection tab and set up the necessary attributes like this SS and click on manage verification keys button and regenerate primary key. And don't forget to save :)

For streaming the above encrypted video, we need to create a JWT auth token which is the tricky part. Following is sample java code to generate the token.

package azurejwttoken;

import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import org.apache.commons.codec.binary.Base64;

/**
 *
 * @author chanaka
 */
public class AzureJWTToken
{

    /**
     * @param args the command line arguments
     */
    public static void main(String[] args)
    {
        // Use the Primary Verification Key which you regenerated
        String          secret              = "YEAIfGy6cIXy1xhcIdgEoBfiWNoI3zq2FCKVQhdZXMcHGN+BAwicafgbRscsVZj1Us3RlWihphY1XMBuQOhlBg==";
        
        long            validitymillis      = 15 * 60 * 1000; // 15 minutes
        long            nowMillis           = System.currentTimeMillis();
        long            expireSeconds       = (nowMillis + validitymillis) / 1000; // Use seconds instead milli seconds
        
        String          jsonHeaderString    = "{\"typ\":\"JWT\",\"alg\":\"HS256\"}";
        
        // Use your ISSUER and SCOPE attributes properly here
        String          jsonBodyString      = "{\"iss\":\"http://mytest.com/\",\"aud\":\"urn:jwttest\",\"exp\":" + expireSeconds + "}";
        
        String          token               = getAuthToken(secret, jsonHeaderString, jsonBodyString);
        
        System.out.println("Token : " + token);
    }
    
    public static String getAuthToken(String secret, String jsonHeaderString, String jsonBodyString)
    {
        String          token       = null;
        
        try
        {
            Mac             sha256_HMAC         = Mac.getInstance("HmacSHA256");
            SecretKeySpec   secret_key          = new SecretKeySpec(Base64.decodeBase64(secret), "HmacSHA256");
            
            sha256_HMAC.init(secret_key);
            
            String          content             = Base64.encodeBase64URLSafeString(jsonHeaderString.getBytes()) + "." + Base64.encodeBase64URLSafeString(jsonBodyString.getBytes());
            String          hash                = Base64.encodeBase64URLSafeString(sha256_HMAC.doFinal(content.getBytes()));
            
            token           = "Bearer%3D" + content + "." + hash;
        }
        catch (NoSuchAlgorithmException ex)
        {
            Logger.getLogger(AzureJWTToken.class.getName()).log(Level.SEVERE, "Error occured while generating AuthToken : NoSuchAlgorithmException", ex);
        }
        catch (InvalidKeyException ex)
        {
            Logger.getLogger(AzureJWTToken.class.getName()).log(Level.SEVERE, "Error occured while generating AuthToken : InvalidKeyException", ex);
        }
        catch (Exception ex)
        {
            Logger.getLogger(AzureJWTToken.class.getName()).log(Level.SEVERE, "Error occured while generating AuthToken", ex);
        }
        
        return token;
    }
    
    /**
     * In case you need to convert base64 string to URL safe manually ()
     * 
     * @param base64EncodedString
     * @return 
     */
    public static String replaceUrlSafeChars(String base64EncodedString)
    {
        base64EncodedString = base64EncodedString.replace('+', '-');
        base64EncodedString = base64EncodedString.replace('/', '_');
        base64EncodedString = base64EncodedString.replace("=", "");
        
        return base64EncodedString;
    }
    
}

Using the JWT site we can verify the generated auth token. Just paste your token to JWT Debugger without the "Bearer%3D" part like this SS.

If it's correct you should be able to play it with Azure media player like this SS. Note that token will expire after 15 minutes according to our code. You can increase that accordingly.

Finally you can stream your video with following hard coded HTML source. Make sure you open the HTML with web server.

<!DOCTYPE html>
<html>
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
    </head>
    <body>
  <iframe src="//aka.ms/azuremediaplayeriframe?url=http%3A%2F%2Fchanaka.streaming.mediaservices.windows.net%2F481ea237-7f1f-48b8-a4c1-096f34c10cd6%2FBest%2520Sweet%2520Minions%2520Video%2520-%2520Let%2520The%2520UFO%2520Go%2520Home.ism%2FManifest&protection=aes&token=Bearer%3DeyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwOi8vbXl0ZXN0LmNvbS8iLCJhdWQiOiJ1cm46and0dGVzdCIsImV4cCI6MTQ1MzM1MjE2MH0.j6pe2Ej9aJJy0AvaqR7OI9FCcNTS1qI8y_EGzNYKT4g&autoplay=false" name="azuremediaplayer" scrolling="no" frameborder="no" align="center" height="280px" width="500px" allowfullscreen></iframe>    </body>
</html><!--
Iframe src dynamic url parameter :
    url : URL encoded encrypted video URL - java.net.URLEncoder.encode(encryptedVedioURL, "UTF-8")
    token : The token generated from the java code
-->

Hope you enjoy. Happy coding... :)

I like to thank Saliya Randunu for providing me lots of resources :)

Getting Started with WSO2 ESB Connectors

2014-04-09T12:21:00.001+05:30

Enterprise Service Bus (ESB) is the infrastructure which provides the implementation of the concept of Service Oriented Architecture (SOA). The connectors are the new concept coming with the wso2 ESB version 4.8.0 and above. Connector is an intermediate API layer, which will represent a real world developer API, while being in the ESB itself. With this post, the initial steps to create a connector will be described.

Download Maven Dependencies:
You need to checkout the source code from here and run maven install command on it. It will download set of required dependencies to your local maven repository. Following are the commands if you have a command line SVN and maven client.

$svn co https://svn.wso2.org/repos/wso2/carbon/platform/branches/turing/components/mediation/mediation-library/connector-template-utils/org.wso2.carbon.mediation.library.connectors.connector-archetype/
$cd org.wso2.carbon.connector/
$mvn clean install

Generate Connector Folder Structure:
Run the following maven command to generate maven project sample connector code.

$mvn archetype:generate -DarchetypeGroupId=org.wso2.carbon -DarchetypeArtifactId=org.wso2.carbon.mediation.library.connectors.connector-archetype -DarchetypeVersion=4.2.0 -DgroupId=org.wso2.carbon.connector -DartifactId=org.wso2.carbon.connector -Dversion=4.2.0

You may need give a name for the connector while processing the above process. (Let's proceed with entering googlebooks as connector name.) If the project creation success, you are ready to develop a connector with the following directory structure.

Fig. 1 : Connector Directory Structure

Before developing, there are some main components which you need to know.

Synapse Template : This is the component which calling to actual end point with the given request data. This will be placed in resources directory.

Proxy : This will handle the request coming inside to the ESB and response coming from the actual end point. This is not a part of connector, but need to invoke the connector.

Request : The request which follows the API published from the connector. This request can be REST or SOAP.

Connector for which API ?
There are large number of services which is provide in the world wide web to do a specific functions. In order to use these for developers, there will be one or several web service APIs like REST, SOAP or a client libraries in one or several languages are provided by the service provider. Out of these, REST will be the simplest one that others since it has less work. Here we are using Google Book REST API which does not need authentication for some of services like volume search.

First we'll go the Google Books REST API documentation and identify the categorization of the services in GoogleBooks API. The services are categories like Bookshelf, Volume, Bookshelf.volumes etc. So we'll create new directory "volume" under resources directory, to place services in the Volume category.

Writing Synapse Template :
You can have a look on the sample Synapse template generated at helloWorld directory. Following synapse template is providing the service at list volume. This should be placed under "volume" directory as listVolume.xml.

<template name="listVolume" xmlns="http://ws.apache.org/ns/synapse">
    <parameter name="searchQuery" description="Full-text search query string." />
    <sequence>
      <property name="uri.var.searchQuery" expression="$func:searchQuery" />
        <call>
            <endpoint>
                <http method="get"
uri-template="https://www.googleapis.com/books/v1/volumes?q={uri.var.searchQuery}" />
            </endpoint>
        </call>
    </sequence>
</template>

Create a component.xml file in "volume" directory, with following content.

<?xml version="1.0" encoding="UTF-8"?>
<component name="googlebooks_volume" type="synapse/template">
    <subComponents>
        <component name="listVolume">
            <file>listVolume.xml</file>
            <description>List volumes matching for the given query.</description>
        </component>
    </subComponents>
</component>

Modify the conector.xml file by adding new dependency as below,

<?xml version="1.0" encoding="UTF-8"?>
<connector>
    <component name="googlebooks" package="org.wso2.carbon.connector" >
         <dependency component="helloWorld"/>
         <dependency component="googlebooks_volume"/>
         <description>wso2 sample connector library</description>
     </component>
</connector>

Now you can build the connector code and test the implemented service foe volume search method. To build the connector, run maven clean install command from the org.wso2.carbon.connector directory.

$mvn clean install

This will generate googlebooks.zip file in the target directory. You can upload this zip file through the ESB interface under connector category. Make sure you enabled the connector after uploading.

Writing Proxy :
Now you can invoke the implemented service for search volumes, by writing a proxy through ESB server interface. (Home>Manage>Services>Add>Proxy Service>Custom Proxy>switch to source view)

<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
       name="googlebooks_listVolume"
       transports="https,http"
       statistics="disable"
       trace="disable"
       startOnLoad="true">
   <target>
      <inSequence>
         <property name="searchQuery" expression="json-eval($.searchQuery)"/>
         <googlebooks.listVolume>
            <searchQuery>{$ctx:searchQuery}</searchQuery>
         </googlebooks.listVolume>
         <respond/>
      </inSequence>
      <outSequence>
         <log/>
         <send/>
      </outSequence>
   </target>
   <description/>
</proxy>

Request :
Using a REST client, send following request, (you can use POSTMAN)

POST /services/googlebooks_listVolume HTTP/1.1
{
    "searchQuery":"aladin"
}

Hope you get the correct response ;)

OAuth2 with LinkedIn

2013-11-21T23:16:00.000+05:30

OAuth2 basically allows third party application, to access protected resource of a specific user. You may have seen some web applications/sites allows you to login through your Google, Facebook, LinkedIn etc accounts. As an example just try this with any mentioned account (Google, Facebook, Yahoo etc.). After giving your username password credentials you will be notified the authorities which the third party application will get if you agreed to proceed.

Some important term definitions in OAuth2:

Resource Owner - The person/entity which capable of granting the access to a protected resource.
Resource Server - The server which contains the protected resources, and capable of deal with access tokens in protected resource requests.
Client - The application which requests for protected resources on behalf of resource owner.
Authorization Server - The server which authorize resource owner, and issue access tokens to the client

Following figure shows basic flow of OAuth2 authentication process.

If you are really interested in digging more details, OAuth2 RFC will be the best resource. I strongly recommend you to go through at least up to 1.6 section in the RFC.

With this blog post, I show you how third party java application can access protected LinkedIn resources.

1. Register an application on LinkedIn. - This basically helps to create a link between LinkedIn and your application (third party application).

2. Get an access token - The access token represent the credentials to access protected resources. So your application does not need to store resource owner's credentials, but the access token.

3. Make API calls - Yes, after obtaining access token you are free to call API methods with that, until it expires.

LinkedIn official documentation contains detailed description on the above steps. You can follow those and familiar with this matter. Make sure you tried both Authorization code request and Access token request.

Now we need our application to store access token. For this we have to do some coding in the application. We can easily get authorization code with a GET request. Then we need a POST request to get access token using the authorization code and other required parameters.

So, we can get the authorization code from request parameters in the redirect uri, and make server side code request to get the access token. For this we have to use a HTTP client library. You can use Apache HTTP client library components to approach this. For process the responce you can use XML parser like DOM or SAX. Try out the sample application yourself after reaing README file. Happy coding !

Download Source Code

Thanks to mkyong for making me comfortable with his SpingMVC example.

Limit textarea character count in IE8

2013-01-08T01:13:00.000+05:30

After long time I'm going to start blogging again. Yes, it's on your favorite browser IE :). As you know, with HTML 5 there is lots of additional features support for major browsers except IE. However it's nice to hear that IE10 is having more support for HTML 5.

With the HTML 5, maxlength attribute was introduced for textarea HTML element. But IE 8 does not have support for this. I did some work around to find a simple solution for this and seems I found it.

Make sure you are using HTML 5 doctype. Note that I have target only IE here, but not the already support browser for maxlength attribute in teaxtarea elements.

<!DOCTYPE HTML>
<html>
  <head>
    <script type="text/javascript">
      function getCaret(el) {
        if (el.selectionStart) {
          return el.selectionStart;
        } else if (document.selection) {
          el.focus();
          var r = document.selection.createRange();
    
          if (r == null) {
            return 0;
          }
    
          var re = el.createTextRange(),
          rc = re.duplicate();
          re.moveToBookmark(r.getBookmark());
          rc.setEndPoint('EndToStart', re);
          return rc.text.length;
        }
        return 0;
      }
 
      function limitPasteData(obj, limit) {
        // Do it with Internet Explorer
        if (window.clipboardData) {
          if (document.selection && document.selection.createRange) {
            obj.focus();
            var modifiedTxtLen;
            // If text has been selected
            if (document.selection.createRange().text.length > 0) {
              modifiedTxtLen = obj.value.length - document.selection.createRange().text.length + window.clipboardData.getData("Text").length;
    
              if (modifiedTxtLen > limit) {
                var neededLen = limit - (obj.value.length - document.selection.createRange().text.length);
                document.selection.createRange().text = window.clipboardData.getData("Text").substring(0, neededLen);
                return false;
              }
            } else {
              modifiedTxtLen = obj.value.length + window.clipboardData.getData("Text").length;
              if (modifiedTxtLen > limit) {
                var neededLen = limit - obj.value.length;
                var neededTxt = window.clipboardData.getData("Text").substring(0, neededLen);
                var mofifiedText = obj.value.substring(0, getCaret(obj)) + neededTxt + obj.value.substring(getCaret(obj), obj.value.length);
                obj.value = mofifiedText;
                return false;
              }
            }
            // If not exceed limit just paste it
            return true;
          }
        }
      }
    
      function limitTypingData(obj, limit) {
        // Other browsers may have support for maxlength attribute
        if (navigator.appName == 'Microsoft Internet Explorer') {
          if (obj.value.length > (limit - 1)) {
   
            if (document.selection && document.selection.createRange) {
              obj.focus();
              // If only text has been selected, allow to type once
              if (document.selection.createRange().text.length > 0) {
                return true;
              }
            }
            return false;
          }
        }
      }
    </script>
  </head>
  <body>
    <textarea id="txtArea" cols="20" rows="5" maxlength="15"
        onpaste="return limitPasteData(this, 15);"
        onkeypress="return limitTypingData(this, 15);" ></textarea>
  </body>
</html>

But remember that the behavior of these functions in IE8 will not 100% equal to default maxlength behavior of supporting browsers. But we can still do the work with this.

Apart from that I would like to notice you that, new line character(s) may be changed in IE than others. IE always take it as '\n\r' and other browsers may take it as '\n'. So that, either you have to handle that or use additional space from the back end database fields.

Have a fun with IE 8 :D

Code to interfaces, not to implementations

2012-09-06T19:43:00.000+05:30

In object oriented programming, interfaces are commonly used to implement more flexible software. Interfaces are very useful to extend and improve a software application. While designing a software its a good practice to use interfaces where applicable.

Lets think about a simple CricketPlayer class (Figure 1) with a method play(). There can be several classes and methods using CricketPlayer class. Think we need to give the support for Football player as well within this software. Then we need to repeat the wheel for new purpose(Figure 2).

Figure 1 : Initial cricket class

Figure 2 :Modification for support football player

But just think if we use common interface (Figure 3) for any type of player, we can easily get rid of the problem. Player interface will have play() method which may be different behaviour for different players and getSport() method which represent the type of the player (sport). CricketPlayer and FootballPlayer subclasses can implement their own behaviours according to related declarations in the Player interface. Coding to this interface will save us with very limited changes, while any kind of player class is going to use in our software. Using an interface software can be extend easily and will be more generic.

Figure 3 : Player interface

Recently I saw practically very useful place where interfaces are using. Most of the industry applications are using the MVC architecture to better maintainability of the code. There may have requirements to use the same applications with different DBMS for different clients. Using an interface with n-tier architecture, this support can be easily implemented.

Following diagram (Figure 4) shows the way of using this stratergy in n-tier architectural application. From the Controller classes which deals with request and responses while dealing with the business logic of the application, calls to the service classes to do specific data manipulation, transaction etc. Service classes calls to Dao classes which deals with data transactions. Service class doesn't directly use either MySqlUserDoa which has MySQL support or XQueryUserDoa which has XQuery support. It only care about BaseUserDao interface, and doesn't care about real object underling the BaseUserDao interface.

Figure 4 : Dao interface

Implementaion under this process is quite simple. Control class read configuration file (or from any other way) and get the Query Language supposed to use according to the requierments. Then create an object of supported Dao class and assigned to a variable of BaseUserDao (interface) type. After this initializing process of BaseUserDao type variable, it can be used by the service classes inside their functions.

BaseUserDao userDao;
if (queryLanguage == 'MySQL') {
    userDao = new MySqlUserDao();
} else if (queryLanguage == 'XQuery') {
    userDao = new XQueryUserDao();
}

Since application is not creating concrete classes (objects are dynamically created in run time and will not directly creating objects for any case), it will not be limited. And since it uses an interface, switching for a new DBMS can be achieved by only adding new file which inherit the interface and modify the object creating process(modify above code) and giving the new DBMS as parameter, using a configuration file or any other way.

Coding to interfaces, not to implementations principle is a good practice for a good software and it make our work more easy. If you are thinkng of creating extendable software application, this practice will very helpful. So, its time to play with interfaces. :)

Singleton with PHP

2012-08-28T18:04:00.000+05:30

        Singleton pattern is under creational design patterns which deals with mechanisms of creating objects according to the situation. I recently used this pattern with PHP. It's very important to get to know use of this pattern.

              Singleton pattern can be used for always keep one object of a specific class while an application is running. It save the memory which is going to allocate for new objects of the class in every where inside the application. Singleton practically used for logger classes which provide global logging access point whithout logging each and every application after the main loggin operation, configuration classes which consist configuration data of an application, access resources which are in shared mode like multi threading applications, Factory design pattern which dynamically create the objects etc.

         It's very easy to implement singleton with PHP (as well as other languages). What we need to do is make a private constructor and control the creating object of this class by a public method. Following code snippet shows how to do this.

class MySingleton {

    private static $instance = null;

    public static function getInstance() {
        if(is_null($this->instance)) {
            $this->instance = new MySingleton($params);
        }
        return $this->instance;
    }

    private function __construct($params) {
        // set parameters
    }

    public function doSomething() {
        // some code
    }

}

Since the constructor is private only the class itself can be access the constructor. getInstance method is accessible for anywhere and since that is a static method it can be used anywhere without creating an object of this class. This function handles the situations of the need of a new instance of MySingleton class by calling the private constructor if needed. Static variable is used because of the restriction of using non-static fields inside static functions.

In the places where needs to call doSomething method in MySingleton class, we call as,

MySingleton::getInstance()->doSomething()

or, if the functions inside the class is frequently used,

$my_singleton = MySingleton::getInstance();
$my_singleton ->doSomething()

It's very simple. Hope this helpful. Have a fun with singleton. :)

සැරයටියෙනුත් යනෙනා තුරා.....

2012-08-25T00:30:00.000+05:30

කෙලිලොලෙන් ගතකරපු අපේ ඒ අතීතය.....
මතකයක් පමණක්ම කර,
ඇයිද නුඹ අපව තනිකර ගිලිහුණේ.....

ඒ අතීතය,
හැමදාමත් අද ඊයේ වගේ දැනේවි,
නුඹේ කතා කනට ඇසේවි,
නුඹේ සිනහව හිතේ මැවේවි,

ඒත්.....
නුඹේ කතාව, ඒ සිනහව
මතු කිසිදිනෙක අප හැබැහින්ම නොවිදීවි.....

නුඹ දමා ගියත් අපව,
මිතුරෙකුව හදේ රැදී සිටී හැම දිනෙක.....

මතු භවයෙදිත් අප මිතුදමින් බැදෙමු
ඒත්.....
මෙලෙස අකාලයේ ලොවම හැර දමා යන්නට නොව,
සැරයටියෙනුත් ගමන් කරනා තෙක්ම.....

GSoC 2012 - 13th Week Progress

2012-08-20T21:31:00.000+05:30

This is the last week we interact with the coding and documenting under GSoC 2012 official time line. In my last week I implemented the feature request simple sort for process list. And improved some tests for PMA_DisplayResults class.

In this week I did some improvements in tests of PMA_DisplayResults class. As the documentation, there were not much things to mention as details in Documentation.html. I only mentioned about the feature of displaying multiple results for queries as well as stored procedures. While I'm implementing the functions phpdoc comments and block comments were improved. So those things make me easy in documenting session. A feature request I proposed to do at the end of the last week was already fixed (My patch has been accepted for that). So I didn't involve with that.

Dieter has suggested to change the class structure for PMA_CommomFunctions class. After discussing in mailing list I decided to make any change after the GSoC 2012 session. I will be stop my further work related to code and documentation improvements under GSoC 2012 session by today.

In next week I'll submit my evaluation form. I wish all GSoCers who passed the mid evaluations to get success this Great event GSoC 2012 with Open source :)

GSoC 2012 - 12th Week Progress

2012-08-13T18:32:00.000+05:30

Its more one week to do the remaining work of GSoC 2012 project. Last week I introduced new links inside information_schema database with modifying a previous code for showing links inside mysql database. As well support for MIME transformations in query results and VIEWs were implemented. More improvements related to VIEWs were done.

In this week I implement the feature request simple sort for process list. That was little older feature request and the description of the feature request itself suggested the fix. But with the time those files were changed and renamed. So I improve the behavior of server_status.php file to do the required thing. Since the sorting needs to specify the table, I used prcesslist table of information_schema database. Apart from these I did some test implementations for the functions in PMA_DisplayResults class.

In my next week I'm intending to improve some tests for PMA_DisplayResults classwhile doing any remaining minor improvements in my codes.

GSoC 2012 - 11th Week Progress

2012-08-06T22:04:00.003+05:30

This is my 11th week of GSoC 2012 project. Its one more week to active work in code level. Last week I did some improvements on showing syntax highlighted SQL statement in PROCESSLIST table in information_schema database and introduce some links inside mysql database related to displaying data fields.

This week as the first task I introduce some links inside information_schema database. In previous week I implement same kind of thing but in more specific way to that task. So I change the previous implementation where it can be used to my new task. I changed the structure of the global variable $GLOBALS['mysql_schema_relation'] and renamed it as $GLOBALS['special_schema_links'] and renamed the file as special_schema_links.lib.php. The new structure of the array can be represented by following snippet.

$GLOBALS['special_schema_links'] = array( 
    // Database name is the major element
    'mysql' => array(
        // Table name
        'db' => array(
            // Column name
            'user' => array(
                // Main url param (can be an array where represent sql)
                'link_param' => 'username',
                // Other url params
                'link_dependancy_params' => array(
                    0 => array(
                        // URL parameter name
                        // (can be array where url param has static value)
                        'param_info' => 'hostname',
                        // Column name related to url param
                        'column_name' => 'host'                        
                    )
                ),
                // Page to link
                'default_page' => 'server_privileges.php'
            )
        )
    )
);

By using this array, links can be generated where needed specially inside schema like mysql, information_schema etc.

          Next I moved into implement feature request for support MIME transformations in query results. Some of the query results which show only field with transformation and large data (blob fields etc) were not displayed their expected transformation in results. PMA has avoided that kind large data comparisons for SQL queries. So that, $where_clause parameter is not generating properly. So, assuming the length of the data fields will not same, I used the length of the data field to generate proper $where_clause and compare inside SQL query. Now query results display the expected out put with their transformations if any.

            Then I moved into give the support for transformed data for VIEWs. In PMA the columns with any transformation are stored inside column_info table of phpmyadmin database. But the problem was these information were not stored in column_info table while creating VIEWs. So I did some changes to store needed details of transformations in VIEWs whie creating them by the link (create view) in browse mode of a result set. A new file called tbl_views.lib.php was introduce to place related functions.

        As well when deleting a column, table, view or database the related information on transformations were not deleted from the column_info table. So I implement a function to delete relevant entries and placed the necessary places to do the work.

In my next week I intend to implement the feature request on simple sort for server_processlist.php.

GSoC 2012 - 10th Week Progress

2012-07-30T22:29:00.001+05:30

This is my 10th week of GSoC 2012 project. In last week I implemented support for displaying results for multiple SQL query statements and stored routines with multiple queries. SQL query window can be used to both cases above.

There were some issues in my previous code. So I first moved into fix one issue. It takes bit more time to fix it than I thought.

Then I moved into next feature implementation. Mainly there were two parts in this feature.

1. Syntax highlighting for SQL in PROCESSLIST table in information_schema database

2. Relations inside mysql database

For the first task, I used a new public property inside PMA_DisplayResults class which contains all the details of columns which need to be apply syntax highlight effect. So that any other table field also can use this feature. Now the SQL field is syntax highlighted in PROCESSLIST table.

Second task is not related to actual relations with foreign keys, it means just link data rows to corresponding pages where possible. As an example, User column of db table in mysql database, can be linked to edit user privileges of corresponding user according to the User column value. A new file called mysql_schema_relation.lib.php has introduced which have a global variable with all the column information of tables which needs to be display as links where possible. From each element of this variable (array), one link can be generated.

$GLOBALS['mysql_schema_relation'] = array(
    'db' => array(        // db table in mysql schema
        'db' => array(    // Db column in db table
            // link_params array contains the get params,
            // if array element is string, value of parameter is set as column value
            // if any of array element is array, url param is the first data of the array and value is second data of the array
            'link_params' => array('db'),
            // This represent the page to link
            'default_page' => 'index.php'
        ),

        'user' => array(    // User column of db table
            'link_params' => array('username'),
            'default_page' => 'server_privileges.php'
        )
    ),

    'proc' => array(
        'db' => array(
            'link_params' => array('db'),
            'default_page' => 'index.php'
        )
    ),

);

Above array is represented as lowercase string array. So when comparing the strings those need to be convert to lowercase. (strtolower() function can be used.)

Now there can be see several links inside mysql schema. In my next week I'll implement the feature, linking from information_schema to phpmyadmin schema.

GSoC 2012 - 9th Week Progress

2012-07-23T23:11:00.000+05:30

        In last week I did some improvements in PMA_DisplayResults class and in test cases. This week I moved into implementation of a feature request. Its on display results for the multiple queries.

         Multiple query execution can be done by using routines ( stored procedures and stored functions) or just executing SQL statement with multiple queries at once. In PMA those could be done by using SQL query window or using routines tab under any table.

          PMA was not supported for executing multiple queries using the SQL query window. While executing it shows the results for the last single query in the multiple SQL statement. That was due to use of mysql_query() function for any case. Since mysql extension is currently not support for the multiple query execution I used mysqli_multi_query() function under mysqli extension. It returns multiple result sets depending on the executed SQL statement. By iterating through the results I displayed this resulted tables in the same page in top to bottom manner. And I made it with very limited operations on the results in same page. Unless it becomes more complex and hard to handle the code. Currently there are two known issue.

1. After executing insert/ edit/ delete query using the query window, page does not show the query window back.
2. After executing multi query SQL statement, browse any table gives notices and some errors.

        After those modifications I moved into routines. That was very cool feature in PMA. But it was not complete. For any routine it shows only one row from only one table. So I improve the behaviour of that feature to return expected outcome according to the routine. Now it shows even multiple results sets with all the needed rows.

        In next week I try to fix those issues in SQL execution mode. And I'll implement new inline transformations for PMA as the next feature implementation.

GSoC 2012 - 8th Week Progress

2012-07-16T13:46:00.000+05:30

I'm pretty sure that most of the GSoCers are very happy in the 8th week of their GSoC project. Because of the happiness of gone through the mid evaluation. In last week I break some functions into more sub functions and introduce global variables (class variables) and __get, __set magic methods for PMA_DisplayResults class with the suggestion of Michal. As well some improvements in tests also done.

According to last week modifications in PMA_DisplayResults class, initialization of the global variables in that class were done inside getTable() method. But as suggested from Michal, I did that initialization part in a separate method called setProperties() and call that function from the sql.php file just before call the getTable() method.

As well I found a issue with my code with the print view screen. I fix those issues both in vertical display mode and horizontal display mode. Then I move into improve unit tests. There were several errors in tests related to PMA_DisplayResults class due to my recent changes. The problem was, needed property values were not set at the run time of tests. I modified those tests to set those properties using magic __set() method. But that was not worked. After searching this issue, I found that, there is a problem with using php property_exists() method is not properly working with the magic methods. So that I introduce a new property $_property_array which contains all the previous properties except $_common_functions. Then I remove previous properties which represent in $_property_array and use php array_key_exists() method inside magic methods.

It takes lot of time to run the all tests in PMA. So that I try to run tests with individual file by giving arguments in command line. But it gives errors again and again due to missing required files and undefined variables, constants etc. After that, I edit phpunit.xml.dist file in order to just run the needed file. With that modification I could run those tests quickly. At the moment previous errors in tests related to PMA_DisplayResults class were fixed.

In next week I'll move into start the work I suggested after mid evaluation. I'll start implementation of display multiple query results.

GSoC 2012 - 7th Week Progress

2012-07-09T20:05:00.002+05:30

This is seventh week of the GSoC 2012 projects. As well this week we need to submit our mid evaluation. So this is a key point of GSoC 2012. Last week I merge the code with master which had Alex's major changes related to his project. And did some modifications regarding mime default function. As well I started to write tests for PMA_DisplayResults class.

In this week I continued writing tests for PMA_DisplayResults class. While those improvements Michal assign me another two tasks.

1. Reduce the large number of parameters passing to functions in PMA_DisplayResults class by storing the variable states inside the class itself

2. Break some functions to more small functions

So, first I did the 2nd task. With that I introduce several sub functions by breaking _getTableHeaders(), _getTableBody() and getTable() functions.

Then I moved into 1st task. I stored the commonly used fields as private properties of the class. There are around 25 private fields in that class now. Since those properties are not initialized when creating the object of this class, most of them were initialized inside the getTable() function.

After I again moved into improving tests. At this point most of the existed tests were failed with the new changes. So I introduce __get() and __set() method for access any private field in PMA_DisplayResult class by out side. Then I use those functions to set private properties while improving tests. But still the failures are remaining in some tests. Apart from that setCommonFunctions() function was removed from PMA_DisplayResults class due not use of that method.

In next week I'll continue test cases as much a possible. But before that I'll do some modifications to PMA_DisplayResults class which was suggested by Michal recently.

GSoC 2012 - 6th Week Progress

2012-07-02T10:17:00.000+05:30

In last week I almost finish the implementing class behavior of PMA_CommonFunctions.class.php but not committed due to some conflicts while merging the code to with master branch.

In this week also I got lots of conflicts while merging the code. Since the functions inside PMA_CommonFunctions class (in master common.lib.php file) were used in many places over the PMA the possibility of occurring a conflict is high. With the merge of Alex's code I again got many conflicts. As well I had to do more modifications because Alex has introduced several new files and they have used the functions in common.lib.php . After did those modifications I committed the code with 4 known issues in master. At the moment two are already fixed in master branch. Others are,

1. When the session expires, after click logout I get following notice,

Notice in ./libraries/relation.lib.php#35
Undefined index: controllink

2. When incorrect login, following warning raised,

Warning in ./libraries/dbi/mysqli.dbi.lib.php#97
mysqli_real_connect(): (28000/1045): Access denied for user 'root'@'localhost' (using password: YES)

After that I moved into finding the possibility of using PMA_mimeDefaultFunction() function which is currently inside the core.lib.php file as a global function, inside the PMA_DisplayaResults class. This function is only used by the functions in PMA_DisplayResults class indirectly with the help of a variable. I moved the PMA_mimeDefaultFunction() function from core.lib.php file to PMA_DisplayResults class and renamed as _mimeDefaultFunction(). The places where calls this function indirectly were modified by adding '$this->' suffix ($default_function = '_mimeDefaultFunction'; $this->$default_function()).

After those improvements I moved into implement tests for the functions in PMA_DisplayResults class. Yasitha has already implemented some tests for some functions. I started to continue that work and finished some tests.

In next week as the major task I'm intending to implement tests for remaining functions in PMA_DisplayResults class. If possible I'll be do some minor improvements in PMA_CommonFunction.class.php

GSoC 2012 - 5th Week Progress

2012-06-25T23:54:00.001+05:30

This is the 5th week of my GSoC 2012 project. In last week I started to implement class behavior in common.lib.php file.

This week I finish remaining part of the implementing class behavior in common.lib.php file. Now the class is named as PMA_CommonFunctions and file is appeared as CommonFunctions.class.php. This class is used singleton pattern to avoid the creation of many objects by several places, which needs more memory. If any function or file with procedural codes contain more than one function inside PMA_CommonFunctions class the class object is assigned to a variable called $common_functions and used that variable to call needed functions inside that class. The classes which use the functions in PMA_CommonFunctions class, were modified by adding getter setter methods to get/set PMA_CommonFunctions class object.

I faced several problems while implementing this part. Some functions were used more than 400 times in many files. All of those cannot be just replaced due to some JS functions also appeared with same name. After completing those modifications, I could only see login screen on my browser. After login nothing displayed in my browser. Then according to guidance of Michal, I enabled write error log to file. So that I could debug the code. Then I found that the error was due to calling non-static methods in side static functions in some classes. Those were replaced by direct calling using the static method (PMA_CommonFunctions::getInstance()->someFunction()) instead of get the use of variable for store class instance.

Then the code was seems work fine. At the moment code is not committed yet due to some conflicts while merging the original PMA code. I'll be able to commit those changes as soon as after resolving them.

In next week I'll start to looking at the possibility of using PMA_mimeDefaultFunction() function (at the moment this is inside core.lib.php file) inside PMA_DisplayResults class as suggested by Marc. If possible unit tests were implement for the functions inside PMA_DisplayResults class.

GSoC 2012 - 4th Week Progress

2012-06-18T18:45:00.000+05:30

In my third week I introduce a class behavior for display_tbl.lib.php file. Now that file has appeared as DisplayResults.class.php and the class inside it is named as PMA_DisplayResults.

Since there was a task not completed in last week, I first completed that. It is removing imported super globals from the functions. There were several functions which used local variables which are imported from $GLOBALS super global array. I removed them and directly used super globals where needed. As well for the super globals which were not change during one execution cycle of the PMA code, I assign them to global variables in the PMA_DisplayResults class (local for that class). That global variables are initialized inside constructor.

As next task, I was moved into fix the bug which was occurred due to breaking _getTableBody() function into subfunctions. It was very helpful for me that Marc had already mentioned the bug is about breaking the feature, which display a link for foreign table value. When I going through the code I saw that was occurred due to incorrect method calling of _getDataCellForNumericFeilds() function (not according to the signature). Now the bug is fixed.

Then I moved into refactoring common.lib.php file. First I remove the HTML renderings inside the function it selves and return a string instead of that. As well some conventions are used according to my proposal. And the all functions names were modified by removing phrase 'PMA_'.

Within next week I'm intending to complete implementing class behavior of common.lib.php file. While refactoring DisplayResults.class.php file I have moved a function (PMA_mimeDefaultFunction) to common.lib.php file and now again move it to core.lib.php file for keep it as global function. I intend to again look at the possibility of get that function back to DisplayResults.class.php file.

GSoC 2012 - 3rd Week Progress

2012-06-12T01:42:00.005+05:30

In my 2nd week, I did some improvements relating to readability and usability of the code including breaking large functions into sub functions. After that step, display_tbl.lib.php file contained many sub functions.

In this week, I mainly focused on applying class behavior to the display_tbl.lib.php file. After getting the confirmation from Michal Čihař for the rough design of that PMA_DisplayResults class, I started my work.

First I search for the functions which are used by the outsiders. There were only two functions (PMA_getTable(), PMA_setConfigParamsForDisplayTable()) were used by sql.php file. Excepting those two functions, private key word was used as access modifier for all other existed functions. For constructor and the above two functions were appeared as public so that any outsider can access those. All existed function names were modified by removing 'PMA_' since they becomes to local functions. The file display_tbl.lib.php was renamed as DisplayResults.class.php .

Then I looked into, remove strings used for conditions in comparisons and introduce constants instead of that. There were many constants after doing this step. If there is any need for change the value of those strings, we can do it by just changing in one place. That is an advantage of using constant.

With these processes there were lots of code violations had been occurred. Most of them were fixed excluding some warnings which were due to very long comparisons inside if conditions.

Introducing global variables for the PMA_DisplayResults class is not finished yet. It will be done in next week. Unexpectedly there was a bug due to one of my previous commits which was pointed by Marc Densile. I am going to fix it in the next week. As well, I'll start to look into refactoring other files related to display query results. Mainly I'll focus on common.lib.php file.

Design for the PMA_DisplayResults class of PMA

2012-05-22T17:14:00.000+05:30

PMA_DisplayResults class will contain the functions specific to the displaying query results and handling special tables like `PROCESSLIST` which provides informations on current threads running in PMA. At the current code base, these functions are appeared as global functions inside display_tbl.lib.php file. After introducing PMA_DisplayResults class instead of this set of functions, the file will be renamed as DisplayResults.class.php .

Class diagram (not the completed one) for the PMA_DisplayResults class

Following code snippet represent some rough implementation of the PMA_DisplayResults class,

class PMA_DisplayResults {
    
    const DIRECTION_LEFT = 'left';
    const DIRECTION_VERTICAL = 'vertical';
    
    private $_db;
    private $_table;
    
    public function __construct($db, $table) {
        $this->_db = $db;
        $this->_table = $table;
    }
    
    public function getTable() {
        
        // Related code for getTable function
        $str .= $this->_getTableHeaders();
        // end of the related code
        
    }
    
    private function _getTableHeaders() {
        //code...
        $msg = PMA_getMessage($this->_db, $this->_table);
        //code...
    }
    
}

As the only outsider accessed Display class, the sql.php file will be use,

// inside sql.php file
$displayResultsObj = new PMA_DisplayResults($GLOBALS['db'], $GLOBALS['table']);
echo $displayResultsObj->getTable();

GSoC 2012 - 2nd Week Progress

2012-05-22T00:38:00.002+05:30

This is my second week of GSoC 2012 project since I have started my coding before the official day. I have started to refactoring the file display_tbl.lib.php in PMA. Last week I remove the HTML rendering inside the functions it selves in this file. Some readability improvements also done.

In this week, I mainly focused on improving the readability of the code inside display_tbl.lib.php file. There were several function which having more than 200 code lines in this file. I broke those functions into sub functions in a way that each sub function do a specific portion for the bigger function. As well the sub functions were not having very large code blocks and those functions were placed just after the referring function. The names of these sub functions were little longer sometimes, because of they express valuable details for the developer at the first time.

In some places there were used similar code segments again and again even in the same functions. Those similar code segments were moved into new sub functions with the help of some additional parameters to separate out the small differences. (PMA_getOperationLinksForVerticleTable, PMA_getColumnParams etc.)

I have mentioned those larger functions which were broken into sub functions with there names below.

PMA_getTableNavigation
- PMA_getMoveBackwardButtonsForTableNavigation
- PMA_getShowAllButtonForTableNavigation
- PMA_getMoveFarwardButtonsForTableNavigation
- PMA_getAdditionalFieldsForTableNavigation

PMA_getTableHeaders
- PMA_getSortByKeyDropDown
- PMA_getDataForResettingColumnOrder
- PMA_getOptionsBlock
- PMA_getFullOrPartialTextButtonOrLink
- PMA_getFormForMultiRowOperations
- PMA_getCommentForRow
- PMA_isInSorted
- PMA_getSortingUrlParams
- PMA_getSortOrderLink
- PMA_getDraggableClassForSortableColumns
- PMA_getDraggableClassForNonSortableColumns

PMA_getTableBody
- PMA_getUrlSqlQuery
- PMA_getColumnParams
- PMA_getVerticalDisplaySupportSegments
- PMA_getModifiedLinks
- PMA_getDeleteAndKillLinks
- PMA_getPlacedLinks
- PMA_getResettedClassForInlineEdit
- PMA_getClassForDateTimeRelatedFields
- PMA_getDataCellForNumericFeilds
- PMA_getDataCellForBlobField
- PMA_getDataCellForGeometryFields
- PMA_getDataCellForNonNumericAndNonBlobFields

PMA_getVerticalTable
- PMA_getOperationLinksForVerticleTable
- PMA_getCheckBoxesForMultipleRowOperations

PMA_getTable
- PMA_getOffsets
- PMA_getSortParams
- PMA_getSortedColumnMessage
- PMA_setMessageInformation
- PMA_getMultiRowOperationLinks

Apart from this I used some coding conventions for improve the readability. If the function is not too small one, an empty space were kept both after starting bracket of the scope and before ending bracket of the scope. Not only for that, wherever I saw the codes are seems packed and hard to read, I put empty lines between code segments. As well I followed the conventions suggested in my proposal.

At next week, I'm going to change my plans by giving the place to most prioritized task at the moment. I am going to make the display_tbl.lib.php file as a PHP class. As well I'm going to reduce the usage of global variables which are used in most of the functions. The strings used inside the functions for comparisons, will be replaced with constants. The design for the class will be discussed with the community on next week.

GSoC 2012 - 1st Week Progress

2012-05-14T18:45:00.001+05:30

This is my first week of Google Summer of Code project which is Refactoring: Displaying query results under phpMyAdmin organization. Since I have my final examination, I started my work before the official day for start coding(21^stMay). It was a great experience that working with world wide open source community. That experience will be very helpful for me to manage my work in the upcoming weeks.

According to my project schedule, I was dealing with, refactoring the file "display_tbl.lib.php" in this week. That is a file with 22 global functions. Some of them are really huge functions. As well, though library files are suppose to provide functions which having logics(calculations, concatenation, manipulation etc), while being in a separate layer (data/model layer), this file have some HTML renderings inside themselves. And the naming convention of some methods were different from the phpMyAdmin (PMA) standards.

In this week, I modified the all the needed functions in "display_tbl.lib.php" file, to remove the HTML renderings inside themselves. Instead of that, that functions are now returning a string which having relevant HTML content for each function. Since inside some of functions are using the functions in "common.lib.php" file and some of them also had the same issue, I modified them as well. The unit tests for each also modified according to the changes I did in corresponding functions.

The names of most of the functions were changed due to changing of the behavior of some functions. Those names were changed with adding "get" phrase. I was also looked in to modify the code to keep necessary indentation, since it improve the readability of the code.

At next week, I'll more look into improve the readability of the code with adding the suggested conventions in my proposal. As well the huge functions will be braked in to several functions. The unit tests also will introduced as possible for those function.

Ready to walk with GSoC 2012

2012-04-25T21:32:00.000+05:30

I am very happy to say that I have been selected for a project in Google Summer of Code 2012. The project idea is "Refactoring: Display query results" which is under phpMyAdmin organization. In this project I'm going to refactor the code which is related to displaying query results and implement some feature requests.

Short description of my project:

In current code base of phpMyAdmin, the code for displaying query results is not much reusable. To improve the ability of re usability of the code, I have suggestions to refactor the code with better approaches. The code is going to be improved by applying Object Oriented Programming concepts. Along with that, new features like support for displaying results for multiple queries at once, introducing some more built-in transformations where it is applicable will be considered.

Now I'm ready to walk with GSoC 2012 and hope to do a great job with phpMyAdmin developers and other GSoCERS. :)

More details on my project can be found at : http://www.google-melange.com/gsoc/proposal/review/google/gsoc2012/chanaka777/1

My git repository can be found at : https://github.com/Chanaka/phpmyadmin

Test Driven Development with PHPUnit

2012-04-15T17:00:00.000+05:30

TDD (Test Driven Development) concept is common practice of most of the companies which involve with the long time projects. It helps to keep the static behavior of functions over the development period. The functions are implemented after implementing test methods (unit tests) for that function. After implementing the function, run the tests and if any test is failing, again modify the function. This process repeats until the all tests are passing.

Test method is a function which check whether the specific behavior of a function is correct. Any function can have at least two test methods. one positive test and one negative test. Most probably a function may have many test methods. The class in which all the test methods are placed is called Test Case. By setting up a test suite for this class, the all tests methods can be run at once.

In long time projects like open source projects, the functions are frequently modified by the developers during a long time. If the modified function is not given the previous behavior of that function, the output for other places will be going wrong. It can be cause to failures. By writing tests this problem can be reduced. Before and after modifying the test cases, by running the tests, developer can be verify the function behave correctly. As well by going through the test cases helps even understand what actually function does.

To follow this TDD concept with Java, JUnit framework can be used. As well with PHP, PHPUnit framework can be used. Let's do some experiments with PHPUnit with Ubuntu OS.

Install PHPUnit framework

$sudo apt-get install phpunit

Now create your test case as a sub class of 'PHPUnit_Framework_TestCase' having just one dummy method to verify we can forward without issue.

<?php

    require_once 'PHPUnit/Framework.php'; // installed file with phpunit framework which is located at : /usr/share/php/PHPUnit/

    class MyClassTest extends PHPUnit_Framework_TestCase {

        // This function just return true
        function testCheckAllWorksFine() {
            $this->assertTrue(true);
        }

    }

?>

Run this MyClassTest.php file as :

$phpunit MyClassTest.php

If the result is as following (Fig. 1), you can proceed the work with phpunit.

Fig. 1

Characters it returns :

.     : pass
F    : fail
E    : error

The functions starting from 'assert' is implemented in the framework. As well there are more useful functions we can use with this framework. You can find more assert functions by visiting http://www.phpunit.de/manual/3.6/en/writing-tests-for-phpunit.html#writing-tests-for-phpunit.assertions.

I have used following useful assert functions for this article :
assertTrue() : Check whether the parameter is True
assertFalse() : Check whether the parameter is False
assertEqual() : Match two results

Start work

1. Requirement : need a function which accept two strings and return the length of the most lengthy string, if both arguments are not strings return false

* First just declare your function properly with arguments

<?php
    class MyClass {   

        function getMaxLength ($str1, $str2) {    

        }

    }
?>

* Now start to write test methods,
a). Check the arguments

<?php

    require_once 'MyClass.php';
    require_once 'PHPUnit/Framework.php';

    class MyClassTest extends PHPUnit_Framework_TestCase {

        var $obj;

        // This function calls before any test merhod run
        function setUp() {
            $this->obj = new MyClass();
        }

        // Positive : Test getMaxLength for correct arguments
        function testGetMaxLengthForAtLeastOneIsString() {
            $paramArray = array("str1", 2);
            $result = $this->obj->getMaxLength($paramArray[0], $paramArray[1]);
            $this->assertEquals(4, $result);
        }

        // Negative : Test getMaxLength for non of arguments are string
        function testGetMaxLengthForNoStrings() {
            $paramArray = array(array(1,2), 12.3);
            $result = $this->obj->getMaxLength($paramArray[0], $paramArray[1]);
            $this->assertFalse($result);
        }

    }

?>

The setUp() will call automatically before running any test method. It can be used for create a fresh objects, load data for databases if using etc. All test methods must be start with 'test'. Using the function name which is going to use by specific test method regardless of the length of the test method name, will be increase the readability and understandability while reviewing later.

b). Check whether the function returns an integer

// Test getMaxLength for output integer for correct arguments
function testGetMaxLengthForCorrectReturnType() {
    $paramArray = array("str1", 2);
    $result = $this->obj->getMaxLength($paramArray[0], $paramArray[1]);
    $this->assertTrue(is_int($result));
}

c). Check whether is returns the length of the most lengthy string

// Test getMaxLength for Length of the lengthy string
function testGetMaxLengthForCorrectLength() {
    $paramArray = array("short_string", "lenghty_string");
    $result = $this->obj->getMaxLength($paramArray[0], $paramArray[1]);
    $this->assertEquals(14, $result);
}

Now the most needed test cases are implemented. From the above test methods, some scenarios like check the function return length of the string for arguments having only one string, is already covered. Now just run the test and see most of them are failing. Its time to start implement the actual function in a way that all te test cases are passing.

Function can be written as :

function getMaxLength($str1, $str2) {   

    if(is_string($str1) || is_string($str2)) {
        if(is_string($str1) && is_string($str2)) {
            $max = (strlen($str1) >= strlen($str2)) ? strlen($str1) : strlen($str2);
        } elseif (is_string($str1)) {
            $max = strlen($str1);
        } else {
            $max = strlen($str2);
        }
        return $max;
    }
    return false;

}

Now again run the test. See whether all the test are passing or not. If not your function may have some issues. Improve the function and again run the test. Do this process until all the test are passing.

The example we discussed was very easy one. But in real applications we have use more complex algorithms and tactics dealing with databases. To check the functionality of the data transaction functions, we can use a test database with some preloaded data. Inside the setUp() function we can create tables and load the data to the test database. As well using the function tearDown() which automatically call after execution of any test method, we can clean the data from the database.

Sometimes applications with MVC architecture have two or more separate layers in Data Layer. In such kind of situation, functions need to test independent from their layers. Lets assume in our application there are two layers in Data Layer. One is data transaction layer which directly deal with the database and other one is data processing layer which process data before or after deal with database through data transaction layer. So, while writing test for a function in data process layer which get some data by calling a function in data transaction layer, we should verify that the correct data set is returned from the function in data transaction layer. To do this we can use Moc Objects.

Lets go through one example.
* getStrings() method in DataTransaction class return an array with two strings (No need of even declare this class)

* getProcessedResult() method in DataProcess class, get the array of two strings by calling the above mentioned function (getStrings()), and return the length of the most lengthy string out of them. This class contain getter and setter methods for use DataTransaction class.

(Declare the getProcessedResult() function and implement getter and setter methods)

<?php

    class DataProcess {

        private $dataTransaction;

        function getDataTransaction() {
            if(is_null($this->dataTransaction)) {
                return new DataTransaction();
            }
            return $this->dataTransaction;
        }

        function setDataTransaction($dataTransaction) {
            $this->dataTransaction = $dataTransaction;
        }
  
        function getProcessedResult() {
    
        }

    }

?>

* Now write the test class using the Moc Object of DataTransaction class

<?php

    require_once 'DataProcess.php';
    require_once 'PHPUnit/Framework.php';

    class TestMoc extends PHPUnit_Framework_TestCase {

     var $dataProcess;

     // This function calls before any test merhod run
     function setUp() {
                $this->dataProcess = new DataProcess();
     }

     public function testGetProcessedResult() {

         // Create a Mock Object for the DataTransaction class
         // mocking only the getStrings() method.
         $dataTransaction = $this->getMock('DataTransaction', array('getStrings'));
  
         // Set up the expectation for the getStrings() method
         // to be called only once and with no arguments
         // and return value is array("abc", "abcde")
         $dataTransaction->expects($this->once())
                         ->method('getStrings')
                         ->with()
                         ->will($this->returnValue(array("abc", "abcde")));
  
         // Set the mocked DataTransaction object to 
            // DataProcess object using setter
         $this->dataProcess->setDataTransaction($dataTransaction);
  
         // Call the getProcessedResult() method on the $dataProcess object
            // which we expect to call the mocked DataTransaction object's
         // getStrings() method with no arguments.
         $this->assertEquals(5, $this->dataProcess->getProcessedResult());

     }

    }

?>

Here, even we didn't implement (even declare) the DataTransaction class, we could run the test using Moc Object. So that we can test functions independent from the layer it contains and other needed functions.

Setting up test suite

Test suit can be used to mange which test cases should run. We can run even all the test methods in several test cases at once. Test suite can be create as following example code snippet.

<?php

    require_once 'PHPUnit/Framework.php';

    class AllTests {

        public static function suite() {

            $suite  = new PHPUnit_Framework_TestSuite("PHPUnit");
            // Add needed test cases
            $suite->addTestFile("MyTestClass1.php");
            $suite->addTestFile("MyTestClass2.php");

            return $suite;

        }

    }

?>

With PHPUnit framework the percentage of the functions which covers from the unit tests can be obtained as a coverage report. Following command can be used to obtain a coverage report. (probably you need to install xDebug : http://xdebug.org/docs/install)

$phpunit --coverage-html ./report AllTests.php

From this report we can find out the uncovered code segments. Some of code segments in MyClass.php has not covered from the unit tests. To cover them again test can be implemented.

So, this is a basic idea of how to follow test driven development with PHPUnit framework. Hope this helpful to you.

Use of Subversion client with sourceforge

2011-12-24T23:17:00.000+05:30

Subversion (SVN) is Centralized, open-source Version Control System. SVN manages files and directories and changes made to them over a period of time, while supporting to shared development with features like rollback code conflict resolve. As well this allows to recover older versions of the software (data) or examine the history of how the code-base changed. SVN is developed in order to overcome the bugs and misfeatures of CVS.

In SVN there are two ends to deal with. One is the Subversion Repository which contains all the managed files and directories, and the other one is commandline or GUI client application to manage files from the client's end. It is important to know the concept of SVN repository before starting work with SVN.

Repository is a file server which is not behave like an usual one. It has it's own revisioning system. After any authorized person change a file in repository, the state of the code-base will change and it is given a index to identify that state which is new code-base. This index is called revision number. At the initial state of the repository revision number is '0'. For any change in repository the revision number will increase by one (Figure 1). So that people can refer the older versions of the code-base.

Figure 1

Usually repository contain three main directories containing the code-base.

trunk - code-base of main development process
branch - copy of specific revision, to implement new features(which takes some time to finish) concurrently and independently on top of that (Figure 2)
tag - some snapshots of the trunk. Can store stable versions of the code-base

Figure 2

Even in branching process, the revision numbers are depend on changes of the branches (Figure 3). How ever at the end of the branching process of a particular branch, containing code should be merge with the main code-base (in trunk).

Figure 3

Now I think you have an idea on how the repository behave. To access SVN repository there are several methods. Following table shows the URL schemes map to each access method.

Scheme	Access Method
file:///	Direct repository access (on local disk)
http://	Access via WebDAV protocol to Subversionaware Apache server
https://	Same as http://, but with SSL encryption
svn://	Access via custom protocol to an svnserve server
svn+ssh://	Same as svn://, but through an SSH tunnel

As I mentioned earlier, the second end of SVN is the application, in client's end. I'll explain how we can generally use SVN client applications with Linux & windows environment.

Setup a repository

We can get use of sourceforge in order to setup a repository. sourceforge is free repository provider for open-source projects. What you have to do is register for the sourceforge and create a project. Make sure to select the version control system as SVN (default is git) while you are creating the project. sourceforge itself can add folders and files. But since we are in another track forget it for a moment. Under the code tab of your project home page, you can see the command you should use to get a checkout for linux based system. (make sure you have login)

With UBUNTU-10.04 OS

SVN commandline client can be used with Linux based systems. After installing the SVN commandline client, files can managed via executing the commands in command prompt.

install SVNcommandline client on your machine

$ sudo apt-get install subversion libapache2-svn

first go inside that directory and add source files to the repository.

$ cd /home/source_folder
$ svn import --username=your_username -m "Meaningfull_message_about_the_source_changes_for_users" url_of_repository

note : the repository url you can obtain from the sourceforge (sourceforge -> your project -> code tab)

In sourceforge project your initial file upload will appear as revision number 2. This is due to sourceforge itself create the general folder structure of repository and it will appear as revision number 1.

To start developing with SVN, first create a new folder and then get a checkout from the repository (copy of the latest code-base in repository)

$ sudo mkdir /var/www/myproject
$ cd /var/www
$ svn co --username your_username url_of_repository myproject

Now you can start further implementation on that checkout. If you modify an existing file it will already prepared to commit (upload) to the repository. But if you add a new file or a directory first you have to make prepare it to commit.

$ svn add new_file_1 new_file_2
$ svn ci -m "Meaningful_Message" new_file_1 existing_file_1 existing_file_2

If you want to commit all the files you changed, you use above code without the file paths. A summery of some useful SVN commands are shown below.

information on usage of SVN

$ svn --help

initial commit

$ svn import --username=your_username -m "Message" URL

get a checkout

$ svn co --username your_username URL project_folder

make prepare to commit the newly added files (unversioned files)

$ svn add file_path

see the states of the files (whether versioned or unversioned)

$ svn st

note: The meaning of the symbols shown for statuses,
? - unversioned file
M - locally modified file
A - schedule to add the file

D - schedule to delete the file

G - merged file

C - conflicted file while updating (file is waiting for user resolve the conflict)

! - file is missing or incomplete (removed by non-svn command)
~ - versioned file obstructed by some file of a different kind

I - file ignored

get latest code to working copy without changing local modifications

$ svn up

note: If any file is about to conflict (overwrite) commandline client itself give some options to user to agree,
p - mark the conflict to be resolved later (postpone)
df - show all changes made to merged file (diff-full)
e - change merged file in an editor (edit)
r - accept merged version of file (resolved)
mf - accept my version of entire file and ignore their changes (mine-full)
tf - accept their version of entire file and lose my changes (theirs-full)
l - launch external tool to resolve conflict (launch)
h - show this list (help)

resolve conflicted file before commit (after postpone conflict we can manually resolve it)

$ svn resolved conflicted_file_path

commit all local changes

$ svn ci -m "Message"

note: to commit specific files, add file paths at the end of the above command separating each by a space

remove all the local changes of a working copy (become to base revision)

$ svn revert -R .

note: to revert specific file replace '-R .' with file path

information on base revision (revision of working copy on top)

$ svn info

modifications between working copy and base revision

$ svn diff

get file differences to a file

$ svn diff > ~/Desktop/path/filename.diff

apply patch (diff file) to working copy

$ svn patch -p0 ~/Desktop/path/filename.diff

With Widows 7 OS

SVN GUI client can be used for windows based systems. TortoiseSVN is the GUI client of SVN. After Installing Tortoise SVN on a machine, files can be managed in UI level. To start the development, go to any directory, under which this project should be created, then right click and select 'SVN checkout' option. Then you can give the repository URL and the local project directory name. If you have already created a project in sourceforge, from the SVN command appeared under 'code' tab, you can fetch out the repository URL.

After any improvement in the code, the corresponding file and all it's parent directories are shown as modified by a red mark on the file or directory. Using the right click options, the modified code can be committed (SVN commit) even while neglecting some of the modified files by selecting them. As well you can see lot of SVN operations can be done from this interface (Figure 4).

Figure 4

So if you are new to SVN, hope this will helpful to you.

Version Control Systems (Source/Revision Control Systems)

2011-12-20T14:05:00.000+05:30

Version Control Systems are playing an important role in software source controlling. Most of the software industry used Version Control Systems to keep their source code in a central space so that any authorized person can access to it and do the changes in source files concurrently. The main purpose of a Version Control System is to enable collaborative editing and sharing of data, and keep the older states of the software code base (keep revisions).

In a project with large number of remote developers, there is a big probability to accidentally overwrite each other's changes in a regular file system (Figure 1). That's how the Version Control Systems come in to the picture in order to solve this problem.

Figure 1

So, to avoid above problem Version Control Systems used a strategy called Lock-Modify-Unlock model (Figure 2). This repository allows only one person to change specific file at a time. To do this it is using a lock system. The user who is going to change a file must lock the file so that any other user cannot lock the file again (it means file can be read but cannot change). After he made the change and commit the file to the repository he must unlock the particular file.

Figure 2

But in this model, the developing process will be unnecessarily serialized, and by mistakes some users may not unlock a file, though others are waiting for unlocking that files. As well concurrent developing of two files by different users cannot have idea of each other file which have some dependencies between them.

Therefore another method called Copy-Modify-Merge model (Figure 3) is used as alternative to this problem. In here, users can concurrently develop the same file and commit it by after merging the new changes in repository to the working copy (locally changed copy).

Figure 3

Version Control Systems such as SVN and CVS are using the above method to file managing.Using this method user have the full control of managing codes in conflicts (overwrite some code).

It's possible to classify the Version Control Systems as Centralized and Distributed. Centralized Version Control Systems means there is only one repository which developers are directly deals with. But in Distributed Version Control Systems are some kind of modification of the concept of Centralized Version Control Systems. The concept is to having two levels of Source Control Systems rather than one. So that, without all ways connecting with the main Source Control System(main repository in server), using an own Source Control System(local host) until completing any specific work (can be a story of project). So that the main repository contains only working copies of the software. As well the operations (commit, revert) are faster due to using the own Source Control System more than Centralized Control System.

Some examples for Version Control Systems are as follows.

(ref : http://www.smashingmagazine.com/2008/09/18/the-top-7-open-source-version-control-systems/)

1). CVS (Concurrent Versioning Systems)

Older technology. Simple system for make sure files and revisions kept up to date.

2). SVN (Subversion)

Popular Version Control System. Most of the open-source projects use SVN.

3). git

A fast populating, open-source Distributed Version Control System.

4). Mercurial

Open-source Distributed Version Control System. Very faster and much simpler system than 'git'.

5). Bazaar

Distributed Version Control System with different types of workflow than other Distributed Version Control Systems.

6). LibreSource

Web portal, with a built-in subversion repositories. Useful for manage collaborative projects.

7). Monotone

Fairly easy to learn. Places higher value on integrity than performance.

I think you may have some idea on Version Control Systems. And I'm willing to have another article soon on use of Subversion client.