Star Tech Corner

ASP.NET MVC View Engine with FreeMarker

2011-04-27T22:58:00.004+08:00

FreeMarker is a quite success template engine in Java world. It is used with Apache Struts, an MVC framework, as a view engine. ASP.NET MVC is using Web Form as the default view engine. The problem is the view become a spaghetti of HTML and C# snippets fairly quickly, just like classic ASP and PHP. FreeMarker.Net is a project that ports the FreeMaker as an ASP.NET MVC view engine

The idea is compiling FreeMarker to .Net assembly with IKVM and create a wrapper to wrap .Net objects so that FreeMarker can understand.

Compiling FreeMarker
It is a strength forward process. It can be done with one command:
ikvmc freemarker.jar -target:library

Separating Necessary IKVM Libraries
Only the following libraries is required for development:

IKVM.OpenJDK.Beans.dll
IKVM.OpenJDK.Charsets.dll
IKVM.OpenJDK.Core.dll
IKVM.OpenJDK.SwingAWT.dll
IKVM.OpenJDK.Text.dll
IKVM.OpenJDK.Util.dll
IKVM.Runtime.dll

Wrapping .Net Object
FreeMarker does not directly deal with the objects. Instead, it deals with the TemplateModel objects. There are a few template models to be implemented:

TemplateBooleanModel
TemplateDateModel
TemplateHashModel
TemplateMethodModelEx
TemplateNumberModel
TemplateScalarModel
TemplateSequenceModel

FreeMarker provides an ObjectWrapper interface to wrap the raw objects into TemplateModel.

The NetObjectModel is actually is a TemplateHashModel

public class NetObjectModel : StringModel, 
                              TemplateModel, 
                              TemplateHashModel {
        Dictionary<string, PropertyInfo> props = 
            new Dictionary<string, PropertyInfo>();
        Dictionary<string, MethodModel> methods = 
            new Dictionary<string, MethodModel>();
        Dictionary<string, ExtensionMethodModel> extensionMethods = 
            new Dictionary<string, ExtensionMethodModel>();

        public NetObjectModel(object data, 
                              NetObjectWrapper wrapper)
            : base(data, wrapper){
            var type = data.GetType();
            foreach (var p in type.GetProperties()) {
                props.Add(p.Name, p);
            }
            foreach (var m in type.GetMethods()) {
                if (!methods.ContainsKey(m.Name)) {
                    methods.Add(m.Name, 
                                new MethodModel(data,  
                                                wrapper, 
                                                m.Name));
                }
            }
        }

        public virtual TemplateModel get(string key) {
            if (props.ContainsKey(key)) {
                return wrapper.wrap(props[key].GetGetMethod()
                                              .Invoke(data, null));
            }
            else if (methods.ContainsKey(key)) {
                return methods[key];
            }
            else if (wrapper.ExtensionTypes.Count > 0) {
                if (!extensionMethods.ContainsKey(key)) {
                    extensionMethods[key] = 
                        new ExtensionMethodModel(data, 
                                                 wrapper, 
                                                 key);
                }
                return extensionMethods[key];
            }
            else {
                return TemplateModel.__Fields.NOTHING;
            }            
        }

        public virtual bool isEmpty() {
            return props.Count == 0;
        }
    }

To adapt the ASP.NET objects, three more template model are created:

HttpApplicationStateModel
HttpRequestModel
HttpSessionStateModel

They are similar but not in the same interface, all of them are like this:

public class HttpApplicationStateModel : NetObjectModel, 
                                         TemplateModel, 
                                         TemplateHashModel {

        public HttpApplicationStateModel(
             object data, 
             NetObjectWrapper wrapper)
            : base(data, wrapper) {
            
        }

        public override TemplateModel get(string key) {
            HttpApplicationStateBase dic = 
                data as HttpApplicationStateBase;
            if (dic.Keys.Cast<string>().Contains(key)) {
                return wrapper.wrap(dic[key]);
            }
            else {
                return base.get(key);
            }
        }

        public override bool isEmpty() {
            IDictionary<string, object> dic = 
                data as IDictionary<string, object>;
            return dic.Count == 0 && base.isEmpty();
        }
    }

View Engine
The view engine is fairly simple, it simply initialize the FreeMarker configuration.

public class FreemarkerViewEngine : VirtualPathProviderViewEngine {
        Configuration config;
        public FreemarkerViewEngine(string rootPath, 
                                    string encoding = "utf-8") {
            config = new Configuration();
            config.setDirectoryForTemplateLoading(
                new java.io.File(rootPath));
            AspNetObjectWrapper wrapper = 
                new AspNetObjectWrapper();

            config.setObjectWrapper(wrapper); 
            base.ViewLocationFormats = 
                new string[] { "~/Views/{1}/{0}.ftl" };
            config.setDefaultEncoding(encoding);
            base.PartialViewLocationFormats = 
                base.ViewLocationFormats;
        }

        protected override IView CreatePartialView(
            ControllerContext controllerContext, 
            string partialPath) {
            return new FreemarkerView(config, partialPath);
        }

        protected override IView CreateView(
            ControllerContext controllerContext, 
            string viewPath, 
            string masterPath) {
            return new FreemarkerView(config, viewPath);
        }
    }

View
FreemarkerView implements IView to render the content. It simply create the dictionary as variable bindings and invoke the FreeMarker.

public class FreemarkerView : IView{
    public class ViewDataContainer : IViewDataContainer {
        private ViewContext context;
        public ViewDataContainer(ViewContext context) {
            this.context = context;
        }

        public ViewDataDictionary ViewData {
            get {
                return context.ViewData;
            }
            set {
                context.ViewData = value;
            }
        }
    }

    private freemarker.template.Configuration config;
    private string viewPath;
    public FreemarkerView(freemarker.template.Configuration config, string viewPath) {
        this.config = config;
        this.viewPath = viewPath;
    }

    public void Render(ViewContext viewContext, 
                        System.IO.TextWriter writer) {
        Template temp = config.getTemplate(viewPath.Substring(2));
            
        Dictionary<string, object> data = 
                        new Dictionary<string, object>{
            {"model", viewContext.ViewData.Model},
            {"session", viewContext.HttpContext.Session},
            {"http", viewContext.HttpContext},
            {"request", viewContext.HttpContext.Request},
            {"application", viewContext.HttpContext.Application},
            {"view", viewContext},
            {"controller", viewContext.Controller},
            {"url", new UrlHelper(viewContext.RequestContext)},
            {"html", new HtmlHelper(viewContext, 
                            new ViewDataContainer(viewContext))},
            {"ajax", new AjaxHelper(viewContext, 
                            new ViewDataContainer(viewContext))},
        };

        Writer output = new JavaTextWriter(writer);
        temp.process(data, output);
        output.flush();
    }
}

Configuration
Configuration is as simple as adding the view engine to the view engine collections.

protected void Application_Start() {
    AreaRegistration.RegisterAllAreas();

    // ****** Optionally, you can remove all other view engines ******
    //ViewEngines.Engines.Clear();
    ViewEngines.Engines.Add(new FreemarkerViewEngine(this.Server.MapPath("~/")));

    RegisterGlobalFilters(GlobalFilters.Filters);
    RegisterRoutes(RouteTable.Routes);
}

Extension Methods
ASP.NET MVC relies on extension method quite heavily. In the NetObjectWrapper, the following code is added to find the extension methods:

public virtual void AddExtensionNamespace(string ns) {
    var types = this.ExtensionTypes;
    foreach (var a in AppDomain.CurrentDomain.GetAssemblies()) {
        try {
            foreach (var t in a.GetExportedTypes()) {
                if (!types.Contains(t) && 
                    t.IsClass && 
                    t.Name.EndsWith("Extensions") && 
                    t.Namespace == ns && 
                    t.GetConstructors().Length == 0) {
                    types.Add(t);
                }
            }
        }
        catch { }
    }
}

In the view engine's constructor, it reads the namespaces specified under system.web/pages sections in web.config:

PagesSection section = 
    (PagesSection)WebConfigurationManager
                      .OpenWebConfiguration("~/")
                      .GetSection("system.web/pages");
if (section != null) {
    foreach (NamespaceInfo info in section.Namespaces) {
        wrapper.AddExtensionNamespace(info.Namespace);
    }
}

An ExtensionMethodModel class is created to find the appropriate method to invoke:

foreach (var type in wrapper.ExtensionTypes) {
    MethodInfo method = type.GetMethod(
        methodName, 
        BindingFlags.Public | BindingFlags.Static, 
        Type.DefaultBinder, 
        argTypes, null);
    if (method != null) {
        cache.Add(key, method);
        return wrapper.wrap(method.Invoke(target, args.ToArray()));
    }
}

Localization
ResourceManagerDirectiveModel (a TemplateDirectiveModel) and ResourceManagerModel are created so that we could do something like this:

<@resource type="Freemarker.Net.MvcWeb.App_GlobalResources.PersonResource, Freemarker.Net.MvcWeb"/>
${res.Title}

ResourceManagerDirectiveModel is getting the ResourceManager from the resource and put it into res template variable:

public void execute(freemarker.core.Environment env, java.util.Map parameters, TemplateModel[] models, TemplateDirectiveBody body) {
    if (parameters.containsKey("type")) {
        TemplateScalarModel scalar = 
            (TemplateScalarModel)parameters.get("type");
        var type = Type.GetType(scalar.getAsString());
        env.setVariable("res", 
            env.getObjectWrapper()
                .wrap(type.GetProperty("ResourceManager", 
                                        BindingFlags.Static | 
                                        BindingFlags.NonPublic | 
                                        BindingFlags.Public)
                        .GetGetMethod(true)
                        .Invoke(null, null)));
    }            
}

Adding More Directives
To all more directives can be added in the future, MEF is used. The directive implementation only needs to export and implement the ImportableDirective interface.

[Export(typeof(ImportableDirective))]
    public class ResourceManagerDirectiveModel : TemplateDirectiveModel, ImportableDirective {

The view engine will import them in the constructor:

public class FreemarkerViewEngine : VirtualPathProviderViewEngine {
        Configuration config;
        [ImportMany]
        IEnumerable<ImportableDirective> directives;
        public FreemarkerViewEngine(string rootPath, 
                                    string encoding = "utf-8") {
            // initialize the config 
            // ...
            // import the extension methods' namespaces
            // ...
            // import the directives
            var dir = new DirectoryInfo(
                Path.Combine(rootPath, "bin"));
            var catalogs = dir.GetFiles("*.dll")
                .Where(o => o.Name != "freemarker.dll" && 
                           !(o.Name.StartsWith("IKVM.") || 
                           o.Name.StartsWith("Freemarker.Net")))
                .Select(o => new AssemblyCatalog(
                             Assembly.LoadFile(o.FullName))
            );
            var container = new CompositionContainer(
                new AggregateCatalog(
                    new AggregateCatalog(catalogs),
                    new AssemblyCatalog(
                      typeof(ImportableDirective).Assembly)
            ));
            container.ComposeParts(this);
        }

The source code is available in CodePlex. Please visit http://freemarkernet.codeplex.com/ and http://www.shaneng.net/Main/FreeMarkerViewEngine for more detail.

P.S.: The bonus of this project is we have not only a new view engine in ASP.NET MVC, but also a new template engine in .Net Framework. Maybe someday we could use FreeMarker to generate code instead of T4 in Visual Studio SDK.

Caching in .Net Framework 4

2011-04-21T23:48:00.013+08:00

We used to have caching in ASP.NET. For non-web application caching, Cache Application Block from Enterprise Library may be the choice. In .Net Framework 4, caching is baked into the library and no longer limited to ASP.NET.

After adding System.Runtime.Caching assembly as the reference in your project, we can cache the data retrieved by a web services:

using System.Runtime.Caching;
class CrmHelper{
    static ObjectCache cache = MemoryCache.Default;
//...
    string key = string.Format("{0}/{1}", 
                          "account", "address1_addresstypecode");
    MetadataService service = new MetadataService();
    service.Credentials = CredentialCache.DefaultCredentials;

    Option[] options = null;
    if (cache.Contains(key)){
        options = (Option[])cache[key];
    else {
        var picklist = (PicklistAttributeMetadata)
            service.RetrieveAttributeMetadata(
                "account", "address1_addresstypecode");
        options = picklist.Options;
        cache.Set(key, options, new CacheItemPolicy {
            AbsoluteExpiration = DateTimeOffset.Now.AddMinutes(2)
        });
    }
//...
}

if (cache.Contains(key)) ... cache.Set(key, data) is going to be a repeating pattern. We could write an extension method to wrap it up:

public static class Extensions {
    public static T Get<T>(this ObjectCache cache, 
                           string key,
                           Func<T> retrieveFunction) {
        if (cache.Contains(key)) {
            return (T)cache[key];
        }
        else {
            var result = retrieveFunction();
            cache.Set(key, result, new CacheItemPolicy {
                AbsoluteExpiration = 
                    DateTimeOffset.Now.AddMinutes(2)
            });
            return result;
        }
    }
}

The previous code is now cleaned up like this:

using System.Runtime.Caching;
class CrmHelper{
    static ObjectCache cache = MemoryCache.Default;
//...
    string key = string.Format("{0}/{1}", 
                          "account", "address1_addresstypecode");
    MetadataService service = new MetadataService();
    service.Credentials = CredentialCache.DefaultCredentials; 

    Option[] options = cache.Get(key, () => {
        var picklist = (PicklistAttributeMetadata)
            service.RetrieveAttributeMetadata(
                "account", "address1_addresstypecode");
        return picklist.Options;
    });
//...
}

It is worth to note that the Cache Application Block from Enterprise Library is deprecating (http://msdn.microsoft.com/en-us/library/ff664753(v=PandP.50).aspx):

Caching Application Block functionality is built into .NET Framework 4.0; therefore the Enterprise Library Caching Application Block will be deprecated in releases after 5.0. You should consider using the .NET 4.0 System.Runtime.Caching classes instead of the Caching Application Block in future development.

If you only want a quick memory cache, the new caching API might already fit your needs. However, Enterprise Library provides some other implementations of caching (e.g. database, file, etc). If you want something more fancy, Enterprise Library could be an alternative.

Refereces:

WCF Channel Factory and Unity 2.0

2011-03-27T10:02:00.008+08:00

Unity 2.0 does not support ChannelFactory in configuration natively. However, we could extend Unity. Inspired by Chris Tavares's FactoryElement extension for static factory configuration, here is the ChannelFactory extension for WCF channel factory configuration in Unity.

The element will get the generic type of the ChannelFactory and calling the CreateChannel() method upon the injection.

public class ChannelFactoryElement : InjectionMemberElement {
        private const string endpointConfigurationNamePropertyName = "endpointConfigurationName";
        private static int numFactories;
        private readonly int factoryNum;

        public ChannelFactoryElement() {
            factoryNum = Interlocked.Increment(ref numFactories);
        }

        [ConfigurationProperty(endpointConfigurationNamePropertyName, IsKey = false, IsRequired = true)]
        public string Name {
            get { return (string)base[endpointConfigurationNamePropertyName]; }
            set { base[endpointConfigurationNamePropertyName] = value; }
        }

        public override string Key {
            get { return "ChannelFactory " + factoryNum; }
        }

        public override IEnumerable<injectionmember> GetInjectionMembers(IUnityContainer container, Type fromType,
            Type toType, string name) {

            Type factoryType = typeof(ChannelFactory<>).MakeGenericType(toType);            
            var constructor = factoryType.GetConstructor(new Type[] { typeof(string) });
            var factory = constructor.Invoke(new object[]{Name});
            var method = factoryType.GetMethod("CreateChannel", Type.EmptyTypes);
            return new InjectionMember[] { new InjectionFactory(o => method.Invoke(factory, null)) };
        }

    }

The extension will register the factory as the element name of ChannelFactoryElement.

public class ChannelFactoryConfigExtension : SectionExtension {
        public override void AddExtensions(SectionExtensionContext context) {
            context.AddElement<ChannelFactoryElement>("factory");
        }
    }

The factory element uses endpointConfigurationName to retrieve the client's endpoint as in ChannelFactory(endpointConfigurationName) constructor.

<unity xmlns="http://schemas.microsoft.com/practices/2010/unity">
    <sectionExtension type="MyNamespace.ChannelFactoryConfigExtension, MyAssembly" />
    <container>      
      <register type="MyNamespace.IMyService, MyAssembly">
        <factory endpointConfigurationName="testing"/>
      </register>
    </container>
  </unity>

PHP-Azure Migration - Part 4

2011-03-25T14:24:00.001+08:00

Up to the last post of the migration, we have done all data and file migration. How about the new files being uploaded from the CMS?

PHP
We are using a very old version of FCKEditor as our rich text editor. Since FCKEditor supports uploading file from the editor, we need to change the code in editor/filemanager/connectors/php/command.php. Here is an example of getting the folder and files for the file browser.

function GetFoldersAndFiles( $resourceType, $currentFolder ) {
        $sServerDir = MapAzureFolder( $resourceType, $currentFolder);
        $blobs = $blobStorageClient->listBlobs($ContainerName, $sServerDir, '/');
 $aFolders = array();
 $aFiles = array();
 $folders = GetSessionFolders($resourceType, $currentFolder);
 foreach($blobs as $blob){
  $name = GetName($blob->Name);
  if ($blob->IsPrefix){
   $folders[$name] = 0;
  } else {     
   $iFileSize = $blob->Size;
   if ( !$iFileSize ) {
    $iFileSize = 0 ;
   }
   if ( $iFileSize > 0 ) {
    $iFileSize = round( $iFileSize / 1024 ) ;
    if ( $iFileSize < 1 ) $iFileSize = 1 ;
   }
    $aFiles[] = '<File name="' . ConvertToXmlAttribute( $name ) . '" size="' . $iFileSize . '" />' ;
  }
 }
 foreach($folders as $name => $value){
  $aFolders[] = '<Folder name="' . ConvertToXmlAttribute( $name ) . '" />' ;
 }
}

It is worth to mention the GetSessionFolders() function. Azure Storage does not have concept of folder. All folders are actually derived by the paths of the files. So, when listing the blobs, we can use $blob->IsPrefix to distinguish the derived folders and the files.

If a user want to create a folder and upload the file to the folder, we need to put the folder into the session. That is how GetSessionFolders() function come from.

When upload a file, we have to handle the Flash format. It seems that modern browsers are intelligent enough to recognize image files like JPEG, GIF, PNG without the content type. When it comes to Flash file, it could not be played properly.

function FileUpload( $resourceType, $currentFolder, $sCommand ) {
...
 $sFileName = 'uploadedfiles/'. strtolower($resourceType).$currentFolder.$sFileName;
 $result = $blobStorageClient->putBlob($ContainerName, $sFileName,  $oFile['tmp_name']);

 if(strtolower($resourceType) == 'flash') {
  $contentType = 'application/x-shockwave-flash';
  $blobStorageClient->setBlobProperties($ContainerName,  $sFileName, null, array('x-ms-blob-content-type' => $contentType));
 }
 $sFileUrl = $currentFolder.$sFileName;
...
}

ASP.NET
For the ASP.NET portal, we have to do the same. Under /editor/filemanager/connectors/aspx/connector.aspx, we changed the first line to

<%@ Page Language="c#" Trace="false" Inherits="FCKAzureAdapter.Connector" AutoEventWireup="false" %>

The connector is simply extending FredCK.FCKeditorV2.FileBrowser.Connector and override the GetFiles(), GetFolders(), and CreateFolder() methods with Azure API. Here is an example of GetFolders():

protected override void GetFolders(System.Xml.XmlNode connectorNode, string resourceType, string currentFolder) {
    string sServerDir = this.ServerMapFolder(resourceType, currentFolder);

    // Create the "Folders" node.
    XmlNode oFoldersNode = XmlUtil.AppendElement(connectorNode, "Folders");

    CloudBlobContainer container = GetCloudBlobContainer(containerPath);
    var dir = container.GetDirectoryReference(folder);

    var items = dir.ListBlobs();
    List<string> dirList = new List<string>();
    foreach (var x in items.OfType<CloudBlobDirectory>())
    {
        dirList.Add(x.Uri.ToString());
    }
    var dirs = dirList.Select(o => o.Substring(o.LastIndexOf('/', o.Length - 2)).Replace("/", "")).ToArray();

    Dictionary<string, List<string>> folderList = 
            HttpContext.Current.Session["Folders"] as Dictionary<string, List<string>>;
    foreach (var dir in dirs)
    {
        // Create the "Folder" node.
        XmlNode oFolderNode = XmlUtil.AppendElement(oFoldersNode, "Folder");
        var dirName = dir.Uri.GetName();
        XmlUtil.SetAttribute(oFolderNode, "name", dirName);
        if (folderList != null && 
            folderList.ContainsKey(resourceType) 
            && folderList[resourceType].Contains(dirName))
        {
            folderList[resourceType].Remove(dirName);
        }
    }

    if (folderList != null && folderList.ContainsKey(resourceType))
    {
        foreach (var folder in folderList[resourceType].Where(o => o != currentFolder && o.StartsWith(currentFolder)))
        {
            var folderName = folder.Substring(currentFolder.Length).Trim('/');
            if (!folderName.Contains('/')) {
                // Create the "Folder" node.
                XmlNode oFolderNode = XmlUtil.AppendElement(oFoldersNode, "Folder");
                XmlUtil.SetAttribute(oFolderNode, "name", folderName);
            }
        }
    }
}

Then, we do the same for editor/filemanager/connectors/aspx/upload.aspx.

Next time, we will discuss what other problems we have encountered.

PHP-Azure Migration - Part 3

2011-03-03T22:49:00.002+08:00

In the previous post, we have fixed all database queries to make the web site start working in SQL Server. The next task is tackling the file I/O without the file system. The file system is not a permanent storage in Windows Azure. We need to copy all resource files (file attachments of articles) from the local file system to Azure Storage.

File Migration
The migration seemed quite straight forward. We can just copying the file to Azure Storage. However, there is no FTP or any batch utilities from Microsoft for batch upload. Given we have 4GB of files with some hierarchical directories, it is infeasible to the files one by one. We used Cloud Storage Studio for the migration.

URL Redirection
The web site and the resource files are now in different servers. We can no longer use relative path. However, since the database is storing the relative path of the resource files, we either change the data (as well as semantics) to store the absolute URIs of the resource files; or redirect the browser to get the files from the correct URLs.

As changing the data would involve a lot of testings in business logics, we decided to create a ASP.NET Module to redirect the requests. Given that

IIS (as well as Windows Azure) allow us to call ASP.NET Module in any web site (including PHP web site)
All files were stored in a particular folder so we could derive the redirect pattern easily

Now, all existing files have been migrated and will be redirected to new URL. Next time, let's take a look how to deal with new uploaded files.

PHP-Azure Migration - Part 2

2011-03-02T22:21:00.006+08:00

In the previous post, I have discussed the database migration from MySQL to SQL Azure. Data migration is only the first part of data processing, the next step is to modify the queries in the web site due the the syntax differences between two database systems.

Driver Adaption
In the ASP.NET frontend, we only need to replace the ADO.NET driver from MySQL to SQL Server driver and it is all done. In the PHP CMS, however, we will need to do some adaptation since PHP did not have a unified API for database driver.

Lucky enough, it is not much work as there is a driver layer in the CMS. There were only three functions we need to change: query, fetch, total which corresponding to mysql_query, mysql_fetch_arry, and mysql_num_rows in MySQL driver and sqlsrv_query, sqlsrv_fetch_array, and sqlsrv_num_rows in SQL Server.

Query Modification
The major differences can be summarizes as follows:

Getting last inserted record identity
Paging
Scalar functions
Aggregation functions
Escaping strings

There is a DAO layer in the CMS that storing all queries. So, all we did is go through all queries in the DAOs and modify them.

1. Getting last inserted record identity
MySQL PHP driver provides mysql_insert_id() to get the auto-increment column value of the last record inserted. There is no equivalent function in SQL Server PHP Driver. However, it is no more than just querying "@@IDENTITY" in SQL Server.

2. Paging
While doing paging in MySQL only takes "LIMIT n OFFSET skip", SQL Server will need

SELECT * FROM (
    SELECT TOP n * FROM (
        SELECT TOP z columns      -- (where z = n + skip)
        FROM table_name
        ORDER BY key ASC
    ) AS a ORDER BY key DESC
) AS b ORDER BY key ASC

3. Scalar functions
MySQL uses NOW() to get the current timestamp. SQL Server uses GETDATE().
MySQL provides RAND() so we could do random order. SQL Server provides NEWID(). Please note that NEWID() generate a GUID so the ordering is random. It does not mean NEWID() is same as RAND() in MySQL.

4. Aggregation functions
MySQL provides many convenient aggregation functions like GROUP_CONCAT(). The queries have to be rewritten for SQL Server or we need to create the corresponding user defined aggregation functions.

5. Escaping strings
Most developers use \' to escape single quote. In fact, both MySQL and SQL Server use two consecutive single quotes to escape a single quote. MySQL just accepting the other formats.

Up to this moment, we discussing the data layer (if you have a data layer in PHP). There is no code change in the business layer yet. This is also an example that how we could modularize a program if we partition layers clearly regardless which platform we are using.

PHP-Azure Migration - Part 1

2011-03-01T23:46:00.009+08:00

Azure is Microsoft's cloud platform. The working models are different in a few ways. We migrated a web site to Azure. The web site consists of a PHP CMS, ASP.NET frontend, and a MySQL database.

Database Migration
There was some discussion among the team about whether using MySQL worker role or SQL Azure. As the final decision is SQL Azure, the first task we need to do is migrating the MySQL database to SQL Azure. The most obvious tools we can work on is Microsoft SQL Server Migration Assistant (SSMA) for MySQL.

SSMA was working fine in SQL Server. It reconstructed the SQL Server equivalent schema from MySQL and copied the data from MySQL to SQL Server. However, there are a few catches when migrating to SQL Azure:
1. Tables without primary keys:
For some reason, some tables in the web site did not have primary keys. The problem is a table in SQL Azure need a clustered index. No primary key means no clustered index (in SSMA sense). We have used a development SQL Server to re-create the schema first. Then, look for the tables without primary keys and add the primary keys back.

2. Large records
The CMS designer has a concept that all data should be put into the database as the backup strategies. So, everything including images files, Flash video files, and video files were stored in some attachment table as well. SSMA could handle most of them but not all. However, when the record is large (say 10MB), it will eventually failed and interrupted the how migration process. It is a very traditional timeout problem. We are just dealing with two:

Command timeout for SQL Server
Idle timeout for MySQL

Since we do not have control for timeout configuration and error handling strategies, we skip those tables during the migration. We dumped out the attachment into file systems. After a few tuning in the upload and download script to make the CMS working, we create a PHP script to migrate only the metadata columns in the attachment tables.

To conclude, the migration were performed in a two steps:

Execute the SQL script to create the tables
SSMA to copy most data
Execute the PHP script to migrate the attachment metadata

The performance was quite satisfactory.

Jeans - Running Java Servlet and JSP Webapp in ASP.NET/IIS - Part 4

2010-11-15T23:07:00.003+08:00

Retrieving a database connection is one of the most popular problem in software development. Usually, we specify the connection string in a configure file and create the database connection with the connection string. The Java way is wrapping connection string in a javax.sql.DataSource interface and put in the application server config. The application server will put the DataSource instance into a JNDI context. So, this is what should be supported.

Bring in the Unity
Microsoft has a Dependency Injection container called Unity. Even though I am not doing dependency injection, I'd like to configure how to instantiate the data source in web.config. Instead of writing my own configuration classes, Unity is a good alternative as an object builder.

Before doing any configuration, the JDBC driver has to be compiled into .Net assembly since it will be loaded by Unity instead of the Java class loader.

Method Injection
When configuring Unity, we could specify the methods to be called during the instance initialization. The first attempt:

<method name="setURL">

    <param name="url" value="..." />

</method>

However, Unity was smart enough to know there is no such parameter name. I fired-off the object explorer to look the actual parameter name. It seemed that IKVM compiled all parameter name with str, str1, str2, etc. The configuration has to be like this:

<method name="setURL">

    <param name="str" value="..." />

</method>

JNDI Context
Having known that a data source with the name jdbc/testDS is the same as java:comp/env/jdbc/testDS, I tried to mimic the behavior with the reference of this article. It turned out the result is an exception:

javax.persistence.PersistenceException: Exception [EclipseLink-7060] (Eclipse Persistence Services - 2.0.2.v20100323-r6872): org.eclipse.persistence.exceptions.ValidationException

Exception Description: Cannot acquire data source [jdbc/testDS].

Internal Exception: javax.naming.NameNotFoundException: Name jdbc is not bound in this Context

    at org.eclipse.persistence.internal.jpa.EntityManagerSetupImpl.deploy(EntityManagerSetupImpl.java:399)

So, I reverted to the simpliest way by just creating the jdbc subcontext in the initial context.

Password
The configuration was set. Time to run the test. It did not work:

javax.persistence.PersistenceException: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.0.2.v20100323-r6872): org.eclipse.persistence.exceptions.DatabaseException

Internal Exception: com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user 'jsptest'.

Error Code: 18456

    at org.eclipse.persistence.internal.jpa.EntityManagerSetupImpl.deploy(EntityManagerSetupImpl.java:399)

Well, the password was copied from persistence and very simple:

<method name="setPassword">

    <param name="str" value="passw0rd" />

</method>

Not sure what happened but when the user and password were put in the URL, it worked.

The new source code has been checked-in in http://jeans.codeplex.com/. The persistence.xml has changed to use the <non-jta-data-source>jdbc/testDS</non-jta-data-source>.

Note: sqljdbc4.jar has been removed from the source repository. Please download it from Microsoft and compile it with IKVM to run the JSP demo site.

Jeans - Running Java Servlet and JSP Webapp in ASP.NET/IIS - Part 3

2010-11-15T00:31:00.000+08:00

Developing Java webapp, we use of Filter for cache control, compression, database connection and transaction handling. So, Filter is another feature I need to support.

Original Thought
In ASP.NET, IHttpModule is like a filter while IHttpHandler is like a servlet. It seemed obvious to modify the existing ServletModule to execute the filter.

Platform Mismatch
The first problem I found is the async design of ASP.NET. The IHttpModule is an event sink and need to implement the event handlers for events like BeginRequest, EndRequest. In contrast, Servlet Filter is using FilterChain to call the filter one by one. I tried to execute the filter chain in BeginRequest event. However, I won't be able to gain the access of Session. Having google around, I found the comprehensive events execute sequence in IHttpModule (http://msdn.microsoft.com/en-us/library/ms178473.aspx):

BeginRequest
AuthenticateRequest
PostAuthenticateRequest
AuthorizeRequest
PostAuthorizeRequest
ResolveRequestCache
PostResolveRequestCache
PostMapRequestHandler
AcquireRequestState
PostAcquireRequestState
PreRequestHandlerExecute
PostRequestHandlerExecute
ReleaseRequestState
PostReleaseRequestState
UpdateRequestCache
PostUpdateRequestCache
EndRequest
PreSendRequestHeaders
PreSendRequestContent

To get the session, the module needs to implement an event handler for PostRequestHandlerExecute event. In order to get the Session, HttpContext.Current.Handler has to be implementing IRequiresSessionState as well.

There is code to wrap the HttpContext.Current.Handler around (http://stackoverflow.com/questions/276355/can-i-access-session-state-from-an-httpmodule). It worked fine but it does not solve the async problem.

From Module to Handler
I managed to run the filter chain in PreRequestHandlerExecute event. It turned out that not only it run the JSP code, but also the source code of the JSP was also append to the result by the default handler. Response.End() has to be invoked. And then some checking for the whether it is my own servlet handler, etc. The code is a bit weird.

Since it does not matter where the filter chain is executed, I moved to code to ServletHandler and revert the changes in ServletModule, it compiled and seemed working with the Hello World JSP. No more Response.End() and checking. Async problem is solved since executing the filter and servlet are in one place. It matched the semantics of Java webapp.

Testing
The simplest way to test is to get a filter that GZIP the output and use fiddler to see if the response is compressed. I grabbed filter from http://onjava.com/pub/a/onjava/2003/11/19/filters.html. Added the setting in web.xml. Somehow it does not work. It turned out it's because the default XML namespace in web.xml generated by Eclipse caused the problem. Removing the namespace made enabled the support of filter.

XML Namespace
The default XML namespace is quite annoying. I am not sure why it is there as it is meaningless. I tried a few way to remove it with the .Net XML API but the namespace somehow stayed there. The final resort was invoking string.Replace() to get rid of it.

The latest source code has been checked-in in http://jeans.codeplex.com/.

Jeans - Running Java Servlet and JSP Webapp in ASP.NET/IIS - Part 2

2010-11-13T00:03:00.001+08:00

One of my mentors said, "If you do not write SQL, you cannot do application development." Even though the statement is not quite true as there are ORM frameworks handling the database communication, the statement emphasizes the importance of database. So, running a pure Hello World JSP is a poor proof of concept to ASP.NET. Running a database connected JSP is more meaningful.

Preparation
The following components were gathered in this phase:

JPA - the current standard of doing ORM in Java.

EclipseLink is the reference implementation. It should serve the purpose well.

SQL Server 2008 Express - As ASP.NET is being used anyway, why don't we work in the whole Microsoft ecosystem?

SQL Server JDBC Driver - Yes, I need a JDBC Driver in .Net...

AdventureWorksLT - Microsoft used to provide sample databases that works with SQL Server. AdventureWorksLT is the simplified version.

One bottleneck I have is I have no JPA experience. I used Hibernate before but JPA is a standard. The problem of being standard is there are many implementations. The only way to understand it is to read the specification. Otherwise, it's hard to understand how it works in general so that it works for most (if not all) implementations.

META-INF/persistence.xml
One of the challenges encountered was loading the persistence.xml. Luckily, IKVM can load the resource files with by specifying -classloader:ikvm.runtime.ClassPathAssemblyClassLoader when the compiling the class files and JAR files into .Net assembly. However, since all files were compiled into a single DLL. Only the first persistence.xml IKVM read can managed to be the resource.

META-INF/orm.xml
The problem of compiling everything into .Net assembly is losing all annotations in the classes. In the other words, you must use META-INF/orm.xml and initialize it in an IHttpModule.

Rework
The limitation of loading the two XML files is quite nasty. It would not feasible for medium size application that might have a few JAR files containing persistence.xml. Creating orm.xml manually is quite a pain as you know the information is in the classes already. To increase the flexbility, ClassPathAssemblyClassLoader is is wrapped by an URLClassLoader with the following classpaths:

WEB-INF\classes
All JAR files in WEB-INF\lib directory

Then, it is invoked to the current thread's context class loader to load the JAR and the precompiled JSP classes directly. It works!

Dual Mode
Now, Jeans support two ways to run a Java webapp:

Native mode - With everything compiled into a single DLL
WAR mode - Run a standard Java webapp directory whose structure is the same as in WAR file given:

JSP are precompiled into servlet and put into WEB-INF/classes folder.
JSP Servlet mappings are stored in jspweb.xml to complement the original web.xml.

The consequence is quite obvious. In native mode, it takes around 50MB of memory to load the Hello World page. In WAR mode, it takes around 100MB of memory the run the same page. No benchmark for the application performance but I believed native mode should run faster.

Jeans - Running Java Servlet and JSP Webapp in ASP.NET/IIS - Part 1

2010-11-12T01:55:00.000+08:00

There are two main camps in enterprise server technology market, Java and .Net Framework. Java side has Servlet, JSP, and Java ServerFace. .Net Framework side has ASP.NET (WebForm and MVC). Both sides are not compatible with each other. This article will describe how to run JSP in IIS natively without connecting to a Java web container.

IKVM
Mr. Jeroen Frijters has implemented an JVM in .NET, IKVM.NET. It seems to be a good start.

Jetty vs. Tomcat
The first attempt was compiling Jetty to execute web applications directly. Digging into the source code and after a few trials, I found that Jetty is using Jasper library, which is Tomcat's JSP engine, to run JSP. Given that .Net does not have the concept of class file, it is unlikely .NET is able to load the create new class and load it into memory in runtime.

JSP Precompiler
Jasper not only is the runtime engine of JSP, it also includes a JSP precompiler to compile JSP into Servlet. Using it is not hard but type the Java command is tedious. It makes sense to use IKVM to create a .Net executable. The Jasper library from Tomcat 5.5 is used:

ant.jar
catalina-ant.jar
commons-el.jar
commons-logging-api-1.1.1.jar
jasper-compiler-jdt.jar
jasper-compiler.jar
jasper-runtime.jar
jsp-api.jar
naming-factory-dbcp.jar
naming-factory.jar
naming-resources.jar
servlet-api.jar

Then compile the precompiler with this command:

ikvmc -classloader:ikvm.runtime.ClassPathAssemblyClassLoader -target:exe -main:org.apache.jasper.JspC -out:bin\jspc.exe *.jar

JSP Compilation
The command-line arguments of JspC is the same as the Java version. The following command can be used (%1 is the directory of the target web application):
jspc -uriroot "%1" -d out -compile -v -webxml "%1\WEB-INF\jspweb.xml" -trimSpaces -source 1.5 -target 1.5

The JSP
The first JSP is a simple Hello World page to test out the JSTL EL:
Hello ${param.name}!

Web Application Compilation
Everything are still in JARs and classes. However .Net Framework needs DLLs. The following command is executed to get everything in .Net assembly:

ikvmc -classloader:ikvm.runtime.ClassPathAssemblyClassLoader -recurse:%1\WEB-INF\classes -out:%1\bin\jsp.dll -reference:%1\bin\jspc.exe %1\WEB-INF\lib\*.jar

Servlet API, in .Net
Before running the DLL, the following classes are necessary in the Servlet API to run the JSP:

HttpServletRequest/Response
ServletInputStream/OutputStream
ServletRequestDispatcher
ServletContext
ServletConfig

and a few more adapters to implement Java I/O.

ASP.NET IHttpModule and IHttpHandler
The final bit is the ASP.NET module to initiate the configuration and the handler to execute the servlets/JSP.

I have made the source code available so you don't need to go through the whole process. Please visit http://jeans.codeplex.com/.

Irrational Decision

2010-05-20T01:01:00.000+08:00

Problem: HTML 5 supports <video> tag. However, standard codec that all browser vendors can agree with.

Opera: Let's do it in Theora.
MS, Apple: H.264 is industrial standard.
Mozilla Foundation: H.264 is not free. Theora is free.
Google: Let me buy out On2 and open it.
Opera, Mozilla Foundation: OK, let's do it in On2, then.
MS, Apple: ...

Now, Opera, Firefox and Chrome will support On2. IE and Safari will still only support H.264. We still do not have the common standard codec for the <video> tag.

Question: MS and Apple were not willing to implement Theora, why they will implement On2? Why the hell Google bought out On2 in the first place?

Sorting NULL LAST in LINQ-to-SQL

2010-01-21T00:11:00.006+08:00

There might be some time that you need to put NULL value into last when sorting a result set. In SQL Server, you can do something like this:


SELECT value
FROM foo
ORDER BY (CASE WHEN x IS NULL THEN 1 ELSE 0 END), x

You might want to ask the mapping of LINQ-to-SQL. Here you go:


foo.OrderBy(i => i.x == null ? 1 : 0)
   .ThenBy(i => i.x);

Web Hosting Experience

2009-12-28T00:51:00.010+08:00

I have recently changed by web hosting from HostMonster to Awardspace. HostMonster served me well these two years until when it was about to renew a month ago. I got notified that my my account got suspended because of "spam/phishing". After checking with the customer service, the reason was some "suspicious" files were found in my web site content.

The suspicious files are actually the executables of my personal projects: the JavaScript obfuscator (JSO) and the solution to convert base64 images in data URI to MIME (Base64Html).

As the projects contributes quite a lot of traffic to the website, I immediately asked for the reason of suspension. They explained that there are viruses found in the executables, blah, blah, blah... After I explained the purpose of my files and verified the MD5 were the same as in my computer, they re-activated the websites.

Everything sounds OK but it already took two days! I believe there are better ways to handle my case:
1. Why did they suspend all my websites? The files are only in one website.
2. Why the whole web sites were get suspended at all? The suspicious files can be redirected to some other safe URL.
3. Why spam/phishing turned out to be a possible virus notification? Why I had to ask to know the real reason of getting suspended? If they cannot distinguish the difference between phishing and virus, it is very unprofessional.
4. There was one statement in the last email I received made me very angry: "The report that we received was from a reputable web security company, so we did take action based on their report." Why couldn't they send me the report to let me take action before suspending my account (thus, suspending all websites)? The action taken is not customer friendly. Suspending account should be the last measure, not the first.

After this incident, I took some time to shop around. I used Awardspace free hosting before so it sounds OK for me. The price is reasonable and it supports ASP.NET. Not sure if it is a good choice since the websites have just migrated for a few days. I did have some trouble on transferring the domain but the customer service responded quickly and nicely.

I might have part 2 of this post to share the experience of website migrations.

P.S.: HostMonster does have a tracked record of over sensitive scam alert (use "oversensitive scam alert" as the keyword to search in Google). So, if you are not living in US (like me), you might get into the same trouble.

Accessing Private Key from PFX (PKCS#12) in .Net Framework

2009-10-10T01:45:00.020+08:00

You may know that .Net Framework has X509Certificate2 class to read X.509 certificate and obtain the public key from the certificate:


var cert = new X509Certificate2("my.cer");
var key = cert.PublicKey.Key as RSACryptoServiceProvider;
var cipher = key.Encrypt(Encoding.UTF8.GetBytes(txtClear.Text), false);
txtCipher.Text = Convert.ToBase64String(cipher);

However, you may not know that X509Certificate2 class can also read the first private key from a PFX file without CryptoAPI:


var cert = new X509Certificate2("my.pfx", "password");
var key = cert.PrivateKey as RSACryptoServiceProvider;
var clear = Convert.FromBase64String(txtCipher.Text);
txtClear.Text = Encoding.UTF8.GetString(key.Decrypt(clear, false));

To read all private keys from PFX, use PKCS12.Read() method below. It uses CryptoAPI and it will return an array of X509Certificate2. You can get the private key to decrypt the cipher as shown above. The full sample code can be download at Shane's Shelf.


using System;
using System.IO;
using System.Runtime.InteropServices;
using System.Security.Cryptography.X509Certificates;
using System.Collections.Generic;

namespace X509Cert
{
    public class PKCS12
    {
        public static X509Certificate2[] Read(string filename, string password)
        {

            FileStream stream = new FileStream(filename, FileMode.Open);
            byte[] buffer = new byte[stream.Length];
            stream.Read(buffer, 0, buffer.Length);
            stream.Close();


            WIN32.CRYPT_DATA_BLOB cryptdata = new WIN32.CRYPT_DATA_BLOB();
            cryptdata.cbData = buffer.Length;
            cryptdata.pbData = Marshal.AllocHGlobal(cryptdata.cbData);
            Marshal.Copy(buffer, 0, cryptdata.pbData, buffer.Length);
            IntPtr hMemStore = WIN32.PFXImportCertStore(ref cryptdata, password, WIN32.CRYPT_USER_KEYSET);
            Marshal.FreeHGlobal(cryptdata.pbData);

            uint provinfosize = 0;

            List<X509Certificate2> certs = new List<X509Certificate2>();

            IntPtr certHandle = IntPtr.Zero;
            while ((certHandle = WIN32.CertEnumCertificatesInStore(hMemStore, certHandle)) != IntPtr.Zero)
            {

                if (WIN32.CertGetCertificateContextProperty(certHandle, WIN32.CERT_KEY_PROV_INFO_PROP_ID, IntPtr.Zero, ref provinfosize))
                {

                    IntPtr info = Marshal.AllocHGlobal((int)provinfosize);

                    if (WIN32.CertGetCertificateContextProperty(certHandle, WIN32.CERT_KEY_PROV_INFO_PROP_ID, info, ref provinfosize))
                    {
                        var certData = new X509Certificate2(certHandle).Export(X509ContentType.SerializedCert);
                        certs.Add(new X509Certificate2(certData));
                    }
                    Marshal.FreeHGlobal(info);

                }
            }

            Marshal.FreeHGlobal(hMemStore);
            return certs.ToArray();

        }
    }

    public class WIN32
    {
        public const uint CRYPT_USER_KEYSET = 0x00001000;
        public const uint CERT_KEY_PROV_INFO_PROP_ID = 0x00000002;

        [DllImport("crypt32.dll", SetLastError = true)]
        public static extern IntPtr PFXImportCertStore(ref CRYPT_DATA_BLOB pPfx, [MarshalAs(UnmanagedType.LPWStr)] String szPassword, uint dwFlags);

        [DllImport("CRYPT32.DLL", EntryPoint = "CertEnumCertificatesInStore", CharSet = CharSet.Auto, SetLastError = true)]
        public static extern IntPtr CertEnumCertificatesInStore(IntPtr storeProvider, IntPtr prevCertContext);

        [DllImport("CRYPT32.DLL", CharSet = CharSet.Auto, SetLastError = true)]
        public static extern bool CertGetCertificateContextProperty(IntPtr pCertContext, uint dwPropId, IntPtr pvData, ref uint pcbData);

        [DllImport("advapi32.dll", EntryPoint = "CryptAcquireContext", CharSet = CharSet.Auto, SetLastError = true)]
        public static extern bool CryptAcquireContext(ref IntPtr phProv, string szContainer, string szProvider, uint dwProvType, uint dwFlags);

        [StructLayout(LayoutKind.Sequential)]
        public struct CRYPT_DATA_BLOB
        {
            public int cbData;
            public IntPtr pbData;
        }

        public WIN32()
        {

        }
    }
}

The above code is altered base on http://www.cnblogs.com/rainlake/archive/2005/09/15/237997.html

MXHR

2009-05-21T21:55:00.010+08:00

Digg is creating a multipart capable XHR library. Good try but not very useful. Here is why:

MXHR response is not compressed by default.
This is illustrated very well in their text only demo. MXHR stream is always slower than normal mode. I opened up the packet sniffer. The normal stream is compressed while the MXHR is not.
Data URI scheme is broken.
The web is broken by IE for a long time. IE6 and IE7 will be still around for a few years. Image data cannot be served in cross-browser manner. The same performance can be achieved by spriting: putting the tiny-images into one image (overhead is small if the image is large enough), preload the combined imaged, and use background-position to get the coordination.
Multipart is not streaming
Well, it is under DUI.Stream namespace. However, it is not streaming. The server is packing up the data into one response. The library is processing the data on-the-fly but the received data is accumulating.
1. If the size of the response is not large (i.e. can be stream in a couple seconds), enveloping the message in JSON or XML could be easier.
2. If the size of the response is large (i.e. comet style), the responseText is going to become very large. We call it memory leak.

HTTP is broken because it is not intended to serve for Web 2.0. That's why there are so many facilities in CSS and JavaScript to patch the protocol. MXHR is definitely not the answer.

E4X Alternative - JSOM

2009-05-04T23:19:00.011+08:00

I have been expecting the popularity of E4X for a long time. However, as long as Internet Explorer is the most popular browser, "new" standards will never get popular (Microsoft is suffered from Windows XP legacy, too). Instead of waiting for E4X, some developers use XPath to query the DOM nodes while the other traverse the DOM node by node. However, it would be good if we could access XML content in JavaScript Object Model (JSOM):


xml='<invoice date="01-20-2000" number="123">' +
    '  <address country="US">' +
    '    <name>John Smith</name>' +
    '    <street>123 George St.</street>' +
    '    <city>Mountain View</city>' +
    '    <state>CA</state>' +
    '    <zip>94041</zip>' +
    '  </address>' +
    '</invoice>';
invoice = JSOM(xml);
alert(invoice.$number);
alert(invoice.address.name);

There is a project, IEE4X, in SourceForge for the same purpose but it does not work quite well for a few cases. So, I grabbed the idea and implement the JSOM library myself. The code can be download at Shane's Shelf.

XML vs. JSON - Size Comparison

2009-04-22T00:02:00.009+08:00

I was wondering how compact is JSON comparing to XML. I grabbed an arbitrary XML from the web and convert it to an equivalent JSON format. It turns out that JSON is around two-third of the XML size.

XML: 596 bytes

JSON: 402 bytes

<?xml version="1.0"?>

<message>

<header>

<to>companyReceiver</to>

<from>companySender</from>

<type>saveInvoice</type>

</header>

<info>

<saveInvoice>

<invoice date="01-20-2000" number="123">

<address country="US">

<name>John Smith</name>

<street>123 George St.</street>

<city>Mountain View</city>

<state>CA</state>

<zip>94041</zip>

</address>

<billTo country="US">

<name>Company A</name>

<street>100 Main St.</street>

<city>Washington</city>

<state>DC</state>

<zip>20015</zip>

</billTo>

<items>

<item number="1">

<name>IBM A20 Laptop</name>

<quantity>1</quantity>

<USPrice>2000.00</USPrice>

</item>

</items>

</invoice>

</saveInvoice>

</info>

</message>

header:{

to:"companyReceiver",

from:"companySender",

type:"saveInvoice”

},

info:{

saveInvoice:{

invoice:{date:"01-20-2000,number:"123",

address:{country:"US",

name:"John Smith",

street:"123 George St.",

city:"Mountain View",

state:"CA",

zip:"94041"

},

billTo:{country="US",

name:"Company A",

street:"100 Main St.",

city:"Washington",

state:"DC",

zip:"20015"

},

items:[

name:"IBM A20 Laptop",

quantity:1,

USPrice:2000.00

Secure WCF Services with Authentication Service

2009-03-14T11:53:00.010+08:00

We can use WCF Authentication Service to authenticate users with ASP.NET membership provider. However, other WCF services are not protected by the authentication service out-of-the-box. That is, WCF services is not using ASP.NET forms authentication.

Fortunately, it is not hard to enable it. The magic point is set the HttpContext.Current.User in Global.asax

public class Global : System.Web.HttpApplication
  {
      // other methods snipped...
      protected void Application_AuthenticateRequest(object sender, EventArgs e)
      {
          HttpCookie ticketCookie = Context.Request.Cookies[FormsAuthentication.FormsCookieName];
          if (null == ticketCookie)
          {
              return;
          }

          FormsAuthenticationTicket ticket = FormsAuthentication.Decrypt(ticketCookie.Value);
          if (null != ticket)
          {
              HttpContext.Current.User = new GenericPrincipal(new FormsIdentity(ticket), null);
          }
      }
  }

In the service you want to protect, set the requirement mode to allowed or required.

    [AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
   public class PrimeService : IPrimeService

Then, throw in the following checking at the beginning of the method.

            if (!HttpContext.Current.User.Identity.IsAuthenticated)
          {
              throw new FaultException<SecurityAccessDeniedException>(new SecurityAccessDeniedException());
          }

That would be good enough for Silverlight client. For .Net WCF client, you need to handle the HTTP cookies by yourself (Authentication Service is using the authentication ticket in cookies). Detail discussion can be found in the article in Shane's Shelf.

Silverlight Multi-part File Upload Form Post

2009-03-03T02:21:00.014+08:00

Silverlight not actually very web friendly. It lacks of built-in support for the current HTML form post protocol. So, we have to write it our own. I have written an extension class and two serializer classes, one for normal form post (DataContractQueryStringSerializer) while the other for multipart upload form post (DataContractMultiPartSerializer).

If you only want to have the working code, just copy the code below. Detail explanation is available at Multi-Part Form Post in Shane's Shelf


  public static class Extensions
  {
      public static void PostFormAsync(this HttpWebRequest request, object parameters, AsyncCallback callback)
      {
          request.Method = "POST";
          request.ContentType = "application/x-www-form-urlencoded";
          request.BeginGetRequestStream(new AsyncCallback(asyncResult =>
          {
              Stream stream = request.EndGetRequestStream(asyncResult);
              DataContractQueryStringSerializer ser = new DataContractQueryStringSerializer();
              ser.WriteObject(stream, parameters);
              stream.Close();
              request.BeginGetResponse(callback, request);
          }), request);
      }

      public static void PostMultiPartAsync(this HttpWebRequest request, object parameters, AsyncCallback callback)
      {
          request.Method = "POST";
          string boundary = "---------------" + DateTime.Now.Ticks.ToString();
          request.ContentType = "multipart/form-data; boundary=" + boundary;
          request.BeginGetRequestStream(new AsyncCallback(asyncResult =>
          {
              Stream stream = request.EndGetRequestStream(asyncResult);

              DataContractMultiPartSerializer ser = new DataContractMultiPartSerializer(boundary);
              ser.WriteObject(stream, parameters);
              stream.Close();
              request.BeginGetResponse(callback, request);
          }), request);
      }
  }

  public class DataContractQueryStringSerializer
  {
      public void WriteObject(Stream stream, object data)
      {
          StreamWriter writer = new StreamWriter(stream);
          if (data != null)
          {
              if (data is Dictionary<string, string>)
              {
                  foreach (var entry in data as Dictionary<string, string>)
                  {
                      writer.Write("{0}={1}&", entry.Key, entry.Value);
                  }
              }
              else
              {
                  foreach (var prop in data.GetType().GetFields())
                  {
                      foreach (var attribute in prop.GetCustomAttributes(true))
                      {
                          if (attribute is DataMemberAttribute)
                          {
                              DataMemberAttribute member = attribute as DataMemberAttribute;
                              writer.Write("{0}={1}&", member.Name ?? prop.Name, prop.GetValue(data));
                          }
                      }
                  }
                  foreach (var prop in data.GetType().GetProperties())
                  {
                      if (prop.CanRead)
                      {
                          foreach (var attribute in prop.GetCustomAttributes(true))
                          {
                              if (attribute is DataMemberAttribute)
                              {
                                  DataMemberAttribute member = attribute as DataMemberAttribute;
                                  writer.Write("{0}={1}&", member.Name ?? prop.Name, prop.GetValue(data, null));
                              }
                          }
                      }
                  }
              }
              writer.Flush();
          }
      }
  }

  public class DataContractMultiPartSerializer
  {
      private string boundary;
      public DataContractMultiPartSerializer(string boundary)
      {
          this.boundary = boundary;
      }

      private void WriteEntry(StreamWriter writer, string key, object value)
      {
          if (value != null)
          {
              writer.Write("--");
              writer.WriteLine(boundary);
              if (value is FileInfo)
              {
                
                  FileInfo f = value as FileInfo;
                  writer.WriteLine(@"Content-Disposition: form-data; name=""{0}""; filename=""{1}""", key, f.Name);
                  writer.WriteLine("Content-Type: application/octet-stream");
                  writer.WriteLine("Content-Length: " + f.Length);
                  writer.WriteLine();
                  writer.Flush();
                  Stream output = writer.BaseStream;
                  Stream input = f.OpenRead();
                  byte[] buffer = new byte[4096];
                  for (int size = input.Read(buffer, 0, buffer.Length); size > 0; size = input.Read(buffer, 0, buffer.Length))
                  {
                      output.Write(buffer, 0, size);
                  }
                  output.Flush();
                  writer.WriteLine();
              }
              else
              {
                  writer.WriteLine(@"Content-Disposition: form-data; name=""{0}""", key);
                  writer.WriteLine();
                  writer.WriteLine(value.ToString());
              }
          }
      }

      public void WriteObject(Stream stream, object data)
      {
          StreamWriter writer = new StreamWriter(stream);
          if (data != null)
          {
              if (data is Dictionary<string, object>)
              {
                  foreach (var entry in data as Dictionary<string, object>)
                  {
                      WriteEntry(writer, entry.Key, entry.Value);
                  }
              }
              else
              {
                  foreach (var prop in data.GetType().GetFields())
                  {
                      foreach (var attribute in prop.GetCustomAttributes(true))
                      {
                          if (attribute is DataMemberAttribute)
                          {
                              DataMemberAttribute member = attribute as DataMemberAttribute;
                              WriteEntry(writer, member.Name ?? prop.Name, prop.GetValue(data));
                          }
                      }
                  }
                  foreach (var prop in data.GetType().GetProperties())
                  {
                      if (prop.CanRead)
                      {
                          foreach (var attribute in prop.GetCustomAttributes(true))
                          {
                              if (attribute is DataMemberAttribute)
                              {
                                  DataMemberAttribute member = attribute as DataMemberAttribute;
                                  WriteEntry(writer, member.Name ?? prop.Name, prop.GetValue(data, null));
                              }
                          }
                      }
                  }
              }
          }
          writer.Write("--");
          writer.Write(boundary);
          writer.WriteLine("--");
          writer.Flush();
      }
  }

The usage is as follows:
First a PHP file


<?php
print_r($_REQUEST);
$src = $_FILES['y']['tmp_name'];
$dest = "C:\\Windows\\Temp\\".$_FILES['y']['name'];
echo $src;
echo "\r\n";
echo $dest;
echo @copy($src, $dest);
?>

Then the Page control

    public partial class Page : UserControl
  {
      public Page()
      {
          InitializeComponent();
          // Create a request object
          HttpWebRequest request = (HttpWebRequest)WebRequest.Create(new Uri("http://localhost/rms/test.php"));
          OpenFileDialog dlg = new OpenFileDialog();
          if (dlg.ShowDialog().Value)
          {
              request.PostMultiPartAsync(new Dictionary<string, object> { { "x", "1" }, { "y", dlg.File } }, new AsyncCallback(asyncResult =>
              {
                  HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(asyncResult);

                  Stream responseStream = response.GetResponseStream();
                  StreamReader reader = new StreamReader(responseStream);
                  this.Dispatcher.BeginInvoke(delegate
                  {
                      // output is a TextBlock
                      output.Text = reader.ReadToEnd();
                      response.Close();
                  });
              }));
          }
      }
  }

Since it is able to serialize data contract, you could actually replace

new Dictionary<string, object> { { "x", "1" }, { "y", dlg.File } }

with

new Point(){X=1, Y=2}

given the point class is like this:

    [DataContract]
  public class Point
  {
      [DataMember]
      public int X { get; set; }
      [DataMember(Name="y")]
      public int Y { get; set; }
  }

Slow Performance of PHP in Windows IIS

2008-12-17T22:37:00.001+08:00

There are a few reasons that PHP runs slowly in Windows. Not because of the request delegation from IIS to PHP via FastCGI or CGI. In fact, the delegation is quite robust in FastCGI.

TCP/IP connection problem between database and web server may be a reason. Some Windows specific behaviors may be another. One of the Windows specific behaviors is the invocation of gethostbyaddr() function.

Here is the high level behavior of gethostbyaddr() function making the invocation slow:
Original

gethostbyaddr() on windows waits for WINS resolution timeout even when you disable WINS on the server.

This means that your DNS server may return "No hostname found" and windows gethostbyaddr() will just sit there, never having asked a WINS server anything, for 3 - 4 seconds. I watched it do just that with a sniffer.

Here is the root cause:
Original

Difference between gethostbyaddr and getnameinfo.

Originally posted by: Gregor The Eye

First at all, sorry for my english. It's not my native language, and, unfortunately, i havn't time to verify all text i enter.

Now, to subject. Computer may have many "names", and two main name groups of names computers have is a "DNS" names and a "NetBIOS" names.

The "NetBIOS" name is aname you assign to your computer using system->network identification->network ID. This name
is displayed in you "network neighborhood" and generally used in local networks to identify computers.

The "DNS" name is a name given to machine in internet. This name is a synonim to machine IP (becouse IP is\s not huma-friendly). "DNS" name is stored in special internet services (so-called "DNS" servers), not at the computer itself. Generally, computer can be shut off, but it's DNS name will be available and produce correct IP.

For example, if I install OS to new computer and call it "EYE", it will have a NetBIOS name "EYE", and, connecting it to my local network, i can access it in "network neighborhood". But, from the internet, i can't access to it using "EYE" name, and instead i must use it's IP - for example - 113.54.25.14. But if i need over users to access it from internet using a human-friendly name, i pay amount of money to sertain organization, and they register a name for it - for example - "www.GregorTheEye.com". Then, user can access my machine using this name, fopr example, writing:

ping www.GregorTheEye.com

or

telnet www.GregorTheEye.com

So, we have 2 names for machine - NetBIOS name and DNS name.
In my local network, following commands will be valid:

ping 113.54.25.14
ping EYE
ping www.GregorTheEye.com

Third command will be valid ONLY if my computer is conected to internet.

And from the internet, valid commands are:

ping 113.54.14
ping www.GregorTheEye.com

As you can see, NetBIOS name is not available from internet.

The main difference between NetBIOS name and DNS name is that DNS name will be available only if computer is connected to the internet and has i's name registered on it. NetBIOS name will be always available to computers that directly connected to target one.

To get DNS name, you must send a request to DNS server (it's IP is written in the system registry if you computer is connected to internet). If DNS server is unavailable, it will take default timeout time to discover this. If it's available, DNS server will return you human-friendly name of target machine if it's exist in database.

To obtain a NetBIOS name, you must send UDP packe to target machine and wait for responce. Becouse UDP is not a guaranteed-to-deliver protocol, responce may ot came, came corrupted, came many times etc. Your program must patiently wait for responce (it's not from 137 port - you must SEND UDP packet to 137 port of target machine, but responce will be returned to the port you specify. Only Win95 no OSR2 has a bug there responce always returns to port 137).

And, finally to Win32 functions -

gethostbyaddr() will first try to connect DNS server, and, if it's unavailable, will try to get NetBIOS name. So, it will get machine name one method or another in most cases.

getnameinfo() will ONLY try a DNS name, no attempts to get a NetBIOS name(). That's the difference.

If you write a program that must get names of local machines as well as names of internet machines (for example, network or port scanner), you MUST use gethostbyaddr(). If you write a program that will operate only with names of remote machines (IRC client, for example) - you MUST use getnameinfo() becouse Microsoft instructs so in lates MSDN releases.

If you have any questions - feel free to contact me. Also, if i write something wrony, i will be very thankfull if you send ma a hint.

Best regards, Gregor The Eye.

However, there is no getnameinfo() function in PHP. One of the nice feature in PHP documentation is the commenting. A commenter contributed a gethostbyaddr_timeout() function that only query the DNS server. You need to pass the DNS address (instead of using the OS specified DNS) to the function, though.

Flash vs. Silverlight vs. JavaFX

2008-12-08T00:55:00.000+08:00

In RIA, Flash is the dominant player while Silverlight and JavaFX are new. However, Silverlight is in a better position than JavaFX.

.Net is a huge platform like Java. However, Microsoft trimmed .Net into a 5MB version to become a Silverlight runtime. On the other hand, Sun only added scripting syntax to Java and reuse the whole Java runtime as well as the branding. In fact, Java applet and Java web start has become the burden of the Java family. Users perceive Java is slow, insecure, and has compatibility issues whether or not any of them are true. So, it is no longer a technology issue. It is a marketing issue.

In fact, when it comes to marketing, Microsoft does it far much better than everyone one else in the technology industry.

Improving ASP.NET Web Performance with Local SMTP Server

2008-11-01T14:58:00.000+08:00

Implementing public web site, we usually need an SMTP server for registration verification, reset password notification, etc. We usually utilize existing email infrastructure to send the email. However, we may not have the control of the infrastructure. The SMTP server may locate overseas or may go down sometime. In these cases, the response of the web server seems slow. We can setup the SMTP in local server to resolve the problem.

How does SMTP works?
There is a special record in DNS called MX record (Mail eXchange record). Using a@b.com as an example. The SMTP server inquires the MX record of b.com to resolve the SMTP server that will receive the email.

SMTP Setup in Windows
Windows comes with SMTP service in XP, 2003, Vista, and 2008. It can be configured with the following steps
1. In IIS Manage, right click Default SMTP Virtual Server, choose Properties
2. Choose Access tab and click the Relay button at the bottom in Relay restriction section. An SMTP relay lets an SMTP client use an SMTP server to forward e-mail messages to a remote domain. By default, it is disabled so you cannot send any email.
3. Add your local host address to allow your local application to send email. You can even add your internal network address so your other machine can use this as the SMTP relay.
4. Optionally, you can prohibit the external connection by configuring the Connection control section.

Since SMTP will queue the email and return, it responses instantly. The web performance also improves instantly, too.

HTML Robustness and Query String Parameter Naming

2008-10-21T20:55:00.000+08:00

All the way back to 10 years ago, HTML was designed for non-professionals to publish information to the world wide web. The parsing and interpreting rules are loose. Web browsers implemented a lot of heuristics to display as much information as possible.

Now, we have been entered the age of Web 2.0 for a few years. The legacy loosen rules remains. One of the rules I have discovered today is the HTML entity. HTML entity without semicolon will be interpreted as if it has semicolon. For example, &lt will be interpreted as <.

The problem is when writing a web application. You will need to concatenate query string in <a> tag like <a href="list.php?a=x&b=y">. The query string parameters cannot be named as one of those HTML entities. So, <a href="list.php?a=x&lt=y"> does not work. The URL will be interpreted as list.php?a=x<=y. Certainly, no one would use lt as the name of a query string parameter. However, some other common names including euro, copy, pound, cent, uml, not, micro, times and divide will not work.

Developer Friendliness

2008-09-25T13:37:00.002+08:00

People usually said that Firefox is more developer friendly than IE or Safari. Currently, there are two embarrassing incidents that flip my perception of Firefox.

1. Text box inside DIV
For some weird reason in Firefox 2, the cursor caret in the text box disappears when the text box is inside a DIV. Web developer has to work with the CSS hack to get the caret back

overflow: hidden; position:fixed;

It is OK if web developer have the total control of the HTML. However, we have web components in .Net and Java Server Face. If such components do not aware of this, user may had a bad experience.

2. Length Required with POST
Content-length is an HTTP header letting the server know how much data it should be expecting in the request. However, Firefox does not send such header to the server with posting with XMLHTTPRequest. Developer need to write it out explicitly

xhr.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded')
xhr.setRequestHeader('Content-length', '0');

XMLHTTPRequest is the key doing AJAX! The HTTP request is usually uncachable when using POST. Why on earth developers need to code it themselves?

I think implementing the standard is important. It relieves the web developers headache. On the other hand, please do not introduce new problem to the developers.