Feroze's musings on Technology

How to share NTLM connection across HttpWebRequest objects

2011-09-04T19:52:00.000-07:00

When writing an application that talks to a server using NTLM authentication, HttpWebRequest does a three legged authentication handshake that looks as follows:

Client	Server
GET / HTTP/1.1	HTTP/1.1 401 Access Denied Connection: close WWW-Authenticate: NTLM <token>
GET / HTTP/1.1 Authorization: NTLM <token1>	HTTP/1.1 401 Access Denied Connection: close WWW-Authenticate: NTLM NTLM <token>
GET / HTTP/1.1 Authorization: NTLM <token2>	HTTP/1.1 200 OK Connection: close

As you can see, by default, HttpWebRequest does not share connections. The reason is that NTLM is a connection authentication protocol, and not a request authentication protocol like BASIC or DIGEST.

In other words, once an NTLM handshake succeeds, the NTLM credentials stick with the connection. So, if you have a situation (for eg, a web server talking to another webserver using NTLM auth) where there are multiple users with different credentials accessing the front-end server, there is a high risk that an authenticated connection for one user might get used by another user due to connection pooling.

In order to force HttpWebRequest to share NTLM connections, you can do the following:

Use UnsafeAuthenticatedConnectionSharing
Use ConnectionGroupName

Setting UnsafeAuthenticatedConnectionSharing will cause all NTLM connections to the same host to be shared.

By setting ConnectionGroupName, you can cause connections to be shared only selectively. In this case, all requests having the same ConnectionGroupName will share connections. You can use this mechanism if you are on the middle-tier scenario.

Related:

System.NET Links and How To's
Tracing with System.Net
The case of multiple NTLM challenges with IIS7

Diagnosing RMI Exception: java.rmi.ConnectException: Connection refused to host: 127.0.0.1

2011-08-09T16:32:00.000-07:00

I have a service that runs on Linux under JBOSS. This service uses JMX (RMI) to talk to a windows box running a java service.

Everything was working ok, until both the Linux and Windows boxes were moved to a different subnet. Then, they started failing with the following exception:

Caused by: java.rmi.ConnectException: Connection refused to host: 127.0.0.1; nested exception is:
java.net.ConnectException: Connection refused
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:601)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:198)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:110)
at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2312)
at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:277)
at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:248)
at com.zillow.core.jmx.ZillowJMXConnectorFactory.connect(ZillowJMXConnectorFactory.java:127)
at com.zillow.core.jmx.ZillowJMXConnectorFactory.connect(ZillowJMXConnectorFactory.java:53)
at com.zillow.bcpserver.BCPServerProxy.afterPropertiesSet(BCPServerProxy.java:123)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1369)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1335)
... 142 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:519)
at java.net.Socket.connect(Socket.java:469)
at java.net.Socket.(Socket.java:366)
at java.net.Socket.(Socket.java:180)
at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:22)
at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:128)
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:595)
... 154 more

I searched for this exception in the search engines, and the only thing I found was people saying that if the local box (i.e linux) did not have a correct IP address for localhost, it would send the 127.0.0.1 address as the "ContactMe" endpoint to the remote destination, and that would fail.

For example, the following links explained that issue:

http://forum.springsource.org/showthread.php?33711-RMI-invocation-attempts-connecting-to-127.0.0.1

However, in my case, that turned out not to be the problem. My Linux boxes (even after the move) had the correct hostname, and the `hostname` command was giving back the correct hostname (and not 127.0.0.1 or localhost).

At this point, there was no more information available through the search engines to diagnose the problem.

So, I decided to debug it myself. I first restarted the service, and used TCPDUMP to do a network sniff on the linux box.

Here, .175 is the Linux box, and .45 is the windows box.

The following is the packet disassembly of the JRMI/ReturnData response being sent by the Windows box to the Linux box.

As you can see, the Windows server is sending back "127.0.0.1" as the CallMe endpoint to the Linux box. The linux box tries to connect to port 1099 on 127.0.0.1 and fails.

Since the machines had been moved to different networks, it is possible that the java service might have lost their IP address and network registration settings.

So, I restarted the java service on the windows box. And viola!, that fixed the problem.

Why isn't HTTPS used more on the web?

2011-03-22T09:56:00.000-07:00

Ars Technica has a great article giving the reasons why HTTPS is not used more on the Web.

The summary is as follows:

1) HTTPS adds some latency to initial connection establishment.
2) HTTPS documents cannot be cached by the browser. While this is not a big issue when the client and server are on the same continents,
3) HTTPS websites cannot be virtual hosts on the same server, and this causes addition in operation costs.

It goes on to add that while these are practical reasons why HTTPS isn't widely deployed, the practical hurdles will fall away eventually leading to a wider adoption of HTTPS.

Via JonesBlog - Apples Retina Display

2010-06-25T15:42:00.000-07:00

Brian Jones analyzes Apple's claim that the Iphone4's display resolution is better than a human eye. Interesting reading.

Multipart Form Upload Helper

2010-03-21T22:58:00.000-07:00

Implementing Multipart-MIME upload helper using WebClient

The WebClient class is a higher level abstraction on top of HttpWebRequest, that abstracts away the common tasks of uploading files, as well as doing Form Posts to HTTP servers. However, it does not provide an easy way to do multipart form uploads.

Multipart form upload is defined by the W3C specification.

Multipart HTML Form Design

Let us start off by designing the HTML Form that will be the target of the multipart upload. As per the HTML spec, the form should have an enctype attribute set to "multipart/form-data". Also, we are going to have a File input control embedded in the form.

Your name: 
Your id: 

What files are you sending?

MIME Part Requirements

In a multipart form upload, each mime part begins with some MIME headers. The following headers are mandatory:

Content-Disposition header
A name attribute that specifies the name of the corresponding control from the HTML form.

For eg, in the form above, if I specify "John" for the name, and "1" for the ID, then I will have the following in the MIME upload

--BD37EC1
Content-Disposition: form-data; name="fname";

john
--BD37EC1
Content-Disposition: form-data; name="id";

acme
--BD37EC1

Ideally, our helper class should be able to write out the boundaries itself, and delegate the task of writing out the headers to each MIME part that is part of the upload.

Our multipart form has two kinds of controls: normal Text input as well as FILE input controls. Therefore, our helper class needs to support both of these types of MIME parts.

Let us start off by thinking about the layour of the MIME part classes.

MIME Part Design

As discussed, we are going to have two types of MIME parts - a normal FORM upload type where we want to specify Name/Value pairs, as well as a FILE part where we want to embed the contents of a file.

In the previous section we saw the basic requirements of a MIME part. It has to have a "Content-Disposition" header. In addition to this, a MIME part that contains file contents should also have a "Content-Type" header.

With this, we can now write out the layout of the base class for these two MIME parts.

/// 
    /// MimePart
    /// Abstract class for all MimeParts
    /// 
    abstract class MimePart
    {
        public string Name { get; set; }

        public abstract string ContentDisposition { get; }

        public abstract string ContentType { get; }

        public abstract void CopyTo(Stream stream);

        public String Boundary
        {
            get;
            set;
        }
    }

This is a good abstraction of a MIME part. Derived classes can define the Content-Type and Content-Disposition, we well as write out the contents of the MIME part into the Stream that is provided. The MIME helper class will do the job of formatting the MIME part and uploading it to a HTTP endpoint.

With this, we can now write out a derived class that can upload textual input data (i.e not file contents)

class NameValuePart : MimePart
    {
        private NameValueCollection nameValues;

        public NameValuePart(NameValueCollection nameValues)
        {
            this.nameValues = nameValues;
        }

        public override void CopyTo(Stream stream)
        {
            string boundary = this.Boundary;
            StringBuilder sb = new StringBuilder();

            foreach (object element in this.nameValues.Keys)
            {
                sb.AppendFormat("--{0}", boundary);
                sb.Append("\r\n");
                sb.AppendFormat("Content-Disposition: form-data; name=\"{0}\";", element);
                sb.Append("\r\n");
                sb.Append("\r\n");
                sb.Append(this.nameValues[element.ToString()]);

                sb.Append("\r\n");

            }

            sb.AppendFormat("--{0}", boundary);
            sb.Append("\r\n");

            //Trace.WriteLine(sb.ToString());
            byte [] data = Encoding.ASCII.GetBytes(sb.ToString());
            stream.Write(data, 0, data.Length);
        }

        public override string ContentDisposition
        {
            get { return "form-data"; }
        }

        public override string ContentType
        {
            get { return String.Empty; }
        }
    }

NameValuePart is an implementation of the MimePart abstract class. It does the job of serializing the Name/Value pairs as a MIME encapsulated part.

Next, we write the implementation of the class the encapsulates file uploads as a MIME part.

class FilePart : MimePart

{

private Stream input;

private String contentType;



public FilePart(Stream input, String name, String contentType)

{

this.input = input;

this.contentType = contentType;

this.Name = name;

}



public override void CopyTo(Stream stream)

{

StringBuilder sb = new StringBuilder();

sb.AppendFormat("Content-Disposition: {0}", this.ContentDisposition);

if (this.Name != null)

sb.Append("; ").AppendFormat("name=\"{0}\"", this.Name);

if (this.FileName != null)

sb.Append("; ").AppendFormat("filename=\"{0}\"", this.FileName);

sb.Append("\r\n");

sb.AppendFormat(this.ContentType);

sb.Append("\r\n");

sb.Append("\r\n");



// serialize the header data.

byte[] buffer = Encoding.ASCII.GetBytes(sb.ToString());

stream.Write(buffer, 0, buffer.Length);



// send the stream.

byte[] readBuffer = new byte[1024];

int read = input.Read(readBuffer, 0, readBuffer.Length);

while (read > 0)

{

stream.Write(readBuffer, 0, read);

read = input.Read(readBuffer, 0, readBuffer.Length);

}



// write the terminating boundary

sb.Length = 0;

sb.Append("\r\n");

sb.AppendFormat("--{0}", this.Boundary);

sb.Append("\r\n");

buffer = Encoding.ASCII.GetBytes(sb.ToString());

stream.Write(buffer, 0, buffer.Length);



}



public override string ContentDisposition

{

get { return "file"; }

}



public override string ContentType

{

get { return String.Format("content-type: {0}", this.contentType); }

}



public String FileName { get; set; }

}

According to the HTML Form upload specification, when the form defines more than one FILE INPUT field, then all the files in the form can be uploaded as one MIME type (of type multipart/mixed) that encapsulates each file, and each file will have it's own MIME encapsulation. In order to support this, we need to create a special class that can encapsulate FILE types.

/// 
        /// Helper class that encapsulates all file uploads
        /// in a mime part.
        /// 
        class FilesCollection : MimePart
        {
            private List files;

            public FilesCollection()
            {
                this.files = new List();
                this.Boundary = MultipartHelper.GetBoundary();
            }

            public int Count
            {
                get { return this.files.Count; }
            }

            public override string ContentDisposition
            {
                get
                {
                    return String.Format("form-data; name=\"{0}\"", this.Name);
                }
            }

            public override string ContentType
            {
                get { return String.Format("multipart/mixed; boundary={0}", this.Boundary); }
            }

            public override void CopyTo(Stream stream)
            {
                // serialize the headers
                StringBuilder sb = new StringBuilder(128);
                sb.Append("Content-Disposition: ").Append(this.ContentDisposition).Append("\r\n");
                sb.Append("Content-Type: ").Append(this.ContentType).Append("\r\n");
                sb.Append("\r\n");
                sb.AppendFormat("--{0}", this.Boundary).Append("\r\n");

                byte[] headerBytes = Encoding.ASCII.GetBytes(sb.ToString());
                stream.Write(headerBytes, 0, headerBytes.Length);
                foreach (FilePart part in files)
                {
                    part.Boundary = this.Boundary;
                    part.CopyTo(stream);
                }
            }

            public void Add(FilePart part)
            {
                this.files.Add(part);
            }
        }

This class does the job of encapsulating FILE parts inside a MIME container.

Now, we just need to write the code that can put this all together.

/// 
    /// Helper class to aid in uploading multipart
    /// entities to HTTP web endpoints.
    /// 
    class MultipartHelper
    {
        private static Random random = new Random(Environment.TickCount);

        private List formData = new List();
        private FilesCollection files = null;
        private MemoryStream bufferStream = new MemoryStream();
        private string boundary;

        public String Boundary { get { return boundary; } }

        public static String GetBoundary()
        {
            return Environment.TickCount.ToString("X");
        }

        public MultipartHelper()
        {
            this.boundary = MultipartHelper.GetBoundary();
        }

        public void Add(NameValuePart part)
        {
            this.formData.Add(part);
            part.Boundary = boundary;
        }

        public void Add(FilePart part)
        {
            if (files == null)
            {
                files = new FilesCollection();
            }
            this.files.Add(part);
        }

        public void Upload(WebClient client, string address, string method)
        {
            // set header
            client.Headers.Add(HttpRequestHeader.ContentType, "multipart/form-data; boundary=" + this.boundary);
            Trace.WriteLine("Content-Type: multipart/form-data; boundary=" + this.boundary + "\r\n");

            // first, serialize the form data
            foreach (NameValuePart part in this.formData)
            {
                part.CopyTo(bufferStream);
            }

            // serialize the files.
            this.files.CopyTo(bufferStream);

            if (this.files.Count > 0)
            {
                // add the terminating boundary.
                StringBuilder sb = new StringBuilder();
                sb.AppendFormat("--{0}", this.Boundary).Append("\r\n");
                byte [] buffer = Encoding.ASCII.GetBytes(sb.ToString());
                bufferStream.Write(buffer, 0, buffer.Length);
            }

            bufferStream.Seek(0, SeekOrigin.Begin);

            Trace.WriteLine(Encoding.ASCII.GetString(bufferStream.ToArray()));
            byte [] response = client.UploadData(address, method, bufferStream.ToArray());
            Trace.WriteLine("----- RESPONSE ------");
            Trace.WriteLine(Encoding.ASCII.GetString(response));
        }

        /// 
        /// Helper class that encapsulates all file uploads
        /// in a mime part.
        /// 
        class FilesCollection : MimePart
        {
            private List files;

            public FilesCollection()
            {
                this.files = new List();
                this.Boundary = MultipartHelper.GetBoundary();
            }

            public int Count
            {
                get { return this.files.Count; }
            }

            public override string ContentDisposition
            {
                get
                {
                    return String.Format("form-data; name=\"{0}\"", this.Name);
                }
            }

            public override string ContentType
            {
                get { return String.Format("multipart/mixed; boundary={0}", this.Boundary); }
            }

            public override void CopyTo(Stream stream)
            {
                // serialize the headers
                StringBuilder sb = new StringBuilder(128);
                sb.Append("Content-Disposition: ").Append(this.ContentDisposition).Append("\r\n");
                sb.Append("Content-Type: ").Append(this.ContentType).Append("\r\n");
                sb.Append("\r\n");
                sb.AppendFormat("--{0}", this.Boundary).Append("\r\n");

                byte[] headerBytes = Encoding.ASCII.GetBytes(sb.ToString());
                stream.Write(headerBytes, 0, headerBytes.Length);
                foreach (FilePart part in files)
                {
                    part.Boundary = this.Boundary;
                    part.CopyTo(stream);
                }
            }

            public void Add(FilePart part)
            {
                this.files.Add(part);
            }
        }
    }

Since the job of serializing to MIME has been delegated to each concrete implementation of MimePart class, the MultipartHelper only has to make sure that it uses the different parts and serializes them into a stream. The stream is then uploaded using the supplied WebClient.

Note that I have put the FilesCollection class as an inner class of the MultipartHelper class, because it is meant to be a helper class that should not be directly accessed by the clients.

The client program for this looks as follows:

class Program
    {
        static void Main(string[] args)
        {
            Trace.Listeners.Add(new ConsoleTraceListener());
            try
            {
                using (StreamWriter sw = new StreamWriter("testfile.txt", false))
                {
                    sw.Write("Hello there!");
                }

                using (Stream iniStream = File.OpenRead(@"c:\platform.ini"))
                using (Stream fileStream = File.OpenRead("testfile.txt"))
                using (WebClient client = new WebClient())
                {
                    MultipartHelper helper = new MultipartHelper();

                    NameValueCollection props = new NameValueCollection();
                    props.Add("fname", "john");
                    props.Add("id", "acme");
                    helper.Add(new NameValuePart(props));

                    FilePart filepart = new FilePart(fileStream, "pics1", "text/plain");
                    filepart.FileName = "1.jpg";
                    helper.Add(filepart);

                    FilePart ini = new FilePart(iniStream, "pics2", "text/plain");
                    ini.FileName = "inifile.ini";
                    helper.Add(ini);

                    helper.Upload(client, "http://localhost/form.aspx", "POST");
                }
            }
            catch (Exception e)
            {
                Trace.WriteLine(e);
            }
        }
    }

If you want to understand what is going on at the wire level, you can create a system.net trace log which will show the details.

Socket.SendFile - Implementing fast file transfer

2009-12-06T14:05:00.000-08:00

Implementing fast file transfer with Socket.SendFile

When writing network applications, we usually have a need to implement file transfer between two hosts. For eg, imaging an FTP client, where the client is downloading or uploading a file from an FTP server. Similarly, you could be uploading an image file (probably a photo) as an attachment to a Blog or a website like Facebook or Flickr.

Usually, file transfer is implemented as a Read/Write pattern, where you read from the source stream and write into the destination stream. Here the source stream is the stream constructed from the Socket, and the target stream is the file, or vice versa if the file is being transferred to a destination server.

The simple Read/Write pattern for file transfer is implemented as follows.

using (FileStream fs = File.OpenRead(filename))
                {
                    byte[] buffer = new byte[1024];
                    int read = fs.Read(buffer, 0, buffer.Length);

                    while (read > 0)
                    {
                        ns.Write(buffer, 0, read);
                        read = fs.Read(buffer, 0, buffer.Length);
                    }
                }

In the .NET framework, there is a better way to do file uploads, which is exposed through Socket.SendFile method. This method exposes the underlying Winsock API TransmitFile. This API is much more powerful and faster in terms of performance.

In order to check the performance difference, I wrote an application that compares the difference in performance between using the Read/Write pattern and the Socket.SendFile method.

Here is the test program:

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Text;
using System.Net;
using System.Net.Sockets;
using System.Net.Cache;
using System.Threading;

namespace socket_sendfile
{
    delegate TimeSpan SendFileDelegate(Socket client, String filename, long fileSize);

    class Header
    {
        public long FileSize { get; set; }
        public int FileNumber { get; set; }

        public void Serialize(Stream stream)
        {
            byte[] buffer = BitConverter.GetBytes(this.FileNumber);
            stream.Write(buffer, 0, buffer.Length);

            buffer = BitConverter.GetBytes(this.FileSize);
            stream.Write(buffer, 0, buffer.Length);
        }

        public static Header Deserialize(Stream stream)
        {
            Header header = new Header();

            byte[] buffer = new byte[4];
            int read = stream.Read(buffer, 0, buffer.Length);
            header.FileNumber = BitConverter.ToInt32(buffer, 0);

            buffer = new byte[sizeof(long)];
            read = stream.Read(buffer, 0, buffer.Length);
            header.FileSize = BitConverter.ToInt64(buffer, 0);
            return header;
        }
    }

    class Program
    {
        private Random rand = new Random();

        static void Main(string[] args)
        {
            Program prog = new Program();

            try
            {
                prog.StartServer();

                using (Socket client = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp))
                {
                    client.Bind(new IPEndPoint(0, 0));
                    client.Connect(new IPEndPoint(IPAddress.Loopback, 8080));

                    prog.Run(client, new SendFileDelegate(prog.SendFile1));

                    Console.WriteLine();
                    prog.Run(client, new SendFileDelegate(prog.SendFile2));
                }
            }
            catch (Exception e)
            {
                Console.Error.WriteLine(e);
            }
        }

        void StartServer()
        {
            Thread serverThread = new Thread(new ThreadStart(this.Server));
            serverThread.Start();
        }

        void Run(Socket client, SendFileDelegate sendFileMethod)
        {
            foreach (long size in this.GetNextSize())
            {
                String filename = Path.GetTempFileName();

                this.CreateFile(filename, size);

                for (int i = 0; i < 10; i++)
                {
                    TimeSpan ts = sendFileMethod(client, filename, size);
                    Console.WriteLine("{0} {1} {2}", i, size, ts.TotalMilliseconds);
                }
            }
        }

        IEnumerable GetNextSize()
        {
            ulong[] sizes = { 1024, 4096, 8192, 16385, 65536, 1048576 };
            for (int i = 0; i < sizes.Length; i++)
            {
                yield return sizes[i];
            }
        }

        void CreateFile(string filename, long size)
        {
            byte[] buffer = new byte[16384];

            // first write out the file.
            using (FileStream tempStream = File.OpenWrite(filename))
            using (BinaryWriter bw = new BinaryWriter(tempStream))
            {
                long remaining = size;
                while (remaining > 0)
                {
                    rand.NextBytes(buffer);
                    int writeSize = buffer.Length;
                    if (writeSize > (int)remaining)
                    {
                        writeSize = (int)remaining;
                    }

                    bw.Write(buffer, 0, writeSize);
                    remaining -= writeSize;
                }
            }
        }

        TimeSpan SendFile1(Socket client, String filename, long fileSize)
        {
            Stopwatch timer = new Stopwatch();

            timer.Start();
            using (NetworkStream ns = new NetworkStream(client))
            {
                Header header = new Header();
                header.FileSize = fileSize;
                header.FileNumber = 1;

                // send the header
                header.Serialize(ns);

                using (FileStream fs = File.OpenRead(filename))
                {
                    byte[] buffer = new byte[1024];
                    int read = fs.Read(buffer, 0, buffer.Length);

                    while (read > 0)
                    {
                        ns.Write(buffer, 0, read);
                        read = fs.Read(buffer, 0, buffer.Length);
                    }
                }
            }

            timer.Stop();

            return timer.Elapsed;
        }

        TimeSpan SendFile2(Socket client, String filename, long fileSize)
        {
            Stopwatch timer = new Stopwatch();

            timer.Start();
            using (NetworkStream ns = new NetworkStream(client))
            {
                Header header = new Header();
                header.FileSize = fileSize;
                header.FileNumber = 1;

                byte[] headerBuffer = null;
                using(MemoryStream ms = new MemoryStream())
                {
                    header.Serialize(ms);
                    ms.Seek(0, SeekOrigin.Begin);
                    headerBuffer = ms.ToArray();
                }

                // send the header
                client.SendFile(filename, headerBuffer, null, TransmitFileOptions.UseDefaultWorkerThread);
                
            }

            timer.Stop();

            return timer.Elapsed;
        }

        void Server()
        {
            byte [] buffer = new byte[1024];
            TcpListener listener = new TcpListener(8080);
            listener.Start();
            using (TcpClient client = listener.AcceptTcpClient())
            using (NetworkStream ns = client.GetStream())
            {
                bool hasData = true;
                while (hasData)
                {
                    // first get the header. Header has the file size.
                    Header header = Header.Deserialize(ns);

                    long remaining = header.FileSize;

                    while (remaining > 0)
                    {
                        int readSize = buffer.Length;
                        if ((long)readSize > remaining)
                            readSize = (int)remaining;

                        int read = ns.Read(buffer, 0, readSize);
                        remaining -= read;
                    }
                }
            }
        }
    }
}

A couple of things to note about this implementation:

1) It uses Message framing to frame file transfers, since it uses the same socket for multiple file transfers. I have used the techniques in Serializing data from .NET to Java to do this. Even though there is no Java app that is involved here, the techniques are the same.

2) The server just drains the incoming stream. It does not save the incoming data to a file. Since we are just interested in benchmarking the performance between the two Send implementations, we should be ok here.

3) The program, which is basically a perf harness, uses a Strategy pattern to change the SendFile method used. That way everything else remains the same, and it just changes the SendFile method to get performance numbers.

Perf Comparison

The following graph shows the performance with the simple Read/Write pattern for file transfer.

The following chart shows the performance when Socket.SendFile is used.

As you can see, there is a huge difference between the two, specially for 1M file size. With Socket.SendFile, it takes max 129ms for upload, whereas without this API, it takes 1000ms for upload. For smaller file sizes, there is not that much of a difference.

There is a huge variance in timings for the SendFile() method for 1M file size, but I havent been able to figure out the reason for that yet. Anyway, the fact that Socket.SendFile() is faster should not be impacted by that.

How to send object from Java to .NET: Alternate implementation

2009-11-15T15:45:00.000-08:00

In the last article, I described how to send an object from a Java application to a .NET application over a Socket connection.

http://ferozedaud.blogspot.com/2009/11/howto-serialize-data-from-object-from.html

The Java implementation was relying on ByteBuffer class to serialize the object in LittleEndian format that the .NET code understands.

However, it turns out that using a ByteBuffer is not necessary. Java's native format is BigEndian. The network byte order is also BigEndian. However, the X86 platform (and .NET) are LittleEndian. So, we only need a way to convert from BigEndian to LittleEndian and vice versa on .NET.

In .NET, the IPAddress.HostToNetworkOrder and IPAddress.NetworkToHostOrder methods are provided for doing this conversion.

On the java application, we can forego the ByteBuffer and use DataInputStream/DataOutputStream directly. This makes the code more concise and easy to understand.

With these changes, the .NET application looks as follows: (I am only including changes to the Read() method, the other stuff is the same.

static void Read(TcpClient client)
        {
            Console.WriteLine("Got connection: {0}", DateTime.Now);
            NetworkStream ns = client.GetStream();
            BinaryReader reader = new BinaryReader(ns);

            Employee emp = new Employee();
            // first read the Id
            emp.Id = IPAddress.NetworkToHostOrder(reader.ReadInt32());

            // length of first name in bytes.
            int length = IPAddress.NetworkToHostOrder(reader.ReadInt32());
            
            // read the name bytes into the byte array.
            // recall that java side is writing two bytes for every character.
            byte[] nameArray = reader.ReadBytes(length);
            emp.FirstName = Encoding.UTF8.GetString(nameArray);

            // last name
            length = IPAddress.NetworkToHostOrder(reader.ReadInt32());
            nameArray = reader.ReadBytes(length);
            emp.LastName = Encoding.UTF8.GetString(nameArray);

            // salary
            emp.Salary = IPAddress.NetworkToHostOrder(reader.ReadInt32());

            Console.WriteLine(emp.Id);
            Console.WriteLine(emp.FirstName);
            Console.WriteLine(emp.LastName);
            Console.WriteLine(emp.Salary);

            System.Threading.Thread.Sleep(5);

            Console.WriteLine("Writing data...");
            // now reflect back the same structure.
            BinaryWriter bw = new BinaryWriter(ns);

            bw.Write(IPAddress.HostToNetworkOrder(emp.Id));
            byte [] data = Encoding.UTF8.GetBytes(emp.FirstName);
            bw.Write(IPAddress.HostToNetworkOrder(data.Length));
            bw.Write(data);
            
            data = Encoding.UTF8.GetBytes(emp.LastName);
            bw.Write(IPAddress.HostToNetworkOrder(data.Length));
            bw.Write(data);

            bw.Write(IPAddress.HostToNetworkOrder(emp.Salary));

            Console.WriteLine("Writing data...DONE");

            client.Client.Shutdown(SocketShutdown.Both);
            ns.Close();
        }

As you can see, we still use BinaryReader/BinaryWriter. However we just call NetworkToHostOrder or HostToNetworkOrder before writing the bytes on the wire.

On the java application, there are changes in the Main() method and the serialize() method. We directly work with DataInputStream/DataOutputStream which are similar to .NET's BinaryReader/BinaryWriter.

public static void main(String [] args)
 { 
  int written = 0;
  EmployeeData emp = new EmployeeData();
  emp.setFirstName("John");
  emp.setLastName("Smith");
  emp.setId(1);
  emp.setSalary(2);
  
  // Create the encoder and decoder for targetEncoding
  Charset charset = Charset.forName("UTF-8");
  CharsetDecoder decoder = charset.newDecoder();
  CharsetEncoder encoder = charset.newEncoder();

  try
  {
   Socket client = new Socket("localhost", 8080);
   
   OutputStream oStream = client.getOutputStream();
   InputStream iStream = client.getInputStream();

   DataOutputStream dos = new DataOutputStream(oStream);

   serializeToStream(dos, emp, encoder);

   DataInputStream dis = new DataInputStream(iStream);

   // now client echoes back the data.
   EmployeeData rEmp = new EmployeeData();
   rEmp.setId(dis.readInt());
   
   int length = dis.readInt();
   System.out.println("FName Length: " + length);
   byte [] stringBuffer = new byte[length];
   dis.read(stringBuffer);
   rEmp.setFirstName(decoder.decode(ByteBuffer.wrap(stringBuffer)).toString());
   
   length = dis.readInt();
   System.out.println("LName Length: " + length);
   stringBuffer = new byte[length];
   dis.read(stringBuffer);
   rEmp.setLastName(decoder.decode(ByteBuffer.wrap(stringBuffer)).toString());
   
   rEmp.setSalary(dis.readInt());
   
   System.out.println("ID: " + rEmp.getId());
   System.out.println("First: " + rEmp.getFirstName());
   System.out.println("Last: " + rEmp.getLastName());
   System.out.println("Salary: " + rEmp.getSalary());
   System.out.flush();
   
   client.close();
   
  } catch(Exception e) {
   e.printStackTrace(System.err);
  } finally {
   System.out.println("Written bytes: " + written);
  }
 }
 
 private static void serializeToStream(DataOutputStream os, EmployeeData emp, CharsetEncoder encoder) throws IOException
 {
  // id
  os.writeInt(emp.getId());
  
  CharBuffer nameBuffer = CharBuffer.wrap(emp.getFirstName().toCharArray());
  ByteBuffer nbBuffer = null;
  
  // length of first name
  try
  {
   nbBuffer = encoder.encode(nameBuffer);
  } 
  catch(CharacterCodingException e)
  {
   throw new ArithmeticException();
  }

  System.out.println(String.format("String [%1$s] #bytes = %2$s", emp.getFirstName(), nbBuffer.limit()));
  os.writeInt(nbBuffer.limit());
  os.write(nbBuffer.array());

  // put lastname
  nameBuffer = CharBuffer.wrap(emp.getLastName().toCharArray());
  nbBuffer = null;
  
  // length of first name
  try
  {
   nbBuffer = encoder.encode(nameBuffer);   
  } 
  catch(CharacterCodingException e)
  {
   throw new ArithmeticException();
  }

  System.out.println(String.format("String [%1$s] #bytes = %2$s", emp.getLastName(), nbBuffer.limit()));
  os.writeInt(nbBuffer.limit());
  os.write(nbBuffer.array());
  
  // salary
  os.writeInt(emp.getSalary());
 }

We do not need to use ByteBuffer anymore - we were only using it because it supports LittleEndian format. The java app always expects to read/write bytes from the wire in BigEndian format (which is java's native format, as well as the Network order for all communications). The .NET application does the conversion from BigEndian to LittleEndian and back.

HOWTO: Serialize data from an object from .NET to Java using custom binary formatting.

2009-11-15T00:34:00.000-08:00

Problem: You want to send data across the network, from a .NET application to a Java application (or vice versa). The data is actually an object that you want to send over the network.

Since these are heterogenous platforms, you cannot use the Runtime specific serialization methods (for eg BinaryFormatter in .NET). You need to roll your own.

You basically have two choices. You can manually transfer the data, by converting the primitive types from one platform into a binary, over the wire representation. On the other side, you need to convert the binary, over the wire representation into a primitive type of the platform.

The other option is to use a serialization method supported by the platform, for eg XmlSerialization. You would then write code on the peer to parse the XML document from the network and convert it into your object representation. Since XML is a portable standard, you can do this fairly easily.

In this article, I will be talking about the first, i.e how to do Binary Serialization manually.

In order to demonstrate this, we will work with the following object in Java.

package net.example;

public class EmployeeData {
 private int id;
 private String firstName;
 private String lastName;
 private int salary;
 
 public void setId(int id) {
  this.id = id;
 }
 public int getId() {
  return id;
 }
 public void setFirstName(String firstName) {
  this.firstName = firstName;
 }
 public String getFirstName() {
  return firstName;
 }
 public void setLastName(String lastName) {
  this.lastName = lastName;
 }
 public String getLastName() {
  return lastName;
 }
 public void setSalary(int salary) {
  this.salary = salary;
 }
 public int getSalary() {
  return salary;
 }
}

This has an equivalent object in .NET

class Employee
    {
        public int Id;
        public String FirstName;
        public String LastName;
        public int Salary;
    }

As you can see, these objects are fairly equivalent in terms of schema.

For each of the platforms, the program will be similar. Each application will receive data that purports to be an Employee record. The app will deserialize the data into an employee record. It will then send the same data back to the peer. The peer will then deserialize the data back into it's equivalent Employee structure. Thus, this will demonstrate a good round-trip between the heterogenous platforms.

First, let us look at the complete application in .NET.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.Net.Sockets;
using System.IO;

namespace netSock
{
    class Employee
    {
        public int Id;
        public String FirstName;
        public String LastName;
        public int Salary;
    }

    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                Run(args);
            }
            catch (Exception e)
            {
                Console.Error.WriteLine(e);
            }
        }

        static void Run(string[] args)
        {
            TcpListener listener = new TcpListener(8080);
            listener.Start();

            while (true)
            {

                using (TcpClient client = listener.AcceptTcpClient())
                {
                    try
                    {
                        Read(client);
                    }
                    catch (Exception e)
                    {
                        Console.Error.WriteLine(e);
                    }
                }
            }
        }

        static void Read(TcpClient client)
        {
            Console.WriteLine("Got connection: {0}", DateTime.Now);
            NetworkStream ns = client.GetStream();
            BinaryReader reader = new BinaryReader(ns);

            Employee emp = new Employee();
            // first read the Id
            emp.Id = reader.ReadInt32();

            // length of first name in bytes.
            int length = reader.ReadInt32();
            
            // read the name bytes into the byte array.
            // recall that java side is writing two bytes for every character.
            byte[] nameArray = reader.ReadBytes(length);
            emp.FirstName = Encoding.UTF8.GetString(nameArray);

            // last name
            length = reader.ReadInt32();
            nameArray = reader.ReadBytes(length);
            emp.LastName = Encoding.UTF8.GetString(nameArray);

            // salary
            emp.Salary = reader.ReadInt32();

            Console.WriteLine(emp.Id);
            Console.WriteLine(emp.FirstName);
            Console.WriteLine(emp.LastName);
            Console.WriteLine(emp.Salary);

            System.Threading.Thread.Sleep(5);

            Console.WriteLine("Writing data...");
            // now reflect back the same structure.
            BinaryWriter bw = new BinaryWriter(ns);

            bw.Write(emp.Id);
            byte [] data = Encoding.UTF8.GetBytes(emp.FirstName);
            bw.Write(data.Length);
            bw.Write(data);
            
            data = Encoding.UTF8.GetBytes(emp.LastName);
            bw.Write(data.Length);
            bw.Write(data);

            bw.Write(emp.Salary);

            Console.WriteLine("Writing data...DONE");

            client.Client.Shutdown(SocketShutdown.Both);
            ns.Close();
        }
    }
}

This program is very simple. It uses a TcpListener to create a server application, and waits for connections. After getting a connection, it reads data from the stream and deserializes it into a Employee class. It then sends the object back to the sender by manually serializing the primitive types into the stream.

The output of this program is as follows (when the java client connects to it):

Got connection: 11/15/2009 12:20:02 AM
1
John
Smith
2
Writing data...
Writing data...DONE

AS you can see, the employee First Name is "John" and Last name is "Smith".

If you have some problem with this, then you can debug it by getting a Tracelog for the application.

Now, let us look at the java example application:

package net.example;
import java.io.DataOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.Socket;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.CharBuffer;
import java.nio.charset.CharacterCodingException;
import java.nio.charset.Charset;
import java.nio.charset.CharsetDecoder;
import java.nio.charset.CharsetEncoder;

public class Main {
 public static void main(String [] args)
 {
  
  int written = 0;
  EmployeeData emp = new EmployeeData();
  emp.setFirstName("John");
  emp.setLastName("Smith");
  emp.setId(1);
  emp.setSalary(2);
  
  // Create the encoder and decoder for targetEncoding
  Charset charset = Charset.forName("UTF-8");
  CharsetDecoder decoder = charset.newDecoder();
  CharsetEncoder encoder = charset.newEncoder();
  byte [] underlyingBuffer = new byte[1024];
  ByteBuffer buffer = ByteBuffer.wrap(underlyingBuffer);
  buffer.order(ByteOrder.LITTLE_ENDIAN);
  try
  {
   Socket client = new Socket("localhost", 8080);
   
   OutputStream oStream = client.getOutputStream();
   InputStream iStream = client.getInputStream();

   serialize(buffer, emp, encoder);
   
   buffer.flip();
   
   int dataToSend = buffer.remaining();
   System.out.println("# bytes = " + dataToSend);
   
   System.out.println("#Bytes in output buffer: " + written + " limit = " + buffer.limit() + " pos = " + buffer.position() + " remaining = " + buffer.remaining());
   
   int remaining = dataToSend;
   while(remaining > 0)
   {
    oStream.write(buffer.get());
    -- remaining;
   }
   
   // now client echoes back the data.
   
   ByteBuffer readBuffer = ByteBuffer.allocate(1024);
   readBuffer.order(ByteOrder.LITTLE_ENDIAN);
   
   int db = iStream.read();
   while(db != -1)
   {
    System.out.println(db);
    readBuffer.put((byte)db);
    db = iStream.read();
   }
   
   int numberOfBytesRead = readBuffer.position();
   
   System.out.println("Number of bytes read: " + numberOfBytesRead);
   
   readBuffer.flip();

   EmployeeData rEmp = new EmployeeData();
   rEmp.setId(readBuffer.getInt());
   
   int length = readBuffer.getInt();
   System.out.println("FName Length: " + length);
   byte [] stringBuffer = new byte[length];
   readBuffer.get(stringBuffer);
   rEmp.setFirstName(decoder.decode(ByteBuffer.wrap(stringBuffer)).toString());
   
   length = readBuffer.getInt();
   System.out.println("LName Length: " + length);
   stringBuffer = new byte[length];
   readBuffer.get(stringBuffer);
   rEmp.setLastName(decoder.decode(ByteBuffer.wrap(stringBuffer)).toString());
   
   rEmp.setSalary(readBuffer.getInt());
   
   System.out.println("ID: " + rEmp.getId());
   System.out.println("First: " + rEmp.getFirstName());
   System.out.println("Last: " + rEmp.getLastName());
   System.out.println("Salary: " + rEmp.getSalary());
   System.out.flush();
   
   client.close();
   
  } catch(Exception e) {
   e.printStackTrace(System.err);
  } finally {
   System.out.println("Written bytes: " + written);
  }
 }
 
 private static void serialize(ByteBuffer buffer, EmployeeData emp, CharsetEncoder encoder)
 {
  // id
  buffer.putInt(emp.getId());
  
  CharBuffer nameBuffer = CharBuffer.wrap(emp.getFirstName().toCharArray());
  ByteBuffer nbBuffer = null;
  
  // length of first name
  try
  {
   nbBuffer = encoder.encode(nameBuffer);
  } 
  catch(CharacterCodingException e)
  {
   throw new ArithmeticException();
  }

  System.out.println(String.format("String [%1$s] #bytes = %2$s", emp.getFirstName(), nbBuffer.limit()));
  buffer.putInt(nbBuffer.limit());
  buffer.put(nbBuffer);

  // put lastname
  nameBuffer = CharBuffer.wrap(emp.getLastName().toCharArray());
  nbBuffer = null;
  
  // length of first name
  try
  {
   nbBuffer = encoder.encode(nameBuffer);   
  } 
  catch(CharacterCodingException e)
  {
   throw new ArithmeticException();
  }

  System.out.println(String.format("String [%1$s] #bytes = %2$s", emp.getLastName(), nbBuffer.limit()));
  buffer.putInt(nbBuffer.limit());
  buffer.put(nbBuffer);
  
  // salary
  buffer.putInt(emp.getSalary());
 }
}

The java client has a similar structure to the .NET app, except for the fact that the .NET app acts like the server, whereas the Java app is just a client. However that difference is not material to the current topic.

The output of the java program is as follows:

String [John] #bytes = 4
String [Smith] #bytes = 5
# bytes = 25
#Bytes in output buffer: 0 limit = 25 pos = 0 remaining = 25
1
0
0
0
4
0
0
0
74
111
104
110
5
0
0
0
83
109
105
116
104
2
0
0
0
Number of bytes read: 25
FName Length: 4
LName Length: 5
ID: 1
First: John
Last: Smith
Salary: 2
Written bytes: 0

Now, let me talk about some important details of this application. First realize, that each platform has it's own way of how it represents primitive datatypes (bytes/integers/floats/characters) in a byte representation in memory. This concept is called Endianness. In the case of Java, the JRE is "Big Endian". While, on the CLR (i.e .NET) the byte representation is "Little Endian". So, when we convert data into a binary representation, we need to make sure that both sides do it the same way.

Since .NET is "Little Endian" I decided to convert everything to Little Endian. On Java, this is very simple. Since I was using ByteBuffer to convert Java primitive types into bytes, I just set the ByteOrder on the ByteBuffer to be LITTLE_ENDIAN.

The second thing to remember is about strings. When you are serializing objects that contain heterogenous datatypes, or types that vary in length, you need a way of distinguishing the end of one type from the begining of another. In the case of strings, I chose the Length. So, when I serialize a string, I first write out the length of the string, and then the entire string. The receiver first reads the length of the string, and then allocates a byte buffer for that string. It then reads in the actual bytes (upto the length of the string) into the array. Note, that the quantity written is not actually the length in Characters, but the length in bytes of the Encoded string.

Which brings me to the final point. String conversion to Bytes can be done in various ways. I chose to use UTF-8 encoding. Again, the encoding used has to be the same on both the sides in order to be able to exchange the data correctly.

I hope this was useful to you. In the next article, I will talk about using XML as a mechanism of transporting objects across heterogenous platforms.

System.Net Links and HOWTOs

2009-11-01T17:26:00.000-08:00

A lot of people have blogged about System.NET tips and tricks. Esp from Microsoft, prior System.Net team members (Jon Cole, Durgaprasad Gorti, Malar Chinnusamy) have authored blog posts on how to do stuff with System.Net.

Non microsoft folks have also stepped up to the plate. For eg, Stephen Cleary has written a very good concise blog on System.Net HOWTOs and FAQs.

In this post, I aim to categorize all of them and link them in one place, so that it will be easy to find for people who need information.

FAQ

Stephen Cleary's .NET TCP/IP Sockets FAQ
How to use a Network Monitor to debug your System.Net application
Creating a tracelog for your application
Proxy Configuration Best Practices in System.Net

Authentication

Primer Part-1

Message Framing

Article on preserving message boundnaries
Synchronous Sample
Async sample
Another excellent sample on message framing by Stephen Cleary

Cookies
How to use CookieCollection/CookieContainer

HTTP Expect-100 Continue Caveats

The following blog post explains the consequences of having Expect100Continue=true for some servers.

Autoproxy support

http://blogs.msdn.com/mahjayar/archive/2005/08/04/447799.aspx

Socket Duplication

Socket duplication is a new feature supported by the 2.0 version of the .NET framework. The following articles describe that in more detail, along with code samples.

Socket Duplication - Part 1
Socket Duplication - Part 2

IPv6

Configuring an IPv4 server to be able to receive connections from IPv6 clients

SMTPClient
http://www.systemnetmail.com/default.aspx (FAQ for system.net.mail namespace and classes)
Sending mail to Gmail a/c with SMTPClient

Socket Applications: Traceroute

Part 1
Part 2
Part 3
Part 4

HOWTO send an object across the network, between a Java app and a .NET app, and back.

http://ferozedaud.blogspot.com/2009/11/howto-serialize-data-from-object-from.html

Write your own search engine using Lucene

2009-10-24T11:51:00.000-07:00

Recently, I have been playing around with Lucene (http://incubator.apache.org/lucene.net/. Lucene is an Open Source project, which is sponsored by the Apache foundation, that gives you all the components necessary to create your own search engine.

I downloaded the latest build, which is versioned 2.0.004 and located at http://incubator.apache.org/lucene.net/download/Incubating-Apache-Lucene.Net-2.0-004-11Mar07.bin.zip.

To start off I wrote a small application that indexes all the INF files in my %WINDIR%\System32 directory, and allows me to search their contents.

Here is an example of how the application works:

Search...
You can enter....
filename|fullpath|lastmodified|contents
>> windows fullpath
Query: fullpath:windows
#hits: 8
----------------
Filename: homepage.inf
FullPath: c:\windows\system32\homepage.inf
Last-Modified: 8/10/2004 3:00:00 AM
----------------
Filename: ieuinit.inf
FullPath: c:\windows\system32\ieuinit.inf
Last-Modified: 6/29/2009 1:40:16 AM
----------------
Filename: mapisvc.inf
FullPath: c:\windows\system32\mapisvc.inf
Last-Modified: 4/14/2006 11:39:08 PM
----------------
Filename: mmdriver.inf
FullPath: c:\windows\system32\mmdriver.inf
Last-Modified: 8/10/2004 3:00:00 AM
----------------
Filename: msxmlx.inf
FullPath: c:\windows\system32\msxmlx.inf
Last-Modified: 8/6/2003 10:15:48 AM
----------------
Filename: pid.inf
FullPath: c:\windows\system32\pid.inf
Last-Modified: 6/20/2007 10:52:36 PM
----------------
Filename: $ncsp$.inf
FullPath: c:\windows\system32\$ncsp$.inf
Last-Modified: 9/23/2005 6:50:22 AM
----------------
Filename: $winnt$.inf
FullPath: c:\windows\system32\$winnt$.inf
Last-Modified: 9/27/2005 8:46:09 PM
Search...
You can enter....
filename|fullpath|lastmodified|contents
>> msxml contents
Query: contents:msxml
#hits: 1
----------------
Filename: msxmlx.inf
FullPath: c:\windows\system32\msxmlx.inf
Last-Modified: 8/6/2003 10:15:48 AM
Search...
You can enter....
filename|fullpath|lastmodified|contents
>>

Here is the code for the application.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Lucene.Net;
using Lucene.Net.Analysis;
using Lucene.Net.Analysis.Standard;
using Lucene.Net.Documents;
using Lucene.Net.Store;
using Lucene.Net.Util;
using Lucene.Net.Index;
using Lucene.Net.Search;
using Lucene.Net.QueryParsers;
using sdir = System.IO.Directory;
using LDirectory = Lucene.Net.Store.Directory;
using System.IO;

namespace searchtest
{
    class Program
    {
        static void Main(string[] args)
        {
            Analyzer analyzer = new StandardAnalyzer();
            LDirectory directory = FSDirectory.GetDirectory("/index.bin", true);
            //Directory directory = new RAMDirectory();
            IndexWriter writer = new IndexWriter(directory, analyzer, true);
            writer.SetMaxFieldLength(25000);

            String [] infs = sdir.GetFiles(@"c:\windows\system32", "*.inf");
            foreach (String inf in infs)
            {
                FileInfo fi = new FileInfo(inf);
                Document doc = new Document();
                doc.Add(new Field("filename", fi.Name, Field.Store.YES, Field.Index.TOKENIZED));
                doc.Add(new Field("fullpath", fi.FullName, Field.Store.YES, Field.Index.TOKENIZED));
                doc.Add(new Field("lastmodified", DateField.DateToString(fi.LastWriteTimeUtc), Field.Store.YES, Field.Index.TOKENIZED));

                using (StreamReader sr = new StreamReader(inf))
                {
                    String text = sr.ReadToEnd();
                    doc.Add(new Field("contents", text, Field.Store.YES,
                        Field.Index.TOKENIZED));
                    writer.AddDocument(doc);
                }
            }

            writer.Close();

            // Now search the index:
            IndexSearcher isearcher = new IndexSearcher(directory);

            while (true)
            {
                Console.WriteLine("Search...");
                Console.WriteLine("You can enter....");
                Console.WriteLine("filename|fullpath|lastmodified|contents");

                Console.Write(">> ");
                String cmd = Console.ReadLine();

                if (cmd == null || cmd.Length == 0)
                    break;

                String fieldname = "contents";
                String predicate = null;

                if (cmd.StartsWith("!"))
                {
                    int index = cmd.LastIndexOf("\"");
                    predicate = cmd.Substring(2, index-2);
                    fieldname = cmd.Substring(index + 1);
                }
                else if (cmd.StartsWith("\""))
                {
                    int index = cmd.LastIndexOf("\"");
                    if (index != -1)
                    {
                        predicate = cmd.Substring(1, index-1);
                        if (++index < cmd.Length)
                        {
                            fieldname = cmd.Substring(index);
                        }
                    }
                }
                else
                {
                    String[] tokens = cmd.Split();
                    if (tokens.Length == 2)
                    {
                        predicate = tokens[0];
                        fieldname = tokens[1];
                    }
                    else if (tokens.Length == 1)
                    {
                        predicate = tokens[0];
                    }
                    else
                    {
                        Console.WriteLine("ERROR:");
                        continue;
                    }
                }

                // Parse a simple query that searches for "text":
                QueryParser parser = new QueryParser(fieldname, analyzer);
                Query query = null;

                try
                {
                    query = parser.Parse(predicate);
                }
                catch (Exception e)
                {
                    Console.WriteLine(e);
                    continue;
                }


                Hits hits = isearcher.Search(query);
                Console.WriteLine("Query: {0}", query.ToString());
                Console.WriteLine("#hits: {0}", hits.Length());
                // Iterate through the results:
                for (int i = 0; i < hits.Length(); i++)
                {
                    Console.WriteLine("----------------");
                    Document hitDoc = hits.Doc(i);
                    Console.WriteLine("Filename: {0}", hitDoc.Get("filename"));
                    Console.WriteLine("FullPath: {0}", hitDoc.Get("fullpath"));
                    Field f = hitDoc.GetField("lastmodified");
                    Console.WriteLine("Last-Modified: {0}", DateField.StringToDate(hitDoc.Get("lastmodified")));
                    //Console.WriteLine(hitDoc.Get("contents"));
                }
            }
            isearcher.Close();
            directory.Close(); 
        }
    }
}

NTLM auth fails with HttpWebRequest/WebClient, but passes with IE

2009-10-02T19:59:00.000-07:00

On public newsgroups, I have seen a lot of postings where people complained that their managed code application, written with HttpWebRequest, and using NTLM auth to talk to a server, would fail. However, Internet explorer running on the same machine would work fine.

Here are some of the threads that show this problem:

http://social.msdn.microsoft.com/Forums/en-US/netfxnetcom/thread/a4aba6c5-6180-441e-ab60-95347fcdc051

In order to root cause this issue, you need to enable logging using System.Net tracelog (http://ferozedaud.blogspot.com/2009/08/tracing-with-systemnet.html) and see the trace. If you see that the client fails with a NotSupported error, when trying to compose a Type2 message (using the response to the previous Type1 message sent by the client).

The second variable here is the operating system on both the client and server. If the OS on the client is >= Vista (for eg, any flavor of Vista or Windows7) and the OS on the server is a version before Vista, then there was a change in the way NTLM works. In vista and later operating systems, NTLM by default now requires 128bit encryption, whereas the prior OS did not.

Ok. So why does IE work on the same machine, and NTLM doesnt?

The difference is the way in which both use the NTLM SSPI package.

When HttpWebRequest uses the package, it asks for NTLMSSP_NEGOTIATE_SEAL and NTLM_NEGOTIATE_SIGN capabilities. This requres encryption. Since 128bit encryption is now required by the OS, this means that the server also has to support 128bit. If the server doesnt, then the authentication will fail.

IE does not ask for SEAL|SIGN capabilities when composing the Type2 message. So, even if the server does not support 128bit encryption, the authentication can still work.

For more details, refer to this thread on stackoverflow:

http://stackoverflow.com/questions/1443617/407-authentication-required-no-challenge-sent/1482442#1482442

Note, that even WindowsXP/Server2003 supports 128bit encryption, just not out of the box. And on Windows7/Vista, even though 128bit is default, it can be changed by modifying the security policy. However, that might now always be possible, esp if the machine is on a domain where the policy is administered by the Domain Admin.

Implementing Traceroute with System.Net - Part IV

2009-09-29T18:26:00.000-07:00

In the previous parts, I showed how to implement traceroute using System.Net.Sockets. We used the Socket class to create a Raw socket, and use that to do a traceroute.

Implementing Traceroute with System.Net Part III
Implementing Traceroute with System.Net Part II
Implementing Traceroute with System.Net Part I

It turns out, that System.Net.NetworkInformation already has a Ping class, that encapsulates all the functionality of sending ICMP ECHO requests and processing replies. It has the necessary switches to tweak the TTL settings as well.

I wrote a small program to demonstrate how we can do a Traceroute using the Ping class.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.Net.Sockets;
using System.Net.NetworkInformation;

namespace PingClassTest
{
    class Program
    {
        static void Main(string[] args)
        {
            Ping pinger = new Ping();
            String data = "0123456789ABCDEF";
            byte[] buffer = Encoding.ASCII.GetBytes(data);
            PingOptions options = new PingOptions();

            IPHostEntry target = Dns.GetHostEntry(args[0]);
            bool isDestination = false;
            for(int i = 1; i <= 30; i++)
            {
                String intermediateHost = null;
                for (int j = 0; j < 3; j++)
                {
                    options.Ttl = i;
                    PingReply reply = pinger.Send(target.AddressList[0], 30, buffer, options);
                    switch (reply.Status)
                    {
                        case IPStatus.TimedOut:
                        case IPStatus.TtlExpired:
                        case IPStatus.Success:
                            Console.Write("<{0}ms\t", reply.RoundtripTime);
                            if (reply.Address.Equals(target.AddressList[0]))
                            {
                                isDestination = true;
                            }
                            intermediateHost = reply.Address.ToString();
                            break;
                        default:
                            Console.Write("*\t");
                            break;

                    }
                }

                Console.WriteLine("\t{0}", intermediateHost);

                if (isDestination)
                {
                    break;
                }
            }javascript:void(0)
        }

        pinger.Dispose();
    }
}

The output of this program, when pinging www.yahoo.com is as follows:

<0ms    <0ms    <0ms            192.168.1.1
<0ms    <0ms    <0ms            98.117.116.1
<0ms    <0ms    <0ms            130.81.138.218
<0ms    <0ms    <0ms            130.81.28.166
<0ms    <0ms    <0ms            130.81.17.56
<0ms    <0ms    <0ms            130.81.17.231
<0ms    <0ms    <0ms            130.81.14.90
<0ms    <0ms    <0ms            216.115.107.57
<0ms    <0ms    <0ms            209.131.32.23
<27ms   <26ms   <26ms           209.131.36.158
Press any key to continue . . .

As you can see, this program gives the exact same functionality as the program that we wrote using the Socket class. The Ping class also supports asynchronous functionality which makes it very useful for high performance applications.

Implementing Traceroute with System.Net - Part III

2009-09-17T18:25:00.000-07:00

In the previous part http://ferozedaud.blogspot.com/2009/09/implementing-traceroute-with-systemnet_07.html I showed a simple implementation of traceroute, that built upon my earlier implementation of Ping.

As promised, in this article, we will see how to make the utility more robust.

One of the things I didnt like about the previous implementation, was that there was no verification of the received request packet. The implementation was not verifying whether the type of the response was correct - an ICMP Echo request should elicit an ICMP Echo response (see the ICMP RFC at http://www.ietf.org/rfc/rfc0792.txt)

So, we add the code for verifying that. According to the RFC, the Echo reply message has a Type Field = 0. Also, since we are using the TTL mechanism for detecting hosts along the route to the destination, we expect that every host but the final one, will reply with a ICMP Time Exceeded message, which has a Message Type = 11.

Also, the other issue with the tool, is that it does not resolve the IPAddress of the intermediate hosts. In a traceroute output, we are mostly interested in the hostnames of the intermediate routers, and not just the IP address. So, we will add that capability as well.

With these changes, here is the changed code. I have only put the changes to the while(true) loop in my original implementation.

while (true)
{
Console.Write("{0}", hop);
bool allTimedOut = true;
for (int i = 0; i < 3; i++)
{
ICMP_PACKET packet = ICMP_PACKET.CreateRequestPacket(111, 222, data);

IPEndPoint epRemote = new IPEndPoint(ipTarget, 0);

pingSocket.SetSocketOption(SocketOptionLevel.IP, SocketOptionName.IpTimeToLive, hop);


stopWatch.Start();
pingSocket.SendTo(packet.Serialize(), epRemote);



byte[] receiveData = new byte[1024];
int read = 0;
try
{
epResponse = new IPEndPoint(0, 0);
read = pingSocket.ReceiveFrom(receiveData, ref epResponse);
stopWatch.Stop();

ICMP_PACKET recvPacket = new ICMP_PACKET(receiveData, 20, read);

ipepResponse = epResponse as IPEndPoint;

if (recvPacket.PacketType == 11)
{
// this is an ICMP Time exceeded message
if (ipepResponse.Address.Equals(ipTarget))
{
// do not expect an ICMP Time Exceeded message from the
// final destination.
Console.Error.Write("\t!!");
}
}
else if (recvPacket.PacketType == 0)
{
// this is an ICMP Echo reply message
// validate that this is coming from the destination host.
if (!ipepResponse.Address.Equals(ipTarget))
{
// do not expect an ICMP Time Exceeded message from the
// final destination.
Console.Error.Write("\t!!");
}
}

Console.Write("\t<<{0}ms", stopWatch.ElapsedMilliseconds);
allTimedOut = false;
}
catch (SocketException e)
{
Console.Write("\t*");
}
}

if (allTimedOut)
{
Console.WriteLine("\tRequest timed out");
}
else
{
String intermediateHost = null;
// now try to resolve the IPAddress
try
{
IPHostEntry hostEntry = Dns.GetHostEntry(ipepResponse.Address);
intermediateHost = hostEntry.HostName;
}
catch (SocketException e)
{
}

Console.WriteLine("\t{0} {1}", ((IPEndPoint)epResponse).Address.ToString(), intermediateHost);

}
++hop;

ipepResponse = epResponse as IPEndPoint;
if (hop > maxHops || ipepResponse.Address.Equals(ipTarget))
{
break;
}
}

In the next part, I will throw some ideas out there, on how this can be extended even more.

Network Programmers Toolchest

2009-09-11T18:09:00.001-07:00

In this post, I want to talk about some of the tools that a programmer doing network programmer might want to learn, in order to do his job better.

First, let us talk about logging tools. No matter which library you use, whether it is System.Net from .NET framework, or CommonsHTTP from the Apache Commons project, you need to have a way of enabling logging to see what is going on in the library.

If you are using System.Net, then you can enable logging using the Logging facility provided in the .NET framework.

http://ferozedaud.blogspot.com/2009/08/tracing-with-systemnet.html

For Apache ComonsHttp library, you can enable logging using the Log4J logging library:

http://logging.apache.org/log4j/index.html

Next, you need a way to capture network traffic in order to see what is going on in the network. Many tools exist that can allow you to do that. I prefer to use Wireshark, but on windows, Netmon is also pretty good.

http://www.wireshark.org
http://blogs.technet.com/netmon

Imagine that you are trying to write code that can simulate loggin in to a website. For eg, Facebook or a line of business application. However, it isnt working - you tried using logging but cant seem to get to the problem. In this case, you might want to use a browser and try out the same scenario, and see what the browser is doing differently.

You can do that by using Wireshark/Netmon to sniff the browser session, but it is not very user friendly. Plus, if the session is encrypted (HTTPS) you cannot even make sense of the network sniff.

In this case, it helps to have a browser extension that can sniff the outgoing requests and incoming responses for you. I like to use Firebug on Firefox, and there are equivalent extensions for Internet Explorer as well.

http://getfirebug.com/

Here are the ones for Internet Explorer.

http://www.fiddler2.com/Fiddler2/version.asp
http://www.httpwatch.com/

Readers: Do you have any favorite tools that I have not covered here? I would like to know about them. Add a comment to this post with the information.

The case of multiple NTLM challenges with IIS7

2009-09-10T08:55:00.000-07:00

A reader posed a question on Stack Overflow about a problem with authentication. The scenario was as follows:

This was a typical 3-tier application, in which the middle-tier (ASP.NET web server) was making a HttpWebRequest to a backend II7 server that required authentication.

When authentication method was set to Digest/Basic/Negotiate, the server worked fine, and the authentication succeeded. However, if the auth method was set to NTLM, the server started to challenge twice.

The reader investigate this, and found that this was caused by a Microsoft Security Update http://support.microsoft.com/kb/957097. Also, he came up with a solution for the problem.

Read more about the problem and it's solution at: http://www.tinyint.com/index.php/2009/08/24/401-error-on-httpwebrequest-with-ntlm-authentication/

Implementing Traceroute with System.Net: Part-II

2009-09-07T19:09:00.001-07:00

In this part, we will implement traceroute according to the principles we discussed in the last post:

http://ferozedaud.blogspot.com/2009/09/implementing-traceroute-with-systemnet.html

First, start with the code for Ping, which is at http://blogs.msdn.com/feroze_daud/archive/2005/10/26/485372.aspx and modify the Main() method as follows:

    static void Main(string[] args)
    {

        if (args.Length != 1)
        {
            Usage();
        }



        if (0 == String.Compare(args[0], "/?", true, CultureInfo.InvariantCulture)

        || 0 == String.Compare(args[0], "-h", true, CultureInfo.InvariantCulture)

        || 0 == String.Compare(args[0], "-?", true, CultureInfo.InvariantCulture))
        {
            Usage();
        }

        string target = args[0];

        IPAddress[] heTarget = Dns.GetHostAddresses(target);

        IPAddress ipTarget = null;

        foreach (IPAddress ip in heTarget)
        {

            if (ip.AddressFamily == AddressFamily.InterNetwork)
            {
                ipTarget = ip;
                break;
            }
        }



        IPAddress[] heSource = Dns.GetHostAddresses(Dns.GetHostName());

        IPAddress ipSource = null;

        foreach (IPAddress ip in heSource)
        {
            byte[] addressBytes = ip.GetAddressBytes();
            if (addressBytes[0] == 169 && addressBytes[1] == 254)
            {
                continue;
            }
            if (ip.AddressFamily == AddressFamily.InterNetwork)
            {
                ipSource = ip;
                break;
            }
        }

        IPEndPoint epLocal = new IPEndPoint(ipSource, 0);

        Socket pingSocket = new Socket(AddressFamily.InterNetwork, SocketType.Raw, ProtocolType.Icmp);

        pingSocket.Bind(epLocal);

        byte[] data = Encoding.ASCII.GetBytes("1234567890abcdef");

        Console.WriteLine("Ping {0}({1}) with {2} bytes of data...", target, ipTarget.ToString(), data.Length);

        Console.WriteLine();

        int hop = 1;
        int maxHops = 30;

        pingSocket.ReceiveTimeout = 30;
        System.Diagnostics.Stopwatch stopWatch = new System.Diagnostics.Stopwatch();
        EndPoint epResponse = (EndPoint)new IPEndPoint(0, 0);

        while (true)
        {
            Console.Write("{0}", hop);
            bool allTimedOut = true;
            for (int i = 0; i < 3; i++)
            {
                ICMP_PACKET packet = ICMP_PACKET.CreateRequestPacket(111, 222, data);

                IPEndPoint epRemote = new IPEndPoint(ipTarget, 0);

                pingSocket.SetSocketOption(SocketOptionLevel.IP, SocketOptionName.IpTimeToLive, hop);


                stopWatch.Start();
                pingSocket.SendTo(packet.Serialize(), epRemote);

                byte[] receiveData = new byte[1024];
                int read = 0;
                try
                {
                    epResponse = new IPEndPoint(0, 0);
                    read = pingSocket.ReceiveFrom(receiveData, ref epResponse);
                    stopWatch.Stop();
                    ICMP_PACKET recvPacket = new ICMP_PACKET(receiveData, 20, read);

                    Console.Write("\<<{0}ms", stopWatch.ElapsedMilliseconds);
                    allTimedOut = false;
                }
                catch (SocketException e)
                {
                    Console.Write("\t*");
                }
            }

            if (allTimedOut)
            {
                Console.WriteLine("\tRequest timed out");
            }
            else
            {
                Console.WriteLine("\t{0}", ((IPEndPoint)epResponse).Address.ToString());
            }
            ++hop;

            IPEndPoint ipepResponse = epResponse as IPEndPoint;
            if (hop > maxHops || ipepResponse.Address.Equals(ipTarget))
            {
                break;
            }
        }

        pingSocket.Close();
    }

As discussed earlier, we have now put a loop around the actual Socket.SendTo() call. This loop will run three times. We will continue sending the packet until we reach 30 hops max, or we have reached the destination. As you have noticed, since we did all the heavy lifting in the Ping program, all we had to do here, was to set the TimeToLive Socket option so that the routers/gateways on the path to the destination respond with a "Time Exceeded" message. When I run it on my machine, here is the output:

C:\>traceroute.exe www.yahoo.com
Ping www.yahoo.com(209.131.36.158) with 16 bytes of data...

1       <0ms    <1ms    <1ms    192.168.1.1
2       <5ms    <10ms   <14ms   <Address removed>
3       <18ms   <23ms   <27ms   <Address removed>
4       <33ms   <38ms   <43ms   <Address removed>
5       <68ms   <92ms   <116ms  <Address removed>
6       <143ms  <171ms  <197ms  <Address removed>
7       <223ms  <250ms  <276ms  <Address removed>
8       <304ms  <331ms  <365ms  <Address removed>
9       <391ms  <417ms  <445ms  <Address removed>
10      <473ms  <510ms  <538ms  <Address removed>

Note, that I have removed the actual IPAddresses from the output, for privacy reasons.

In the next part, we will discuss how we can make this utility more robust.

Implementing traceroute with System.Net: Part-I

2009-09-04T20:17:00.000-07:00

In the last part, I started by giving links to my implementation of the Ping utility, that used System.Net.Sockets.

http://ferozedaud.blogspot.com/2009/08/implementing-traceroute-with-systemnet.html

In this part, we will talk about traceroute. Traceroute is a general purpose utility that is used to discover a network path from the source to the destination. Traceroute is similar to Ping, in that it uses the ICMP protocol to send ICMP Echo request packets to the server. However, one key difference, is that it additionally uses the IP Time To Live (TTL) mechanism to specify the lifetime of the outgoing packets.

When the TTL expires on a packet, the receiving host must send an ICMP "Time Exceeded" message to the sender. The sender looks at this packet, and gets the IPAddress/HostName of the host that responded.

So, the basic algorithm for this goes as follows:

while (reply.address != dest.address)
int ttl = 1;
for i = 1 to 3
send a packet to the host at address dest.address with TTL=ttl
wait for reply
if (timeout) then
print "*"
else
print ipaddress of host thatresponded
// increase the TTL
ttl = ttl + 1;
end

This should be pretty easy to implement, given that we have already implemented a PING client in previous episodes, that does all the heavy lifting for us, in terms of marshalling the ICMP packet from managed code to network byte order, etc.

http://blogs.msdn.com/feroze_daud/archive/2005/10/20/483088.aspx
http://blogs.msdn.com/feroze_daud/archive/2005/10/23/483976.aspx
http://blogs.msdn.com/feroze_daud/archive/2005/10/24/484260.aspx
http://blogs.msdn.com/feroze_daud/archive/2005/10/26/485372.aspx

In the next part, we will modify the Ping utility to convert it into a Traceroute implementation.

Tracing with System.Net

2009-08-27T09:56:00.000-07:00

In my old blog, I had written a config file that shows how to enable trace logging for System.Net. Since it is useful, here is the config file again..

<?xml version="1.0" encoding="UTF-8" ?>

    <configuration>
    <system.diagnostics>
    <trace autoflush="true" />

    <sources>
    <source name="System.Net">

    <listeners>
        <add name="System.Net"/>
    </listeners>
    </source>
    <source name="System.Net.Sockets">
    <listeners>
        <add name="System.Net"/>
    </listeners>
    </source>
    <source name="System.Net.Cache">
    <listeners>
        <add name="System.Net"/>
    </listeners>
    </source>
    </sources>
    <sharedListeners>
        <add
            name="System.Net"
            type="System.Diagnostics.TextWriterTraceListener"
            initializeData="System.Net.trace.log"
        />
    </sharedListeners>
    <switches>
        <add name="System.Net" value="Verbose" />
        <add name="System.Net.Sockets" value="Verbose" />               
        <add name="System.Net.Cache" value="Verbose" />
    </switches>
</system.diagnostics>
</configuration>

NOTE: If you are doing logging inside of the ASP.NET process, make sure to give the ASP.NET process identity WRITE permissions to the directory where you want the log to be written.

Things I did at Microsoft: Outbound Dialing Rules for Exchange Unified Messaging

2009-08-23T14:16:00.001-07:00

Continuing my series on things that I did at Microsoft: in this part, I talk about outbound dialing rules that I implemented for Exchange Server 2007.

Exchange Unified Messaging server provides the functionality to place outbound calls. For eg, imagine a customer calling the corporate switchboard, and looking up the person he wants to talk to, using the AutoAttendant. Imagine that the target person is not in the US, and in order to call him, a long distance call will need to be placed.

It is possible that the company might want to prevent that call and instead route the caller to an operator. This is implemented by the concept of dialing rules.

The UM administrator can configure dialing rules on the AutoAttendant, DialPlan or Mailbox Policy. When the UM server needs to place an outbound call, it consults the dialing rules, and figures out whether the call is allowed. It also figures out the effective number to be called - this number will have the correct outside line access code (for eg: prepending the target number with a '9' to signify to the PBX that the numbe is an external number).

More details about this feature can be found in this article: http://msexchangeteam.com/archive/2007/01/29/432440.aspx which discusses how to create dialing rules.

Implementing Traceroute with System.Net - Introduction

2009-08-23T14:06:00.000-07:00

Previously, I have shown how you can implement a Ping utility using System.Net.Sockets.

The following are the links to the four part article that describe the implementation.

http://blogs.msdn.com/feroze_daud/archive/2005/10/20/483088.aspx
http://blogs.msdn.com/feroze_daud/archive/2005/10/23/483976.aspx
http://blogs.msdn.com/feroze_daud/archive/2005/10/24/484260.aspx
http://blogs.msdn.com/feroze_daud/archive/2005/10/26/485372.aspx

Ping is used to figure out if a destination host is responding on the network.

With a few modifications, we can change ping so that it shows us the routes that a packet is taking through the network.

In the next part, we will look at traceroute, and see how we can use the principles in writing our own traceroute utility from scratch.

Dell refutes Microsoft's Netbook return claims

2009-08-17T09:52:00.000-07:00

At the open source world conference, Dell's Senior Product Manager, Todd Finch refuted microsoft's claim that return rates for Linux netbooks have been much higher than windows netbooks.

http://www.crn.com/hardware/219300174;jsessionid=ULPLFEHHWWC

Originally, I thought that the return rate made sense, because people buying cheap netbooks would find the task of learning Linux/Ubuntu UI a huge task. But it appears that the people buying these netbooks are computer savvy, if Dells experience is to be believed.

However, to get a fuller picture, we also need the story across all hardware vendors that are selling netbooks.

Things I did at Microsoft - Call Answering Rules

2009-08-13T19:14:00.001-07:00

Larry Osterman started a series of posts on his blog, where he talks about the stuff he was doing in the past. So, I thought it would be a good idea to talk about suff I did at Microsoft.

One of the things that I am really proud about, is a feature I implemented for Exchange2009, which is called "Call Answering Rules". The best way to explain this, is to think about it as the equivalent of the Exchange/Outlook rules, but for your phone. This allows you to create rules that govern how incoming calls to your phone are routed.

Some of the examples of rules you can create are:

1) Forward all calls with a given caller-id to my cell phone.

2) If my client calls and I am in a meeting, forward to voicemail.

3) If a call comes, and my Free-Busy status says "Out of office", then call two other numbers configured by me.

I implemented the rules engine for this feature. You can get more details about the feature from this blog http://msexchangeteam.com/archive/2009/06/22/451667.aspx

Hi!

2009-08-13T18:36:00.000-07:00

Welcome to my blog. Here, I will post my thoughts on Software, as well as code snippets and other stuff.

For my previous blog, you can see http://blogs.msdn.com/feroze_daud