ServerFramework.com

Windows 8 Registered I/O Performance - 10 Gigabit networking... - AsynchronousEvents

2012-03-26T15:44:35Z

When I switched to looking at the performance of the more advanced RIO server designs that use IOCP it quickly became apparent that even multiple 1 Gigabit connections weren't enough of a challenge to give me any meaningful figures; my traditional IOCP datagram servers were easily able to keep up and increasing the workload per datagram required such high workloads that the tests became meaningless. So, we've brought forward the purchase of the hardware that we intended to use for our private cloud scalability testing and we now have 2 Intel 10 Gigabit AT2 cards. Switch prices are still prohibitive for lab use and so these two cards are directly connected, point to point.

The good news is that we now have a 10 Gigabit network link between two of our test servers. The bad news is that I now have to work out how to use it, the traditional datagram generation program that I was previously using to test simply doesn't scale to saturate the new link.

Windows 8 Registered I/O Performance - AsynchronousEvents

2012-03-15T15:15:00Z

I've been looking at the Windows 8 Registered I/O Networking Extensions since October when they first made an appearance as part of the Windows 8 Developer Preview. Whilst exploring and understanding the new API I spent some time putting together some simple UDP servers using the various notification styles that RIO provides. I then put together some equally simple UDP servers using the "traditional" APIs so that I could compare performance.

Of course these comparisons should be taken as preliminary since we're working with a beta version of the operating system. However, though I wouldn't put much weight in the exact numbers until we have a non-beta OS to test on, it's useful to see how things are coming along and familiarise ourselves with the designs that might be required to take advantage of RIO once it ships. The main thing to take away from these discussions on RIO's performance are the example server designs, the testing methods and a general understanding of why RIO performs better than the traditional Windows networking APIs. With this you can run your own tests, build your own servers and get value from using RIO where appropriate.

Do bear in mind that I'm learning as I go here, RIO is a new API and there is precious little in the way of documentation about how and why to use the API. Comments and suggestions are more than welcome, feel free to put me straight if I've made some mistakes, and submit code to help make these examples better.

How to test RIO's performance

The tests consist of sending a large number of datagrams to the server under test. We send two sizes of datagram, the test datagram and the shutdown datagram. The server counts the datagrams that it receives and the time taken. It shuts down as soon as it receives a shutdown datagram. The servers that we are using for these tests are detailed here and the datagram generator is available here.

Whilst the numbers that the servers report are useful for getting a rough idea of how the various API's compare they're not the whole story. It's useful to look at performance counter logs that are taken whilst the test server is running. The CPU usage of the server under test, and the entire machine, are useful indicators of how much further we could push a given server. The number of datagrams received, and dropped by the network and Winsock are useful to see, as is the non-paged pool usage, etc.

To make the testing repeatable I've put together some simple scripts which create the required performance logs using logman, the command line interface to perfmon. This means that for each test run we can run a single command which creates and starts a performance counter log, runs the server and then stops the performance counter log. It would be nice to include custom performance counters in each of the example servers so that we can see more of what's going on inside, but whilst easy to do, using our Performance Counters Option pack, that's beyond the scope of these tests.

The test client, or clients for when we're using two network links into the test machine, are started manually. We could automate this with winrs, as we've done in the past, but these tests don't really warrant that level of complexity.

Our test system

Our test system consists of a dual Xenon E5620 @ 2.40GHz, that's 16 CPUs in 2 Numa nodes with 16GB of memory. The machine has four 1Gb Ethernet network intefaces, a Broadcom BCM571C NetXtreme II GigE with two channels and a Intel 82576 Gigabit dual port adapter. We're using the Intel adapter for all of the tests shown here, sometimes using one NIC and sometimes two.

Windows Server 8 beta Datacentre edition is running directly on the hardware.

The client hardware is less impressive, but both client machines can push their 1Gb network interfaces to around 98% whilst running our datagram generator and that's more than enough for our purposes here.

The first tests

To get a feel for how the RIO API differs from the traditional API's the first test will compare a polled RIO server with a traditional, blocking, polled server. The code for the servers is available here along with some commentary on their designs. You'll need Visual Studio 11 to build the examples.

The test scripts, mentioned above, can be downloaded from here. Each server has its own script and a text file that details the performance counters to capture during the test run. All of the scripts call a common script which sets up the performance counter log and then starts the server. You shouldn't start the clients until the server is running and has output its configuration details. Once the server receives its first datagram it will display "TimingStarted" and when it has received a shutdown datagram it will display "TimingStopped" and display the number of datagrams that it managed to receive, the time taken and the datagrams per second. You need to copy the x64 release builds of the example servers into the same directory of the test scripts and then be sure to run the batch file and not the exe directly.

As an initial test we will run the traditional UDP server with one test client. We'll set the test client to send 10,000,000 datagrams, which takes a little over one minute. Once the test was completed the server reported that it had processed 9,952,510 datagrams in 86,880ms, a rate of 114,000 per second. Running the RIO polled server example with the same network load the results were broadly similar; 9,932,228 datagrams in 86,681ms, a rate of 114,000 per second.

At first glance it seems that RIO isn't so impressive, however we need to remind ourselves of what these example servers are doing; all they're doing is pulling datagrams off of the wire as fast as they can. They're both doing so on a single CPU of a 16 CPU machine and, from these results, it seems that on, this hardware, both APIs can quite easily handle a single saturated 1Gb network link.

Digging deeper into RIO's performance

Whilst the two servers at first appear to behave almost identically under the load it's only when we start looking at the performance counters that we can see that actually the two APIs have completely different performance characteristics.

Here's the graph for the traditional UDP server. Note the thick blue line, that's the amount of time the process spends in kernel mode, on average 37.133% of its time. The graph from the RIO server is a little different. The thick blue line is still there, it's just that it's 0 most of the time. The average is 0.167%. Another thing worth noting is that the spinning that the RIO server does is obvious from the fact that it uses 100% of a CPU (see the thick red line) and that most of that is spent in user mode code (the dotted green line that runs across the thick red line).

Another thing that is interesting to see is the non-paged pool usage; the RIO server uses a fixed amount for the life of the process, 8,064 bytes, the traditional server uses 4,192 bytes for most of the time but has some random peaks, the highest of which is 133,656 bytes.

Increasing the network load

Running two clients, each sending 10,000,000 datagrams to different network cards on the test machine gives us similar figures, the traditional server remains ahead of the RIO server with 19,847,578 datagrams to 19,279,842 datagrams. It seems that with the given hardware both APIs are capable of dealing with two saturated 1Gb links.

Increasing the work done per datagram received

The example servers all have a DoWork() function which allows us to add some "processing" for each datagram that is received, this gives us a slightly more realistic test as, except for discard servers, most servers need to do some work with each datagram that arrives.

Running the tests again, this time with a 'workload' of 100 gives the following results.

Traditional Server

6,386,294 datagrams out of 10,000,000 on 1 1Gb link, 63%
4,824,707 datagrams out of 20,000,000 on 2 1Gb links, 24%
38,830,887 datagrams out of 100,000,000 on 2 1Gb links, 38%

RIO Server

9,985,323 datagrams out of 10,000,000 on 1 1Gb link, 99%
19,730,003 datagrams out of 20,000,000 on 2 1Gb links, 98%
93,640,607 datagrams out of 100,000,000 on 2 1Gb links, 93%

Clearly a more realistic example allows the RIO API to show what it's capable of. Note that I ran a longer test, with two clients each sending 50,000,000 datagrams as the second test showed some results that seemed to imply that the traditional server had become overwhelmed near the end of the test. The longer test was to see if it could recover, it didn't it simply entered the overwhelmed state and stayed there until the end of the test. This is possibly due to the socket's recv buffer filling up.

As can be seen from these graphs, the traditional server quickly gets into a state where it is dropping vast numbers of datagrams (thick pink line) whilst burning more user mode CPU than kernel mode CPU having maxed out the single CPU that it's running on. The RIO server doesn't drop any datagrams and the graph looks surprisingly like the previous one with no load.

To look at how the performance of the RIO server degraded as the workload per datagram increases I ran some more tests.

RIO Server, 10,000,000 datagrams

9,985,323 datagrams, 99% at a workload of 100
9,888,733 datagrams, 98% at a workload of 300
7,174,653 datagrams, 71% at a workload of 500
5,573,046 datagrams, 55% at a workload of 700
4,361,820 datagrams, 43% at a workload of 1000
2,927,590 datagrams, 29% at a workload of 2000

And just to compare...
Traditional server, 10,000,000 datagrams

2,522,667 datagrams, 25% at a workload of 1000

Some conclusions

Bear in mind that these results are specific to the test machine I was running on and that we're testing on a beta version of the Windows 8 Server operating system. Even so, the figures are impressive. The lack of kernel mode transitions allow much more CPU to be used for real work on each datagram that arrives. The registering of I/O buffers once at program start up reduces the work done per operation and also means that your server will use a known amount of non-paged pool rather than a completely variable amount. Though non-paged pool is more plentiful than it used to be pre-Vista this is likely still an advantage.

The RIO API isn't especially complicated but your server designs will be different. The simple polling example server that we used here is unlikely to be an ideal choice as it uses 100% of its CPU for the whole time that the server is running. It's also a little unfair to compare RIO to such a simple traditional server but; there are better alternatives, but it's a useful line in the sand. As we'll see in the following performance articles there are better, and more scalable ways to use both APIs.

If you're interested in digging deeper into the results used in this article then all of the performance logs taken whilst running the tests are available here.

Windows 8 Registered I/O - Generating load for the performance tests - AsynchronousEvents

2012-03-14T17:15:00Z

Now that we have five example servers, four RIO designs and a traditional polled UDP design, we can begin to look at how the RIO API performs compared to the traditional APIs. Of course these comparisons should be taken as preliminary since we're working with a beta version of the operating system. However, though I wouldn't put much weight in the exact numbers until we have a non-beta OS to test on, it's useful to see how things are coming along and familiarise ourselves with the designs that might be required to take advantage of RIO once it ships.

Sending a stream of datagrams

Before we can compare performance we need to be able to push the example servers hard. We do this by sending a stream of datagrams at them as fast as we can for a period of time. The servers start timing when they get the first datagram and then count the number of datagrams that they process. The test finishes by sending a series of smaller datagrams at the server. When the server sees one of these smaller datagrams it shuts down and reports on the time taken and the number of datagrams processed and the rate at which they were processed.

All we need to be able to do to stress the servers is to send datagrams at a rate that gets close to 100% utilisation of a 1Gb Ethernet link. This is fairly simple to achieve using the traditional blocking sockets API.

   for (size_t i = 0; i < DATAGRAMS_TO_SEND; ++i)
   {
      if (SOCKET_ERROR == ::WSASendTo(
         s,
         &buf,
         1,
         &bytesSent,
         flags,
         reinterpret_cast<sockaddr *>(&addr),
         sizeof(addr),
         0,
         0))
      {
         ErrorExit("WSASend");
      }
   }

There's not much more to it than that. We use similar code to setup and clean up, but if you've been following along with the other examples then there's nothing that needs to be explained about that.

The code for this example can be downloaded from here. This code requires Visual Studio 11, but would work with earlier compilers if you have a Windows SDK that supports RIO. Note that Shared.h and Constants.h contain helper functions and tuning constants for ALL of the examples and so there will be code in there that is not used by this example. You should be able to unzip each example into the same directory structure so that they all share the same shared headers. This allows you to tune all of the examples the same so that any performance comparisons make sense. This program can be run on versions of Windows prior to Windows 8, which is useful for testing as you only need one machine set up with the beta of Windows 8 server.

Windows 8 Registered I/O - Traditional Polled UDP Example Server - AsynchronousEvents

2012-03-14T14:00:00Z

This article presents the fifth in my series of example servers for comparing the performance of the Windows 8 Registered I/O Networking extensions, RIO, and traditional Windows networking APIs. This example server is a traditional polled UDP design that we can use to compare to the RIO polled UDP example server. I've been looking at the Windows 8 Registered I/O Networking Extensions since October when they first made an appearance as part of the Windows 8 Developer Preview. Whilst exploring and understanding the new API I spent some time putting together some simple UDP servers using the various notification styles that RIO provides. I then put together some equally simple UDP servers using the "traditional" APIs so that I could compare performance. This series of blog posts describes each of the example servers in turn. You can find an index to all of the articles about the Windows 8 Registered I/O example servers here.

A traditional polled UDP server

This server is probably the simplest UDP server you could have. It's pretty much just a tight loop around a blocking call to WSARecv(). There's none of the complexity required by RIO for registering memory buffers for I/O and so we use a single buffer that we create on the stack.

   do
   {
      workValue += DoWork(g_workIterations);

      if (SOCKET_ERROR == ::WSARecv(
         s,
         &buf,
         1,
         &bytesRecvd,
         &flags,
         0,
         0))
      {
         ErrorExit("WSARecv");
      }

      if (bytesRecvd == EXPECTED_DATA_SIZE)
      {
         g_packets++;
      }
      else
      {
         done = true;
      }
   }
   while (!done);

There is some added complexity to allow us to compare performance, and this is similar to the RIO server examples. We can add an arbitrary processing overhead to each datagram by setting g_workIterations to a non zero value and we count each datagram that arrives and stop the test when a datagram of an unexpected size is received.

Setting up for the datagram processing loop

As with the RIO examples we do some setup before we can process datagrams. See the polled RIO example server for details of how and why we set up the timing system and initialise Winsock, and for details on our error handling policy.

int _tmain(int argc, _TCHAR* argv[])
{
   SetupTiming("Simple polled UDP");

   InitialiseWinsock();

   SOCKET s = CreateSocket();

   Bind(s, PORT);

   SetSocketRecvBufferToMaximum(s);

   bool done = false;

   CHAR buffer[RECV_BUFFER_SIZE];

   WSABUF buf;

   buf.buf = buffer;
   buf.len = RECV_BUFFER_SIZE;

   DWORD bytesRecvd = 0;

   DWORD flags = 0;

   if (SOCKET_ERROR == ::WSARecv(
      s,
      &buf,
      1,
      &bytesRecvd,
      &flags,
      0,
      0))
   {
      ErrorExit("WSARecv");
   }

   g_packets++;

   StartTiming();

   int workValue = 0;

We then create a traditional blocking UDP socket, bind it to a port, set its receive buffer size to the maximum and create our receive buffer on the stack, set up our WSABUF and call WSARecv() for the first time. We make this call outside of our processing loop so that we can start timing when we get the first datagram. This code then proceeds into the processing loop, shown above, and processes datagrams until the test is complete and a datagram of an unexpected size is received.

The code for CreateSocket(), Bind() and SetSocketRecvBufferToMaximum() can be found in Shared.h and remember that the use of globals isn't clever, it's simply convenient for some of the other example servers that use the shared code.

After the processing loop

Once the performance test completes we stop our timing and report the results.

   StopTiming();

   PrintTimings();

   return workValue;
}

Join in

Comments and suggestions are more than welcome. I'm learning as I go here and I'm quite likely to have made some mistakes or come up with some erroneous conclusions, feel free to put me straight and help make these examples better.

Windows 8 Registered I/O - Multi threaded RIO IOCP UDP Example Server - AsynchronousEvents

2012-03-12T18:10:00Z

This article presents the fourth in my series of example servers using the Windows 8 Registered I/O Networking extensions, RIO. This example server, like the last example, uses the I/O Completion Port notification method to handle RIO completions, but where the last example used only a single thread to service the IOCP this one uses multiple thread to scale the load . I've been looking at the Windows 8 Registered I/O Networking Extensions since October when they first made an appearance as part of the Windows 8 Developer Preview. Whilst exploring and understanding the new API I spent some time putting together some simple UDP servers using the various notification styles that RIO provides. I then put together some equally simple UDP servers using the "traditional" APIs so that I could compare performance. This series of blog posts describes each of the example servers in turn. You can find an index to all of the articles about the Windows 8 Registered I/O example servers here.

Using an I/O Completion Port for RIO completions

As I mentioned back in October, there are three ways to receive completion notifications from RIO; polling, event driven and via an I/O Completion Port. Using an IOCP for RIO completions allows you to easily scale your completion handling across multiple threads as we do here and this is the first of my example servers that allows for more than one thread to be used to process completions.

Creating an IOCP driven RIO completion queue

We start by initialising things in the same way that we did with the earlier example RIO servers. In fact, this initialisation is identical to the previous IOCP example except for one thing.

int _tmain(int argc, _TCHAR* argv[])
{
   SetupTiming("RIO IOCP UDP");

   InitialiseWinsock();

   CreateRIOSocket();

   g_hIOCP = ::CreateIoCompletionPort(
      INVALID_HANDLE_VALUE,
      0,
      0,
      0);

   OVERLAPPED overlapped;

   RIO_NOTIFICATION_COMPLETION completionType;

   completionType.Type = RIO_IOCP_COMPLETION;
   completionType.Iocp.IocpHandle = g_hIOCP;
   completionType.Iocp.CompletionKey = (void*)1;
   completionType.Iocp.Overlapped = &overlapped;

   g_queue = g_rio.RIOCreateCompletionQueue(
      RIO_PENDING_RECVS,
      &completionType);

   if (g_queue == RIO_INVALID_CQ)
   {
      ErrorExit("RIOCreateCompletionQueue");
   }

With the previous design we passed 0 as the completion key. This server passes 1. This is an arbitrary change purely to allow us to post completions of 0 to cause all of the threads waiting on the completion queue to shut down. This is a common idiom with normal, non-RIO, IOCP designs as it's more usual that the completion key is a pointer to a "per operation" data structure. A RIO design with multiple completion queues would likely use the completion key for "per queue" data.

Creating a RIO request queue

Creating the request queue and posting our receives is identical to the polled example. The only difference is how we handle the completions.

Starting our worker threads

This example is the first to use more than just a single thread. Because of this we need to have a way to start and manage our worker threads and for them to communicate with each other and the main thread.

   CreateIOCPThreads(NUM_IOCP_THREADS);

   INT notifyResult = g_rio.RIONotify(g_queue);

   if (notifyResult != ERROR_SUCCESS)
   {
      ErrorExit("RIONotify", notifyResult);
   }

   WaitForProcessingStarted();

First we call CreateIOCPThreads(), which is shown below, this creates some events that the threads will use to communicate and then creates and starts the threads themselves. As with the earlier examples, we use globals for convenience and not as an example of good design.

inline void CreateIOCPThreads(
   const DWORD numThreads)
{
   g_hStartedEvent = ::CreateEvent(0, TRUE, FALSE, 0);

   if (0 == g_hStartedEvent)
   {
      ErrorExit("CreateEvent");
   }

   g_hStoppedEvent = ::CreateEvent(0, TRUE, FALSE, 0);

   if (0 == g_hStoppedEvent)
   {
      ErrorExit("CreateEvent");
   }

   // Start our worker threads

   for (DWORD i = 0; i < numThreads; ++i)
   {
      unsigned int notUsed;

      const uintptr_t result = ::_beginthreadex(
         0,
         0,
         ThreadFunction,
         0,
         0,
         &notUsed);

      if (result == 0)
      {
         ErrorExit("_beginthreadex", errno);
      }

      g_threads.push_back(reinterpret_cast<handle>(result));
   }

   cout << numThreads << " threads running" << endl;
}

The main thread then calls RIONotify() to enable notifications and then waits for the first datagram to be processed before it starts the timer.

inline void WaitForProcessingStarted()
{
   if (WAIT_OBJECT_0 != ::WaitForSingleObject(
      g_hStartedEvent,
      INFINITE))
   {
      ErrorExit("WaitForSingleObject");
   }

   StartTiming();
}

Calling RIODequeueCompletion() and processing results

This example's processing loop is similar to the previous examples, especially the single threaded IOCP example server. It's slightly more complicated due to the fact that it's being run on a separate thread.

unsigned int __stdcall ThreadFunction(
   void *pV)
{
   DWORD numberOfBytes = 0;

   ULONG_PTR completionKey = 0;

   OVERLAPPED *pOverlapped = 0;

   const DWORD recvFlags = 0;

   if (!::GetQueuedCompletionStatus(
      g_hIOCP,
      &numberOfBytes,
      &completionKey,
      &pOverlapped,
      INFINITE))
   {
      ErrorExit("GetQueuedCompletionStatus");
   }

   int workValue = 0;

   if (completionKey == 1)
   {
      RIORESULT results[RIO_MAX_RESULTS];

      bool done = false;

      ::SetEvent(g_hStartedEvent);

      ULONG numResults = g_rio.RIODequeueCompletion(
         g_queue,
         results,
         RIO_MAX_RESULTS);

      if (0 == numResults ||
          RIO_CORRUPT_CQ == numResults)
      {
         ErrorExit("RIODequeueCompletion");
      }

      INT notifyResult = g_rio.RIONotify(g_queue);

      if (notifyResult != ERROR_SUCCESS)
      {
         ErrorExit("RIONotify", notifyResult);
      }

      do
      {
         for (DWORD i = 0; i < numResults; ++i)
         {
            EXTENDED_RIO_BUF *pBuffer = reinterpret_cast<EXTENDED_RIO_BUF *>(results[i].RequestContext);

            if (results[i].BytesTransferred == EXPECTED_DATA_SIZE)
            {
               ::InterlockedIncrement(&g_packets);

               workValue += DoWork(g_workIterations);

               if (!g_rio.RIOReceive(
                  g_requestQueue,
                  pBuffer,
                  1,
                  recvFlags,
                  pBuffer))
               {
                  ErrorExit("RIOReceive");
               }

               done = false;
            }
            else
            {
               done = true;
            }
         }

         if (!done)
         {
            if (!::GetQueuedCompletionStatus(
               g_hIOCP,
               &numberOfBytes,
               &completionKey,
               &pOverlapped,
               INFINITE))
            {
               ErrorExit("GetQueuedCompletionStatus");
            }

            if (completionKey == 0)
            {
               done = true;
            }
            else
            {
               numResults = g_rio.RIODequeueCompletion(
                  g_queue,
                  results,
                  RIO_MAX_RESULTS);

               if (0 == numResults ||
                   RIO_CORRUPT_CQ == numResults)
               {
                  ErrorExit("RIODequeueCompletion");
               }

               INT notifyResult = g_rio.RIONotify(g_queue);

               if (notifyResult != ERROR_SUCCESS)
               {
                  ErrorExit("RIONotify", notifyResult);
               }
            }
         }
      }
      while (!done);
   }

   ::SetEvent(g_hStoppedEvent);

   return workValue;
}

The first thing we do is wait for a completion. Once we have a completion we dequeue the results and then call RIONotify() to allow more completions to occur. It's important to realise that until we call RIONotify() no further completions will be posted to the I/O Completion Port and that this effectively acts as synchronisation around the calls to RIODequeueCompletion(). With this design only one thread can ever be calling RIODequeueCompletion() at a time, which is a good thing as the documentation for RIODequeueCompletion() states that this is a requirement for users of the API.

Remember that this example is made more complex due to the way we profile the servers. See the explanation in the completion handling section of the polled RIO server example for details of why this is.

Shutting down and displaying results

Whilst our worker threads are processing datagrams our main thread is waiting for the performance test to end.

   WaitForProcessingStopped();

   StopIOCPThreads();

   PrintTimings();

   return 0;
}

Our thread function's main loop can exit in two ways. Firstly because a datagram arrives that isn't of the expected size, thus signalling the end of the performance test, and secondly if GetQueuedCompletionStatus() returns a completion key of 0 which means that the main thread has posted completions to request that we shut down. This means that the first time a "shutdown" datagram arrives the first thread that begins processing it will shut down and set the g_hStoppedEvent event. The main thread is currently waiting for this event, and will wake when the event is set and shut the rest of the worker threads down. Once all of the threads have terminated the main thread will display details of the datagrams received and the test timings.

inline void WaitForProcessingStopped()
{
   if (WAIT_OBJECT_0 != ::WaitForSingleObject(
      g_hStoppedEvent,
      INFINITE))
   {
      ErrorExit("WaitForSingleObject");
   }

   StopTiming();
}

inline void StopIOCPThreads()
{
   // Tell all threads to exit

   for (Threads::const_iterator it = g_threads.begin(),
      end = g_threads.end();
      it != end;
      ++it)
   {
      if (0 == ::PostQueuedCompletionStatus(
         g_hIOCP,
         0,
         0,
         0))
      {
         ErrorExit("PostQueuedCompletionStatus");
      }
   }

   cout << "Threads stopping" << endl;

   // Wait for all threads to exit

   for (Threads::const_iterator it = g_threads.begin(),
      end = g_threads.end();
      it != end;
      ++it)
   {
      HANDLE hThread = *it;

      if (WAIT_OBJECT_0 != ::WaitForSingleObject(
         hThread,
         INFINITE))
      {
         ErrorExit("WaitForSingleObject");
      }

      ::CloseHandle(hThread);
   }   

   cout << "Threads stopped" << endl;
}

Unexpected performance issues...

The slight problem with this design is that it's not actually as performant in some scenarios as we might like it to be. As it stands, the fact that we can scale out across multiple threads is a plus point but the fact that the operations that we have to perform to achieve that scaling are considerably more expensive is a problem. This is more of an issue when we're looking for a general purpose solution which works as well for low throughput and fast processing of each datagram as it does for high throughput and/or slow processing. Luckily there are a couple of things we can do to fix this, but we'll look at those once we've done some performance comparisons and seen the problems first hand.

Join in

Windows 8 Registered I/O - Single threaded RIO IOCP UDP Example Server - AsynchronousEvents

2012-03-12T15:45:00Z

This article presents the third in my series of example servers using the Windows 8 Registered I/O Networking extensions, RIO. This example server uses the I/O Completion Port notification method to handle RIO completions, but only uses a single thread to service the IOCP. I've been looking at the Windows 8 Registered I/O Networking Extensions since October when they first made an appearance as part of the Windows 8 Developer Preview. Whilst exploring and understanding the new API I spent some time putting together some simple UDP servers using the various notification styles that RIO provides. I then put together some equally simple UDP servers using the "traditional" APIs so that I could compare performance. This series of blog posts describes each of the example servers in turn. You can find an index to all of the articles about the Windows 8 Registered I/O example servers here.

Using an I/O Completion Port for RIO completions

As I mentioned back in October, there are three ways to receive completion notifications from RIO; polling, event driven and via an I/O Completion Port. Using an IOCP for RIO completions allows you to easily scale your completion handling across multiple threads, though in this first IOCP example server we use a single thread so as to allow us to compare the performance against the polled and event driven servers. The next example server will adapt this server for multiple threads and allow us to scale our completion processing across more CPUs.

Creating an IOCP driven RIO completion queue

We start by initialising things in the same way that we did with the earlier example RIO servers.

int _tmain(int argc, _TCHAR* argv[])
{
   SetupTiming("RIO IOCP UDP");

   InitialiseWinsock();

   CreateRIOSocket();

   g_hIOCP = ::CreateIoCompletionPort(
      INVALID_HANDLE_VALUE,
      0,
      0,
      0);

   OVERLAPPED overlapped;

   RIO_NOTIFICATION_COMPLETION completionType;

   completionType.Type = RIO_IOCP_COMPLETION;
   completionType.Iocp.IocpHandle = g_hIOCP;
   completionType.Iocp.CompletionKey = (void*)0;
   completionType.Iocp.Overlapped = &overlapped;

   g_queue = g_rio.RIOCreateCompletionQueue(
      RIO_PENDING_RECVS,
      &completionType);

   if (g_queue == RIO_INVALID_CQ)
   {
      ErrorExit("RIOCreateCompletionQueue");
   }

Once that is done we create an IO Completion Port and then create a RIO completion queue which uses the IOCP for notification. In this simple design we have no need for a completion key as we only have a single completion queue so there's no need to differentiate between completion types. We also use a plain old OVERLAPPED rather than extending it to carry more information. More complex designs could use either the completion key, or an extended overlapped structure to pass queue specific information to our completion handler in much the same way that we do with normal IOCP server designs.

Creating a RIO request queue

Creating the request queue and posting our receives is identical to the polled example. The only difference is how we handle the completions.

Calling RIODequeueCompletion() and processing results

Processing completions is almost identical to processing event driven completions. We simply change the call to WaitForSingleObject() that we were using in the event driven example to the following to retrieve a completion notification from the IOCP.

   DWORD numberOfBytes = 0;

   ULONG_PTR completionKey = 0;

   OVERLAPPED *pOverlapped = 0;

   if (!::GetQueuedCompletionStatus(
      g_hIOCP,
      &numberOfBytes,
      &completionKey,
      &pOverlapped,
      INFINITE))
   {
      ErrorExit("GetQueuedCompletionStatus");
   }

Everything else is identical. Things change somewhat when we switch to using multiple threads for our completion handling.

Join in

Windows 8 Registered I/O - Single threaded RIO Event Driven UDP Example Server - AsynchronousEvents

2012-03-09T22:25:00Z

This article presents the second in my series of example servers using the Windows 8 Registered I/O Networking extensions, RIO. This example server uses the event driven notification method to handle RIO completions. I've been looking at the Windows 8 Registered I/O Networking Extensions since October when they first made an appearance as part of the Windows 8 Developer Preview. Whilst exploring and understanding the new API I spent some time putting together some simple UDP servers using the various notification styles that RIO provides. I then put together some equally simple UDP servers using the "traditional" APIs so that I could compare performance. This series of blog posts describes each of the example servers in turn. You can find an index to all of the articles about the Windows 8 Registered I/O example servers here.

Using an event for RIO completions

As I mentioned back in October, there are three ways to receive completion notifications from RIO; polling, event driven and via an I/O Completion Port. Using the event driven approach is similar to using the polling approach that I described in the previous article except that the server doesn't burn CPU in a tight polling loop.

Creating an event driven RIO completion queue

We start by initialising things in the same way that we did with the earlier example RIO servers.

int _tmain(int argc, _TCHAR* argv[])
{
   SetupTiming("RIO Event Driven UDP");

   InitialiseWinsock();

   CreateRIOSocket();

   HANDLE hEvent = WSACreateEvent();

   if (hEvent == WSA_INVALID_EVENT)
   {
      ErrorExit("WSACreateEvent");
   }

   RIO_NOTIFICATION_COMPLETION completionType;

   completionType.Type = RIO_EVENT_COMPLETION;
   completionType.Event.EventHandle = hEvent;
   completionType.Event.NotifyReset = TRUE;

   g_queue = g_rio.RIOCreateCompletionQueue(
      RIO_PENDING_RECVS,
      &completionType);

   if (g_queue == RIO_INVALID_CQ)
   {
      ErrorExit("RIOCreateCompletionQueue");
   }

Once that is done we create an event and then create a RIO completion queue which uses the event for notification. The event is signalled when there are completions to process and reset when we call RIONotify().

Creating a RIO request queue

Creating the request queue and posting our receives is identical to the polled example. The only difference is how we handle the completions.

Calling RIODequeueCompletion() and processing results

The processing loop is, again, similar to the polled example. Unsurprisingly rather than polling we wait on the event and dequeue the completions once the event is set. This reduced the amount of CPU used as there's no need to spin whilst waiting for new datagrams to process. The only complication is that we need to call RIONotify() to indicate that we're ready to process more completions. Note that in a real server you would probably want to wait on your completions available event and a 'we're ready to shut down' event so that you can shut the sever down cleanly.

   bool done = false;

   DWORD recvFlags = 0;

   RIORESULT results[RIO_MAX_RESULTS];

   const INT notifyResult = g_rio.RIONotify(g_queue);

   if (notifyResult != ERROR_SUCCESS)
   {
      ErrorExit("RIONotify");
   }

   const DWORD waitResult = WaitForSingleObject(
      hEvent,
      INFINITE);

   if (waitResult != WAIT_OBJECT_0)
   {
      ErrorExit("WaitForSingleObject");
   }

   ULONG numResults = g_rio.RIODequeueCompletion(
      g_queue,
      results,
      RIO_MAX_RESULTS);

   if (0 == numResults ||
       RIO_CORRUPT_CQ == numResults)
   {
      ErrorExit("RIODequeueCompletion");
   }

   StartTiming();

   int workValue = 0;

   bool running = true;

   do
   {
      for (DWORD i = 0; i < numResults; ++i)
      {
         EXTENDED_RIO_BUF *pBuffer = reinterpret_cast<EXTENDED_RIO_BUF *>(results[i].RequestContext);

         if (results[i].BytesTransferred == EXPECTED_DATA_SIZE)
         {
            g_packets++;

            workValue += DoWork(g_workIterations);

            if (!g_rio.RIOReceive(
               g_requestQueue,
               pBuffer,
               1,
               recvFlags,
               pBuffer))
            {
               ErrorExit("RIOReceive");
            }

            done = false;
         }
         else
         {
            done = true;
         }
      }

      if (!done)
      {
         const INT notifyResult = g_rio.RIONotify(g_queue);

         if (notifyResult != ERROR_SUCCESS)
         {
            ErrorExit("RIONotify");
         }

         const DWORD waitResult = WaitForSingleObject(
            hEvent,
            INFINITE);

         if (waitResult != WAIT_OBJECT_0)
         {
            ErrorExit("WaitForSingleObject");
         }

         numResults = g_rio.RIODequeueCompletion(
            g_queue,
            results,
            RIO_MAX_RESULTS);

         if (0 == numResults ||
             RIO_CORRUPT_CQ == numResults)
         {
            ErrorExit("RIODequeueCompletion");
         }
      }
   }
   while (!done);

   StopTiming();

   PrintTimings();

   return workValue;
}

As before, the structure of the processing loop is complicated somewhat by the fact that we want to start and stop the timing for the performance testing, and the DoWork() function can be used to add 'processing overhead' to each datagram. This can be configured using the g_workIterations which is defined in Constants.h. With this set to 0 there is no overhead and we can compare how quickly each API can receive datagrams. Setting larger values will affect how the various multi-threaded examples perform and can be useful if you're unable to saturate the test machine's network interfaces.

This example can be optimised slightly so that we revert to straight polling as long as calling RIODequeueCompletion() returns us at least one result. We'll look at this variation after we've studied the performance of the example shown here.

Join in

Windows 8 Registered I/O - Simple RIO Polled UDP Example Server - AsynchronousEvents

2012-03-07T11:28:00Z

Polling RIO for completions

As I mentioned back in October, there are three ways to receive completion notifications from RIO; polling, event driven and via an I/O Completion Port. The first is the simplest though it burns CPU time even when no datagrams are being received.

At its simplest a polled RIO server obtains datagrams to process like this:

   RIORESULT results[RIO_MAX_RESULTS];

   ULONG numResults = 0;

   do
   {
      numResults = g_rio.RIODequeueCompletion(
         g_queue,
         results,
         RIO_MAX_RESULTS);

      if (0 == numResults)
      {
         YieldProcessor();
      }
      else if (RIO_CORRUPT_CQ == numResults)
      {
         ErrorExit("RIODequeueCompletion");
      }
   }
   while (0 == numResults);

You then loop over the results array and process each result in turn before looping back to dequeue more completions.

Getting to the point where you can call RIODequeueCompletion() takes a bit of setting up though...

Creating a RIO completion queue

The examples are each stand alone but can share two common header files. The first, Constants.h, contains all constants that are used to tune the examples. The second, Shared.h, contains inline helper functions which hide some of the complexity and allow the individual example programs to focus on the area of the API that they're demonstrating. We use several of these helper functions as we prepare to create our RIO completion queue.

int _tmain(int argc, _TCHAR* argv[])
{
   SetupTiming("RIO polled UDP");

   InitialiseWinsock();

   CreateRIOSocket();

   g_queue = g_rio.RIOCreateCompletionQueue(RIO_PENDING_RECVS, 0);

   if (g_queue == RIO_INVALID_CQ)
   {
      ErrorExit("RIOCreateCompletionQueue");
   }

So that we can compare the performance of these examples we first call SetupTiming(), this prepares us for calling StartTiming() and StopTiming() later in the program. SetupTiming() locks this thread to a single CPU, using SetThreadAffinityMask() as recommended by the help for QueryPerformanceCounter(), once this is done we call QueryPerformanceFrequency() and store the resulting value for use by StopTiming().

With that done we initialise Winsock and then create our RIO socket. Note that these examples use quite a few global variables for convenience. This isn't how I would suggest you write production code, but it's convenient for these examples as it makes much of the code simpler and allows us to focus on the RIO API. For example, CreateRIOSocket() does the following, creating a socket and assigning it to g_s, we then bind the socket to PORT, which is a constant that is defined in Constants.h, and then initialising the RIO API function table so that we can use it through g_rio.

inline void CreateRIOSocket()
{
   g_s = CreateSocket(WSA_FLAG_REGISTERED_IO);

   Bind(g_s, PORT);

   InitialiseRIO(g_s);
}

inline SOCKET CreateSocket(
   const DWORD flags = 0)
{
   g_s = ::WSASocket(
      AF_INET,
      SOCK_DGRAM,
      IPPROTO_UDP,
      NULL,
      0,
      flags);

   if (g_s == INVALID_SOCKET)
   {
      ErrorExit("WSASocket");
   }

   return g_s;
}

inline void InitialiseRIO(
   SOCKET s)
{
   GUID functionTableId = WSAID_MULTIPLE_RIO;

   DWORD dwBytes = 0;

   bool ok = true;

   if (0 != WSAIoctl(
      s,
      SIO_GET_MULTIPLE_EXTENSION_FUNCTION_POINTER,
      &functionTableId,
      sizeof(GUID),
      (void**)&g_rio,
      sizeof(g_rio),
      &dwBytes,
      0,
      0))
   {
      ErrorExit("WSAIoctl");
   }
}

As you can see, we check all API calls for failure and report errors via our ErrorExit() functions.

Finally we can create the RIO completion queue. We use the tunable constant RIO_PENDING_RECVS to specify how large the queue should be.

Creating a RIO request queue

Setting up a RIO request queue seems to be one of the few places where RIO API changes between the Windows 8 Developer Preview and the Windows 8 Server Beta are visible to us. With the Developer Preview we could pass any value for the maxReceiveDataBuffers and maxSendDataBuffers whereas in the beta RIOCreateRequestQueue() only accepts a value of 1. See my earlier posting on RIO for more details, but the documentation and the API are now in sync and the API currently doesn't support scatter/gather I/O.

   ULONG maxOutstandingReceive = RIO_PENDING_RECVS;
   ULONG maxReceiveDataBuffers = 1;
   ULONG maxOutstandingSend = 0;
   ULONG maxSendDataBuffers = 1;

   void *pContext = 0;

   g_requestQueue = g_rio.RIOCreateRequestQueue(
      g_s,
      maxOutstandingReceive,
      maxReceiveDataBuffers,
      maxOutstandingSend,
      maxSendDataBuffers,
      g_queue,
      g_queue,
      pContext);

   if (g_requestQueue == RIO_INVALID_RQ)
   {
      ErrorExit("RIOCreateRequestQueue");
   }

   PostRIORecvs(RECV_BUFFER_SIZE, RIO_PENDING_RECVS);

Once the request queue has been created we can post some read requests. Note that we specify the size of the buffers to use and the number of receives that we want to have pending, both of these values can be changed easily in Constants.h.

Registering buffers and posting RIO read requests

Before we can issue some read requests we need to register some I/O buffers. The PostRIORecvs() function is complicated by the fact that we're trying to keep things simple (!) and by the fact that the example is tunable for different buffer sizes and the number of receives that we may want pending. We loop allocating and registering buffers and then slicing the buffer into buffer slices. We use an extended RIO_BUF structure so that we can pass an "operation" code with each buffer slice. These examples don't use this operation, but a real server might need to pass additional information with each I/O request, especially if it's using a single completion queue for reads and writes. We deliberately leak our EXTENDED_RIO_BUF structures but this isn't much of a problem in this example as they're in use from this point until the program exits.

inline void PostRIORecvs(
   const DWORD recvBufferSize,
   const DWORD pendingRecvs)
{
   DWORD totalBuffersAllocated = 0;

   while (totalBuffersAllocated < pendingRecvs)
   {
      DWORD bufferSize = 0;
   
      DWORD receiveBuffersAllocated = 0;

      char *pBuffer = AllocateBufferSpace(
         recvBufferSize,
         pendingRecvs,
         bufferSize,
         receiveBuffersAllocated);

      totalBuffersAllocated += receiveBuffersAllocated;

      RIO_BUFFERID id = g_rio.RIORegisterBuffer(
         pBuffer,
         static_cast<DWORD>(bufferSize));

      if (id == RIO_INVALID_BUFFERID)
      {
         ErrorExit("RIORegisterBuffer");
      }

      DWORD offset = 0;

      const DWORD recvFlags = 0;

      EXTENDED_RIO_BUF *pBufs = new EXTENDED_RIO_BUF[receiveBuffersAllocated];

      for (DWORD i = 0; i < receiveBuffersAllocated; ++i)
      {
         // now split into buffer slices and post our recvs

         EXTENDED_RIO_BUF *pBuffer = pBufs + i;

         pBuffer->operation = 0;
         pBuffer->BufferId = id;
         pBuffer->Offset = offset;
         pBuffer->Length = recvBufferSize;

         offset += recvBufferSize;

         if (!g_rio.RIOReceive(g_requestQueue, pBuffer, 1, recvFlags, pBuffer))
         {
            ErrorExit("RIOReceive");
         }
      }

      if (totalBuffersAllocated != pendingRecvs)
      {
         cout << pendingRecvs << " receives pending" << endl;
      }
   }

   cout << totalBuffersAllocated << " total receives pending" << endl;
}

Before we can register our I/O buffer we need to allocate the memory that we will be using. As I mentioned back in October when I was first looking at the RIO API, it's important to allocate your I/O buffer memory in a particular way so that you use memory efficiently for RIO's registered I/O buffers.

inline char *AllocateBufferSpace(
   const DWORD recvBufferSize,
   const DWORD pendingRecvs,
   DWORD &bufferSize,
   DWORD &receiveBuffersAllocated)
{
   const DWORD preferredNumaNode = 0;

   const SIZE_T largePageMinimum = USE_LARGE_PAGES ? ::GetLargePageMinimum() : 0;

   SYSTEM_INFO systemInfo;

   ::GetSystemInfo(&systemInfo);

   systemInfo.dwAllocationGranularity;
   
   const unsigned __int64 granularity = (largePageMinimum == 0 ? systemInfo.dwAllocationGranularity : largePageMinimum);

   const unsigned __int64 desiredSize = recvBufferSize * pendingRecvs;

   unsigned __int64 actualSize = RoundUp(desiredSize, granularity);

   if (actualSize > std::numeric_limits<DWORD>::max())
   {
      actualSize = (std::numeric_limits<DWORD>::max() / granularity) * granularity;
   }

   receiveBuffersAllocated = std::min<DWORD>(pendingRecvs, static_cast<DWORD>(actualSize / recvBufferSize));

   bufferSize = static_cast<DWORD>(actualSize);

   char *pBuffer = reinterpret_cast<char *>(VirtualAllocExNuma(
      GetCurrentProcess(),
      0,
      bufferSize,
      MEM_COMMIT |
      MEM_RESERVE  |
      (largePageMinimum != 0 ? MEM_LARGE_PAGES : 0),
      PAGE_READWRITE,
      preferredNumaNode));

   if (pBuffer == 0)
   {
      ErrorExit("VirtualAlloc");
   }

   return pBuffer;
}

Our allocation function is again slightly more complex than it need be, but that complexity allows us to explore various options by simply changing our configuration constants; you can ignore things like the USE_LARGE_PAGES flag and the fact that we're allocating to a preferred NUMA node unless you're interested in the details and your hardware supports these features. The important thing is that we allocate in terms of the system's allocation granularity and that we use a variant of VirtualAlloc() to do. Once again, in the name of simplicity, we leak these buffers (which will be in use for the life of the program) and allow program exit to clean them up.

Calling RIODequeueCompletion() and processing results

We finally have queues created, buffers registered and reads pending. Processing these reads in our simple polled RIO server is fairly straight forward. First we enter a polling loop for completions and spin until completions are available. Once we have at least one completion we call StartTiming() to start our performance timing. We then process the completion results. Our performance tests are simple, we send a number of datagrams of EXPECTED_DATA_SIZE and then indicate that the test is complete by sending a series of datagrams of a different size. Once our servers receive an unexpected sized datagram they consider the test to be complete and shutdown. Thus our main completion loop is the do/while loop below. We process datagrams, issue new reads and then dequeue more results. Once we're done we stop our timer and display details about the time taken and the number of datagrams that we processed.

bool done = false;

   DWORD recvFlags = 0;

   RIORESULT results[RIO_MAX_RESULTS];

   ULONG numResults = 0;

   do
   {
      numResults = g_rio.RIODequeueCompletion(
         g_queue,
         results,
         RIO_MAX_RESULTS);

      if (0 == numResults)
      {
         YieldProcessor();
      }
      else if (RIO_CORRUPT_CQ == numResults)
      {
         ErrorExit("RIODequeueCompletion");
      }
   }
   while (0 == numResults);

   StartTiming();

   int workValue = 0;

   bool running = true;

   do
   {
      for (DWORD i = 0; i < numResults; ++i)
      {
         EXTENDED_RIO_BUF *pBuffer = reinterpret_cast<EXTENDED_RIO_BUF *>(results[i].RequestContext);

         if (results[i].BytesTransferred == EXPECTED_DATA_SIZE)
         {
            g_packets++;

            workValue += DoWork(g_workIterations);

            if (!g_rio.RIOReceive(
               g_requestQueue,
               pBuffer,
               1,
               recvFlags,
               pBuffer))
            {
               ErrorExit("RIOReceive");
            }

            done = false;
         }
         else
         {
            done = true;
         }
      }

      if (!done)
      {
         do
         {
            numResults = g_rio.RIODequeueCompletion(
               g_queue,
               results,
               RIO_MAX_RESULTS);

            if (0 == numResults)
            {
               YieldProcessor();
            }
            else if (RIO_CORRUPT_CQ == numResults)
            {
               ErrorExit("RIODequeueCompletion");
            }
         }
         while (0 == numResults);
      }
   }
   while (!done);

   StopTiming();

   PrintTimings();

   return workValue;
}

The DoWork() function above can be used to add 'processing overhead' to each datagram. This can be configured using the g_workIterations which is defined in Constants.h. With this set to 0 there is no overhead and we can compare how quickly each API can receive datagrams. Setting larger values will affect how the various multi-threaded examples perform and can be useful if you're unable to saturate the test machine's network interfaces.

Join in

Windows 8 Registered I/O Example UDP Servers - AsynchronousEvents

2012-03-07T11:27:00Z

RIO API demonstration

The examples are simple in that they do the bare minimum to demonstrate the APIs in question but they are configurable so that you can tune them to the hardware on which you're running them. You can run them to compare the maximum speed at which you can pull UDP datagrams off of the wire using each API and then adjust the examples so that they do a specific amount of "work" with each datagram to simulate a slightly more realistic scenario.

Simplified error handling

Error handling is limited, we display an error and exit the program, but we don't skip error checking, all API calls are checked for errors. The examples are each stand alone but can share two common header files. The first, Constants.h, contains all constants that are used to tune the examples. The second, Shared.h, contains inline helper functions which hide some of the complexity and allow the individual example programs to focus on the area of the API that they're demonstrating.

This is the index page

I will be blogging about the construction of the various examples over the next few weeks and updating this entry as an index page for all of the examples. I've listed the examples that I'll be talking about and I'll link to each blog post as they go live. Once I've presented the RIO examples I'll present the more traditional examples and finally some performance comparisons.

RIO server examples

RIO Polled UDP - A server which uses a single thread and a tight loop to poll for RIO completions.
RIO Event Driven UDP - A server which uses a single thread and event driven notifications to handle RIO completions.
RIO IOCP UDP - A server which uses a single thread and I/O Completion Port notifications to handle RIO completions.
RIO IOCP MT UDP - A server which uses a configurable number of threads and I/O Completion Port notifications to handle RIO completions.

Traditional server examples

Simple Polled UDP - A server which uses a single thread and a tight loop to poll WSARecv() for datagrams.
IOCP UDP - A server which uses a single thread and I/O Completion Port notifications to handle overlapped WSARecv() completions.
IOCP MT UDP - A server which uses a configurable number of threads and I/O Completion Port notifications to handle overlapped WSARecv() completions.

A simple UDP datagram traffic generator

Simple UDP traffic generator - A client which uses a single thread and a tight loop send datagrams using WSASendTo(), this easily saturates a 1000BASE-T, 1Gb ethernet connection.

Test scripts

Test scripts - These simple scripts create performance counter logs and run the test servers.

Performance Test results

The first tests - Where we compare the simple polled traditional server with the polled RIO server.

Join in

Latest release of The Server Framework: 6.5.4 - AsynchronousEvents

2012-02-14T08:28:41Z

Version 6.5.4 of The Server Framework was released today.

This release contains two important bug fixes and a selection of minor improvements. If you run your code on Vista/Windows Server 2003 or later and you don't explicitly disable FILE_SKIP_COMPLETION_PORT_ON_SUCCESS in your Config.h then you should install this update.

This release includes the following, see the release notes, here, for full details of all changes.

Bug fix. If FILE_SKIP_COMPLETION_PORT_ON_SUCCESS was enabled but JetByteTools::Socket::CanEnableSkipCompletionPortOnSuccess() returned false then the the code that handled issuing read and write calls would fail if ERROR_SUCCESS was returned because it would assume that FILE_SKIP_COMPLETION_PORT_ON_SUCCESS was enabled and that it should handle the completion directly but a completion would have been posted to the IOCP and so the completion would get handled twice. We now correctly whether we have actually enabled FILE_SKIP_COMPLETION_PORT_ON_SUCCESS rather than just whether we want to enable it.
Change to JetByteTools::Socket::CConnectionMaintainingStreamSocketConnectionFilter so that we do not attempt to maintain a connection if the reconnect delay is 0.
Added the concept of being able to force a write request to go via the I/O pool even if marshalling is currently turned off.
Added JetByteTools::Socket::IManageStreamSocketConnectionFilters::TryRequestWrite()
Changed how JetByteTools::Socket::CFlowControlStreamSocketConnectionFilter issues write requests and how it deals with write failure due to socket closure. We now purge any queued data when we detect the socket has been closed, rather than continuing to try and send more.
Added JetByteTools::Socket::IDatagramSendSocket which is a common base class for JetByteTools::Socket::IDatagramSocket and JetByteTools::Socket::IDatagramServerSocket.
Added JetByteTools::Socket::IFilterableStreamSocket::CanIssueFilteredWrite() which is now called instead of JetByteTools::Socket::IStreamSocketEx::CanWrite() by JetByteTools::Socket::CFilteringStreamSocketConnectionManagerBase::TryRequestWrite(). This removes a race condition during the shutdown of the write side of a socket in situations where filtering is being used and the filter wishes to write to the socket after the application level code has requested that the write side of the socket be shut down. We always tracked the outstanding write count before actually issuing the shutdown and the filter could manage this to allow it to be able to send after a shutdown had been requested BUT the filtered send could still fail as the socket's write shutdown flag would be set. This new function does not check the write shutdown flag and so allows the filter to write successfully.
Changed JetByteTools::IO::IAllocateBufferHandles::Flush() so that it returns a bool indicating if buffers were active when the flush was done. This brings it in line with JetByteTools::IO::IAllocateBuffers.
Changed the JetByteTools::IO::IAsyncIOStream::Write() methods so that they take an optional bool that enables you to force the write to go via the I/O pool even if I/O marshalling is turned off.
Changed JetByteTools::IO::CAsyncFileLog to monitor its own write thread to remove the chance that it might hang during destruction if the thread has terminated due to an exception.

The advantage of having lots of clients and clients with lots of clients - AsynchronousEvents

2012-02-08T12:11:02Z

Our Secretive Online Game Company client uses The Server Framework for their custom application server for the games industry. They have thousands of users who run their server on a very diverse set of hardware. This is great for us as it really helps to shake down The Server Framework. There's nothing like running your multi-threaded code on lots of different hardware to help find all of the hidden race conditions and whatever. I'm pleased that we have so few bug reports coming in from our clients. Especially knowing that our Online Game Company client has the latest code out in the field or at least in use internally on their cloud system. Unfortunately one of their clients has recently exposed a latent bug in a rarely used corner of The Server Framework.

One of the features of The Server Framework is that we track new features in various Windows operating systems so that you can take advantage of them simply by upgrading. Often you only need to adjust your Config.h file to enable powerful new features. One of these features is FILE_SKIP_COMPLETION_PORT_ON_SUCCESS, you can read more about it over on Len's blog. This allows some optimisation in thread scheduling and context switching and generally improves performance on busy servers.

Back in April 2011 we updated our FILE_SKIP_COMPLETION_PORT_ON_SUCCESS support to include protection from incompatible networking providers, see this Microsoft Knowledge Base article for details of the potential problem. In addition to the compile time support for FILE_SKIP_COMPLETION_PORT_ON_SUCCESS which can be used to turn the feature on and off in The Server Framework, we added a run time check to ensure that the machine on which the code was running did not have any incompatible networking providers installed.

Unfortunately this code wasn't tested as well as it could be and there was a bug in it which leads to problems on systems that have an incompatible networking provider installed and FILE_SKIP_COMPLETION_PORT_ON_SUCCESS support turned on. This leads to the completions for some operations being processed twice and subsequent reference counting (over-release) problems with the corresponding socket and buffer structures.

We've fixed this issue and it will be included in a 6.5.4 release which is currently in test. If you think you're suffering from the problems caused by this and need the fix immediately then please get in touch.

The offending networking provider in this case was the "AVSDA" provider which, a quick web search suggests, is part of Avira Anti virus.

WASP download of XP versions now fixed - AsynchronousEvents

2012-02-02T14:58:40Z

I've just noticed a problem with downloading the XP versions of WASP.

This is now fixed. The XP versions can now be downloaded correctly again from here. Sorry for any inconvenience caused.

Latest release of The Server Framework: 6.5.3 - AsynchronousEvents

2011-12-14T08:47:43Z

Version 6.5.3 of The Server Framework was released today.

This release updates the WebSockets Option pack to the final version of the protocol as detailed in RFC 6455 which was released yesterday. There is also a bug fix to WebSocket status reason processing. If you have 6.5 or 6.5.1 or 6.5.2 and you are NOT using WebSockets then you probably don't need this release.

This release includes the following, see the release notes, here, for full details of all changes.

Updated to support RFC 6455 - added close status codes 1011 and 1015.
Fixed a bug in the handling of long status result messages, we now truncate them correctly.

New client profile: Smart Moves Software Systems - Online gaming - AsynchronousEvents

2011-11-23T08:38:32Z

We have a new client profile available here for a new client who selected The Server Framework to help it expand its online gaming platform to incorporate a WebSockets interface.

A new release of WASP, now with SSL/TLS support - AsynchronousEvents

2011-11-22T10:10:47Z

We've just released a new version of WASP, our pluggable application server platform. This release is built with release 6.5.2 of The Server Framework and includes support for secure TCP connections using SSL/TLS via our SChannel Option pack.

Setting up a secure TCP endpoint with WASP is easy, simply add the Secure configuration option to the <EndPoint> node like this:

<?xml version="1.0" encoding="Windows-1252"?>
<Configuration>
  <WASP>
    <TCP>
      <Endpoints>
        <EndPoint
          Name="Echo Server"
          Port="5050"
          HandlerDLL="[CONFIG]\EchoServer.dll"
          Secure="true">
        </EndPoint>
      </Endpoints>
    </TCP>
  </WASP>
</Configuration>

This tells WASP to secure the endpoint using a default certificate called "Wasp" that is located in the "MY" certificate store. You can add a self signed test certificate using the standard Microsoft "make cert" utility, makecert.exe and a simple script which creates and installs the correct type of certificate can be downloaded from here.

if you do not want to use a certificate called "Wasp" in the "MY" certificate store then you can configure the certificate used by adding the StoreName, CertificateName and UseMachineStore config values.

<?xml version="1.0" encoding="Windows-1252"?>
<Configuration>
  <WASP>
    <TCP>
      <Endpoints>
        <EndPoint
          Name="Echo Server"
          Port="5050"
          HandlerDLL="[CONFIG]\EchoServer.dll"
          Secure="true"
          StoreName="OurSpecialStore"
          CertificateName="OurCertificate"
          UseMachineStore="true">
        </EndPoint>
      </Endpoints>
    </TCP>
  </WASP>
</Configuration>

Testing your new secure endpoint can be done using either our OpenSSL server test or our SChannel server test. These are example clients that ship with The Server Framework and that allow you to create thousands of concurrent connections and control how they send data to a server. This is an easy way to build a test system for your server as all of the complexity of managing and controlling the connections is done for you and you simply have to adjust the messages that are generated and how the response validation is done. The default message that is built is an network byte order integer length prefixed message and so this program can be used to stress test WASP with either of the first two example plugins that were discussed in the tutorial.

You can download the SChannelEchoServerTest program from here. See our for details of how to run this tool.