All Your Base Are Belong To Us

Performance Gains with Data Parallelism: Using SIMD Instructions from C#

Sasha Goldshtein — Thu, 24 May 2012 13:03:12 GMT

This is a short excerpt (with slight modifications) from Chapter 10 of Pro .NET Performance, scheduled to appear in August 2012. I might be publishing a few more of these before and after the book is out.

Theoretically, .NET developers should never be concerned with optimizations tailored to a specific processor or instruction set. After all, the purpose of IL and JIT compilation is to allow managed applications to run on any hardware that has the .NET Framework installed, and to remain indifferent to operating system bitness, processor features, and instruction sets. However, squeezing the last bits of performance from managed applications may require reasoning at the assembly language level. At other times, understanding processor-specific features is a first step for even more significant performance gains.

Data-level parallelism, also known as Single Instruction Multiple Data (SIMD), is a feature of modern processors that enables the execution of a single instruction on a large set of data (larger than the machine word). The de-facto standard for SIMD instruction sets is SSE (Streaming SIMD Extensions), used by Intel processors since Pentium III. This instruction set adds new 128-bit registers (with the XMM prefix) as well as instructions that can operate on them. Recent Intel processors introduced Advanced Vector Extensions (AVX), which is an extension of SSE that offers 256-bit registers and even more SIMD instructions. Some examples of SSE instructions include:

Integer and floating-point arithmetic
Comparisons, shuffling, data type conversion (integer to floating-point)
Bitwise operations
Minimum, maximum, conditional copies, CRC32, population count (introduced in SSE4 and later)

You might be wondering whether instructions operating on these “new” registers are slower than their standard counterparts. If that were the case, any performance gains would be deceiving. Fortunately, that is not the case. On Intel i7 processors, a floating-point addition (FADD) instruction on 32-bit registers has throughput of one instruction per cycle and latency of 3 cycles. The equivalent ADDPS instruction on 128-bit registers also has throughput of one instruction per cycle and latency of 3 cycles.

Using these instructions in high-performance loops can provide up to 8-fold performance gains compared to naïve sequential programs that operate on a single floating-point or integer value at a time. For example, consider the following (admittedly trivial) code:

//Assume that A, B, C are equal-size float arrays
for (int i = 0; i < A.length; ++i) {
C[i] = A[i] + B[i];
}

The standard code emitted by the JIT in this scenario is the following:

; ESI has A, EDI has B, ECX has C, EDX has i
xor edx,edx
cmp dword ptr [esi+4],0
jle END_LOOP
NEXT_ITERATION:
fld dword ptr [esi+edx*4+8] ; load A[i], no range check
cmp edx,dword ptr [edi+4]   ; range check accessing B[i]
jae OUT_OF_RANGE
fadd dword ptr [edi+edx*4+8]; add B[i]
cmp edx,dword ptr [ecx+4]   ; range check accessing C[i]
jae OUT_OF_RANGE
fstp dword ptr [ecx+edx*4+8]; store into C[i]
inc edx
cmp dword ptr [esi+4],edx   ; are we done yet?
jg NEXT_ITERATION
END_LOOP:

Each loop iteration performs a single FADD instruction that adds two 32-bit floating-point numbers. However, by using 128-bit SSE instructions, four iterations of the loop can be issued at a time, as follows (the code below performs no range checks and assumes that the number of iterations is equally divisible by 4):

xor edx, edx
NEXT_ITERATION:
; copy 16 bytes from B to xmm1
movups xmm1, xmmword ptr [edi+edx*4+8]
; copy 16 bytes from A to xmm0
movups xmm0, xmmword ptr [esi+edx*4+8]
; add xmm0 to xmm1 and store the result in xmm1
addps xmm1, xmm0
; copy 16 bytes from xmm1 to C
movups xmmword ptr [ecx+edx*4+8], xmm1
add edx, 4 ; increase loop index by 4
cmp edx, dword ptr [esi+4]
jg NEXT_ITERATION

On an AVX processor, we could move even more data around in each iteration (with the 256-bit YMM* registers), for an even bigger performance improvement:

xor edx, edx
NEXT_ITERATION:
; copy 32 bytes from B to ymm1
vmovups ymm1, ymmword ptr [edi+edx*4+8]
; copy 32 bytes from A to ymm0
vmovups ymm0, ymmword ptr [esi+edx*4+8]
; add ymm0 to ymm1 and store the result in ymm1
vaddps ymm1, ymm1, ymm0
; copy 32 bytes from ymm1 to C
vmovups ymmword ptr [ecx+edx*4+8], ymm1
add edx, 8 ; increase loop index by 8
cmp edx, dword ptr [esi+4]
jg NEXT_ITERATION

The SIMD instructions used in these examples are only the tip of the iceberg. Modern applications and games use SIMD instructions to perform complex operations, including scalar product, shuffling data around in registers and memory, checksum calculation, and many others. Intel’s AVX portal is a good way to learn thoroughly what AVX can offer.

The JIT compiler uses only a small number of SSE instructions, even though they are available on practically every processor manufactured in the last 10 years. Specifically, the JIT compiler uses the SSE MOVQ instruction to copy medium-sized structures through the XMM* registers (for large structures, REP MOVS is used instead), uses SSE2 instructions for floating point to integer conversion, and other corner cases. The JIT compiler does not auto-vectorize loops by unifying iterations, as we did manually in the preceding code.

Unfortunately, C# doesn’t offer any keywords for embedding inline assembly code into your managed programs. Although you could factor out performance-sensitive parts to a C++ module and use .NET interoperability to access it, this is often clumsy and introduces a small performance penalty as the interoperability boundaries are crossed. There are two other approaches for embedding SIMD code without resorting to interoperability.

A brute-force way to run arbitrary machine code from a managed application (albeit with a light interoperability layer) is to dynamically emit the machine code and then call it. The Marshal.GetDelegateForFunctionPointer method is key, as it returns a managed delegate pointing to an unmanaged memory location, which may contain arbitrary code. The following code allocates virtual memory with the EXECUTE_READWRITE page protection, which enables us to copy code bytes into memory and then execute them. The result, on my Intel i7-860 CPU, is a more than 2-fold improvement in execution time!

[UnmanagedFunctionPointer(CallingConvention.StdCall)]
delegate void VectorAddDelegate(
float[] C, float[] B, float[] A, int length);

[DllImport("kernel32.dll", SetLastError = true)]
static extern IntPtr VirtualAlloc(
IntPtr lpAddress, UIntPtr dwSize,
IntPtr flAllocationType, IntPtr flProtect);

//This array of bytes has been produced from
//the SSE assembly version – it is a complete
//function that accepts four parameters (three
//vectors and length) and adds the vectors

byte[] sseAssemblyBytes = {
0x8b, 0x5c, 0x24, 0x10, 0x8b, 0x74, 0x24, 0x0c, 0x8b,
0x7c, 0x24, 0x08, 0x8b, 0x4c, 0x24, 0x04, 0x31, 0xd2,
0x0f, 0x10, 0x0c, 0x97, 0x0f, 0x10, 0x04, 0x96, 0x0f,
0x58, 0xc8, 0x0f, 0x11, 0x0c, 0x91, 0x83, 0xc2, 0x04,
0x39, 0xda, 0x7f, 0xea, 0xc2, 0x10, 0x00 };

IntPtr codeBuffer = VirtualAlloc(
IntPtr.Zero, new UIntPtr((uint)sseAssemblyBytes.Length),
0x1000 | 0x2000, //MEM_COMMIT | MEM_RESERVE
0x40 //EXECUTE_READWRITE
);

Marshal.Copy(sseAssemblyBytes, 0,
codeBuffer, sseAssemblyBytes.Length);

VectorAddDelegate addVectors = (VectorAddDelegate)
Marshal.GetDelegateForFunctionPointer(
codeBuffer, typeof(VectorAddDelegate));
//We can now use ‘addVectors’ to add vectors!

A completely different approach, which unfortunately isn’t available on the Microsoft CLR, is extending the JIT compiler to emit SIMD instructions. This is the approach taken by Mono.Simd. Managed code developers who use the Mono .NET runtime can reference the Mono.Simd assembly and use JIT compiler support that converts operations on types such as Vector16b or Vector4f to the appropriate SSE instructions.

I am posting short updates and links on Twitter as well as on this blog. You can follow me: @goldshtn

Obtaining mscordacwks.dll for CLR Versions You Don’t Have

Sasha Goldshtein — Sat, 19 May 2012 14:46:55 GMT

Note: This blog post assumes that you can capture and analyze managed dumps in WinDbg using SOS, and have encountered a bizarre technical problem when using dumps from a production environment. If this assumption is incorrect, feel free to peruse my .NET Debugging Resources link post.

When debugging managed dumps in WinDbg, you will need to load the SOS version that is compatible with the CLR version in the dump. SOS, in turn, requires the CLR “data access DLL” (mscordacwks.dll), which is a debugging helper shipping with the .NET Framework. If SOS and/or mscordacwks.dll are missing or have the wrong version, you will receive an error similar to the following when trying to run most SOS commands:

Failed to load data access DLL, 0x80004005
Verify that 1) you have a recent build of the debugger (6.2.14 or newer)
2) the file mscordacwks.dll that matches your version of mscorwks.dll is in the version directory
3) or, if you are debugging a dump file, verify that the file mscordacwks_<arch>_<arch>_<version>.dll is on your symbol path.
4) you are debugging on the same architecture as the dump file. For example, an IA64 dump file must be debugged on an IA64 machine.

You can also run the debugger command .cordll to control the debugger's load of mscordacwks.dll. .cordll -ve -u -l will do a verbose reload. If that succeeds, the SOS command should work on retry.

If you are debugging a minidump, you need to make sure that your executable path is pointing to mscorwks.dll as well.

Some CLR versions have been indexed by the Microsoft public symbol server, so you can instruct the debugger to download everything automatically by setting your symbol path and image path to the Microsoft symbol server.

However, some CLR versions have not been indexed and cannot be retrieved automatically—and this is where you need to crawl the Web and find the binaries manually. Today I had the chance to encounter a CLR version, 2.0.50727.3607, which WinDbg couldn’t retrieve from the Microsoft symbol server:

SYMSRV: http://msdl.microsoft.com/download/symbols/
mscordacwks_x86_x86_2.0.50727.3607.dll/4ADD5446590000/
mscordacwks_x86_x86_2.0.50727.3607.dll not found

Doug Stewart has been collecting updates to CLR 2.0 for several years now, and has an entry on his post for the 3607 CLR build, associated with KB article 976569.

If you go to the KB article, you’ll be able to download the update, an executable file. Instead of launching it, open it in an application that understands self-executing archives, such as 7-Zip:

Locate the .msp file and open it again:

One of the .cab files contains the files we are looking for. If you got the wrong .cab file, try the other one :-)

Now all that’s left is to extract them to a temporary directory, rename them to mscorwdacwks.dll and mscorwks.dll, and issue a .cordll -u -ve -lp <path> command in WinDbg:

0:014> .cordll -u -ve -lp D:\temp\3607
CLRDLL: Loaded DLL D:\temp\3607\mscordacwks.dll
CLR DLL status: Loaded DLL D:\temp\3607\mscordacwks.dll

Voila—you can run any SOS commands now.

I am posting short updates and links on Twitter as well as on this blog. You can follow me: @goldshtn

“Fitting” Performance into the Software Development Lifecycle

Sasha Goldshtein — Sun, 13 May 2012 09:22:47 GMT

This is a short excerpt from Chapter 1 of Pro .NET Performance, scheduled to appear in August 2012. I might be publishing a few more of these before and after the book is out. We have an Amazon page and a cover image now!

Where do you fit performance in the software development lifecycle? This innocent question carries the mind baggage of having to retrofit performance into an existing process. Although it is possible, a healthier approach is to consider every step of the development lifecycle an opportunity to understand the application’s performance better—first, the performance goals and important metrics; next, whether the application meets or exceeds its goals; and finally, whether maintenance, user loads, and requirement changes introduce any regressions.

During the requirements gathering phase, you should start thinking about the performance goals you would like to set.

During the architecture phase, you should refine the performance metrics important for your application and define concrete performance goals.

During the development phase, you should frequently perform exploratory performance testing on prototype code or partially complete features to verify that you are well within the system’s performance goals.

During the testing phase, you should perform significant load testing and performance testing to validate completely your system’s performance goals.

During subsequent development and maintenance, you should perform additional load testing and performance testing with every release (and preferably on a daily or weekly basis) to quickly identify any performance regressions introduced into the system.

Taking the time to develop a suite of automatic load tests and performance tests, to set up an isolated lab environment in which to run them, and to analyze their results carefully to make sure no regressions are introduced is a very time-consuming process. Nevertheless, the performance benefits gained from systematically measuring and improving performance and making sure regressions don’t creep slowly into the system is worth the initial investment in having a robust performance development process.

I am posting short updates and links on Twitter as well as on this blog. You can follow me: @goldshtn

Pinpointing Memory Leaks with CLR Profiler Heap Graphs

Sasha Goldshtein — Fri, 04 May 2012 08:43:52 GMT

CLR Profiler is a free Microsoft tool for diagnosing memory-related performance problems in managed applications. In this post, I’m using CLR Profiler v4.0, which you can download here.

I talked about CLR Profiler here as a post-mortem diagnostic tool that can open log files generated by SOS.dll’s !TraverseHeap command and present a reference graph of all live objects. This in itself is a little-known feature of CLR Profiler; it is even less known that CLR Profiler can generate these reference graphs live, and compare them automatically to show you where a memory leak is coming from.

All you need to do is run your application under CLR Profiler, and click the “Show heap now” button periodically. This is similar to the “Take snapshot” functionality in ANTS Memory Profiler and other tools. When the application terminates, you click the “Heap Graph” button in the Summary view.

This produces a reference graph in which you can see the differences between snapshots. This is the end of the graph, which makes it evident that almost all the retained objects are strings, held by string arrays and FileInformation objects:

And this is the beginning of the graph, which makes it clear (with some experience deciphering root reference chains) that the majority of objects are retained by a static EventHandler:

If you zoom into the snapshots, you’ll see three colors, indicating the amount of memory allocated and retained between the snapshots. For example, the darkest pink objects were created between the second and the third snapshots.

There are several advanced options available for further exploration. For example, you can view only new objects and then see who allocated them (which call stack in your program is responsible for creating them). You could even look at a GC timeline and see which objects were alive at every point in time, as well as who allocated them:

To summarize, CLR Profiler still has plenty of hidden gems and you should consider using it—especially in simple scenarios. After all, it’s hard to beat its price :-)

I am posting short updates and links on Twitter as well as on this blog. You can follow me: @goldshtn

Setting Up an Offline Production Debugging Environment

Sasha Goldshtein — Wed, 02 May 2012 07:22:05 GMT

If you’re doing production debugging in a closed environment—closed-down servers with no Internet access, or if you work at an institution that doesn’t have unrestricted Internet access from developer machines, this post will help you set up an offline production debugging environment. Incidentally, this post will also help if you’re going to host SELA’s .NET Debugging or C++ Debugging courses, and want to make sure your workstations are ready for the numerous hands-on labs.

First and foremost, you are going to need all the tools you plan using for production debugging. At the very least, this includes:

Debugging Tools for Windows
Note there are 32-bit and 64-bit versions, you need whatever your application is using. For several releases now, the installation package is only available through the WDK or the Windows SDK, but you can follow the instructions here to configure the SDK web installer to download only the Debugging Tools for Windows installation package.
CLR Profiler
The v4.0 version works for CLR 2.0 applications, so that’s the one you need.
Sysinternals Suite
If you’re really low on disk space, the least you need are Process Explorer, Process Monitor, and VMMap.
.NET disassembler – any of the following will do:

RedGate .NET Reflector [paid]
ILSpy [free, open source]
JustDecompile [free]

Application Verifier
Note there are 32-bit and 64-bit versions.
Debugging extensions

Next, you’re going to need debugging symbols for your environment. This includes debugging symbols for your own code, but more importantly, symbols for Windows, the CLR, and the .NET Framework versions your application is using.

In an online environment, obtaining symbols is trivial through the Microsoft symbol server. In an offline environment, you need to jump through a few hoops to make sure you have all the symbols set up, so here goes.

Windows symbols are available for download as symbol packages for various OS versions. As always, you need to make sure you’re downloading symbols for the precise operating system version on which your application runs. If there are several platforms you’re using, you’ll need symbols for all of them.

If you installed any hotfixes or updates from Windows Update, you’ll need to download symbols for them manually. Use the symchk.exe utility that ships with the Debugging Tools for Windows to obtain manually symbols on an Internet-connected machine. Follow these instructions to generate a text file on the disconnected machine and use it on a connected machine to download symbols.

.NET Framework symbols are available online on the Reference Source Code Center. These packages contain source code (!) and symbol files for the .NET Framework assemblies: mscorlib.dll, System.dll, and others.

CLR symbols are not available online as part of the .NET Framework symbol packages. You won’t usually need them unless you’re exploring CLR internals or dealing with very specific problems that require inspecting unmanaged call stacks of managed threads. To obtain CLR symbols, you’ll have to use the same technique mentioned above with symchk.exe.

Key CLR binaries and SOS.DLL are required for properly debugging dump files on a different machine. If you have access to the original machine, you can simply copy mscorwks.dll/clr.dll, SOS.dll and mscordacwks.dll from the %windir%\Microsoft.NET\Framework* directory corresponding to your .NET version. If you don’t have access to the original machine, you’ll have to track down the installation package for the appropriate CLR version and retrieve these binaries from it—I’ll cover this in a future post.

After following these instructions, you should have a fully functional offline production debugging environment. Additions and comments are welcome!

I am posting short updates and links on Twitter as well as on this blog. You can follow me: @goldshtn

Second Meeting of the Jerusalem .NET/C++ User Group

Sasha Goldshtein — Mon, 30 Apr 2012 07:22:38 GMT

The second meeting of the Jerusalem .NET C++ User Group will take place on May 29. This time we’ll be talking about advanced C++ topics. If you’re a developer in the Jerusalem area working with C++, this is the user group for you! Slides from the previous meeting’s talks are available online if you’d like to catch up on what you might have missed last time.

The agenda for this meeting is as follows:

18:00-18:15 – Networking and refreshments
18:15-19:00 – Portable lock-free use of STL containers (Adi Shavit)
19:00-19:15 – Networking and refreshments
19:15-20:00 – Undocumented native debugging tricks (Ofek Shilon)

Undocumented native debugging tricks
Visual Studio provides great debugging facilities as is – but, it also contains a tremendous wealth of useful debugging features, that just never matured enough to be documented and supported. Such hidden features range from enhanced interaction with the debugee, improved debugging productivity, better state diagnostics, better exploration of code flow, and more. The talk would survey as many of those goodies as time permits.

Portable lock-free use of standard STL containers
Parallel and concurrent hardware is becoming more and more pervasive and, in turn, demand for high performance and fast response of these concurrent systems is growing. One modern go-to solution are lock-free data structures. A lot of research has recently produced many lock-free and wait-free data-structures. However, these are often data-structures for very specific use-cases. Portable, well-tested and suitably licensed implementations are still hard to come by. I'll present a scheme for providing portable lock-free access to, and modification of, existing and standard STL containers. The scheme can also be adapted to most other non-STL container. With a compliant C++11 compiler, only the std library is required. On older compilers can use boost to provide the missing std components. This scheme leverages standard atomic reference-counting (via shared_ptr<>)to provide automatic garbage collection.

The Case of The Unquoted Command Line: Process Monitor and MPGO.EXE

Sasha Goldshtein — Sat, 28 Apr 2012 11:49:16 GMT

A few months ago I wrote about MPGO, a new Microsoft tool that ships with .NET 4.5 and enables profile-guided optimization of managed assemblies. Specifically, MPGO optimizes the layout of native images for managed assemblies, which reduces startup times, working set sizes, and page fault costs.

Shameless plug: Pro .NET Performance has a large section dedicated to improving startup performance, and I’ve written about this before: Using Prefetch to improve startup performance, Rebasing and compression.

Anyway, I was experimenting with MPGO and encountered a strange error when working with some of my assemblies. Specifically:

C:\Temp\mpgo test>mpgo -scenario App.exe -assemblylist App.exe -OutDir .

Error: Timeout or error when trying to instrument assembly C:\Temp\mpgo test\App.exe (FFFFFFFF)
Profile information will not be collected for this assembly.
This will not prevent information from being collected for other assemblies

Error: Timeout or Error when trying to remove instrumented assembly C:\Temp\mpgo test\App.exe (FFFFFFFF)
Failed to merge collected profile data into assembly C:\Temp\mpgo test\App.exe:
Unexpected Internal Exception The file "C:\Temp\mpgo test\App.ibc" does not exist

This error struck me as odd, because the application in question was really simple, and I certainly didn’t know what error FFFFFFFF meant. I tried specifying the full path to the executable, and got a different error instead:

C:\Temp\mpgo test>mpgo -scenario "C:\temp\mpgo test\App.exe" -assemblylist "C:\temp\mpgo test\App.exe" -OutDir .

Session executable does not appear to exist.

This produced a somewhat simpler error, although still completely misleading because App.exe exists in the specified directory. However, because the full path changed MPGO’s behavior, I suspected that something in its file access code or command-line parsing has gone awry.

This is where I launched Process Monitor and configured it to watch all events generated by MPGO.EXE. When using the latter command line, I saw very quickly that MPGO was trying (and failing) to access “C:\Temp\mpgo”, which is a directory that doesn’t exist—I explicitly gave it “C:\Temp\mpgo test\App.exe” as a parameter!

At this point I just had to look at the call stack to see what’s going on. MPGO.EXE is a managed application, but Process Monitor can’t give you managed call stacks (yet?), so I had to attach WinDbg and !u the raw addresses to see where the file access was coming from.

0:000> !u 0x600f8b
Normal JIT generated code
MPGO.Harness.PreRunValidation()
Begin 00600e10, size 22d
00600e10 55              push    ebp
00600e11 8bec            mov     ebp,esp
…snipped…
00600f7e 8b4e04          mov     ecx,dword ptr [esi+4]
00600f81 ba01000000      mov     edx,1
00600f86 e8051e7f59      call    mscorlib_ni+0x2f2d90 (59df2d90) (System.IO.File.InternalExistsHelper(System.String, Boolean), mdToken: 06004402)
00600f8b 85c0            test    eax,eax
00600f8d 7541            jne     00600fd0
00600f8f b9fc432a00      mov     ecx,2A43FCh (MT: MPGO.MpgoException)
00600f94 e88310c9ff      call    0029201c (JitHelp: CORINFO_HELP_NEWSFAST)
…snipped…

Next, I turned to the PreRunValidation method in Reflector and discovered that the exception is thrown because MPGO thinks my executable file does not exist, from a line that looks like this:

if (!File.Exists(this.Exe))
{
MpgoException.Throw("Err_ExeMissing");
}

By inspecting all assignments to this.Exe, I found the following in the Harness class constructor:

Match match = Regex.Match(args[i], "(?<exe>(?:\"[^\"]+\")|[^ ]+)(?: (?<args>.*))?");
if (!match.Success)
{
Usage.Show("Err_CantParseScenario_0", args[i]);
}
this.Exe = match.Groups["exe"].ToString();
if (this.Exe.StartsWith("\""))
{
this.Exe = this.Exe.Trim(new char[] { '"' });
}
this.Exe = Path.GetFullPath(this.Exe);
this.Args = match.Groups["args"].ToString();

When applied to a quoted path, such as “C:\temp\mpgo test\App.exe”, this code produces an unquoted executable name and considers everything that follows the space to be the executable’s arguments…

There is a different bug when using the former command line. In that case, I configured Process Monitor to log process (and thread) creation/deletion events, and noticed that MPGO.EXE launches NGEN.EXE with an invalid command line:

The full command line is:

"C:\Windows\Microsoft.NET\Framework\v4.0.30319\ngen.exe" install C:\Temp\mpgo test\App.exe /tuning /ExeConfig:C:\Temp\mpgo test\App.exe

…which confuses NGEN, because it treats “C:\Temp\mpgo” as the application name and the rest is an invalid command line. This time, the problem is that when launching NGEN, MPGO does not bother to quote the application name.

To summarize: the purpose of this post is not to bash MPGO—the bug has been reported to the team responsible and will likely be fixed in the next Visual Studio 11 drop. What I wanted to show is that Process Monitor is quite useful in determining why applications you aren’t familiar with are behaving incorrectly. Also, figuring out proper quoting of command line parameters is really nasty (on Windows).

I am posting short updates and links on Twitter as well as on this blog. You can follow me: @goldshtn

Deep Dive into WinRT: MSDN Session

Sasha Goldshtein — Mon, 23 Apr 2012 11:33:20 GMT

Thanks for coming to my session on WinRT internals today at Microsoft Raanana! Preparing for this session has been very interesting for me, especially as I was mucking around with vtable pointers for the Pro .NET Performance book anyway :-)

Deep Dive into WinRT

In this session we talked about the following:

Refreshment of how COM objects work
WinRT object layout and relationship to COM
The WinRT type system and threading model
Asynchronous operations in WinRT
Windows metadata files and projecting WinRT APIs to C#, C++/CX and JavaScript
Designing and developing WinRT components
Performance interoperability tips for WinRT
Profiling WinRT applications with Visual Studio Profiler

Here’s my favorite slide—as close to assembly language as they let me:

You can download the slides (PDF) and demos right now; the session’s video recording (Hebrew) will be available later on Channel 9.

I am posting short updates and links on Twitter as well as on this blog. You can follow me: @goldshtn

Restoring a Computer From Windows Home Server Backup

Sasha Goldshtein — Sat, 14 Apr 2012 09:44:42 GMT

The other day I had the immensely fun experience of restoring my Alienware m15x laptop from backup. It’s the second time I’ve restored a computer from backup since I have my trusty Acer easyStore Windows Home Server system. It just sits quietly in the corner, gathering dust and backing up my documents, work, and memories, claiming no reward but an additional 2TB hard drive every six months.

The general restore process is very simple:

Burn a copy of the Windows Home Server Computer Restore CD
Boot the faulty machine from the CD
Let it find your home server over the network
Choose which machine to restore and which backup to use
Wait a few hours and you have a restored machine

The only potential pitfall in this process is that you need the restore CD environment to recognize your hardware. At the very least, it must be able to recognize the hard drive and the Ethernet network card, or it won’t be able to download the backup from your Home Server and restore the system from it.

With every backup, Windows Home Server captures the set of critical drivers required for the restore. Before restoring the system, you can get these drivers from the Home Server backup—they will be in a folder called Windows Home Server Drivers for Restore on your system drive when you view the Home Server backup from the Windows Home Server console. These drivers will work great in the recovery environment if you are restoring a 32-bit system—the recovery environment is 32-bit.

Unfortunately, if you are restoring a 64-bit system, you will need to find 32-bit drivers for the hard drive and network card manually. This can be a rather tedious process. You can usually find drivers on the hardware manufacturer’s website.

When restoring my Alienware m15x, the only hardware device with a missing driver was the Intel Ethernet NIC. You’ll need the Windows Vista 32-bit drivers, as that’s what the recovery environment is based on. Download them from here and you’ll be good to go!

Oh, and by the way—backups are important, but they are worthless if you never try restoring from backup. You’d think it should only take an hour and then spend a whole weekend sorting out driver problems, like I did :-)

I am posting short updates and links on Twitter as well as on this blog. You can follow me: @goldshtn

Honored to be Renominated as Microsoft MVP for 2012

Sasha Goldshtein — Wed, 04 Apr 2012 07:34:30 GMT

On April 1 (yes, it has the potential of being an April Fool’s every time!) I received the renomination letter—I am honored to receive the Microsoft MVP Award in Visual C# for 2012.

As I wrote a year ago, 2011 was an exciting year for C#—I had several opportunities to talk about async methods and the parallelism revolution last year, and 2012—the year of Windows 8—promises to be even more interesting for C# as we tackle the development of Metro-style apps.

I wouldn’t be writing this post if it weren’t for the help and support of my colleagues, friends, and managers at SELA and our business partners and friends at Microsoft Israel. Specifically, David Bassa and Ishai Ram are still the best managers and friends a tech professional can have, and Guy Burstein of Microsoft DPE continuously amazes me by shaping the Israeli developer community with MSDN events and user group meetings.

Thanks for reading—I am looking forward for the rest of 2012 and hope to continue providing you with interesting information on gory CLR internals and debugging problems :-)

I am posting short updates and links on Twitter as well as on this blog. You can follow me: @goldshtn

What AnyCPU Really Means As Of .NET 4.5 and Visual Studio 11

Sasha Goldshtein — Wed, 04 Apr 2012 06:59:15 GMT

The 32-bit and 64-bit development story on Windows seemingly never stops causing problems for developers. It’s been a decade since 64-bit processors have started popping up in the Windows consumer environment, but we just can’t get it right. If you forget some of the gory details, here are a couple of reminders:

On a 64-bit Windows system, both the 32-bit and 64-bit versions of system DLLs are stored. The 64-bit DLLs are in C:\Windows\System32, and the 32-bit DLLs are in C:\Windows\SysWOW64.
When a 32-bit process opens a file in C:\Program Files, it actually reads/writes to C:\Program Files (x86).
There are separate views of (most of) the registry for 32-bit and 64-bit applications. You can change the 64-bit registry location and it wouldn’t be visible to 32-bit applications.

These differences are hardly elegant as they are, but they allow 32-bit applications to run successfully on a 64-bit Windows system. While unmanaged applications always had to choose the native target—x86, x64, or ia64 in the Visual Studio case—managed code has the additional choice of AnyCPU.

What AnyCPU used to mean up to .NET 4.0 (and Visual Studio 2010) is the following:

If the process runs on a 32-bit Windows system, it runs as a 32-bit process. IL is compiled to x86 machine code.
If the process runs on a 64-bit Windows system, it runs as a 64-bit process. IL is compiled to x64 machine code.
If the process runs on an Itanium Windows system (has anyone got one? ;-)), it runs as a 64-bit process. IL is compiled to Itanium machine code.

Prior to Visual Studio 2010, AnyCPU was the default for most .NET projects, which was confusing to some developers: when they ran the application on a 64-bit Windows system, the process was a 64-bit process, which may cause unexpected results. For example, if the application relies on an unmanaged DLL of which only a 32-bit version is available, its 64-bit version won’t be able to load that component.

In Visual Studio 2010, x86 (and not AnyCPU) became the default for most .NET projects—but otherwise the semantics haven’t changed.

In .NET 4.5 and Visual Studio 11 the cheese has been moved. The default for most .NET projects is again AnyCPU, but there is more than one meaning to AnyCPU now. There is an additional sub-type of AnyCPU, “Any CPU 32-bit preferred”, which is the new default (overall, there are now five options for the /platform C# compiler switch: x86, Itanium, x64, anycpu, and anycpu32bitpreferred). When using that flavor of AnyCPU, the semantics are the following:

If the process runs on a 32-bit Windows system, it runs as a 32-bit process. IL is compiled to x86 machine code.
If the process runs on a 64-bit Windows system, it runs as a 32-bit process. IL is compiled to x86 machine code.
If the process runs on an ARM Windows system, it runs as a 32-bit process. IL is compiled to ARM machine code.

The difference, then, between “Any CPU 32-bit preferred” and “x86” is only this: a .NET application compiled to x86 will fail to run on an ARM Windows system, but an “Any CPU 32-bit preferred” application will run successfully.

To inspect these changes, I created a new C# console application in Visual Studio 11 that prints the values of Environment.Is64BitOperatingSystem and Environment.Is64BitProcess. When I ran it on my 64-bit Windows system, the result was as follows:

Is64BitOperatingSystem = True
Is64BitProcess = False

Inspecting the project’s properties shows the following (in the current Visual Studio UI “Prefer 32-bit” is grayed out and unchecked, where in actuality it is enabled…):

Inspecting the executable with CorFlags.exe shows the following:

Version   : v4.0.30319
CLR Header: 2.5
PE        : PE32
CorFlags : 131075
ILONLY    : 1
32BITREQ : 0
32BITPREF : 1
Signed    : 0

After changing the 32BITPREF setting with CorFlags.exe (using the /32bitpref- option), the output was as follows:

Is64BitOperatingSystem = True
Is64BitProcess = True

I am posting short updates and links on Twitter as well as on this blog. You can follow me:@goldshtn

Slides from the First Jerusalem .NET/C++ Meeting

Sasha Goldshtein — Sun, 25 Mar 2012 10:13:03 GMT

Last Tuesday we hosted the first meeting of the Jerusalem .NET/C++ User Group. I forgot to take pictures, but we were a nice group of hardcore C++ developers eager to learn about the new C++ 11 features, debugging C++ code in production, and some memory management tricks relevant for C++ real-time applications.

As promised, below are the presentations from the event. My presentation on C++11 covers lambdas, auto variables, rvalue references and even touches briefly on the subject of variadic templates:

C++11

View more presentations from goldshtn.

Noam Sheffer’s presentation covers the fundamentals of debugging C++ applications in production—generating crash dumps and hang dumps, performing basic analysis in WinDbg, and inspecting memory dumps to solve more complex problems:

C++ Production Debugging

View more presentations from goldshtn.

Finally, Vladimir Oster talked about memory management in C++ real-time applications—overloading operators new and delete, allocating from a static memory pool, using placement new correctly, and other topics. Unfortunately, I can’t share his slides online, but ping me directly if you attended the session and would like to receive a soft copy.

Virtual Method Dispatch and Object Layout Changes in CLR 4.0

Sasha Goldshtein — Thu, 15 Mar 2012 10:37:28 GMT

As part of the upcoming Pro .NET Performance book, there’s quite a bit of research we need to do on various facets of CLR internals. During my research for the Type Internals chapter I discovered a change in CLR object layout and virtual method dispatch as of CLR 4.0 – possibly not the most exciting of changes but one that invalidates most of the existing material on this subject, such as the popular JIT and Run article or the Advanced .NET Debugging book.

First, a quick overview of how reference type instances are laid out on the heap. Suppose that we have an Employee class with two instance fields, _name and _id, as well as a virtual method called Work. On a 32-bit managed heap, an Employee instance occupies 16 bytes, and has the following layout (each cell = 4 bytes):

Object Header Word (Sync Block Index)

Method Table Pointer

The _name field

The _id field

The method table pointer points to Employee’s method table, which contains, among other things, code addresses for Employee’s methods. On CLR 2.0, the method table has roughly the following layout:

Flags, Size, EEClass, Module Ptr, etc.

Interface Map Pointer

+0x28 Object.ToString

+0x2c Object.Equals

+0x30 Object.GetHashCode

+0x34 Object.Finalize

+0x38 Employee.Work

Employee..ctor

Interface MTs implemented by Employee

The code addresses for Employee’s virtual methods (including those inherited and possibly overridden from any base classes) allow virtual method dispatch to proceed as follows (assuming that the object reference is in ECX):

mov eax, dword ptr [ecx]
call dword ptr [eax+38]

The key here is that the order of methods in the method table and the offset of overridden methods from the beginning of the method table remains constant in all derived classes. For example, suppose that Manager derives from Employee and overrides the Work method. The method table for Manager would have the following layout:

Flags, Size, EEClass, Module Ptr, etc.

Interface Map Pointer

+0x28 Object.ToString

+0x2c Object.Equals

+0x30 Object.GetHashCode

+0x34 Object.Finalize

+0x38 Manager.Work

Manager..ctor

Interfaces implemented by Manager

If you inspect carefully the invocation sequence for Employee.Work, it turns out that the same instructions can be used to invoke Manager.Work – indeed, the caller’s instruction sequence should not depend on whether Manager overrides the Work method, and whether the referenced instance is of type Employee or of type Manager. This is the key to polymorphism.

However, as of CLR 4.0, the method table layout has changed. Several fields have been moved around or removed completely. Specifically, the invariant offset where virtual methods are laid out is no longer constant, because the list of interfaces implemented by the type precedes them (and can vary in derived classes). For example, this is a possible layout for Employee’s method table on CLR 4.0 – assuming that it implements three interfaces:

Flags, Size, EEClass, Module Ptr, etc.

+0x24 Pointer to Interface List

+0x28 Pointer to Methods

More Miscellanea

Interfaces implemented by Employee

+0x40 Object.ToString

+0x44 Object.Equals

+0x48 Object.GetHashCode

+0x4c Object.Finalize

+0x50 Employee.Work

Fortunately, the “Pointer to Methods” field is at a constant offset from the beginning of the method table, and always points to the first entry in the code address list, which is the code address for Object.ToString. The offset of any virtual methods from that address is constant in all derived classes. In other words, the JIT can use a slightly longer method invocation sequence to call virtual methods in CLR 4.0:

mov eax, dword ptr [ecx]
mov eax, dword ptr [eax+28]
call dword ptr [eax+10]

In Manager’s method table, the Work method may have a different offset from the beginning of the method table because Manager is free to implement additional interfaces, but the “Pointer to Methods” field is at the same offset and accommodates for this difference:

Flags, Size, EEClass, Module Ptr, etc.

+0x24 Pointer to Interface List

+0x28 Pointer to Methods

More Miscellanea

Interfaces implemented by Manager

+0x44 Object.ToString

+0x48 Object.Equals

+0x4c Object.GetHashCode

+0x50 Object.Finalize

+0x54 Manager.Work

This allows the same form of dispatch to work for Manager objects as well.

Needless to say, relying on the details shown in this blog post would be as futile as relying on any prior material on this subject. Internal details such as object layout and code generated by the JIT are subject to change at any time between CLR releases.

I am posting short updates and links on Twitter as well as on this blog. You can follow me: @goldshtn
This blog post was also cross-posted to CodeProject.

Pro .NET Performance

Sasha Goldshtein — Tue, 13 Mar 2012 05:34:17 GMT

It is customary to apologize for a prolonged blog silence, so here goes: I’ve been very busy with three extremely interesting projects, one of which is the SELA Developer Practice that is very near. Here’s another:

It’s been more than five years since I wrote a SELA course called Effective C#, which quickly turned to another course called .NET Performance. Since then, I trained this course dozens of times, spoke about performance optimization at conferences, delivered a one-day summary at several tutorials and workshops. All along I was thinking that there is no single decent textbook that combines the deep dive into CLR internals I am teaching in the course with real tips for performance measurement and optimization. The time was ripe to write one.

I am very happy to announce that I am writing a book on .NET Performance! Together with Dima Zurbalev, my esteemed colleague and coauthor, we are trying to produce a relevant guide for seasoned .NET developers looking to improve the performance of their applications and understand much better how and why the CLR does things.

The chapter list is not set in stone, and we don’t have yet a cover to show off, but we’re looking at around 350 pages across the following chapters:

Performance Metrics
Performance Measurement Tools
Type Internals
Garbage Collection
Collections and Generics
Concurrency and Parallelism
Networking, I/O, and Serialization
Unsafe Code and Interoperability
Algorithm Optimization
Performance Patterns
Web Application Performance – guest chapter by Ido Flatow, an outstanding Web application expert and renowned international speaker

The book will be published by Apress around August (2012), somewhere before or near the release of Visual Studio 11 and .NET 4.5, which should be a very exciting time.

I am posting short updates and links on Twitter as well as on this blog. You can follow me: @goldshtn

Finalization Queue or F-Reachable Queue? Find Out with SOS

Sasha Goldshtein — Sat, 25 Feb 2012 15:45:59 GMT

It’s 2012, so time for another post related to finalization. This so-often-abused CLR feature has popped up here in the past. A quick recap:

This time, we’ll see how to determine whether a particular object is in the finalization queue (which means it hasn’t been scheduled for finalization yet) or in the f-reachable queue (which means it’s waiting for the finalizer thread to run its finalizer).

Let’s fire up our trusty WinDbg and SOS and look at some heap objects:

0:003> !dumpheap -stat
...snipped…
000007ff00023b78     1340        32160 MemoryLeak.Schedule
000007ff00023aa0     1340        32160 MemoryLeak.Employee
000007fef5ff7b08      435        44400 System.String
000007fef5fe58f8      310        67120 System.Object[]
00000000005387f0      577      2680608      Free
000007fef5fffb48     1345     13432600 System.Byte[]
Total 6231 objects

Okay, what are these schedules, employees, and byte arrays running around?

0:003> .foreach (obj {!dumpheap -mt 000007fef5fffb48 -short}) {!gcroot obj; .echo -----}
…edited for clarity…
Finalizer queue:Root:000000002a021a8(MemoryLeak.Employee)->
0000000002a021c0(MemoryLeak.Schedule)->
0000000002a021d8(System.Byte[])
-----
Finalizer queue:Root:000000002a07058(MemoryLeak.Employee)->
0000000002a07070(MemoryLeak.Schedule)->
0000000002a07088(System.Byte[])
-----
Finalizer queue:Root:000000002a0bf08(MemoryLeak.Employee)->
0000000002a0bf20(MemoryLeak.Schedule)->
0000000002a0bf38(System.Byte[])
…many more of these snipped…

Now, are these Employee objects rooted at the finalization queue or the f-reachable queue? Unfortunately, !gcroot does not tell. However, !FinalizeQueue shows the queue statistics:

0:003> !FinalizeQueue
SyncBlocks to be cleaned up: 0
MTA Interfaces to be released: 0
STA Interfaces to be released: 0
----------------------------------
generation 0 has 370 finalizable objects
(0000000000d29030->0000000000d29bc0)
generation 1 has 4 finalizable objects
(0000000000d29010->0000000000d29030)
generation 2 has 8 finalizable objects
(0000000000d28fd0->0000000000d29010)
Ready for finalization 571 objects
(0000000000d29bc0->0000000000d2ad98)
Statistics:
…snipped…

Note the information on finalizable objects vs. objects that are ready for finalization. The former are in the finalization queue, the latter are in the f-reachable queue.

But now suppose that you have an individual object and you want to determine whether it’s in the finalization queue or the f-reachable queue. All you need to do is check in which array it is contained by searching memory with the s command:

0:003> s -q 0000000000d29bc0 0000000000d2ad98 0000000002a0bf08
00000000`00d29da8 00000000`02a0bf08 00000000`02a10db8
0:003> s -q 0000000000d28fd0 0000000000d29bc0 0000000002a0bf08

Okay, so 0000000002a0bf08 is in the f-reachable queue, waiting for the finalizer thread.

I am posting short updates and links on Twitter as well as on this blog. You can follow me: @goldshtn
This blog post was also cross-posted to CodeProject.