The rule can be addressed at organizational level in an interesting way. I have come across an idea of a project manager who was responsible for multiple teams dealing with significant amounts of legacy code. They introduced some kind of a gamification to the development process. The teams were supposed to register as many improvements in code as they could and a team who got the biggest number won the game. The prize was some extra budget to spend on team party. Such idea may not be applicable in all organizations, but it clearly shows how to methodically approach the problem of technical debt at management level.

Although I do not immediately recommend the idea of gamification, but I certainly recommend creating some static (not assigned to any sprint) ticket for all the improvements and encourage developers to make even smallest refactor commits under such ticket during they normal development tasks. Below I would like to show some basic indicators that in my opinion qualify for being improved as soon as they are discovered.

1. Improper naming causing an API ambiguity

   I see a few problems here. When I first saw client code calling GetValue I thought it returns some custom, domain specific type. I needed to search for a method returning string and I skipped GetValue, because it did not look like it returns a string. It was only later that I realized it actually does return a string. If it returns a string, it should be named appropriately.

   More general observation here is that we have 3 ways of converting the type into a string. In my particular case I had 10 usages of GetValue, 45 usages of the operator and 0 usages of ToString in the codebase. When talking to the maintainers, I was told there was a convention not to use the ToString method. That situation clearly shows some adjustments are needed both at the level of code quality and at the development process level. I have nothing against operator overloading, however it is not very frequently used in business code. The code readability is a top priority in such case and being as explicit as possible is in fact beneficial from the perspective of the long term maintenance.

   The unused method should obviously be removed, and the one returning a string should be named ToString. I would keep the overloaded operators, because why not, but I still am a little bit hesitant about using them in new code. It is cool language feature when you write code, but it appears not so cool when you have to read it. Even here, I would consider sacrificing the code aesthetics of the operator in favor on simplistic ToString.

2. Misused pattern causing an API ambiguity

   This one is very similar to the previous one, as it boils down to the fact, that we can instantiate an object in two ways. When I was introducing some modifications to the code, at first I was forced to answer the question: should I use the constructor or the Create method. Of course, it turns out, indeed there is a slight difference, because Create returns a result object, which is a common way to model logic in a kind of functional way. But still, at the level of API surface we do not see the difference clearly enough.

   The gist of that case is, there is a pattern in tactical (I mean, at the level of the actual source code) Domain Driven Design to use private constructors and provide static factory methods. The purpose of that is primarily to prevent default construction of an object that would render it in a default state that is not meaningful from the business point of view. Also, factory methods can have more expressive names to indicate some specific extra tasks they do.

   The constructor should be made private and the factory method can be named CreateAsResult, if the wrapper type is prevalent in the code base.

The ideas behind such improvements can actually be very simple. Some of them have to do with trivial, but extremely relevant conclusions about engineering a software. For example:

• any piece of code that slows down a programmer maintaining the code can potentially be considered not good enough
• the code is written once, but read multiple times and thus, when writing a code, we should optimize for the easiness of reading it

The vital part of that mindset of clearly expressing intention is proper naming. I highly recommend watching excellent presentation CppCon 2019: Kate Gregory “Naming is Hard: Let’s Do Better”. It helps develop a proper way of thinking when writing a code.

I came up with the following solution: let’s create a new MVC endpoint authenticated using existing attributes, but with token included in the URL. Then use a browser’s session to communicate with Hangfire Dashboard and mark a request as authenticated, if it is such. A user accesses the dashboard by navigating to the new endpoint, then if authentication succeeds, they are redirected into the main dashboard URL which is authenticated just by a flag set in the session. The biggest advantage of this solution is it requires no changes in the existing authentication and authorization mechanisms.

1. Create a new MVC controller in the back-end Web Api application. Use whatever authorization techniques and attributes are already used in the application for the API controllers

2. Enable the session mechanism. It may look strange for the Web Api, but anyways. Call app.UseSession before app.UseMvc
3. In the controller’s action set some flag in the session. This way, it will be set if, and only if the authentication and authorization succeeds
4. Do the redirect to the Hangfire Dashboard endpoint
5. Create a class that implements IDashboardAuthorizationFilter. It is the customization for the authentication of Hangfire Dashboard. Try to read the flag from the session and decide if the request is authenticated

   Use Authorization = new[] { new HangfireAuthorizationFilter() } in the DashboardOptions

6. Now the most important part that enables the existing token-based authentication to work with the Hangfire Dashboard. Create new middleware class that rewrites the token from the URL into the headers. It will allow the existing authentication mechanism do its job without any modifications

   Call app.UseMiddleware<TokenFromUrlMiddleware> before app.UseAuthentication.

Though, there are some caveats for this solution.

• The solution may require some additional session setup in multi-instance back-end configuration. By default, the session is stored in memory. Each instance will have its own copy of the session store causing the session flag to be unrecognizable between the instances, if it is not configured to use a distributed cache like e.g. Redis
• The security of the token included in the URL is disputable. However, as for my architectural drivers, it is acceptable, because the application is internal
• There are some rough edges if the application is hosted in a virtual subdirectory of the domain using Kestrel, not IIS. Please notice the Redirect action begins with / which is the root of the domain. We must adjust it accordingly if a subdirectory approach is used (append the subdirectory name). Also, we must somehow inform Hangfire about the subdirectory. If a subdirectory is used, then the real URL of the dashboard is not the main Hangfire path set in its options, but actually the path with prefixed with a subdirectory. The dashboard is a black box from our point of view, and we cannot influence the way it makes its own HTTP requests. The only way of configuring its behavior is by using its apis. We use PrefixPath property of the DashboardOptions to configure this
• In my setup I also had to use IgnoreAntiforgeryToken = true because of some errors which occurred only on the containerized environment under Kubernetes. The final settings are as follows:

• Due to the discrepancies between containerized environment and local one, it is worth considering the separate, conditionally compiled setup calls for the DEBUG local build and the RELEASE build. This way we can skip the prefixes required for the subdirectory based hosting if we run locally
• There in an interesting SO post describing the differences between the Path and PathBase properties of the HttpRequest. These are used internally by the Hangfire to dynamically generate URLs for the requests sent by the dashboard. It turns out that, these properties are used to detect the subdirectory part of the URL. They behave differently under IIS and Kestrel, unless the particular middleware is additionally plugged into the pipeline
• By default, the session cookie expires 20 minutes after closing the browser’s tab or right after closing the entire browser
• One can imagine a very unlikely corner case, when the real token is invalidated while the Hangfire Session in open. In such case, the dashboard will remain logged in. I consider those properties as acceptable, though

The story begins with SQL error: SQLEXCEPTION: Transaction was deadlocked on lock resources with another process and has been chosen as the deadlock victim.

1. In any reasonably modern version of SQL Server Management Studio there is an XEvent session system_health under Management → Extended Events. It allows for viewing some important server logs, among which xml_deadlock_report in particularly interesting. It is very important to have an access to the production instance of database server in order to be able to watch the logs.



2. In this particular case, these xml_deadlock_reports contained one suspicious attribute: isolationlevel = Serializable (4) and the SQL code was a SELECT. I would not expect my SELECTs running with Serializable isolation level.



3. The isolation level is an attribute of a connection between a client and the database server. A connection is called session in SQL Server terminology. An explicit BEGIN TRAN is not necessary for the isolation level to be applied. Every SQL statement runs in its own statement-wide transaction. However, for such narrow-scoped transactions, in practice it may not make any difference whether you raise the isolation level or not. The difference can be observed when a transaction is explicit and spans multiple SQL statements.
4. The cause of setting the serialization level to Serializable was the behaviour of the TransactionScope [1]. If you use it, it raises the isolation level. It is just a peculiarity of this very API of the .NET framework. It is good to know this.
5. SQL Server, at last in 2012 and some (I am not sure exactly which ones) later versions, does not reset the isolation level when ADO.NET disposes of a connection. A connection returns back to the connection pool [2] and is reused by subsequent SqlConnection objects unless they have different connection string.
6. The connection pool size, if the connection pooling is active, poses the limit of how many concurrent connections to a database server a .NET application can make. If there are no free connections in the pool, an exception is thrown [3].
7. Eliminating the usage of TransactionScope did not solve the issue. Even if you run SELECTs under the default Read Committed isolation level, these still issues Shared locks which may deadlock with Exclusive locks of UPDATEs. In any reasonably high production data traffic, where SELECTs span multiple tables, which are also very frequently updated, it is highly probable, that a deadlock will occur.
8. The difference between running SELECT under Serializable isolation level and Read Committed level is that in the former, the locks are kept from the moment of executing the SELECT until the transaction ends. You can observe it by manually beginning a Serializable transaction, running any SELECT and observing dm_tran_locks DMV and only then committing (or rolling back, whatever) the transaction. With Read Committed level locks are not kept until an explicit transaction ends, they are released immediately after execution of a SELECT finishes. These are the same kind of locks, Shared locks. This implies one cannot observe the difference between executing a SELECT under Serializable and Read Committed, when there is no explicit transaction and thus, there is only a statement-wide transaction which releases locks immediately after the results are returned.
9. Setting isolation level of Read Uncommitted is practically equivalent to running a SELECT WITH(NOLOCK) hint, even if you don’t explicitly open a transaction.
10. In Entity Framework a SqlConnection is opened for every materialization of the query, the results are returned, and the connection is immediately closed and returned back to the pool [5]. The connection lifetime is by no means related to the scope of DbContext object. I can see a kind of similarity between how Entity Framework uses SqlConnections and how ASP.NET makes use of threads when executing async methods. A thread is released on every await and can be used for doing something more valuable than waiting. Similarly, a SqlConnection is released right after materialization and can be used for executing different command, in different request (in case of ASP.NET) even before DbContext is disposed of.
11. It is not that obvious how to reset the isolation level of the connection. You see, every time your C# code using Entity Framework results in sending a SQL to the SQL Server, it can take different connection from the pool (if anyone knows if there is any ordering applied when retrieving connections from the pool, please feel free to comment). It may or may not be the same connection you used previously. Consequently, it is not easy to ‘catch’ the underlying connection using Entity Framework. You can call BeginTransaction every time you use DbContext, and then you are guaranteed to own the connection for all your SQL commands. But that way you are forcing opening transaction when you don’t really need one. What I recommend is to handle StateChange event of DbConnection object as described in [4]. You can do it either on opening the connection or on closing it.
12. In SQL Server you can monitor open sessions with the following query:

References:

[1]    https://stackoverflow.com/questions/11292763/why-is-system-transactions-transactionscope-default-isolationlevel-serializable

[2]    https://stackoverflow.com/questions/9851415/sql-server-isolation-level-leaks-across-pooled-connections

[3]    https://docs.microsoft.com/en-us/dotnet/framework/data/adonet/sql-server-connection-pooling

[4]    https://stackoverflow.com/questions/28442558/entity-framework-and-transactionscope-doesnt-revert-the-isolation-level-after-d

[5]    https://docs.microsoft.com/en-us/previous-versions/dotnet/netframework-4.0/bb896325(v=vs.100)#connections-and-the-entity-framework

It did not work. Or… did it? The picture above shows only generated SQL, but I promise the results show correctly the one record. The problem is, the predicate has been applied in memory after having pulled all the records from table A into memory as well. I am not going to explain what it means for any reasonably sized system. The correct way of doing this is to use Expression<Func<A, bool>>.

The explanation is in fact really obvious for anyone deeply understanding how ORMs work. The data structure which allows for inspecting a predicate on the fly and building final SQL query is Expression. There is already an infrastructure for so-called expression visitors. Please also note, that you can always get your Func from Expression<Func> by calling Compile method on it.

• An entity A can have an entity B and C
• An entity A can have only entity B
• An entity A can exist without B and C
• An entity B has not null property Active

I am implementing the domain with the following SQL. I omit foreign key constraints for brevity.

Now let’s suppose my task is to perform validity check according to special rules. I am given an Id of an entity A as an input and I have to check:

1. If the entity exists and
2. If it is valid

The existence will be checked by simply looking if corresponding row is present in the result set, and for validity check I will write simple CASE statement. These are my rules for my example data:

• A.1 exists and has active B.10 and has C.100 => exists, correct
• A.2 exists and has inactive B.20 and has C.200 => exists, incorrect
• A.3 exists and has active B.30 and has C.300 => exists, correct
• A.4 exists and has active B.40 and DOES NOT HAVE C => exists, incorrect
• A.5 exists and DOES NOT HAVE NEITHER B NOR C => exists, incorrect
• A.6 does not exist, incorrect

I write the following query to do the task:

My rules include checking if B.Active is true, so I just put this into WHERE. The result is:

AId  Correct 
---- --------
1    1       
3    1       
4    0       

The problem is, I have been given the exact set of Ids of A to check: 1, 2, 3, 4, 5, 6. But my result does not include 2, 5, 6. My application logic fails here, because it considers those A records as missing. For 6 this is fine, because it is absent in table A, but 2 and 5 must be present in the results for my validity check. The fix is extremely easy:

Now the result is:

AId  Correct 
---- --------
1    1       
2    0       
3    1       
4    0       
5    0       

It is very easy to understand, that WHERE is applied to filter all the results, no matter what my intention for JOIN was. When a record is LEFT JOINed, the condition is not met, because values from B are null. But I still need to have A record in my results. Thus, what I have to do is to include my condition in JOIN.

It is also very easy to fall into this trap of thoughtlessly writing all intended conditions in the WHERE clause.

1. It is possible to create Nuget package upon build. This option is actually available also from the VS2017 Project properties GUI. Add this code to csproj.

2. It is possible to add local folder as Nuget feed. The folder can also be current user’s profile. This one is actually not Core specific. Nuget.config should look like this:

3. You can compile for multiple targets in .NET Core compatible csproj. Please note the trailing s in the tag name. You can also conditionally include items in csproj. Use the following snippets:

   and:

   There is a reference documentation for the available targets: here.

4. The listening port in Kestrel can be configured in multiple ways. It can be read from environment variable or can be passed as command line argument. An asterisk is required to bind to physical interfaces. It is needed e.g. when trying to display the application from mobile phone when being served from development machine. The following are equivalent:

   set ASPNETCORE_URLS=http://*:11399
   --urls http://*:11399
   

5. The preferred way to pass hosting parameters to Kestrel is launchSettings.json file located in Properties of the solution root. You can select a profile defined there when running:

   dotnet run --launch-profile "Dev"
   

   dotnet run is used to build and run from the directory where csproj resides. It is not a good idea to run the app’s dll directly. Settings file can be missing from bin folder and/or launch profile may not be present there.

Tmux running under Git Bash default terminal with two shell processes

I know everyone uses Cmder, but it didn’t work for me. It hung a few times, it has way too many options, it has issues sending signal to kill a process. I gave up on using it. I work with carefully configured default Windows console and believe it or not, it serves the purpose. I also know you can use Windows Subsystem For Linux under Windows 10, which is truly amazing, but I am just talking about the cases where you need standard Git for Windows installation.

When I worked with Unix I liked GNU Screen, which is terminal multiplexer. It gives you a bunch of keyboard shortcuts to create separate shell processes under the same terminal window. The problem is, it is not available under GIT Bash. But it turns out, its alternative — Tmux is.

I did a little research and have found that GIT Bash uses MINGW compilation of GNU tools. It uses only selected ones. You can install the whole distribution of the tools from https://www.msys2.org/ and run a command to install Tmux. And then copy some files to installation folder of Git. This is what you do:

1. Install before-mentioned msys2 package and run bash shell
2. Install tmux using the following command: pacman -S tmux
3. Go to msys2 directory, in my case it is C:\msys64\usr\bin
4. Copy tmux.exe and msys-event-2-1-4.dll to your Git for Windows directory, mine is C:\Program Files\Git\usr\bin. Please note, that in future, you can see this file with the version number higher than 2-1-4

And you are ready to go. Please note, that I do this on 64-bit installations of Git and MSYS . Now when you run Git Bash enter tmux. My most frequently used commands are:

• CTRL+B, (release and then) C — create new shell within existing terminal window
• CTRL+B, N — switch between shells
• CTRL+B, a digit — switch to the chosen shell by the corresponding number
• CTRL+B, " — split current window horizontally into panels (panels are inside windows)
• CTRL+B, o — switch between panels in current window
• CTRL+B, x — close panel

This is everything you need to know to start using it. Simple. There are many other options which you can explore yourself, for example here http://hyperpolyglot.org/multiplexers.

Update 1: Users in comments are reporting the method not always works. If you have any experiences with this method please feel free to comment, so that we can figure out what are the circumstances under which it works

Update2: I managed to run this on Windows 7, Windows 2012 R2 and Windows 10. My Git installation is set up to use MinTTy console and tmux works only when run from this console, not from default Windows command line console. Still haven’t figured out what are precise requirements for this trick

In the video above, you can see, that we cannot drag and drop a file onto a PowerShell script. Conversely, we can easily do this with bash script having installed Git for Windows.

I needed to perform some trivial conversion within a SQL script, i.e. replace ROLLBACK with COMMIT. I thought I would implement it with PowerShell. I am not going to comment on the implementation itself, even though it turned out to be not that obvious. Then I realized, it would be nice, if I could drag and drop a file in question to apply the conversions on it.

This does not work with default configuration of PowerShell. I did not have time to hack it somewhere in the registry, as I assume it is doable. I switched to old, good bash shell instead.

It’s a pity I couldn’t do that with Windows native scripting technology. It is very interesting, that MinGW port of shell has been so carefully implemented, that even dragging and dropping works in non-native environment.

I recall the book Pragmatic Programmer: From Journeyman To Master. There is a whole subchapter about the power of Unix tools. The conclusion was, that over time we would come across plenty of distinguished file formats and tools to manipulate data stored with them. Some of them may become forgotten and abandoned years later, making it difficult to extract or modify the data. Some may work only on specific platforms.

But standard Unix tools like shell scripting, Perl, AWK will always exist. I should say: not only will they always exist, but also they will thrive and support almost every platform you can imagine. They work on plain text, which is easy to process. I am a strong proponent of before-mentioned technologies and I have successfully used them many times in everyday work. It sounds particularly strange among .NET developers, but this what it looks like. The PowerShell simply did not do the trick for me. Perl did. As it always does.

For the sake of the future reference I am including the actual scripts:

PowerShell script:

Bash script running Perl:

Now let’s open SSMS (BTW, since version 2016 there is an installer decoupled from SQL Server available here) with two separate tabs. Please consider, that each tab has its own connection. Then execute the following statement to increase isolation level in both tabs:

The following code tries to resemble application level function which does a bunch of possibly time consuming things. These are emulated with WAITFOR instruction. But the point is that the transaction does both SELECT and UPDATE on the same table having those time consuming things in between.

Let’s put the code in both tabs and then execute one tab and the second tab. After waiting more than 10 seconds, which is the delay in code, we will observe an error message on the first tab:

Msg 1205, Level 13, State 56, Line 8
Transaction (Process ID 54) was deadlocked on lock resources with another 
process and has been chosen as the deadlock victim. Rerun the transaction.

This situation occurred in a web application where concurrent execution of methods is pretty common. For application developer it is very easy to be tricked into thinking that having set SERIALIZABLE isolation level we magically make sequential execution of our SQL code. But this is wrong. By setting SERIALIZABLE level we do not automatically switch the behavior of the code wrapped with transaction to the behavior of lock statement known from C# (technically lock is a monitor).

I would advise having a closer look at the instructions wrapped in transaction. In real application the execution flow is much more `polluted` with an ORM calls, but my simplified code from above just tries to model common scenario of reads followed by writes. What happens here is that SQL Server takes a reader lock on the table after executing the SELECT. When we execute the code again in another session we have one more reader lock taken on the table. Now when the first session passes waitfor and comes to UPDATE it needs to take a writer lock and waits (I am purposely using generic vocabulary instead of SQL Server specific one — these locks inside database engine all have their names). We observe the first tab waits more than 10 seconds. This is because when the first tab reaches its UPDATE it needs to take writer lock, but it is locked by the SELECT in the second tab. Conversely, the second’s tab UPDATE waits for the lock taken by the SELECT in the first tab. This is deadlock which fortunately is detected by the engine.

The problem is caused by the lock taken witch SELECT instruction having SERIALIZABLE isolation level set. The lock is not taken in this place with READ COMMITED which is the default level.

I am writing about this for the following reasons:

• This is very simple scenario from the application point of view: to read some data, update the data, do some things and have all of this wrapped with a transaction.
• It is very easy to make wrong assumption, that SERIALIZABLE level guarantees that our SQL code will be executed sequentially. But it only guarantees, that if the transactions execute, their observable effects will be as if they both executed sequentially i.e. one after another. But it is your job to make them actually execute not run into a deadlock.

Error message from TypeScript compiler

This one attacked me yesterday by the end of the day. This is a TypeScript project with some instrumentation done with gulp. The command shown attempts to launch TypeScript compiler. The console is standard git bash for Windows. I didn’t realize what was going on there at the first sight. The messages were confusing and mysterious. I made sure I didn’t have any changes in the repository comparing to the HEAD. I opened the folder in Visual Studio Code and everything was fine. It was only the compiler which saw the problems.

The next day I figured out there is a small detail in my path. Just take a look at directory names casing:

TypeScript compiler ran successfully

After changing the directory using names with proper casing everything was fine. What actually happened was that, I somehow opened the git bash providing the current directory name using lower case letters. It is not that difficult to do that. This is a flaw in the MINGW implementation of bash, I think. It just allows you to change the directory using its name written with whatever case. That is not the problem itself, because cmd.exe allows you to do so as well. The problem is that it then stores the path (probably in PWD environment variable). Some cross-platform tools, which are case sensitive on other platforms, when executed from git bash with mismatched working directory casing may begin to treat such path as separate one, different from the original one. Especially tools which process files using their paths relative to the current directory, like TypeScript compiler for instance.

This can possibly be a wider class of problems and I guess there are other development tools which behave like that when launched from git bash under before mentioned conditions.

It prints A does activity despite ia variable storing reference to an object of type B and also despite explicitly declaring hiding of base method. It was not clear to me why is it so. It is obvious that the type B has its own implementation, so why is it not run here? To overcome this I initially created base class declared as abstract:

It prints B does activity, but it also is overcomplicated. Then I came up with simpler solution — it turns out we have to explicitly mark class B as implementing IActivity.

It prints B does activity, but it is not perfect. Method hiding is not a good practice. Finally I ended up with more elegant (and the simplest, I guess) solution:

In here we are using virtual and override modifiers to clearly specify an intention of specializing the method. It is better than previous one, because just by looking at class A we are already informed that further specialization is going to occur.

The real usage scenario is that we have an interface representing Entity Framework Core context. We have two distinct implementations: one uses real database and the other uses in-memory database for tests. The latter inherits from the former because what inheritance is all about is copying code. We just want to have the same methods for in-memory implementation like for regular one, but with some slight modifications e.g. in methods executing raw SQL. We also have to use an interface to refer to these objects, because this is how dependency injection containers work.

As you can see, what might seem purely theoretical, actually is used in real world line of business application. Although I have been pretty confident I understand the principles of inheritance in object oriented programming since my undergrad computer science lectures, but as I mentioned, I was surprised discovering that we have to explicitly put : IActivity on class B. The implementation has already been there. Anyway, this example encourages me to be always prepared to verify assumptions I make.

This will compile and run, however it will not distinguish ProductCodeCentreNumPair instances. I.e. it will not do the grouping, but produce a sequence of IGroupings, each for corresponding source item. The reason is self-evident, if we try to think for a while. This custom class does not have custom equality comparison logic implemented. Default logic is based on ReferenceEquals so, as each separate object resides at different memory address, they all will be recognized as not equal to each other. Even if they contain the same values in their fields (strings behave like value types, although they are reference types). I used the following set of overridden methods to provide my custom equality comparison logic to solve the problem. It is important to note, that GetHashCode is also needed in order for the grouping to work.

Alternatively you can use anonymous types, I mean:

will just work. This is because instances of anonymous classes have automatically generated equality comparison logic based on values of their fields. Contrary to ReferenceEquals based implementation generated for typical named classes. They are most frequently used in the context of comparisons, so it seems reasonable.

One more alternative is to use a structure instead of a class. But structures should only be used if their fields are value types, because only then you can benefit from binary comparison of their value. And even having structs instead of classes requires implementing custom GetHashCode. By not implementing it, there is a risk that 1) the auto-generated implementation will use reflection or 2) will not be well distributed across int leading to performance problems when adding to HashSet.

After an investigation and some experiments it turned out that bad data structure was to blame. The loop did some lookup-heavy operations over List<T>. Which are obviously O(n), as list is implemented over array. Substitution of List<T> for Hashset<T> caused dramatic reduction of execution time to a dozen or so of seconds. This is not as surprising as it may seem because Hashset<T> is implemented over hashtable and has O(1) lookup time. These are some data structures fundamentals taught in every decent computer science lecture. The operation in a loop looked innocent and the author did not bother to try to anticipate future load of the logic. A programmer should always keep in mind the estimation of the amount of data to be processed by their implementation and think of appropriate data structure. By appropriate data structure I mean choice between fundamental structures like array (vector), linked list, hashtable, binary tree. This can be the first important conclusion, but I encouraged my colleagues to perform profiling. The results were not so obvious.

Although the measurement of timing of the entire loop with 16 thousand iterations showed clearly that hashtable based implementation performs orders of magnitude better, but when we stepped over individual loop instructions there was almost no difference in their timings. The loop consisted of several .Where calls over entire collection. These calls took something around 10 milliseconds in both List<T> and Hashset<T> implementations. If we had not bothered to measure entire loop execution, it would have been pretty easy to draw conclusion there is no difference! Even performing such step measurement during the development can me misleading, because at first sight, does 10 milliseconds look suspicious? Of course not. Not only does it look unsuspicious but also it works well. At least in test environment with test data.

As I understand it, the timings might have been similar, because we measured only some beginning iterations of the loop. If the data we are searching for are at the beginning of the array, the lookup can obviously be fast. For some edge cases even faster than doing hashing and looking up corresponding list of slots in a hashtable.

For me there are two important lessons learned here:

• Always think of data structures and amount of data to be processed.
• When doing performance profiling, measure everything. Concentrate on amortized timings of entire scope of code in question. Do not try to reason about individual subtotals.

Right click on toolbar ⟶ Customize ⟶ Commands tab ⟶ Toolbar ⟶ Debug ⟶ Add command ⟶ Debug ⟶ Detach all

An important caveat is also to use Multiple startup projects configured in the solution properties. A web application typically consists of many projects, so it is convenient to run them all at once.

Having IIS Express continuously running in the background I would like to introduce how to quickly start debugging. Here I also prefer custom toolbar button:

Right click on toolbar ⟶ Customize ⟶ Commands tab ⟶ Toolbar ⟶ Standard ⟶ Add command ⟶ Debug ⟶ Attach to process

Or even better, keyboard binding:

Tools ⟶ Options ⟶ Environment ⟶ Keyboard ⟶ Debug.AttachtoProcess

And now we face the most difficult part: choosing correct iisexpress.exe process. It is likely there will be a few of them. We can search for the PID manually by right clicking IIS Express icon ⟶ Show All applications and we can view the PID. But this may drive you crazy as you have to do this many times. I recommend simple cmd script which invokes PowerShell command to query WMI:

@echo off
powershell.exe -Command "Get-WmiObject Win32_Process -Filter """CommandLine LIKE '%%webapi%%' AND Name = 'iisexpress.exe'""" | Select ProcessId | ForEach { $_.ProcessId }"

Here I am searching for the process whose path contains “webapi” string. I have to use triple quotes and double percent signs because that is how cmd’s escaping works. The final pipe to ForEach is not necessary, it serves the purpose of formatting the output to raw number, not a fragment of a table. I always have running cmd window so I put this little script into PATH variable and I can view desired PID instantaneously. By the way, Windows Management Instrumentation is tremendously powerful interface for obtaining information about anything in the operating system.

Knowing the PID you can scroll down the process list by simply pressing i letter and then you can visually distinguish the instance with a relevant PID.

We can see the simplest usage of Entity framework here. There is a Test class and a TestChild class which contains a reference to an instance of Test named Parent. This reference is marked as virtual so that Entity Framework in instructed to load an instance in a lazy manner, i.e. upon first usage of that reference. In DDL model obviously TestId column is a foreign key to Test table.

I create an entity object, save it into database and then I retrieve it at line 21. Because the class uses virtual properties, Entity Framework dynamically creates some custom type in order to be able to implement lazy behavior underneath.

Now let’s suppose I have a need to modify something in an object retrieved from a database. It turns out I can modify Value property, which is of pure string type. What is more, I can also modify Parent property, but… the modification is not preserved!. This program throws at line 25 because an assignment from line 24 is silently ignored by the framework.

I actually have been trapped by this when I was in need of modifying some collection in complicated object graph. I am deeply disappointed the Entity Framework on one hand allows modification of non-virtual properties, but on the other hand it ignores virtual ones. This can make a developer run into a trouble.

Of course I am aware it is not good practice to work on objects of data model classes. I recovered myself from this situation with AutoMapper. But this is kind of a quirk, and a skilled developer has to hesitate to even try to modify something returned by Entity Framework.

The link to before-mentioned SO:
http://stackoverflow.com/questions/18900308/angularjs-dynamic-ng-pattern-validation

I have created a codepen of this case:

[codepen_embed height=”265″ theme_id=”0″ slug_hash=”wWwEdw” default_tab=”js” user=”przemsen”]See the Pen AngularJS ng-pattern — an excellent example of duck typing by Przemyslaw S. (@przemsen) on CodePen.[/codepen_embed]

In the codepen you can see a JavaScript object set up in ng-pattern directive, which should typically be a regular expression. The point is that AngularJS check for being a regular expression is simply a call to .test() function. So we can just attach such test function to an object and implement whatever custom validation logic we need. In here we can see the beauty of duck typing allowing us to freely customize behavior of the framework.

I said this could be a beginning of a discussion on how to actually understand duck typing. It is not that obvious and programmers tend to understand it intuitively rather than precisely, though it is not a term with any kind of formal definition. I recommend Eric Lippert’s blog post on the subject: https://ericlippert.com/2014/01/02/what-is-duck-typing/.

Let’s try to parse a JavaScript Date object:

What draws my attention is the time value. It is 01:00 which may look strange. But it is not if we correlate this value with time zone information which is stored along with JavaScript object. The time zone information is an inherent part of the object and it comes from the browser, which obviously derives it from the operating system’s culture settings. It turns out these two pieces of information are essential when making AJAX calls, because then .toJSON() method is called. I am making this assumption based on the example behaviour of ngResource library, but other frameworks or libraries probably do the same, because they must somehow convert JavaScript Date object to a universal text format to be able to send it via HTTP. By the way, .toJSON() returns the same result as to .toISOString().

What we have got here is UTC-normalized date and time value. The time zone offset of actual time value used with time zone information allow us to normalize the date when sending it to a server. The most important thing here is that the values stored in Date object are expressed in local time zone i.e. the browser’s one. This implies some strange consequences like the one of being in UTC-xx:xx time zones. Let’s try the same example after setting time zone to UTC-01:00.

The problem here is that we have actually ended up with parsed values which are different from their original textual representation i.e. March 1st versus February 28th. But it is still OK providing that our date processing logic relies on normalized values:

However, it can be misleading when we try to get individual date components. Here we try to get day component.

But in general the object still can serve the purpose if we rely on normalized values and call appropriate methods.

The problem is that not all Date constructors behave in the same way. Let’s try this one:

Now the parsed date although it still contains time zone information derived from the operating system, but it does not contain time value modified by the corresponding offset. In the first example we have 1am time which corresponds to GMT+01:00, here we have just 00:00 time and, of course, we still do have GMT+01:00 time zone information. This time zone information without correctly shifted time value is actually catastrophic. Look what happens when .toJSON() is called:

The result is wrong date sent to a server. This is not the end of the observation. The same phenomenon can also happen in the other way round, i.e. when we are transferring date values from a server to a client. Now let’s assume the server sent the following date and we are parsing it. Please keep in mind that the actual process of parsing may happen implicitly in some framework’s code, for instance when specifying data source for Kendo grid. So the one who parses it for us can be a framework’s code.

As we see, this constructor results in shifted time value just like the Date(“2015-03-01”) one. But when considering displaying these retrieved values we inevitably have to answer the question, whether we aim at showing local time or the server time. We have to remember that in case when the client’s browser is in GMT-xx:xx time zone and we try to show the parsed value (like in c.getDate() example), not the normalized one, this may result in wrong date displayed in front of the user. I say `may`, because this can really be a desired behaviour depending on the requirements. For example, in Angularwe can enforce displaying normalized value by providing optional time zone parameter to $filter('date').

Here we do not worry about internal, actual component values of object c whose prototype is Date. It internally may store February 28th but it does not matter. $filter is told to output values for UTC time. It is also worth mentioning that the Date constructor also assumes that its argument in specified in UTC time. So we populate the Date object with UTC value and return also UTC value not worrying about internal representation which is local. This approach results in the output date and time being equal to the intended input one.

As a conclusion I should write some recommendations on how to use Date object and what to avoid. But honestly I cannot say I gained enough knowledge in this area to make any kind of guidelines. I can only afford making some general statements. Just pay attention to what your library components like, for instance, a date picker control operates on. Is their output a Date object or string representation of the date? What do you do with this output? Is their input a Date object or a string representation and they do the parsing on their own? Just examine carefully and do not blindly trust your framework. I personally do not accept situation when something works and I do now know why it does. I tend to dig into details and try to find the nitty gritty. I have always believed deep research (even one which takes much time) and understanding of underlying technology is worthwhile and I often recall the post by Scott Hanselman who also appears to follow this principle.

I had several attempts to clean up and organise this huge collection. I have tried to remove dead ones and to group them in folders. I have tried using keywords and descriptions to be able to search more effectively. But with no success. Now I have something about dozen of folders, but I still find myself in trouble when I need to search for particular piece of information. The problem boils down to that: I absolutely remember what the site is about, I am absolutely sure I have it in my collection but I cannot find it because either it has some strange title or words in URL are meaningless (Firefox searches only within titles and urls, because obviously that is all it can do).

I realized I need a tool which is much more powerful when it comes to bookmarks searching. I could not find anything to satisfy my requirements so I implemented it myself. Today I am introducing BookmarksBase which is an open source tool written in C# to solve this issue.

BookmarksBase embraces a concept that may seem ridiculous: why don’t we pull all textual contents from all sites in bookmarks. Do you think it is lots of data? How much it would be? Even if you were to sacrifice a few hundreds of megs in order to be able to search really effectively, isn’t it worth that space?

Well, it turns out it takes much less space than I originally expected and the tool works surprisingly fast, although it is implemented in managed code without any distinguished optimizations. First we have to run separate tool to collect data (BookmarksBase Importer). Downloading + parsing takes maybe a minute or two. Produced index file containing all text from all sites in bookmarks, which I call bookmarksbase.xml in my case is only 12 MiB (over 1000 bookmarks). Then we can run BookmarksBase Search that allows us to perform actual searching within contents/addresses/titles. Surely, when you have bookmarksbase.xml created you can run whatever serves the purpose for you e.g. grep, findstr (in Windows) or any kind of decent text editor that can handle big amounts of text. I crafted XML so that it can be easily readable by human: there is new lines, and the text is preserved in nice column of fixed width (thanks to Lynx — see source for details).

More details and download link are available on GitHub

However, when you work as a .NET developer, chances are sooner or later you will come across a solution implemented with PowerShell. It could be, for instance, a deployment script and you will have to maintain it. This happened to me a while ago. Although modification that I committed was relatively simple and I made it up rather quickly with little help of Google, I decided to dig into the subject and check few more things out. What I found after a bit of random research was quite impressive to me. I would like to share three main features I found so far and I consider valuable in a scripting technology. At the bottom of this post I also put some code snippets for quick reference how to accomplish particular tasks.

1. Out-GridView

In PowerShell you can manipulate format of the output in many ways. You can generate HTML, CSV, white space formatted text tables etc. But there is also an option to view output of a command with WPF grid that has built-in filter. Look at the effect of Get-Process | Out-GridView command — this is functionality you get out of the box with just a few keystrokes!

Out-GridView

2. Embedding C# code

This feature seems quite powerful. If you need more advanced techniques in your script you can basically implement them inline using C# and then just invoke them.

Add-Type @'
using System;
using System.IO;
using System.Text;
      
public static class Program
{
    public static void Main()
    {
        Console.WriteLine("Hello World!");
    }
}
'@
 
[Program]::Main()

3. XML parsing done simply right

Any time I had to do some XML parsing in my scripts using other languages I always felt somewhat confused. This is not sort of things that you just recall from your head and type as a code. You have to use specific APIs, you have to call them in specific way, in specific order etc. I do not mean this complicated in any way, it is not, but it is cumbersome in many languages. I always had to look things up in a cheat-sheet. Not any more From now on, I will always lean toward the simplest, and perhaps basically the best implementation of XML parsing:

$d = [xml][/xml] "12"
$d.a.b

This outputs 1. Yes, it is as simple as that. You basically call member properties with appropriate names that match XML nodes.

I am sharing these features because I did not imagine a scripting language can offer something as powerful. And this possibly is only a tip of an iceberg, as I just scratched the surface of PowerShell world. I also suggest checking out little script I wrote to explore PowerShell functionalities: managesites.ps1. It may be useful for ASP.NET developers — it allows you to delete sites from IIS Express config file.

Miscellaneous code snippets:

• if (test-path "c:\deploy"){ "aaa" }
• $f="\file.txt";(gc $f) -replace "a","d" | out-file $f — this one is particularily important, because equivalent functionality of in-line editing in MinGW implementation of Perl and SED seems not to work correctly
• foreach ($line in [System.IO.File]::ReadLines($filename)){ }
• -match regex
• ( Invoke-WebRequest URL | select content | ft -autosize -wrap | out-string )
• reflection.assembly]::LoadWithPartialName("Microsoft.VisualBasic") | Out-Null
$input = [Microsoft.VisualBasic.Interaction]::InputBox("Prompt", "Title", "Default", -1, -1);
• foreach ($file in dir *.vhd) { }
• Set-ExecutionPolicy unrestricted

Indeed, there are reasonable technical reasons why even powered off machine is billed too. When you create a virtual machine you consume data center resources and they have to remain allocated for you e.g. IP address, CPU cores, storage etc. It does not matter if it is running as this resources still must be reserved and ready for you.

The solution for this problem is to use Azure Powershell to control your virtual machines. The default options of stopping command does also what is called deallocation and then the payment counter stops.

Below I present quick reference of relevant commands.

1. You need to “log in” to your Azure account from PowerShell. You do this either with Import-AzurePublishSettingsFile filename or with Add-AzureAccount commands. Use the former if you would like to use profile settings file downloaded from the portal, and use the latter if you prefer to just type Microsoft account credentials and have the shell store them for you. In both cases credentials are stored in C:\Users\**name**\AppData\Roaming\Windows Azure Powershell.
2. Use Get-AzureSubscription to list your subscriptions.
3. Use Select-AzureSubscription -SubscriptionName **name** to switch the shell to apply following commands to this subscription.
4. Use Get-AzureVM to list your virtual machines, their names and their states.
5. Use Stop-AzureVM -ServiceName **name** -Name **name** to shut down and deallocate a virtual machine.
6. Use Start-AzureVM -ServiceName **name** -Name **name** to power on a virtual machine.

When you close the shell, and open it again you do not have to log in to your Microsoft Account again, but before you are able to control virtual machines you have to select subscription first.

.gitconfig

https://github.com/przemsen/main/blob/master/configs/.gitconfig (raw)

.bashrc

https://github.com/przemsen/main/blob/master/configs/.bashrc (raw)

.vimrc

https://github.com/przemsen/main/blob/master/configs/.vimrc (raw)

git-prompt.sh — Git Bash for Windows

https://github.com/przemsen/main/blob/master/configs/git-prompt.sh (raw)

Main GitHub repository

https://github.com/przemsen/main

https://github.com/przemsen.

And the first and very basic project is:

https://github.com/przemsen/WebThermometer.

WebThermometer

WebThermometer is a WPF application to be used as a desktop gadget. It repeatedly downloads (default is 5 min. interval) current temperature from arbitrary web site and displays it. I personally find it useful as I like to observe current weather conditions right from my computer. I tried to write in a way so that it can easily be modified for use with other data sources. You can also download already compiled and ready to run version from my Polish blog.

My plan is to successively select some of my entire projects and some code snippets which in my opinion are and/or will somehow be valuable to show and demonstrate. You can freely modify and recompile all of the published code providing that you specify it has originally been authored by me.

PS. Today auto updating mechanism of my WordPress failed (apparently this sometimes happens) and I ended up with damaged entire installation. I restored from backup and I apologize for deleting few comments since last 2 months.

This code works and does what it supposed to do. However, I had a slight inconvenience while debugging it. I tend to frequently use Visual Studio DEBUG->Exceptions->CLR Exceptions Thrown (check this out!) functionality which to me is invaluable tool for diagnosing actual source of an exception. The code base relied heavily on this very ConvToInt method, thus it generated lots of exceptions and caused Visual Studio to break in with the debugger over and over again. I then had to disable CLR Exceptions Thrown to protect myself from being hit by flying exceptions all the time. Having switched this off I ended up with somehow incomplete diagnosing capabilities. It is bad either way. So, what I did was basically simple refactoring:

This method also works. One can even argue for better performance of this code, because throwing exceptions is considered to be slow. And this is also correct. Although performance was not key factor here (for line of business applications rarely is), but I measured it anyway. I ran both methods in a for loop 5 million times in release mode having wrapped them with appropriate calls to the methods of Stopwatch class. The results are surely not surprising. For valid string representations of a number, the former method (i.e. one using System.Convert) gave the average result of

663 milliseconds

and the latter (i.e. one using TryParse) gave the average result of

642 milliseconds

We can safely assume both methods have the same performance in this case. Then I ran the test with a not valid string representation of a number (i.e. passing “x” as an argument). Now the TryParse version gave the average result of:

546 milliseconds

And the System.Convert version, which indeed repeatedly threw exceptions gave the (not average, I ran this once) result of

233739 milliseconds

That is a huge difference in 3 orders of magnitude. Then I was fairly convinced my small and undoubtedly not impressive refactoring was right and justified. Except that it is not correct. It has worked well and has been successfully tested. But after a few weeks, when a different set of use cases was being tested, the application called ConvToInt with -1 in the second argument. It turned out, that the method returned 0, not the -1 for invalid string representations of a number. What I want to convey here is:

TryParse sets its out argument to 0, even if it returns false and did not successfully convert a string value to a number.

I scanned the code base and have found this pattern a few times. Apparently I was not the only programmer to not know this little fact about TryParse method. Of course, it is well documented (http://msdn.microsoft.com/en-us/library/f02979c7.aspx). The problem with this very API to me seems even more serious. The 0 value is supposed to be the most frequently used number value when it comes to string conversion failure in general. However, in a construct like this above, it comes from TryParse itself, despite the fact that it is provided by the caller and, more importantly, is primarily expected to be used as a default number value in case of failure. One can easily get into trouble when he or she expects (and passes as argument) different default value, e.g. -1 and still receives 0 because TryParse works this way by definition. Obviously the solution here is to add an if statement:

The performance does not get significantly worse because of this one conditional statement, I measured it and it is roughly the same.

The lessons learned here:

• Exceptions actually ARE EXPENSIVE. This is NOT a myth.
• Do not rely on the value passed as out variable to TryParse method in case of a failure. Always back up yourself with an if statement and check for the failure.
• More general one: learn the APIs, go to the documentation, do not simply assume you know what the method does. Even if it comes to basics. The descriptive and somewhat verbose method name can still turn into an evil when ran under edge cases. Always be 100% sure about what is the contract between the API authors and you, i.e. what the API actually does.

The solution is to define custom tool in .gitconfig with Kdiff3 executable and custom command line parameters. Here is the configuration which I use on Windows:

[merge]
    tool = kdiff3NoAuto
    conflictstyle = diff3

[mergetool "kdiff3NoAuto"]
    cmd = C:/Program\\ Files\\ \\(x86\\)/KDiff3/kdiff3.exe --L1 \"$MERGED (Base)\" --L2 \"$MERGED (Local)\" --L3 \"$MERGED (Remote)\" -o \"$MERGED\" \"$BASE\" \"$LOCAL\" \"$REMOTE\"

1. WCF Non-HTTP Activation service must be installed in Add/Remove programs applet of the Control Panel. It is not obvious, that it is a component of the operating system itself, not of the IIS.
2. The TCP listener processes must be running. Check netstat -a to see if there is a process listening on the port of your choice (the default is 808), check the following system services and start them if need be: Net.Tcp Listener Service and Net.Tcp Port Sharing. I have observed cases, where those services were unexpectedly shut down, e.g. after restart of the operating system.
3. IIS management: the application must have net.tcp protocol enabled in its properties, as well as the site must have bindings for that protocol configured. If you have large number of services you can use my simplistic C# program which parses the IIS global configuration file — applicationHist.config. Link to my OneDrive
4. If this is first try of enabling the net.tcp protocol, run the following tools to ensure the components of the .NET Framework are correctly set up: c:\Windows\Microsoft.NET\Framework\v4.0.30319\aspnet_regiis.exe -i and c:\Windows\Microsoft.NET\Framework\v4.0.30319\servicemodelreg.exe -ia. Use Framework64 for 64 bit system.
5. Make sure that you are not running on the default endpoint configuration. The default configuration can be recognized in the WSDL of the service. It contains <msf:protectionlevel>EncryptAndSign</msf:protectionlevel> code which is responsible for default authorization settings. These defaults manifest themsevles in a strange symptom of the service working in Test Client and not working in target client application. It is caused by Test Client having successfully recognized default binding configuration from WSDL whereas the target application uses your custom configuration and it is very likely these two do not match.
6. Check for the equality of the service names in an .svc file and in the Web.config file (assuming that declarative binding is used instead of programmatically created one) in section <system.serviceModel> -> <services> -> <service>
7. Make sure that IIS has created a virtual directory for the application. Create it from Visual Studio by pressing appropriate button in the application’s properties.

In my every day work I tend to use command prompt a lot. I have both cmd.exe and bash (from Git for Windows) opened all the time. My typical environment comprises numerous directories, I mean more than one hundred. Many of them share parts of their names. The names are long, dozens of characters. I have to move between them over and over again. The problem is that it is not feasible to type longish directory name many times manually.

Now, let’s suppose we have the following (shortened, of course) subdirectories structure of a directory which we are in at the moment:

..
aa
bbaa
ccaadd
eeaaff

When I would like to change the directory to bbaa in cmd.exe I type cd *aa* <Tab> <Tab>, I get the second result of auto completion, I press <Enter> and I have moved to bbaa. If I press <tab> three times, I get ccaadd. And when I press <tab> four times, I get eeaaff. This feature is brilliant. The auto completion works with wildcards and matches not only beginnings of a name. Last, but not least, it allows to cycle through suggestions while in-line editing a command.

The most important part here is: not only beginning of a name (which is, as far as I know, the behavior of a typical Unix shell) AND also ability to have the suggestions inserted in place, not only displayed them below the command prompt.

A Unix shell also does match wildcards, but only displays matched names. It does not offer (or I am not aware of it) a way to instantaneously pass matched name to a command. It only lists relevant suggestions and a user have to then manually re-edit the command so that it has desired argument. cmd.exe is better in that it allows a user to cycle through suggestions while in-line editing command argument. Which is great when it comes to long names of which only some parts can be conveniently memorized by a human.

I propose the following function which could be appended to .bashrc.

function cdg() { ls -d */ | grep -i "$1" | awk "{printf(\"%d : %s\n\", NR, \$0)}"; read choice; if [ "$choice" == "0" ]; then : ; else cd "`ls -d */ | grep -i \"$1\" | awk \"NR==$choice\"`"; fi; }

It is a simplistic function that searches through ls results with grep, parses them with awk and finally picks one of them and calls cd.

Now we can type cdg aa and we get all possible choices:

1 : aa/
2 : bbaa/
3 : ccaadd/
4 : eeaaff/

We simply type the number and we are done being moved to the desired directory. Without the need to manually re-enter the cd command with proper argument. Obviously, in cmd.exe we get this nice auto completion for every command typed in the interpreter, and my solution only solves changing directory use case in bash.

2014.02.02 UPDATE 1: After some deeper research, it turned out that the behavior of cmd.exe can be achieved in bash as well. The following line should be included into .bashrc:

bind '"\t":menu-complete'

However, my solution still serves the purpose as it uses grep -i which makes it case insensitive and thus renders it useful.

2014.02.06 UPDATE 2: I have experienced the second reason my solution is still relevant. It is much faster than pressing <tab> and waiting for the shell to suggest names. This can be observed in an environment with number of directories greater than a few, where MinGW tooling tends to be slow in general.

The error message gives absolutely no clue what the real problem is. After some research I was aware of the existence of this limitation, but I could not believe the path issue can end up with such error message. Eventually, I decided to conduct in-depth investigation with some advanced debugging tools to confirm the root cause of the problem. I followed the great advice from Case of the Unexplained series by Mark Russinovich: when in doubt, run Process Monitor. The picture below shows file system activity of devenv.exe process when it is opening solution containing suspicious projects.

• We can see the querying directory event and part of its results.
• The results comprise file names in alphabetical order. These files should be opened by Visual Studio and thus the appropriate project references should be loaded. All of these files are expected to take part in consecutive file system events.
• File names in green frames are both in the result list of querying directory and are then opened by the devenv.exe process. As expected.
• File names in red frames are in the result list of querying directory, but are missing from file system activity events further (second window in the background). And this causes the problem.
• All files that are missing have path length longer than 260 characters.
• All files that are correctly opened by devenv.exe and loaded into the project and whose file system events are visible in Process Monitor have path length shorter than 260 characters. Obviously, the example shows only some of them, but I analyzed them all to draw the conclusion:

This proves that project references that point to dependencies with path length longer than 260 characters were not loaded and thus prevented the whole project from being loaded properly. After moving the solution to the root directory of the drive (the single letter, e.g. C, is indeed the shortest directory name possible) the problem was solved.

To wrap up:

1. Be aware that 260 characters limit for path does exist in Windows operating system.
2. The observable symptoms of hitting this limitation probably will not help you solve the problem.
3. The solution in most cases could be to rename one or more directories or to make a symbolic link in the root directory of the drive.
4. The problem described above occured under Windows 7 SP1 64-bit and Visual Studio 2012 Professional.
5. Do not hesitate to use Process Monitor. It is incredibly powerful tool when it comes to solving wide range of operating system problems.

However when the script was run by cron, it failed at the line where it prints some string containing Unicode characters. The line executes normally when running from the shell. I suspected there is some issue with the standard output of the processes run by the cron, because in such case there is no meaningful notion of standard output.

As it turns out, programs executed by cron have no terminal attached to their standard output. According to this Stack Exchange post the standard output is sent to the system’s mail system and thus delivered to a user this way. One can easily verify this by issuing tty command in the cron and redirecting the output to a file. Something similar to this is not a terminal (message translated directly from my system with Polish locale) should be observed.

The further explanation goes as follows: if there is no terminal attached to the process, the Python interpreter cannot detect the encoding of the terminal (it has nothing to do with the system’s environmental variables describing locale. They are all global and are part of the process’ environment, but they do not affect the non-terminal device being attached to standard output). You can verify this by running a Python script that tries to output terminal’s encoding: print sys.stdout.encoding. The None can be observed. So, the interpreter falls back to Ascii encoding and crashes when printing Unicode characters.

The solution in this case was to enforce the interpreter to use Unicode encoder for standard output.

UTF8Writer = codecs.getwriter('utf8')
sys.stdout = UTF8Writer(sys.stdout)

The output is discarded at all, but these lines prevent the interpreter from using default Ascii encoder which is not appropriate for printing Unicode string.

These are my quick observations of what to do and what not to do when writing either shell scripts or interpeter scripts under Unix/Linux environment.

• DO NOT USE DOT CHARACTER in script’s code. Use variables, assign them a value once and then refer to them in the code. Relying on external assumptions that script will be executed in certain directory is bad and will hurt. It is extremely likely you will forget the (in fact, unnecessary) requirements when executing script elsewhere, e.g. from cron.
• If you absolutely have to refer to some file located in the same directory as the script itself, consider writing another script whose solely purpose is to change the directory to meet target’s expectations and then immediately execute it. Although it would be strange to write shell script to execute another shell script, this technique can be used with other interepreted languages e.g. Python or Perl scripts.
• DO NOT USE TILDE CHARACTER in script’s code. It has not obvious expansion rules. For instance, it works at the beginning of the word, but not in the middle. That also means, it is very likely to forget the rules and have the tilde character not expanded in location where it is expected to. Use the value of HOME variable and expand it in one of traditional ways.

I have already come across learning materials that suggest the otherwise. Let’s have a look at [1] (my own translation from Polish):

With a using clause we end up having code which is proof against exceptions

and [2]:

A finally block can be used to handle any exception

In my opinion the fact, that in the very case of try..finally apart from catch an exception is simply thrown out of the scope, is not stressed enough in the literature and claims like these above can be misleading.

Going a little bit further, I consider this as a little semantic pitfall in the language. When we think of a try statement, we immediately recall exception handling mechanism. However this time this is not the case. Maybe other languages have better (i.e. more meaningful) way of expressing the intent of a code being executed at the end of a scope. Have a look at scope(exit) and scope(failure) instructions in D language in [3].

[1]. Polish magazine “Programista”, issue 6/2013 (13) p. 26

[2]. “Programming in C#, A primer, second edition”, chapter 18.8

[3]. Three Unlikely Successful Features of D

As a C# developer, you are hopefully aware of the outer variable trap issue. Yet, in the version 5.0 of the language, the behaviour of a closure has been slightly altered. Let’s take a look at first (regarding comment “switches”) half of the following code:

Now let’s try to compile it using version 3.5 and 4.5 of the compiler. This assumes default installations, the file is named Program.cs and, of course, as 4.5 version of run-time is in-place update, it resides in the directory named after 4.0.

1. c:\Windows\Microsoft.NET\Framework\v3.5\csc.exe Program.cs && Program
2. c:\Windows\Microsoft.NET\Framework\v4.0.30319\csc.exe Program.cs && Program

The former example results in 1 1 printed into the console and the latter results in 0 1 printed. The 1 1 result is caused by typical outer variable trap, where the lambdas are kind of bound to the captured variable itself (which in the end has a value of 1), but not to its value at the time of creation of the lambda. The breaking change introduced in version 5.0 of the language brings the behaviour to what actually might have been expected — capture the value indicated by the sequence of the code being executed. However, this works only inside foreach loop.

By switching the comments (deleting the first slash) you can prove standard for loop behaves exactly the same in both versions of compiler and results in printing 2 2 indicating outer variable trap.

I have to admit I am really impressed by the ideas of Pi use cases that people come up with all around the world (e.g. this list). Raspberry Pi is a microcomputer that almost every engineer passionate about computer science and/or electronics could hardly resist playing around with. Yet, I do not have plenty of creative ideas what to do with it. But, as it turns out, it can perform extremely well doing even simple tasks.

I began my set up with downloading and flashing Raspbian Linux distro into rather high class SD card. It is important to pay attention to card’s class because difference between 3-4 MB/sec and 10 MB/sec transfer speed does matter. It is pretty easy to flash Raspbian and to do the initial set up. You do not have to install it, because you are flashing ready to run image of working system. Then I installed additional packages with services:

• VNC server allows me to detach monitor, keyboard and mouse from the device and to connect to its desktop remotely. This tutorial was the starting point for me.
• SSH server is also a must-have. I can can easily login to the shell with Putty or from Android device using JuiceSSH.
• Last but not least, I started Samba server, which is the main “server role” for my Raspberry Pi.
• I also started nginx web server to be able to access shared folder with a web browser.

I use my Raspberry Pi as a file exchange server between all my devices including laptops and Android devices. As for me, it is extremely useful. I do not have to power on my laptop to send a file to or from the tablet. Now I have the computer that is always on and just serves my files. On Windows I have mapped network drive to the Samba share and on Android device I use X-Plore file manager which has an option to connect to SMB share. One important thing is to set an anonymous Samba share so that Android clients do not have to enter Windows credentials. Just in case, here are the smb.conf contents that work great for me:

workgroup = HOME
security = share
guest account = pi 
[LAN_EXCHANGE]
comment = LAN EXCHANGE
path = /home/pi/LAN_EXCHANGE
browseable = yes
read only = no
guest ok = yes
create mask = 0666
directory mask = 0777

And please do not tell me there already exist inventions like this WiFi “Drives”. I am pretty sure they not only are more expensive, but also have nothing to do with aforementioned protocols which IT pros use

However, the originally compiled executable does not work on modern Windows system. I decided to check if it is possible to fix this. I took my Visual Basic 6 installation CD, started snapshotted virtual machine running Windows XP and opened source project files. The application had simple HTML editor and used a few really esoteric OCX controls. In fact, they were not essential for the application’s core functionality. After removing unnecessary references from the project and disabling some of the UI components, I recompiled the source code so that it uses only “out of the box” Visual Basic libraries. The Windows operating system includes Visual Basic 6 runtime DLL even now and finally I managed to run the application on my Windows 7 64-bit machine.

If you want to give it a try, you can download it from my Skydrive

The problem here is that it is really difficult to guess the actual device. In such situations I would like to recommend rather obsucre Microsoft tool called USB View. It is not very easy to find a download, because the original version of the tool was created back in Windows 9x days, however it still works, has updated versions and could be particularly useful. Just look at the screen showing all connected USB devices:

Now I can quickly decode the device numbers and connect the right one, which is card reader, to the virtual machine. BTW, it also good to know the internal architecture of modern laptops — their equipment are most likely connected to the mother board via USB.

UPDATE: USB View is part of Windows SDK, it is installed along with debuggers. Link to Windows SDK for Windows 8.1

What concerns me is the way the idea is implemented. They provide dedicated browser extension working with GMail or MS Outlook extension. So far so good, but if you use neither of these you have to “sync” your Banatag account with your e-mail account. It boils down to simply giving your log-in and password information to Bananatag. Then they act as a proxy for sending e-mails and they attach appropriate images on the fly.

Both solutions sound not so good to me. I prefer not to install many extenstions to keep my software installations as lightweight as possible. I typically work on more than one machine and installing extensions is not always possible. Last but not least, giving my credentials to some company is unacceptable from security point of view.

However, the idea is so simple that it is possible almost for everyone to start up completely own tracking web application. In this post I provide complete instructions as well as small, self-contained source code written in pure ASP.NET without any external dependencies.

First, you need free web hosting provider supporting ASP.NET. I have come across somee.com which seems to be fairly good. They offer 150MB space and require you to access web page at last once a month. After signing up and creating new web site you end up with address like http://(yourname).somee.com. Then you can download source code from my Skydrive which is pr.ashx. It stands for pixel recorder, because it is an application which serves one pixel image and records each event in randomly named text file. Simply upload that file into the root of your newly created somee.com account. Now you can access it from the Internet typing http://(yourname).somee.com/pr.ashx.

When the application starts for the first time it creates uniquely named text file and provides a link to it. Now you can start using it. When you are writing an e-mail insert an image from a URL to have it tracked. Please make sure that you force your e-mail composing app not to attach image to the message body, but rather to preserve the reference to your web application serving the image. The URL is as simple as this: http://(yourname).somee.com/pr.ashx?token=1. Where token is a number between 1 and 100. You can further adjust the range in source code. Now every time someone attempts to access this URL, one-pixel image is served and the time stamp is written into unique text file. You can use it from anywhere in any e-mail composing app, web mail or desktop. You can view your log and check from anywhere using any web browser whether an e-mail have already been read. Just remember what token number was associated with particular message and keep in mind some e-mail providers do not download any images by default (e.g. GMail — user has to click button to download any images included in the message body). Have fun!