Code Kata is a term coined by Dave Thomas, co-author of the book The Pragmatic Programmer, in a bow to the Japanese concept of kata in the martial arts. A code kata is an exercise in programming which helps a programmer hone their skills through practice and repetition.

There’s a nice project over here on Github containing descriptions and code for a bunch of Katas, including the string calculator  that Roy introduces. Roy’s site has the string calculator kata executed in a bunch of different languages, including AutoHotkey!

The catch is that it’s not always clear you’re mixing framework versions in a solution. Adding a new project will default against the newest framework version. Which is not constant over a solution’s life time. The result can be quite strange. I managed to get away with a circular reference which was not seen because one project was using a version of the other built against an older framework version. And was thus considered a different assembly. Up to the moment I checked the framework version of all projects.

The only good way to solve this descend into hell would be having all projects in the solution on one version. In a big solution with a lot of projects this can be quite tedious. I would love to be able to set the version on the solution level. Or else to have a nice ‘refactoring’ tool to the work for me. R# are you listening ?

I’ve already explained the problem in my blog, and also in White-Books available on the NDepend website. (see white-book Partitioning code base through .NET assemblies and Visual Studio projects page 4 CopyLocal = true is evil)

For example, in 2009 by fixing the problem on the NUnit code base, I’ve been able in less than an hour, to shrink the compilation time from 32 seconds to 12 seconds.

So what is the consequence of Nancy framework relying on CopyLocal = true: A huge number of cloned version on Nancy.DLL for each VS project referencing it. After a recompilation, I can count 56 occurences of Nancy.dll that weights 842 KB each, it means more than 46MB wasted. Apparently it doesn’t affect so much the VS compilation duration, I guess the VS team made VS 2012 more intelligent in the sense that if it has already parsed a reference assembly for a project, it doesn’t parse it for others, which is a good thing.

I tried to post something on the Nancy user group, without success, hence this post. To fix this issue, it is as simple as:

• making sure that each project compile in a ..\bin\Debug and ..\bin\Release folder
• from VS projects, reference the DLLs in the ..\bin\Debug directory, and set CopyLocal = false for each reference

Additional, I found ..\bin to be a good folder to host test assemblies. This way they can reference the application assemblies that live in ..\bin\Debug and the indirection can be handled with a App.config file like:

On Nancy code structure

One very good point is that Nancy.dll is a single DLL that only depends on .NET Fx assemblies so it is optimally packaged.

On the other hand, this large DLL made of more than 7.000 lines of code and almost 300 classes, doesn’t abide by any kind of layered architecture. The consequence is that each namespace depends on all other namespaces.

As a consequences classes are not layered at all, there is no high level or low level classes.

Let’s have a look at the dependency matrix below. It is made of the 287 Nancy.dll classes, in mode indirect dependency. A blue cells means that the class in column depends (directly or indirectly) on the class in row, a green cell means the opposite, and a black cell means that the 2 classes are dependents on each others. A perfectly layered set of classes (which is not wished, but what should be approached) would be a lower triangular matrix blue, and a higher triangular matrix green. Here the matrix has blue and green cells mixed equally in both lower and higher triangulars, which is another way to figured out that classes are not layered.

It means that it must be difficult to unit-test a set of cohesive classes (i.e a component) in isolation from the others (even by using mocks that don’t help in case or re-entrant dependency). It also means that maintenance and evolution must be impaired by this absence of code structure, because touching a single class can virtually have an unexpected effect anywhere in the code base.

An Event Sourcing Perspective

Let’s start with why you want to update an event? An event is a fact that happened at a point in time with the understanding of it from that point in time, a new understanding of the fact would be a new fact(naturally this does not apply to politicians). To update a previous event is generally a very bad idea. Many want to go back and update events to new versions, this is not the best way to handle versioning!

The prefered mechanism from an event sourcing perspective is to write a new face that supercedes the old fact. As an example I could write that event 7 is a mistake, this is a correction, I might as well put in a comment “this was due to bug #1342″ (similar to a journal entry in accounting). There are a few reasons this is a better way of handling things.

The first is my ability to look back at my history. If I were to change the fact and I look back at that point in time I have changed what it means. What about others who made decisions at that point in time? I can no longer see what it was they made decisions off of. Beyond this I might have a very valid query to ask your event streams of “how long on average does it take us to find bugs in previous events”.

The second model leads us to two types of queries supported on event streams (as-of vs as-at).

Beyond that with Event Sourcing the updating of an event can be inherently evil. How do you update any projections that the update occured? What about other subscribers who may be listening to the streams? Any easy answer might be to replay fully all involved with the stream but this quickly falls apart.

These are the primary reasons why the Event Store does not support an update operation on an event. There are however some wonderful architectural benefits that come from this particular constraint.

Architectural Goodness

If we prevent an event from ever being updated, what would the cachability of that event be? Yes it would be infinite. The Event Store supports a RESTful API (ATOM). All events served from the event store have infinite cachability, what does that mean?

Imagine you have a projection updating into a SQL table that has been running for the past eight weeks. You make a change and need to restart it (replaying from event 0). When the replay occurs and it requests events from the Event Store where do they likely come from? Your hard drive! You don’t make requests to the Event Store for them.

Beyond the events being infinitely cachable if you look through our atom implementation in fact every single request we serve with the exception of the head uri (http://somewhere.com/streams/{stream}) is also infinitely cachable. In other words when you want to reread $all (say for 5m events) you will hit exactly one non-cachable request!

This is very important when we start talking about scalability and performance. The Event Store can pretty easily serve 3-5k atom requests/second on a laptop (per node in clustered version) but how many will actually get to the Event Store? In order to scale you focus on commoditized reverse proxies in front of the Event Store not scaling the Event Store itself. nginx or varnish can easily saturate your network, just drop them in front only head calls make it through (and there is even a setting per stream to allow caching for x seconds of head links).

This is often a hard lesson to learn for developers. More often than not you should not try to scale your own software but instead prefer to scale commoditized things. Building performant and scalable things is hard, the smaller the surface area the better. Which is a more complex problem a basic reverse proxy or your business domain?

This also affects performance of replays for subscribers as you can place proxies near the subscribers (local http cache is a great start!). This is especially true for say an occasionally connected system. Gmail uses this trick to provide “offlining out of the box” for web apps. Since much of the data will already be in the http cache your hits will be hitting it, in many cases you can build say a mobile app with no further support.

Over Atom if we allowed updates, NO uris could be cachable!

This is all cool but I actually need to update!

Occasionally there might be a valid reason why an event actually needs to be updated, I am not sure what they are but I imagine there must be some that exist. There are a few ways this can actually be done.

The generally accepted way of handling the scenario while staying within the no-update constraintis to create an entire new stream, copy all the events from the old stream (manipulating as they are copied). Then delete the old stream. This may seem like a PITA but remember all of the discussion above (especially about updating subscribers!).

Some however may be using the TCP API and are willing to take the pains and complexity that come from subscribers (you can imagine they have none). In this one case, updates would be acceptable and simpler than adding new events. We have been going back and forth on whether or not to support this. It would not be much work at all for us but I imagine that it would be misused 1000 times for every 1 time it was used reasonably. I am reminded of the examples of being able to call .NET code from a biztalk orchestration or being able to execute .NET code inside my SQL database both have valid uses but should rarely be used. Perhaps we will make a command line parameter –run-with-scissors-updates or make people build from source to enable.

I’m a TDD wannabe.  I’ve done small tasks using TDD, and I’ve written plenty of tests, but most just get shuffled around, never included in my CI builds, and basically the entire thing is a mess…  Perhaps it’s because I’ve been writing code for startups, but most likely it’s because I’ve just not taken the time to learn the right way.  I’ve never truly seen the benefits that it can bring.  So I saw this class as a great chance to learn from the master.

I’m going to blog about my experiences going through this course.  I’ve been through the first two hours, and so far I really like the pace and Roy’s teaching style.  Roy is snarky-funny, and so the content is entertaining to boot.

So here are some of the more interesting things that I’ve learned from the course so far…

Difference between Integration Tests and Unit Tests

Although the generally accepted definition of the difference may be up to some debate, Roy’s definition is simple, I like it, and it’s useful for me.  Roy says that a unit test is all in memory.  It should pass on any new developer’s machine, and is not tied to disk, databases or anything outside of the memory scope of the test.  This is important because confidence in your unit tests is paramount.  If you cannot trust that a failing test is due to a bug, i.e. if it’s due to some setup on disk, etc. it diminishes the usefulness of your test suite and developers cannot quickly setup to work on a codebase.

Arrange, Act, Assert

The three A’s of unit testing was new to me, and  is a pattern for writing tests that Roy introduces:

Each method should group these functional sections, separated by blank lines:

1. Arrange all necessary preconditions and inputs.
2. Act on the object or method under test.
3. Assert that the expected results have occurred.

This approach, by eliminating assert code intermixed with code that sets up or acts upon your objects, reduces smelly tests by separating what is being tested from all the other stuff.  See, I need stuff like this to help me fall into a pit of success of testing.  

There’s a lot of good stuff in there about creating readable and maintainable tests.  Roy also delves into best practices for naming tests, offers some good hints on how to setup Resharper live templates, a good deal of information on why he prefers TestDriven.Net as a test runner (and how to use it to debug methods directly!) including an interesting history lesson on MSFT vs  TestDriven.Net’s Jamie Cansdale.

With over nine hours of course total, I’m expecting to learn a lot.  Oh, and apparently serial killers practice TDD.  :)

-Brendan

Here’s the scenario: you have an application with several images and those images are stored on a remote server. The sources of each image is something like http://someserver.com/myimage.png.

Looking at a Windows 8 store app as an example, the good news is that your page will display immediately. Your images may display immediately. Or more likely, they will begin to display after the page has loaded. You may see the images incrementally appearing. This does not make for a good user experience.

The good news is that WinJS implements the Promise Pattern and XMLHttpRequest via the WinJS.xhr object. This means we can pre-fetch images and resolve them to blob URL’s and then bind our UI to the blob URL instead of the raw image url.

Using a stock WinJS Navigation Project, the following code is placed in the app.addEventListener(“activated”, function (args) {} handler:

var promises = [];

//Load the blob for the default image in the event a specified image URL does not work.
WinJS.xhr({ url: "images/no-available-image.png", responseType: "blob" }).done(function 
(data) {
 var noImageBlob = URL.createObjectURL(data.response);
  //The imageResults.json file is a static result from the Bing Images API.
  WinJS.xhr({ url: "js/imageResults.json" }).done(function (data) {
     var results = JSON.parse(data.responseText);
     //Loop through the results to create the promise array.
     results.d.results[0].Image.forEach(function (element, index, array) {
     promises[index] = WinJS.xhr({ url: element.MediaUrl, responseType: "blob" })
        .then(function (e) {
           //When the promise executes, an imageBlob property is created
           //holding the image blob URL
           results.d.results[0].Image[index].imageBlob = URL.createObjectURL(e.response);
        }, function error(e) {
           //If an error occurs when trying to retrieve the image
           //the No Image BLob URL created on line 28 is used instead
           results.d.results[0].Image[index].imageBlob = noImageBlob;
        });
     });
     //The join method kicks off the promises handed to it.
     //The promises can run in any order and finish at any time. 
     //The done event for the join method fires after all of the promises
     //in the promise array have been fulfilled.
     WinJS.Promise.join(promises).done(function (e) {
        //Need to create a global namespace to hold a reference
        //to the images array. To facilitate data binding, 
        //the WinJS.BindingList method is invoked, passing the 
        //image array.
        WinJS.Namespace.define("ImageList",
           {
              Images: new WinJS.Binding.List(results.d.results[0].Image)
           });;
           //Now that all of the pre-processing has occurred, we can now
           //navigate to the home page that displays the images.
           if (app.sessionState.history) {
              nav.history = app.sessionState.history;
           }
           args.setPromise(WinJS.UI.processAll().then(function () {
              if (nav.location) {
                 nav.history.current.initialPlaceholder = true;
               return nav.navigate(nav.location, nav.state);
           }
           else {
                return nav.navigate(Application.navigator.home);
           }
        }));
     });
  });
);

JavasScript, being a dynamic language, we can simply augment the object from the parsed JSON. In this example, I used a static file from the Bing Image Search API. You will have to conform the object references to the specific API you are working with. In the previous code, we wait until all of the images have been fetch. In an async world, this can be a challenge. We don’t know what order the requests will be processed. In some cases, a server status 500 could result. This approach affords us an opportunity to handle that error with a place holder image. If on the other hand, you deal with the raw URL’s directly, you have no such opportunity to intercept errors.

On the HTML markup side, I have this code to bind an image tag’s src to the newly created imageBlob property. For this example, the raw URL that is returned in the response is MediaUrl.

<section aria-label="Main content" role="main">
   <div id="ImagesTemplate" data-win-control="WinJS.Binding.Template">
      <div style="width: 150px; height: 100px;">
          <img src="#" style="width: 60px; height: 60px;" 
              data-win-bind="alt: MediaUrl; src: imageBlob" />
      <div>
       <h4 data-win-bind="innerText: MediaUrl"></h4>
      </div>
   </div>
</section>
<div id="ImageListView" 
   data-win-control="WinJS.UI.ListView"  
   data-win-options="{itemDataSource : ImageList.Images.dataSource, 
                      itemTemplate: select('#ImagesTemplate')}">
</div>

In the markup, I have a WinJS ListView control that uses the ImageList.Images.dataSource as its datasource. The ListView also references a simple template to display the output.

Here is the result (looks a bit skewed due to this page’s dimensions):

When the page displays, because the images have already been pre-fetched, they will immediately display as well. This makes for a more pleasant user experience.

Looking at the first image, it has a raw URL of:

http://www.israbox.com/uploads/posts/2011-04/1303738133_john-coltrane.jpg

When pre-fetched, the blob URL is:

blob:354BAAC6-B8D8-4172-A283-4DF2F037FECD

Getting in the flow: According to wikipedia, flow is the mental state of operation in which a person performing an activity is fully immersed in a feeling of energized focus, full involvement, and enjoyment in the process of the activity. In essence, flow is characterized by complete absorption in what one does. Focus, immersion, being concentrated, involvement …  being everyday in the flow by coding hours and hours, contribute a lot to a solid positive state of mind, it is a bit like meditation.  These are moments where one can completely forget about minor everyday pesky stuff, but also forget for a while more serious problem in life everyone has to face. Being in the flow is the condition to solve challenging problems and to create beautiful pieces of engineering. Being in the flow can lead to addiction but it is not addiction. This is essential to control when to check-in in the flow and when check-out, making sure to not be disturbed meantime.

Being creative: Being a software engineer is one of the most mainstream way of being paid for being creative. Often, writing software is deemed to be an artistic activity. A programmer has to be humble, because this is a kind of art not understood by the mass. But being humble is a chance to become wiser and increase self-confidence. Also, knowing you are going to be creative for a while, is an excellent motivation to overcome the first step effort to jump in the flow. But the truth is that for every passionate programmer, there is a background thread in the mind in charge of creativity (often running at sleep-time), that makes it so that in the morning the envy to create what you have in mind is too strong.

Becoming an expert: It is common to hear that a programmer must know numerous technologies, that its skill is to learn how to learn new technologies. On this I disagree because what makes me really happy is to master completely a technology and exercise daily my expertise. I used to master all the tricky details of C++ and COM and it was fun. Before that I used to master some assembly level programming and it was fun, I was not even getting paid for that. In 2002/2003 and then  2005 I wrote two editions of a 1.000 pages book on .NET and C# and writing it has been one of the most blessed moment in my career. Since then I capitalize on this knowledge every single hour of coding, letting me focus my thoughts on problems to solve, and not on all the non-trivial things a complex platform like .NET is actually performing under my feet. Of course I am constantly discovering new details about surrounding technologies, like functional programming paradigms through the prism of functional paradigm introduced in .NET languages. But I know what is my core knowledge, both in terms of technologies and in term of program design skills. And as long as I won’t be forced to change my core skills, relying on my expertise to express my creativity and making a living on top of it is a source of personal achievement.

Meet inspiring people sharing the same passion: I imagine meeting peers is a source of happiness for every expert in something. This is also why investing in a solid programming particular set of skills is a positive thing. Not only respect from others programmers arise, but it lets have great exchange with smart people as passionate as you. The importance of flow, underlined above, also comes with the disadvantage of being often alone with your thoughts. Most programmers enjoy working alone anyway, but for those who need a bit of more social activity, having an expertise in something is also a great way to become partly a teacher (in professional or academic spheres), partly an architect and contribute to important decisions, partly a team-leader and be responsible for project progressing, partly a consultant, and be able to share your knowledge in a pleasant social environment. I put the word partly in italic because if your social activity make it so that you are not writing code meant to be run in production anymore, you shouldn’t consider you as a programmer anymore and you’ll loose a great deal of the points I am mentioning here. If you need social interactions all day long, programming is not for you. This is also why (sadly) there are so few women in software, because evolution designed them to be much more social beings than men.

Being involved in something that make sense: Here also my position might be a bit different of what is widely accepted. I agree that for juniors, it is important to multiply the opportunities to work in several different teams and companies, to get an idea of what they like and what they don’t, to be influenced by several inspiring mentors. But once you become seniors, working on the same application in the long term, where you feel well programming in,  polishing it days after days, seeing its evolution across years, maintaining it in a clean state by adopting modern paradigms like automatic tests, DbC or relentless refactoring, having your word to say about strategic decisions, personally I found this being a great source of daily happiness and a great motivation to involve myself! In addition, working hard to achieving important milestones regularly, is an excellent way to give a sense to your professional career, which is (much) more the exception than the rule.

Work wherever, whenever you like: A 2KG laptop with the proper tools set installed, a few hours of electricity every 24 hours, this is all what a programmer needs to do his job well. A descent internet connection is often appreciated, and there are today only few points on earth where internet is not available at all somehow. Programming might be probably the less demanding working activity in terms of time and space requirement. Getting in a flow in a 12 hours plane flying across the planet, scheduling half a year to live and work in a paradise tropical island, avoiding traffic jam by staying at home for work (in your pijama), delaying programming in the night or early morning to take care of the kids and their education, all this is not only possible but pretty common actually. Most serious software companies let some of their skilled engineers work wherever they prefer. Did you know that many of the great minds behind Visual Studio didn’t actually moved with their family to Seattle, but still live in their preferred location, sometime far away from the US?

Make a decent living doing something you like: Last but not least, everywhere, skilled programmers get paid above the average salary in their countries, and if we take the example of a developing country like India, good programmers get a much much higher income than the average. On top of that, a skilled programmer have pretty close to zero chances to remain unemployed for a long time. This also means that if you don’t like your current position, it is easy to find another better suited one. This situation is made possible because less than two decades ago, the modern civilization realized that IT is the condition for its development. It is a fact that many of the richest persons in the world were originally programmers and every motivated programmer has the potential to create its own ISV business. A programmer can also decide to make more money by coding for the financial industry, or even bet on a startup and potentially makes millions in a few years.

All the next big things will consume even more IT, this includes human genome analysis for the mass, the entire medicine that will be deeply impacted by that, more prevalent portable devices, more sophisticated entertainments, augmented reality, robots and automated machines to do surgery, to assist the increasing growing number of old people, artificial intelligence in the long term, and certainly everything nobody hasn’t imagined yet. After all 20 years ago, nobody anticipated the impact of Google. 10 years ago nobody anticipated the impact of Facebook and smart phones.

Ok enough getting in the flow of writing my passion for programming, I have some code to write before the weekend

For example, if you look at our Web API sample, you’ll see there’s a bunch of friction if you just try to get Web Api working from scratch.

• You need to add using statements for each namespace you want to use. This is a lot more painful than one might think when you don’t have intellisense.
• You need to configure web API, this involves creating a host, defining default routes etc. Adding lots of object creation and such starts to make the script pretty hairy. Not impossible, but painful when there’s no template.
• You need to teach Web Api how to resolve controllers in script by implementing a custom controller resolver.

Now pull in the Web Api script pack (scriptcs –install scriptcs.webapi) using the Require<WebApi>() function and your boiler plate code evaporates to this:

The script pack does all of the following to make the experience better:

• Removes the need for using statements for common namespaces. The script pack provides those which is why you don’t have to add the web api namespaces in your example above.
• Adds dll and nuget package references that bring the dependencies the framework needs.
• Removes general boilerplate code. In the previous sample you need to create a host, define routes etc as I mentioned. In this case the script pack creates the host for you and configure with the default routes. You can customize if you need to.
• Provide APIs to fill gaps that prevent the framework from working well in script / supporting dynamically emitted assemblies. The Web Api script pack brings in and configures a custom controller resolver for you.

We’re just getting started with the work we’ve done with script packs, but they are a really nice extensibility point and really take advantage of nuget as a delivery mechanism. The community has been rising to the occasion and building out quite the gallery as well.

There’s some great posts about script packs covering topics like how to build them or even use them from the REPL that you should really check out.

• http://blog.martindoms.com/2013/05/14/building-scriptcs-script-pack/
• http://scottksmith.com/blog/2013/05/14/deeper-dive-into-scriptcs/

Have fun exploring the new world of scripting in C# with scriptcs!

1. Standardizing the look and feel. It is far more consistent and maintainable to set attributes (including a css class) and event handlers in one centralized place.
2. Simplify the script. Often a part of the logic the script will follow is already known server side. Instead of writing everything out in javascript rendering the intended statements leads to a leaner client. There is an example in my previous post on building post-data.
3. Decoupling the libraries used. At the moment we are using the Telerik MVC suite. In my previous post I described how our html helpers build standardized Telerik components for our views. In the not to far future we want to switch to the Telerik Kendo suite. Having wrapped up the library dependency in our Html helper will make this switch a lot easier to implement.

What has evolved is the way we work with the model. In MVC the implementation of the controller and the view is clear. When it comes to the implementation of the model there are almost as many different styles of implementation as there are programmers. In general the model can bring any data to the view you can imagine. Not only the source of the data varies, from plain sql to a C# property, also the use of the model’s data varies. It can be a customers name from the database. Or it can be the string representation of some html attribute needed for a fancy picker. Here data and presentation start to get mixed up. Our extensions needed information for the Html-Id. The original Html helper had a custom constructor to get that specific data from the model into the helper. Which required to create our own htmlhelper when starting the view and use that one instead of the standard @html. As seen in the eposHtml in the previous story. It would be cleaner if our extension methods could be satisfied with the default html helper. It would also be cleaner to keep a better separation between ‘real’ data and presentation.

The model is available in every HtmlHelper extension method.

It’s a property of the ViewData.

In our case we needed something to give the control an unique Id. The Id method builds that Id. Previously we passed the Id-base in the constructor, which lead to the custom helper. A far more elegant solution is using a very basic IOC-DI pattern. As implemented By the Id method

The method queries the model first for the IProvideCompositeId interface, in case the model does not implement that it is queried for the IprovideId interface. Resulting in a string which can be safely used in an HtmlId. (A negative number would lead to a ‘–’ in the string, which is not accepted in an Html Id).

These interfaces are very straightforward

In case the model is going to be used in a view requiring unique Id’s the model has to implement one of these interfaces.

Working this way:

• We can use our custom html extensions in the default html helper
• Specific data from the model is available inside our extensions
• The model and the view do not get entangled

The code is no big deal. I know. But the model is something whose horizons are still not in sight.

As its name suggests this rule matches fields assigned from multiple methods. Bugs due to corrupted state are often the consequence of fields anarchically assigned.

On many code bases tested, I found the threshold of 4 methods writer to be the best balanced number between too many false positives, and a decent amount of suspicious matches to look after at code-reviewing time. Of course this threshold is easily customizable.

Several parameters can be taken account.

• Is the field static? in which case it can be pretty serious if many methods assign it.
• Does the field type is a value or a reference type? In the case of a reference type, such situation might reveal a potential NullReferenceException scenario luring.
• Is the field assigned outside from its class. Since another rule prevents fields to be visible from outside its class, this case has been eliminated.
• How many assigner methods of the parent class are visible outside the class? How many  methods calling directly or indirectly an assigner method are visible outside the class?
• Can a particular class instance be accessed from multiple threads? in which case the situation sounds pretty alarming.

Not all cases are taken account since static analysis have limits and for code review warning purposes the rule algorithm must be kept easily understandable. Here is the rule code and comments that contain insight about how to generally fix such issue.

I’d be curious to read your suggestions and comments on that.

And here is what it looks like when we run the rules on the NUnit code base:

Before you go further, if you are wondering what all the scriptcs hype is about please check out Scott Hanselman’s great post and his new Tekpub video.

Last few days I’ve been working on a new REPL experience for scriptcs and now it’s in! REPLs are nothing new to dynamic languages but they have not really been available in C# with one exception, Mono has a great REPL. Roslyn introduces the Roslyn interactive window in VS which is also a REPL which runs in the editor.

This REPL is different than both in that it is specific for scriptcs, and like the rest of the scriptcs experience, there’ s no IDE required. Basically it combines the goodness of scriptcs (nuget) with an interactive experience. You can just install some nuget packages and type code which instantly executes. For example imagine just pulling in HttpClient and then just doing some http requests!

Below you can see I am installing the mongo nuget package. Then running scriptcs by itself and typing in some simple code to work with the package.

And then we have pretty error handling.

Thanks to the Roslyn team’s efforts layering a REPL on top was AMAZINGLY easy. The REPL part itself was like 15 lines of code! And thanks to the nuget team for even making this a remote possibility through having an awesome package ecosystem. I look forward to ALSO seeing the mono version soon

If you wanna try it, you can grab the latest from github at http://github.com/scriptcs/scriptcs in the dev branch.

It will also be on our chocolately nightly builds tomorrow: http://www.myget.org/F/scriptcsnightly. It should be on our public feed very soon there after.

Practical ReST – the book

I’ve been talking for years about writing a brain dump of what I know of ReST. The project has started, you can pledge to buy your copy today (in various editions) and get content as early as next month!

The response has been amazing, with three quarters of the minimum funding pledged in 48 hours. Of course, the more contributors the more time I can allocate to the book, the quicker the book will be in your hands, so don’t hesitate!

Writing a book in itself is a big challenge, and the scope is very ambitious. But it’s also a foray into self-publishing, and I expect it to be an amazing opportunity for learning.

Practical ReST – the class

I’ve been giving a ReST class for a while, but I’m rewriting it so it follows the same structure as the book, and make it open to people with many technological backgrounds.

First date should be in May in London, and I expect Warsaw, Vilnius and other locations to pop up this year.

If you want to see a workshop organised in your city (or your company), pop me an email.

I’m also launching a referral program *and* group rebates, so if you’re a training company and you want to add the course to your books, you can too.

Hanselminutes podcast

Scott was nice enough to have me on his show this week, talking about Practical ReST. It’s published and ready to be listened to, and while I was half-way through an asthma attack and having microphone problems, Scott saves the day by being a great host.

Cancer Research UK

You may remember that I asked for your contributions to help me raise money for cancer research UK. I can now announced that we raised more than 700£, and that both my objectives of non-drinking were achieved. I cannot thank you all enough for your generosity.

serialseb.com

It had to happen. I now have my own web site. You cannot imagine how difficult it is for me to put work and effort in talking about what I did rather than what I want to do. But you will find on there a link to all my previous talks, with videos and slides (where applicable), as well as where you’ll find me next, my future talks and where classes will be delivered next.

What next?

ReST will be my focus for the next few months. I have a couple of other projects up my sleeve, but this is enough announcements for now, don’t you think?

If you haven’t seen it, it looks exactly like this:

The reason this is happening is because by default we don’t allow errors to propagate back to the user. This is in order to prevent you publishing to production and leaking details of your system.

Let’s assume I just published a simple express app, but I made an error. Below you can see that instead of requiring winston, I required win$ton.

When I publish, I’ll see the page cannot be displayed error.

iisnode.yml to the rescue

To find out what is going on, you can enable displaying errors using our YAML configuration file otherwise known as iisnode.yml. If you use our azure-cli tool, and create a site in a directory where there is a node app, then we automatically create it for you. If not, you can create one really easy with any text editor. The entries you need to put in the file / update are below.

Once you save that file, go republish to Windows Azure. If the file did not exist and this is an Azure Website, do an “azure site restart”. if the file already existed, you won’t need to do anything.

Now with debugger errors enabled I see the following:

Bingo!

Don’t go to production this way, but DO use tail for websites.

Make sure to disable debugger errors when you go to production. You can leave logging enabled however, because another command you can still use is “azure site log tail” which will stream logs directly to your console realtime. Also if you are a cloud service, you can remote in to view the logs written locally.

Using “azure site log tail” watch what happens when I refresh the page.

azure-log tail has many more uses. You can use it to capture any log output that your applications give you in a realtime manner. For example here’s a screenshot of me tailing an app using socket.io and Service Bus in Azure.

Yes it is awesome!

PS: Today “tail” works only for node apps, but that is going to change VERY shortly.

If you are using our azure-cli then you might have though of creating automation scripts to package up common tasks. A while ago I posted on how you can achieve some script-ability just using bash tools. Basically the techniques I listed there involve piping text from command to command and using tools like grep and awk to parse the results in order to feed to the next command.

It turns out there’s another way, and one that is much closer to the kind of fidelity you an achieve with Powershell. Sound interesting? keep reading. It turns out the majority (like 99%) of our cmds can return JSON objects if you apply the –json switch.

For example if I do “azure site list –json” here’s what I get.

That means that I can take that result and parse it into a JSON object which I can then easily manipulate. Potentially I could even pipe results in from one cmd to the other. I could even imagine being able to define cmds as JSON objects so that I could script out my tasks in a more object friendly fashion. I could then wrap that all up in a nice pretty box to be used by myself or others.

And that is what azure-scripty is about. (Kudos to @JpScripter for reminding me I need to post.)

azure-scripty gives you a JSON oriented API that you can use within a node script for automation tasks. It lets you combine the power of raw node with the capabilities of our azure-cli. One thing that is also really nice is it leverages the knowledge you already have of the CLI. As it says on the github page, “if you know the CLI you know scripty.”

azure-scripty offers you the following:

• Works anywhere node works i.e. Windows, Linux, Mac, Nodecopter
• Author using pure string command or JSON object command styles.
• Uses the standard node callback model
• Batching multiple commands together.
• Piping results from one command to the other

Here’s a few examples to give you an idea:

String command style

This will list out my websites in json format.

Object command style

This sample creates a new mobile service using the object oriented style. Notice you can pass fixed position args, with all other named args being by convention.

Batching

This sample shows creating a web site and a mobile service using the object oriented style.

Piping

One of my favorites. This example shows using piping to pipe the list of sites into the stop command. Notice that :Name, that’s plugging in the Name parameter from the returned site.

Get it and please give feedback

1. npm install azure-scripty
2. go check the README.
3. Start scripting!

Looking forward to your feedback and your contributions!

One thing we can agree on, there is a massive lack of maturity in this business. At one time or another, including yours truly, has been guilty of an off-color joke. Most of the times, at community gatherings like conferences; we are typically amongst friends and feel comfortable that we can be ourselves, letting our guard down. At the end of the day, we must rely on the fact that in this business, we have to have a thick skin and that we are adults. That in no way is meant to justify offensive comments and that somebody like Adria didn’t have a right to be offended. She did have a right to be offended. And to that, it’s not for anybody else to question it. It’s enough for anybody to say they were offended – and others should respect that. That said, does it mean the offended have an open ended license to exact whatever retribution they want on the offenders? No…. In these community gatherings, there is a level of privacy, within the community cocoon; we have a reasonable expectation to.

We see the t-shirt shirt = “Fork you”. We see on open source projects: “Fork me on GitHub”. Doesn’t take a rocket scientist to see the double entendre here. If one is to where that shirt in public, does that give anybody who is offended license to snap a picture, tweet it, and then hope that some misfortune befalls that person? I don’t think such license is granted.

Consider this YouTube video from Ms. Richard’s: http://www.youtube.com/watch?v=nMpJSKbsmgQ. There’s a saying in the law that in matters of equity, you have to have “Clean Hands.” Is it permissible for her to be offended as she was when she makes a video that could be taken as offensive by others? At about 3:03 into the video, she talks about an incident that was done to her – that could very well be taken as offensive, and she let it pass. In this case, she decided to not let something pass…something that WAS NOT said directly to her. If I was prosecuting this case, and I was going to depose Ms. Richards, this is a line of questioning I would surly explore to great details. What was her real motivation here?

This gets to my first question:

Was there a reasonable expectation of privacy by these two gentlemen?

I believe there was. They were having a private conversation, which Ms. Richards overheard. Yes, this was a public place. Did she have the right to express her indignation at those comments to these two gentlemen? Absolutely yes. In fact, that would have been the more impactful thing to do. That would be a proportional response. Reporting the incident to the PyCon staff is not appropriate. They are not there to police what people say. Note, these two guys didn’t say the comment to her. She thrust herself into their conversation. By the same token, the two guys allowed her to confront them. She didn’t. Instead, she took their picture and posted a tweet. And from there, SHE took what was a private thing and made it public. She made the situation worse. In that regard, she violated the privacy rights of these two men, notwithstanding their comments. Speaking of those comments, only her, the two guys and maybe one or two others know what the precise words were. The first result, at least one of the guys lost his job.

Next question..
Does the guy who got fired have a cause of action against Ms. Richards?

This is the part you want to pay attention to as you should think very carefully about the consequences of tweeting something that is private and further, identifying specific people where harm could come to them. First, did these two guys have a right to say what they said, no matter how stupid it was? Yes, they had a right. There is such a thing as a First Amendment last I checked. It’s also not illegal to be stupid. Did the guy who got fired, his employer, have a right to fire him? If he works in an employment at-will state, the answer is yes. Is he totally out of luck? No.. He can sue Ms. Richards for Tortious Interference with an Employment Relationship. She knew and intended a certain level of a negative outcome to these two gentlemen. That’s malice. While his employer may be on good ground to fire him, he can nevertheless sue Ms. Richards to recover damages. There may also be a cause of action on defamation grounds. These two guys are being labeled as sexists when all that may have happened was a joke, off color as it may have been, that offended this one person. That too may be the basis of a cause of action.

Next question..
Is there any other entity that could be liable?

As SendGrid has already admitted, Ms. Richard’s was there in her capacity as a representative as a SendGrid Employee. Ms. Richard’s did not do SendGrid any favors when she tweeted that she had SendGrid’s full support – a claim that SendGrid didn’t dispute. In the law, there is the legal doctrine of “Respondeat Superior.” It effectively means that the master is liable for the actions of his agent. Ms. Richards has a large Twitter following and that is due in least in part to her former job as a developer evangelist for SendGrid. There’s a strong argument to be made that SendGrid is liable, at least in part, for Ms. Richard’s actions – regardless of the fact that they terminated her services.

This whole episode illustrates again the importance of how we react emotionally and the legal consequences therein. If I’m the gentlemen who lost his job, I’m exploring my legal options – and there are legal options. Above all, this is a teachable moment. If you are offended at something somebody does, consider the context. 99.99% of the time, these matters are best handled privately. Takes more courage and is more impactful when you communicate your disdain to the offending party directly. If your goal is to change behavior that will more likely have the desired effect. Sure, you may be told to eff-off. That’s a risk…a risk worth taking.

How about conference organizers?

I understand that PyCon has modified their code of condut… Didn’t know that such a thing existed. I would always suggest to any conference organizer to NOT get into the business of policing their attendees. Rather, I’d rely on the venue staff for that. Obviously, if somebody is disruptive, boot their tails out. These guys didn’t appear to be disruptive at all for the record – not that it really matters in this case.

A word about the “Brogrammer” culture..

You probably know about it. If not, look it up on our favorite search engine. The fact is, guys are often not the sharpest knives in the drawer socially and are often in need of an increase in maturity level. Women have a tough enough time in this business. I know several women that do a lot of work in getting more girls interested in software development. Sad as it is, Adria Richards did more to set that effort back than the two guys she was offended by. That’s a perfect example of irony!! It’s really a shame. Ms. Richards looks to be a pretty sharp individual – one who appears to have better common sense than she displayed here. Learn from it folks.

As soon as it shipped we started to get reports of issues using our SDK and CLI. We’ve investigated this and uncovered that there are some things we need to fix. In the meanwhile, node 0.10.x is not currently supported. We’d ask everyone using our SDK and CLI to stick to node 0.8.x for the time being.

If you are using our Mac or Windows installers you should be fine as they each include private copies of node with working versions.

We are on top of this and will be working on this in our next sprint. We’ll keep you posted and let you know as soon as we have it working which we expect to be very soon.

Thanks for your understanding and sorry for any inconvenience this has caused.

Nate soon after posted:

It’s funny, you never see someone with the title “Master Craftsman” at a startup. I wonder why?

— Nate Kohari (@nkohari) March 5, 2013

then

I take it all back. I would strongly encourage startups to use TDD while developing any products that compete with ours.

— Nate Kohari (@nkohari) March 5, 2013

I wanted to write a few comments about TDD in startups. Good code is the least of the risks in a startup. Sorry but worrying about technical debt making us go slower when we have a two month runway and likely will pivot four times to quote Bob.

Captain Sulu when the Klingon power moon of Praxis exploded and a young Lieutenant asked whether they should notify Star-Fleet: “Are you kidding?” ARE YOU KIDDING?

One of the biggest mistakes in my career was building something appropriate…

It was just after Hurricane Katrina. I was living in a hotel. An acquaintance asked me if we could hack together this business idea they had for a trading system. He had the knowledge but not the know how. I said sure, hell I was living in a hotel!

In less than two weeks we had an algorithmic trading system. It was a monstrosity of a source base. It was literally a winforms app connected directly to the stock market. UI interactions happened off events directly from the feed! Everything was in code behinds (including the algos!) Due to the nature of the protocol if anything failed during the day and crashed the app (say bad parsing of a string?) the day for the trader was over as they could not restart.

But after two weeks we put it in front of a trader who started using it. We made about 70-80k$ the first month. We had blundered into the pit of success. A few months later I moved up with the company. We decided that we were going to “do things right”. While keeping the original version running and limping along as stable as we could keep it while adding just a few features.

We ended up with a redundant multi-user architecture nine months or so later, it was really quite a beautiful system. If a client/server crashed, no big deal just sign it back on, multiple clients? no problem. We moved from a third party provider to a direct exchange link (faster and more information!). We had > 95% code coverage on our core stuff, integration suites including a fake stock exchange that actually sent packets over UDP so we could force various problems with retry reconnects etc/errors. We were very stable and had a proper clean architecture.

In fact you could say that we were dealing with what Bob describes in:

As time passes your estimates will grow. You’ll find it harder and harder to add new features. You will find more and more bugs accumulating. You’ll start to parse the bugs into critical and acceptable (as if any bug is acceptable!) You’ll create modules that are so fragile you won’t trust yourself, or anyone else, to modify them; so you’ll work around them. You’ll build a festering pile of code that, with every passing week, requires more and more effort just to keep running. Forward progress will slow and falter. It may even reverse as each release becomes buggier and buggier, and less and less stable. Catastrophes will become more and more common as errors, that should never have happened, create corruptions and damage that take huge traunches of time to repair.

We had built a production prototype and were suffering all the pain described by Bob. We were paying down our debt in an “intelligent” way much the way many companies that start with production prototypes do.

However this is still a naive viewpoint. What really mattered was that after our nine months of beautiful architecture and coding work we were making approximately 10k/month more than what our stupid production prototype made for all of its shortcomings.

We would have been better off making 30 new production prototypes of different strategies and “throwing shit at the wall” to see what worked than spending any time beyond a bit of stabilization of the first. How many new business opportunities would we have found?

There are some lessons here.

1) If we had started with a nine month project it never would have been done

2) A Production Prototype is common as a Minimum Viable Product. Yes testing, engineering, or properly architecting will likely slow you down on a production prototype.

3) Even if you succeed you are often better to stabilize your Production Prototype than to “build it right”. Be very careful about taking the “build it right” point of view.

4) Context is important!

Never underestimate the value of working software.

BUT……

Where are the apps? Granted, Words with Friends is finally in store. The new ESPN App is nice. But what about Taxi Magic and Pandora? How about a better FaceBook and Twitter app? Instagram? I was just at an event where somebody was showing Vine – a really cool iOS app. I looked for it on the Windows App Store…. How about Instapaper???

I’ve taken the plunge to give the Windows Phone a fair shake. I love the UI. What is needed are apps…apps people want – which will drive the platform. Whether Microsoft wants to admit it, the Apple AppStore is the store by which the Windows Store is measured. There’s A LOT of noise on the Windows Store. Note – quantity is not quality. If there is a top app in iOS, Windows needs to have that app. Microsoft needs to be focused on those vendors to get these apps built.

Another nit is the difference between Windows 8 and Windows Phone. It would be great if those two worlds could come closer together from a development standpoint. On the plus side, the process by which apps are certified for the store is simple. The Windows App Certification Kit is a nice utility and easy to use. If your app fails, there is clear guidance on what to fix.

None of this is to say there are not useful apps on the Windows/Windows Phone Stores – there are. And when it comes to things like Live Tiles, Microsoft nailed that one. A very useful feature – which makes the negative things especially frustrating to deal with.

At the end of the day, like many, I want to see Windows 8 succeed. I think and believe there to be a great amount of potential. But to succeed, the public needs to see the apps that are available in other ecosystems. Until that day comes, Windows 8, and in particular Windows Phone, will continue to be third in a 3 horse race – which shouldn’t be the case.

Disclaimer: I am not the first person to create a tool for C# scripting. There are a long list of projects from too many people to name, and I respect all of them A big part of why I did this was also to mess with Roslyn.

Learnings from node

For the past 18 months I’ve been living mostly in text editors and script getting fully absorbed in node.js. Recently I started to think a lot about my C# development experience and if there are things I could bring back from my learning’s with node / JavaScript

• No projects, just script- One of the things I love about node.js is you don’t need a project. You can just jump in a folder start creating js files and go to town.
• No IDE requirement, you can just use a text editor.
• Packages over assemblies – In node, when you want to get something you use npm to download the packages.It’s super simple. You just have your app and your local node_modules folder and you are good to go.
• No compilation – This is a big one. With node, I just run node.exe and my app and it works. I don’t have to first create an executable to run, I just run.

On first glance it might seem like this is just oil and water with C#.

Thanks to Roslyn and Nuget however I no longer think that’s the case.

What is Roslyn?

Roslyn is a compiler as a service. It gives you an API that lets you compile and analyze code on the fly. Additionally it gives you hooks to dynamically refactor and generate code. Beyond that it also includes really powerful scripting support allowing you to author applications and components as loose C# scripts. You don’t need a class if you don’t want one and you don’t need to compile. And you don’t need to use Visual Studio, you can host Roslyn in your own applications and components and it’s even got nuget packages.. You can read more on Roslyn at MSDN or in a nice post from Filip on Roslyn and Web API

Roslyn is a breath of fresh air in terms of it’s simplicity. If you have worked with the existing horrendous mechanisms for doing this like the Codedom you are going to love Roslyn. You can literally be up and running with Roslyn having it compile scripts for you in a matter of minutes.

How does Roslyn help?

I can cross off three bullets

• No projects, just script- One of the things I love about node.js is you don’t need a project. You can just jump in a folder start creating js files and go to town.
• No IDE requirement, you can just use a text editor.
• Packages over assemblies – In node, when you want to get something you use npm to download the packages.It’s super simple. You just have your app and your local node_modules folder and you are good to go.
• No compilation – This is a big one. With node, I just run node.exe and my app and it works. I don’t have to first create an executable to run, I just run.

Now yes it does compile, but that’s all taken care of for me, I don’t have to think about it.

Packages vs Assemblies

For the last bullet I looked to nuget. The nuget command line provides an easy way to install packages right from the command line into the same folder as a set of scripts. However, the challenge is how to make those scripts see those packages? Today Nuget works great with a csproj and within Visual Studio or using MS Build. But what if you don’t have Visual Studio or a csproj? Well when you install packages using the cmd line you get folders for each package that at some level actually do contain binaries. So, if you can find those binaries and get them loaded up into Rosyln, you are golden. Yes this doesn’t get rid of assemblies, but I don’t have to manage them individually, I just install packages.

• Packages over assemblies – In node, when you want to get something you use npm to download the packages.It’s super simple. You just have your app and your local node_modules folder and you are good to go.

Roslyn + Nuget = scriptcs

Having crossed off all my bullets, I then decided to spike a little with Roslyn and out came Scriptcs: https://github.com/glennblock/scriptcs

scriptcs is a little command line tool which lets you have the following work flow.

• You author your “app” as a .csx file or csharp script in your favorite editor. There’s no project.
• You add dependencies to your app by simply installing nuget packages from the command line. By convention the dlls within those packages will be loaded up. No need for references as the idea is if it’s there you want it.
• You add GAC references or other dll references (not nuget packages) by using the “r:” Roslyn syntax, i.e.”r: System.Net”.
• You run your app using “scriptcs” passing the csx file.

For example here is a snippet of setting up a web api host.

Back to reality, though the future looks bright.

At this point I would say this is just an experiment. I am not claiming you can build an enterprise app with this or anything like that. I am using this more as an exploration to understand what a c# script experience from outside VS could look like.

From here there’s a lot more places to go. One would be allowing you to have includes of other csx files as you probably don’t want your whole app in a single file! Beyond that Roslyn offers a ton of rewriting, AST analysis or even auto-refactoring capabilities that I am sure can be leveraged in scriptcs but are not.

Seeing this definitely makes me excited about where things can go in the future. I’d love to hear your thoughts.

• explore two folders containing older and newer versions of assemblies to compare
• create on-the-fly two NDepend projets and analyze them
• compare the two older/newer code base snapshots, which lead to the creation of a ICompareContext object
• use this compare-context object to explore code diff and changes

The power offered by a compare-context object is barely exposed here. We only reports 3 lists of assemblies added/removed/changed, which can already be a useful information. But much more can be done with a compare-context object in hands. I wrote recently about how Code Query LINQ can be used to write rules to detect code regression issue. This post explains how code-diff can be used to detect regression, including API Breaking Changes, loss in code quality, recent code not well covered by tests… All CQLinq queries/rules exposed are based on NDepend.CodeModel API. And thus, they can all be integrated and combined into a program that starts like the one above, and then uses a compare-context object to explore code diff.

A subtle points is that when one writes a CQLinq rule involving code diff, one doesn’t deal with a compare-context object. This is an astute we use to make CQLinq rules/queries as fluent, concise and elegant as possible. Indeed, a CQLinq pre-compiling phase takes care to transform calls to extension methods, into calls involving the compare-context. Hence the following CQLinq rule that reports methods that were complex, and that became even more complex….

…is actually transformed into this expression, where the usage of the compare-context is made obvious.

Thus the logic of this query could be easily integrated into our program this way:

In the real-world, for companies that have build servers and that care for the quality of code delivered, this ability to write custom programs that can be included into the build process chain to focus on the quality of new or refactored code can certainly be a nice option.

But one can also see value in writing a custom-tooling program to be ran on the developer desktop. Such program could also use the special method TryCompareSourceWith() that can open a source file diff tool on the right location of two versions of the same source file (the source file diff tool is the one configured in NDepend > Tools > Options > Source Files Compare Tool). For example the open-source PowerTool Code Review Methods Changed uses this TryCompareSourceWith() method to show diff in code during a code review session. Certainly a more sophisticated program could help reviewing code diff, for example with the ability to assign some users to approve some diff, and persist in a DB such approval or disapproval actions.

Enjoy!