Check it out at http://blog.house-of-code.com.

All the old posts have been moved to the new blog (and the old posts will redirect to their new URL).

If you follow me with RSS, please re-subscribe to http://blog.house-of-code.com/rss.

Thank you all for reading!

My latest crusade was about ignored tests. When I first came aboard the company, we had many. Since I considered them a broken window, I started handling them one by one. 
After a couple of days – Great Success – no more ignored tests.  Since then it has been two years, and we’ve been trying to get the ignored tests in check… unsuccessfully: 
First we started to register the ignored tests introduced in each commit.
Next we sent a daily personal email to each developer with their own ignored tests. 
Then, we sent it to their managers. 
And, as a final move – we went directly to the developers. Then we heard them say: “I know,  I’ll fix them once I finish my super-urgent feature…”. 
Days, weeks and months passed by, and nothing was fixed.

I then realized something simple: you can’t make people do things. Even if they understand the importance of the issue at hand, they will assign it a different priority than you. And most of these issues are seemingly  not something they enjoy doing.
I wanted to find a way to encourage developers and rally them to my cause.

I started with small “hackathon”-like days during which we solved a specific issue, like dead code or improving tests execution time.
It worked for a short time, but it didn’t achieve the impact I wanted (not too many people are excited about “hey, we deleted a bunch of code… hurray for us!”). The worst part was that it was a one-time thing: the developers stopped addressing those issues once the hackathon was over.

I needed to return to the basics – understand the user.

For starters, if you don’t know how to do something, at least make sure you are measuring it. So we started to measure some statistics about the issues (number of ignored tests per team, number of dead endpoints, etc…) and display them on our office monitors. 
Although it was very helpful to me, it was not clear enough for other people in the company what those stats meant.
Next, I started to create a backlog of our technical debt tasks. This didn’t work as well (I got the “it’s in the backlog” answer from both developers and team leads all the time, but no one seemed to try and get things out of the backlog).

Finally, it struck me: Why not create a healthy competition between the groups and teams? This way, we’ll exterminate the “threat” and reward people for doing good deeds for our ecosystem and codebase. 
People love to win – and  even better – hate to lose. This can create the momentum I wanted and it will go all the way up to the group leads.

So I started thinking about the concept for the first competition –  exterminating ignored tests. I wanted a cool and funny way to display it on the boards and make everyone connected, resulting in “Game of Code”:

Next to each group’s sigil, you can see the number of ignored tests, with the leading group painted green.

I’m getting great feedback, and many questions and discussions about this issue – which is exactly what I wanted: developers, product and managers are all talking about ignored tests, trying to understand what “this cool Game of Thrones thing on the monitor” is all about.

And, of course, the groups are starting to churn the ignored tests at a high rate – after two weeks we are left with only a few remaining tests, and as the deadline nears, the rate increases.

I also spotted a few tricksters who tried to cheat the system (our developers are very creative), but  this is fine as well – the important thing is that people are paying attention and trying to understand the underlying issue.

People also mentioned that the competition itself is not fair – it’s not meant to be. It’s not meant to be fair or balanced. The competition is only an excuse.  It is meant to be a fun game for all to play, and at the end, we all win!

So, what is the difference?

Clear, single goal everyone can understand easily.

Displaying the goal and exposing it to everyone (including the company’s CEO) in a clear, understandable manner, creates a larger impact, raises awareness and puts the effort in everyone’s faces.

Competition and Gameplay are very good motivators. It’s a constructive way to encourage engagement in the most unexpected ways.

How does it all unfold in the long run?
I’ll let you know. 
All I know is when you play the Game of Code, we all win.

What impresses me even more is the way Google are handling the product. If you ever read “The Lean Startup” by Eric Ries (if you didn’t, you should…), you can see a lot of concepts that are used to develop this innovation.

Sell your product

First concept that is introduced in the book is to charge money for your product. It creates a commitment from the clients, establishes the product has valuable in the eyes of the customer and creates an environment in which people who use the product actually want it.

An added benefit to this is that you can get immediate feedback from the ones who purchased it. Since the customer pied for it, he will not ignore things that are bothering him.

Although in this case the price is pretty steep, I think it donates to the feeling of exclusivity of the product (more on that later).

MVP

Glass is a great MVP. It achieved what many products before tried and failed – on the go display in front the user.

The purpose of a MVP is to create the minimum requirements for the product, and then throwing it to the real world to see if it answers the users expectations.

In this case you get a HUD (Heads Up Display) which connects to the internet and your phone, which is still quite awkward to use.

But that’s the point – I’m sure someone thought “Hey, wouldn’t it be nice to use Head gestures to scroll?” – but once it was out in the field, he got the feedback from the user saying: “Hey, when I’m on a bus or in a meeting, I can’t really start to bob my head to scroll for what I need”.

The amazing thing here, that the MVP by itself is a platform for change by the actual users. Apps are being develop to enrich and fill the blanks Google left or missed. Some of them, in turn, will become part of the system, thus improving the product by the community knowledge and experience.

Use the Power Users

One of my eureka moments when reading the Lean Startup book was that although we strive to build a product to accommodate most users, we need the power users to give us feedback. They will be the most invested users. They will give the best feedback (uncensored and uncompromising).

And when they feel they are heard, they will be the ones who will advocate your product and help you to spread it.

In this case, Google has done a wonderful job by creating an exclusive group of users for there product (as they are doing all the time – remember Gmail beta?), which are invested (since they pied a lot of money for it) and are not afraid to criticize it when it doesn’t meet their expectations.

Although it seems trivial, think how hard it was for you to get users to use your product (and pay for it).

Feedback loop

All of the above serves only one purpose: to get feedback. To experiment. To understand what the user really need, not what we think he wants.

When you search the web about Glass, you get a ton of reviews, criticism and suggestions – good and bad. I’m sure the developers of Glass are gaining a lot from the feedback (UX, looks, app development, etc…).

But it’s not only a technical experiment. It’s also a grand social experiment meant to see how we, as humans, will react to this sort of product. This is a kind of product which changes our way of thinking and behavior – someone is always recording us, search data about us and our conversation topics while we talk to him. How will people react to this? How will governments and security agencies will? A very interesting question that I’m sure the management at Google are looking at the world wide reaction with a magnifying Glass (pun intended).

Entrepreneurship

Although almost everyone considers Google as one of the world’s trendsetters and a very innovative company, it is still amazing to see projects like this – which has a lot of risk and I’m guessing costs quite a lot, can happen within such a big company.

Lean Startup talks about how entrepreneurship can happen inside a big company or enterprise – islands of innovation, and how to create a culture for think and executing these kind of things.

It amazes me that Google is doing these all the time (with quite a lot of failures) and continuing to do so to find the next grand thing.

In conclusion

I don’t know if Glass will be successful or not, but I’m quite sure that from this experiment, Google (and other companies) are learning a lot.

I believe this is the way developing products should work, since the road to creating a successful one, is bumpy and long. What makes a successful product? This is the question all of us are trying to answer all the time.

Now, think about it for a minute. Can you use this concepts on your next idea? For your next feature?

Then a friend at work gave me a great idea: Why not dress up as H.A.L. 9000? I’ve used H.A.L. character to brand our Continuous Deployment efforts (especially in the launch day we had for the process a few months back), so this sounded like a awesome.

I had two ways to go: the easy way, and the cool way.

The easy way was to print H.A.L. on a t-shirt – cool but mundane. Why not create a working version – with the halo and all??!!

So, I started to collect the parts I needed:

For the halo: Netduino and a very bright red LED (“Wait!”, you might think, “Why not simply use a blinking light from Christmas decorations, or something from a kid’s toy? Well, I wanted the halo to be random and soft, and I couldn’t find one that matches my needs… And again, much cooler doing it with Netduino!), some long copper wires to connect it all, a 9V battery and a power connector for Netduino which a friend patched up.

I’ve used Netduino Plus 2 that I had for some time now. The challenge here was to create the halo so the light will be smoothly dimmed – in just the right way. For that I couldn’t just use a simple digital port, since it will create only an on-off effect. I consulted a friend which suggested using PWM, which is available as part of the Netduino framework (not the .Net micro framework). In essence, what it does is to blink the LED for short interval to create a virtual change in the light’s strength. playing with the interval with a sin function creates the dimming effect I wanted:

var led = new PWMPWMChannels.PWM_ONBOARD_LED, /*Frequency*/ 1000, /* Duty cycle - aka interval */ .5, false);
led.Start();

while(true)
{
    /* One on-off cycle */
    for (var startValue = /* Low end of the sin wave */ 5.212; startValue < 10.495; startValue = startValue + 0.1)
    {
        var ledIntensity = Math.Sin(startValue) * .5 + .5;
        led.DutyCycle = ledIntensity;
    }
}

To add the randomness I added Thread.Sleep at the top of the sin and in the beginning of a cycle:

const double initialIntensity = 5.212;
const double topIntesity = 7.8;
const double finalIntensity = 10.495;

var led = new PWM(channel, 1000, .5, false);
led.Start();

while (true)
{
    Thread.Sleep(RandomSleepTime(MaxOffTime));

    for (var startValue = initialIntensity; startValue < finalIntensity; startValue = startValue + 0.1)
    {
        var ledIntensity = Math.Sin(startValue) * .5 + .5;
        led.DutyCycle = ledIntensity;

        Thread.Sleep(FadeInterval);

        // While Sin is in max value, so is the led intensity. So, to create the random pulsing affect, we need to pause.
        if (startValue > topIntesity - 0.09 && startValue < topIntesity + 0.09)
            Thread.Sleep(RandomSleepTime(MaxOnTime));
    }
}

The entire code needed for this available at https://github.com/ysa23/Netduino-HAL

For the actual construct, I used a piece of cardboard to hold the LED and wiring. To create the halo effect to spread evenly, I created a cone from a piece of black paper, cut a hole in the middle (for the LED) and covered it with a printed picture of H.A.L. All of this will be based on a lid from a coffee cup, which I’ve cut to let the wires through:

The hardest part in all this was to find the transparent dome to create the final touch. For this I have to thank my wife who found a top from a milkshake cup.

The frame is constructed from a piece of flexible cardboard I colored in silver, shaped it as a cone, and cut a hole to fit the construct in.

After a couple of hours of glue (three types of glue!!!), I came up with this:

All of this was sewed on a black t-shirt I bought by my wife (at 1AM). The wires were also sewed from within the shirt running to the waist. The wires were connected to the Netduino which I put in a small pack attached to my belt. The end result was:

After it was all done, we went to sleep happy and excited about the accomplishment. At 10:00AM we woke up – late as hell to the party…

I can’t wait for next year… Do you have any ideas?

Both concepts are used a lot, and have several ways to defining them: Jezz Hubmble, Martin Fowler , Crisp’s Blog

I’ll try to redefine these from another point of view – the product’s point of view:

Continuous Delivery: The ability to deliver features to production whenever we want

Continuous Deployment: The ability to deploy our current without ANY manual intervention

The way I think about it is that Continuous Delivery defines the requirement and Continuous Deployment is an implementation. Deliver is a more abstract term

Another type of implementation for Continuous Delivery is Releasable Trunk – In this implementation our trunk/master branch is always in a state in which it is verified and valid for production deployment, but the deployment it self is triggered manually while someone is “over sighting” the deployment.

The decision to use one implementation or another is a product decision.

Remember that whatever you do, before you deploy, the version you want to deploy needs to be validated and verified (Site down == no money).

Why should we have someone behind the wheel?

Manual processes are slow! Manual processes are error prone!

When someone asks you the annoying question – “Why do you need to automate?” –  recite this statements above.

Besides these, automation has positive side effects on our process:

Improves the process

Automation requires us to take a deep look into the process and its components which usually results in finding the weak spots. This in turn makes us fix and improve them.

Improves code quality

Automatic build of our code leads to less mistakes heading to everyone’s work space (you’ll kick someone for committing code which breaks compilation). Continuous Integration of our features leads us to fix integration errors sooner in the development process, forces us to change design and code (and sometimes spec) very early in the process. Continuous Deployment causes us to deal with backward compatibility in a granular level – feature level, instead of a version level. This in turn, makes us to create “safer” code (feature switches, silent deployments/gradual rollout).

Improves testing quality

Since  we don’t have a man in the middle, validation of a version is done using automated testing. When we start to automate the validations we need to do for each version, we are actually making sure they will happen in each version before deployment (repetitive testing). If we fix a bug and we usually don’t want it to happen again, we write a test for it, and that’s it. It won’t happen again. If one test runs wild – causes missed validation (aka flakiness) and the whole automated process stops because of that – you’ll have to improve the test if you want your feature in production.

Sense of ownership

We feel safe if the ownership of the deployment is in someone (usually else’s) hands. It create a sense of virtual safety – “he is responsible for what is going to happen with the version. If there’s a bug, he is responsible to find it.”. This is called the bystander effect. As you can understand, this is not the culture you want. Collective ownership – everyone is responsible for production and the codebase. If we remove that someone that deploys the version, imagine how it will make the developers feel. We want to create a sense of – “no one is their – so I’ll need to be their” – this in turn makes the ownership of a version to become collective – among many people (or a blamefest…).

The need for speed

Imagine you had 200 testers, and 1000 test cases. How long would it take you to validate a version? What if you had 10 servers you need to remove from the Load Balancer, stop, copy-paste (at best) your new code, restart the site, add back to the LB? How long would it take you? What if you had 20?

A developer wants to automate because, well, we are a lazy bunch – “first time do it manually, second time write a script, third time write a framework”. We want to do our job – which is to code features. Everything else is waste. This is why we love Continuous Integration – it reduces the waste of integration. While Continuous Delivery = give us product power, Continuous Deployment = removing waste from the process.

Continuous Deployment: from commit to production automatically. (In other words – all you need is to code).

In the next post, we’ll take a deeper dive into the building blocks that composes Continuous Deployment.

In my dream each commit a developer does, is automatically tested and deployed to production without any manual intervention, without affecting the users of our app.

Each commit is tested in the CI before it is deployed. Once the CI is green, the deployment starts automatically (including all the commits in that successful build). Version is deployed on a staging server on which smoke tests are performed to validate the version on real data, and only after they pass the version is deployed to production gradually.

This vision is called Continuous Deployment – every developer’s dream…

Why is this dream so appealing to so many in the software industry?

Product wise – We want to expose features to our users as fast as we can and create an effective feedback loop, so our product will get better all the time, quickly.

Development wise – I always like to say that developers are a lazy bunch (in a good way). We don’t like repeating the same thing again and again. Deployments are like that – a repeatable task – before you go you want to validate your version with a suite of tests (usually the same suite is used + the tests you just wrote for your feature) and deploy it (without breaking our product). So we will automate the tests – both intrusive tests (ones that create data) that will run on our CI and smoke tests to validate that the changes did not break the product in production, and automate the deployment process. We only want to be involved if there is a problem in one of these.

In the next couple of weeks I’ll explain the road we are taking in order to bring us closer to this dream.

The next post will discuss the differences between continuous deployment and continuous delivery.

If the calls where made from the browser on the client side then it is simple – use fiddler in order to inspect and analyze the calls.

If you try to do the same with calls made from IIS, you will see nothing by default. Why?

Fiddler is a proxy server for your calls. When you open it it creates a simple listener on 127.0.0.1:8888 and listens for calls. It also registers it self in the registry under HKCU so all calls from processes of the current user are first redirected to fiddler’s address.

You see where I getting at?

IIS runs under a different user and as windows service, so the calls are not redirected to fiddler.

The most simple solution for this problem is to configure your app to use a proxy for it’s calls in the web.config file:

   1: <system.net>

   2:       <defaultProxy>

   3:              <proxy usesystemdefault="False" proxyaddress="http://127.0.0.1:8888" />

   4:       </defaultProxy>

   5: </system.net>

This configuration make all the external calls IIS makes to fiddler. Just make sure that fiddler is running when this configuration is set – if not your site will not work…

You can read more about it here.

After a series of interviews I was hired by Sears Israel as a software developer for Sears Social.

Sears Social is a new social experience which creates a new shopping experience using your social connections.

After only a couple of months, I have to tell you – I made the right choice. This place is amazing! Great professional people, Agile practices and great atmosphere.

It was pretty amazing for me to see first hand how to practice continues deployment, informative DSM with 25 participants which takes less then 10 minutes and a great bunch of people which makes you smile when you go to the office in the morning and even a bigger smile when you leave in the evening.

So stay tuned…

In the last couple of months I’ve been thinking about where I would like to work – what is the ideal workplace for me?

• Experienced and talented people from whom I can learn
• An interesting an compelling product – I think it is very important to believe I what you do.
• Agile environment, a team and management which believes in the agile process
• A place where I can have fun doing what I love – writing lots of code

If you are looking for a developer who loves the job and wants to write great code, and you are in Israel , you can contact me and I will send you my CV.

We have a web app which can send messages between the users that are connected to it. A user can send a message to any number of users.

The implementation is simple:

• When a user sends a message:
  • The message is saved in the database
  • Cache value is inserted for each recipient. The saved value is a time stamp which indicates when the user last received a message.
• Users that are suppose to see the message:
• The client is polling the server and asks if the server has new messages. It does that with an initial time which is loaded when the page is loaded and saved in JavaScript.
• If there is something new the client asks the server for the new messages (according to saved time) and the server returns the messages from the database and the last received time from the cache.
• Client time stamp is updated with the received time and continues polling

Pretty simple…

Unfortunately the implementation has a critical bug which causes clients to lose messages.

Consider this code:

using (var t = OpenTransaction())

{

    SaveMessageToDb(message);

    foreach(var user in message.Recipients)

    {

        Cache[user.Code] = message.SendDate;

    }

    t.Commit();

}

Can you spot the bug?

Let say we have 1000 users that will receive the message. After saving the message to the db the code updates the cache for each client. Each client polls the server at a constant interval. Let say that user AAA polls the server – his cache key is updated but the loop that updates the cache is not yet over. The user send a request to get the new messages but because the transaction wasn’t committed so the new messages will not be returned from a query – the loop that updates the cache is still going. The client then updates the last received time – and so the user will never get the message.

I must say that this was a very frustrating bug…

The solution was composed with changes in both the client and the server:

Client:

We changed the client script so that if the server claims that there are new messages but the clients receives an empty list of messages then the received time is not updated and another request is sent to the server until new messages are received.

Server:

What we need to do here is to make the cache join the transaction. In this case we couldn’t because the transaction was a local oracle transaction – not a distributed transaction. If it was we could use System.Transactions to implement a transactional cache.

In this case we used an event which is triggered when the transaction is commited (in TransactionScope there is the AfterCommit event, in our case we had to implement this ourselves). It looks something like this:

using (var t = OpenTransaction())

{

    SaveMessageToDb(message);

    t.AfterCommit = (sender,e) => 

                    {

                        foreach(var user in message.Recipients)

                        {

                            Cache[user.Code] = message.SendDate;

                        }

                    }

    t.Commit();

}

Way not simply update the cache outside the transaction?

Because we don’t actually know when the transaction is committed. Maybe another method calls this method with a transaction of her own and the commit is actually done there (the OpenTransaction method actually returns a current transaction if one exists already).

Important:

Our solution was infect a little more complex because we wanted other modules that are in the transaction to be able to get the updated value from the cache even if the transaction wasn’t committed yet:

using (var t = OpenTransaction())

{

    SaveMessageToDb(message);

    t.AfterCommit = (sender,e) => 

                    {

                        foreach(var user in message.Recipients)

                        {

                            Cache[user.Code] = message.SendDate;

                        }

                    }

    DoSomthingThatUsesCache();

    t.Commit();

}

So we created a class which wraps the cache, and contains a local dictionary which contains updated values, and access them when needed. If a value is updated in a transaction then wrapper registered to the event and updates the real cache when the transaction is committed. We called the class TransactiveCache:

using (var t = OpenTransaction())

{

    SaveMessageToDb(message);

    foreach(var user in message.Recipients)

    {

        TransactiveCache[user.Code] = message.SendDate;

    }    

    DoSomthingThatUsesCache();

    t.Commit();

}