Mike Chaliy's Personal Site

Where do we keep our application configuration settings?

Fri, 20 Jan 2012 16:33:50 +0200

In code.

Precisely this is not true. We have single configuration setting. This is environment name where code will work (e.g. Local, Staging, Live). Everything else is configured in code based on environment name, machine name and sometimes on phase of the moon.

What we do when some reconfiguration needed? We redeploy application. 8 minutes and we are live (my bad btw, we need to optimize this, 8 minutes is huge time, especially in some sorts of fackups).

One of the huge benefits (and why I write this post now) is that you can always take your app and execute in any other host. For example right now application works in console exe to fix some data with our domain model.

Жартома ми називаємо цей стиль “гопники”. Я точно не знаю як це перекласти. Мені подобається.

Web-related development tools I used in 2011

Mon, 02 Jan 2012 15:26:00 +0200

Client side web is quite new for me. Last year was interesting in this area. Let me list some tools:

Bootstrap, from Twitter - this is killer. You add link to precompiled CSS file in your page and it starts look just well. For utility sites where you do not want to spend time on design this is mast have. Another related project, jQuery UI Bootstrap, adds Bootstrap look and feel to jQuery UI.

Knockout - magic toolkit that brings MVVM for you. As for me too much magic. But overall is very nice. Thanks to Roman Gomolko for introduction. Also some people suggested that Backbone.js is similar but with less magic, so this is something I will try this year. Another library I need to try is RxJS.

CoffeScript - nice syntax sugar for JavaScript, write less. However I wouldn’t try if integration to Visual Studio exists. Mindscape Web Workbench - extension that provides syntax highlighting and what is more useful, compilation to JavaScript when you press Ctrl+S (it uses code generator). So technically you add foo.coffee file and then reference foo.js, very frictionless.

ReSharper JavaScript. Few years ago I decided to go with GWT, just because maintanance large JavaScript application was nightmare. Now I am slowly change my mind and JavaScript support in ReSharper is one of the best helpers on this journey :). Unfortunately, it does not support CoffeeScript :(. Anyway, this is example how tool can change your mind about another tool.

WCF Web API - next generation of WCF that will make HTTP stuff more user friendly. It quite unstable(it’s preview version), also Glen Block is not on it anymore, so I cannot say that everything is light. However I really like it and use it in some of my pet projects with success.

ELMAH - have to point this tool, but I do not really like it. First of all it look like alien in ASP.NET MVC world (fix). Second it is not useful. There is no daily digest, no filter on UI, no grouping or another stuff that could help to analyze errors. This is another time I am thinking about another pet project.

And of course JQuery with many small JQuery plugins.

Something like this…

PsGet News #1: features, new modules, challenges

Sun, 18 Dec 2011 15:43:00 +0200

Some new features:

Install-Module now has default alias inmo, inmo posh-git will do the job
Install-Module now supports tab completion, try inmo posh , watch slow motion video.

Sidenote: to update PsGet itslef, execute install-module psget -force. PsGet is also on PsGet :).

Some new modules:

Find-String , more details from author
Send-Growl, growl is notification system, if you have it installed you can now use this module to post notifications easy
pswatch, cmdlet continuously monitors a directory tree and write to the output the path of the file that has changed.

New challenges:

Some interesting modules are hosted on CodePlex (e.g. PowerTab, PSCX). Unfortunately, CodePlex does not provide direct links to download this modules. There is no clear solution for this issue except to host compiled versions of this modules directly in PsGet repository. This is something we discuss now.
TabExpansion, Prompt this stuff could conflict in some modules. For example, try install posh-git and posh-hg, you will need to fix issues manually. I added ticket to resolve this stuff.

In news:

Haacked - Better Git with PowerShell
Scott Muc - Adopting PsGet for Powershell Module Management
José F. Romaniello - Continuous Testing scripts with Growl

Script to commit with message taken from JIRA ticket

Sat, 12 Nov 2011 12:43:08 +0200

Every day we commit. In some cases we commit resolutions to JIRA tickets. In most cases commit messge is title of the ticket and may be few additional words about what actually was done. Considering this, I wrote small script. It connects to JIRA, takes title of the tocket, optionally appends what you will say to it and eventually commit everything to Mercurial. Usage is simple:

Commit-Project ABB-499 "Foo bar"

This will commit with message from issue ABB-499 and “Foo bar” at the end. Last message is optional.

This script requires PsGet.

Enjoy.

Health monitoring for our production

Sat, 12 Nov 2011 12:14:56 +0200

Few weeks ago we have small fail, when on staging environment one of our schedulers failed to start, so part of our background tasks just didn’t started. Few months ago we have had an issue, when wrong configuration on one of the web facing boxes have invalid serialization configuration and so public API calls to load balanced farm was failing on random basis. Why I am talking about this? Well, week ago we eventually decided that we need kind of health monitoring for our environment.

To the time being we already have some integration tests (NUnit) that test our public endpoints and environment stuff. The only problem that they run only after deployment builds.

The idea is simple. Schedule integration tests to run every XX minutes.

I think most build servers can do this without problems. I do not know why we didn’t made this ages ago.

Now, I am about to write some tests that will use heuristics based on usage data of our services, so we probably be able to early identify if something goes wrong. Later we will expand this to check database health, disk usage and so on.

If this stuff will work for us, we will open source our library with assertions for environment.

Enjoy.

How do we automatically deploy with Mercurial and TeamCity

Mon, 18 Jul 2011 23:30:00 +0300

Few weeks ago we moved from SVN to Mercurial. This transition triggered some changes to our deployment process and we decided to play with something interesting. We decided to implement fully automatic deployment. So the idea:

First of all we have three permanent branches:

default, this is a Mercurial name of trunk, main development occurs here. If you need something big you make branch, and after work is complete you merge back to default.
staging, this is common place where code is ready for staging lives. In most cases code first appear in default and then get merged to staging. Sometimes if we need fast fix on staging, we do it in staging branch and then merge back to default. We also experimenting with merging code from named feature branches, this scenario works, but requires some communication.
live, similar to staging, but dedicated to live. In most cases code comes from staging.

Second is we have TeamCity configured to pull changes from this branches and if something changed automatically build. In case of staging and live branches build also includes deployment to servers. Deployment is implemented using simple Powershell scripts and robocopy.

Easy and it works. Now deployment is just simple commit to the corresponding branch.

Some less important notes:

If after deployment, we found issue and want rollback, two options here. First is reverting changes in corresponding branch (this is easy, because each build has its own commit). Second is using nice “Run custom build” feature of the TeamCity. You just need to select previous commit and run build again.
Messages of the commits to staging and live should be descriptive, as they are gone be the only information you see on build server.
This is easy to write script that will automate sequence of commands to commit to staging or live branch. This is one from me.

Enjoy!

Execute business logic in separate application domain

Tue, 12 Jul 2011 01:41:00 +0300

From time to time, I am trying to simplify code that I write on daily basis. One of the ideas that keeps bugging me is using more static stuff. As you may know, general rule of thumbs is to reduce static state, singletons and other static code. This is true. So everything I will write later should be considered as experimental.

One of the problems with static code is shared state. For example, you have class with static field, two threads access to it… and boom! you have permanently hidden problem. Traditional mitigation to this problem is just not to use static state. But what if we try to create static “instances”? If I am right we have two options, first is to just start new process and second is to create new AppDomain right in our current execution process. So I decided to give a try to second option. Test code is bellow:


using System;
using System.Diagnostics;

public class Program
{
    public static class Runner
    {
        public static int Test1 = 0;
        public static void CodeInChildAppDomain()
        {
            Console.WriteLine("Test {0} + {1}", 
                  AppDomain.CurrentDomain.FriendlyName, ++Test1);
        }
    }

    static void Main()
    {        
        var s = new Stopwatch();
        s.Start();

        for (var i = 0; i 
            {
                Runner.CodeInChildAppDomain();
                Runner.CodeInChildAppDomain();
            });    
        }

        s.Stop();
        Console.WriteLine("Average: {0}ms", s.Elapsed.TotalMilliseconds / 1000);       
    }
}

gist

Runner.CodeInChildAppDomain will run in child domain. Static field Test1 is for demonstration. Code will run 1000 cycles and average time is about 4ms. Adding Rx assembly to the Runner increased average up to 6ms.

Conclusion is simple, if 6ms is ok for each your operation you can try. Enjoy.

IEnumerable could be considered harmful...

Tue, 05 Jul 2011 23:35:54 +0300

… well, of course only under certain circumstances, but hey, I am quite experienced developer, and today I once again wrote code that killed app in unexpected way. So the point of this post, do not undervalue lazy nature of the IEnumerable. Or do not use IEnumerable as arguments in your API. Both is OK.

Later is explanation for idiots like me. So take a look on this code:

Simple LINQ query and simple usage of the results. Problem comes when you start using results. This is very easy to write code that will re-evaluate your nice LINQ stuff again and again. And this very easy to introduce unexpected side-effect. And this quite hard to find this side-effect.

In my case I have had quite large LINQ query, that among other things called method that was responsible to find some entity and if not found, create it. And query was evaluating six! times for each item. As the result six new records in database, every time I hit this code. After I figure out what is wrong, fix was an easy job. However I still really ashamed with this code.

From now I think to try arrays. This is does not seem to kill API usability, and even looks more compact. That’s it. Hope you do not write such stupid code.

PsGet got its very own central directory

Sun, 01 May 2011 22:25:00 +0300

There are two good news. First of all, full install of the PsGet can now look like:

(new-object Net.WebClient).DownloadString("http://bit.ly/GetPsGet") | iex

I also thinking to replace http://bit.ly/GetPsGet with something like http://g.psget.net. With its own name it should look more trusted to install.

Second and much more notable is actually the fact that PsGet now supports simple central directory of the modules. Of course, for now it almost empty. But I will try to fill it with some intersting modules. So what we have…

To install a PowerShell environment for Git from https://github.com/dahlbyk/posh-git/ execute

install-module posh-git

To install psake, a build automation tool… now with less XML…

install-module psake

To install poshcode utilities from poshcode.org

install-module poshcode

Does it look nice? I hope so :) You can also search for modules. Wildcards are supported:

 Get-PsGetModuleInfo ps* | format-list

Btw, right now is good time to start suggesting modules you want to see in directory.

P.S. Also I am thinking about short alias for install-module (e.g. install or psget or .. )

Download files with PowerShell

Tue, 26 Apr 2011 15:25:48 +0300

Few days ago I made introduction to PsUrl - small PowerShell module to GET and POST stuff to Web. One of my intents was to resolve situation when there was no build it PowerShell way to download files. I was wrong. PowerShell has everything you need.

BitsTransfer

This is warper for BITS API available in Windows. And yes this module comes bundled with PowerShell 2.0

Import-Module BitsTransfer
Start-BitsTransfer http://download.microsoft.com/download/9/5/A/95A9616B-7A37-4AF6-BC36-D6EA96C8DAAE/dotNetFx40_Full_x86_x64.exe

This will download file and save it to dotNetFx40_Full_x86_x64.exe…

help about_bits_cmdlets

Happy downloading!

Transfer files with PowerShell Remoting

Tue, 19 Apr 2011 13:20:00 +0300

I am in progress to setup full server remotely from PowerShell command line. Now I have IIS installed, Tasks scheduled, .NET 4.0 installed and so on. What I need next is to get some files needed to setup application. I have them on my local computer. So few options here: a) updalod to Dropbox, and download them with PsGet b) Upload them with PowerShell Remoting. First option is too easy. So i decided to go with second.

Ok, so SHH can easy transfer files. What about PowerShell? Well, almost nothing. There is nothing built-in. In community I found two options: PModem File Transfer Protocol, should work, but unfortunately file with implementation is not here any more. So second option. Blog post More PowerShell Remoting is not strictly about transferring files, but it has nice idea. You can actually use Invoke-Command with script that will have file embedded as a string. Of course, this is not bad to have this idea packaged to PowerShell module. This is how PsRemoteFiles micro project started.

PsRemoteFiles

In short words, it takes file on client, encode it as base64, then generate script with encoded data included and execute it on the remote server. Easy!

copy-remoteItem Tests.ps1 -Credential "Comp1\User01" -ComputerName "Comp1"

For now, implementation is quite naїve, for example it loads file in memory twice. This is not really big problem if you have small files, but with big one… Anyway, next plans is to chunk file transfer. Also name does not seem to express what exactly this command do. So I have some work to do :).

Project home page is https://github.com/chaliy/psremotefiles

Enjoy!

Introducing PsUrl and PsGet

Sun, 17 Apr 2011 11:41:00 +0300

For about three weeks I am sick with PowerShell. Of course, I used it for few years now, but now everything is different. I am automating stuff that I never though about. And this automation requires some helpers for comfortable work. This post is all about such utilities.

PsUrl

This story started from simple task, I needed download stuff and I have list of URLs. So the idea is Get-Content Urls.txt | .... The problem with ..., there is more or less nothing to download stuff from the web in PS. Of course we can use WebClient from .NET.

(New-Object Net.WebClient).DownloadString($Url)

Well, this works, but not really super native, for example you cannot pipe URL directly from Get-Content. So I decided to implement simple wrapper and package it as a module. This is how PsUrl micro project started. After installing PsUrl.psm1 module, you will get few commands. Most interesting is Get-Url.

Get-Url http://example.com

This will basically download stuff from example.com and return it as a string. Of course you can pipe in and pipe out.

Get-Content Urls.txt | Get-Url $_
Get-Url http://example.com | Set-Content example.html

Easy. The most visible problem here, is that you need install it. This is not super tricky, but rather annoying. This is how PsGet micro project started.

PsGet

Inspired by NuGet, I started to think how to simplify module installation in PS. I found Install-Module script by Joel Bennett. It takes any single file module and install it to user modules or for global use. Do you see what I see? If you combine it with download stuff I already have, installation of the modules can became simple as

Import-Module PsGet
Install-Module https://github.com/chaliy/psurl/raw/master/PsUrl/PsUrl.psm1

Nice? Last command actually installs previous PsUrl module. I have huge plans (and hopes) about this project. Right now it supports installing multi file modules, zipped packages from web or local file. Will support NuGet, possible registry of modules.

And btw, it has single line installer.

(new-object System.Net.WebClient).DownloadString("https://github.com/chaliy/psget/raw/master/GetPsGet.ps1") | invoke-expression

Downloads and installs PsGet for your user account.

That is all for now, in few days I will describe how remotely install and fully configure web server without RDP. Enjoy!