Erata.NET

Metrics.NET Update

2014-10-18T00:00:00+00:00

Since March, i've been working on this Metrics.NET library.

I'm using it in a few production projects and i've become addicted to having everything measured and having the metrics information available when making decisions. Especialy when writing systems that are asyncronous where it's difficult to rely only on logging for understanding what is going one.

What it is

Metrics.NET library is a .NET Port, with lots of additional functionality, of the awesome Java metrics library by Coda Hale.

To paraphrase the Java library description - Metrics is a library which gives you unparalleled insight into what your code does in production. Metrics provides a powerful toolkit of ways to measure the behavior of critical components in your production environment.

The full description and documentation is available on the GitHub Wiki.

Plans for the future

For the near future, my focus will be on a solution for remote, persistent metrics. This includes the integration with Graphite/Ganglia/InfluxDb and maybe others. But for some odd reason none of this existing solutions runs on windows. Which is strange since they are all written in cross platform languages. We now run C# on unix, but we can't run go or python on windows.

And since I have to deal mostly with production running on windows I'm also considering storing metrics in the EventStore or even maybe in something like LMDB or Ayende's Voron. Or even postgres if it comes to that.

Any feedback, especialy on the public API of the library would be appreciated.

Metrics.NET

2014-04-13T00:00:00+00:00

For the past couple of weeks I've been working on a new pet project, a .NET port of Coda Hale's Metrics library.

The Metrics.NET library provides a way of instrumenting applications with custom metrics (timers, historams, counters etc) that can be reported in various ways and can provide insights on what is happening inside a running application.

The library is published on NuGet can be installed with the following command:

Install-Package Metrics.NET

Supported runtimes: .NET 4.5.1, .NET 4.5, .NET 4.0, Mono 3.4.0 ( tested on OsX ).

The API of the library might change until a 1.X version will be made available.

Documentation:

Quick Usage Sample

public class SampleMetrics
{
    private readonly Timer timer = 
      Metric.Timer("Requests", SamplingType.FavourRecent, Unit.Requests);

    public void Request(int i)
    {
        // measure until disposed
        using (this.timer.NewContext()) 
        {
            // do some work
        }
    }
}

For more information on the library see the GitHub repository and the GitHub Wiki.

The Expert

2014-03-30T00:00:00+00:00

I'm sure everybody felt like this expert at least some times. It makes me laugh and cry an laugh and cry...

Please share with everybody.

NodaTime - Coherent date time types for .NET

2014-02-19T00:00:00+00:00

For the last two weeks I've been working on an internal project. Being a low risk project, I had the option of introducing new technology that was harder to introduce in other projects. I'm hoping to write a few posts about my experiences with NancyFx, EventStore & NodaTime, to name a few of the things I'm using.

One library I've been looking for a chance to try is the NodaTime library, written by the famous Jon Skeet.

The experience of using NodaTime in a few words:

extremely low learning curve
perfect API
everything is where you expected it to be and named as you expect it to be named

I can't say that I've experienced major issues with the native DateTime form .NET. But I have often felt that it is not a perfect match for representing a concept or not being clear enough in a public APIs. When trying to represent a particular day, or a time in a timezone or UTC the DateTime abstraction did not capture the full intent.

So far the most useful types for me from NodaTime have been: Instant & Duration.

Instant represents a specific point on a clear and unambiguous time axis. This is a common representation also found in other languages - like the time_t type in C - representing the number of ticks from the Unix epoch (January 1st 1970 00:00 UTC).

The "zero point" used everywhere in NodaTime is the Unix epoch: midnight at the start of January 1st 1970, UTC. (UTC itself did not exist in 1970, but that's another matter.)

Duration represents an exact number of ticks on the same time axis. This is similar to the TimeSpan type from .net.

I have also used a LocalDate type to represent a birth date.

As I've said I'm using Nancy to expose an HTTP API and EventStore for persistence. For moving data across this boundaries I'm using the awesome and now almost standard Json.NET library. My first worry was that I'm going to have to convert the types from NodaTime to and from a format supported by API and events consumers.

Almost as expected, I've found that NodaTime already provides a NuGet package for this. Flawless Json.NET serialisation is available by installing the NodaTime.Serialization.JsonNet package.

Conclusion

If you value the expressiveness and correctness of your code, I encourage you to give NodaTime a try. It risks becoming an addiction, but it is definitely worth it.

As a starting point continue reading the excellent User Guide for NodaTime.

In the end I'd like to thank Jon Skeet and the rest of the authors that made this awesome library available for the rest of us.

Moving to Jekyll

2014-02-08T00:00:00+00:00

With hope that this will also mean writing more often, I've finally decided to move to a modern blogging platform.

Moving away from Wordpress

Wordpress is a mature blogging platform, full of extraordinary features. You get automatic updates, easy updates for plugins, nice interface to manage the whole thing and a lot of available plugins & themes.

Why move away from this?

It requires hosting, it is kinda slow and it is not exactly low ceremony.

Why Jekyll

You get to host it for free on Github pages.
You get to write markdown.
You get full control over the theme.
You are free to use modern web technologies.
You get to manage content with GIT.
It's low ceremony - write, commit & push to publish.

Update

I've finished migrating posts, and content to Jekyll. Even if some of the content required manual fiddling, it has not been a difficult experience. Now waiting for Google Webmaster's Tools to tell me what I've screwed up.

One thing I've noticed is that my blog now is blazing fast - which is normal, considering it's just a bunch of static web pages hosted on Github.

Great slides from Rob Ashton

2013-07-02T00:00:00+00:00

Verry good advice by Rob Ashton about TDD and testing in general. My favorite line from the slides: "Listen to the pain"

Listen to the pain

Focus on the outside, testing in ASP.NET MVC from Rob Ashton

The Repository Pattern

2013-06-19T00:00:00+00:00

In the context of a question asked on the RavenDb mailing list I've decided to share my thoughts on the Repository Pattern.

It is a common mistake to consider the Repository pattern a generic "good practice", without actually considering what it is good about it.

According to Martin Fowler's P of EAA a Repository

"Mediates between the domain and data mapping layers using a collection-like interface for accessing domain objects."

This makes a lot of sense, and even if you are dealing with an anemic domain model, you don't want to have data mapping code mixed in with your domain object code. The key here is "data mapping code". You don't want code that deals with the internal representation of your data storage (whatever your actual storage is) mixed with code that performs business operations on your objects.

Back in the day, when we used to write SQL code in our apps, it was essential that you implemented a layer that encapsulated the SQL queries and SQL mapping code from the rest of the application. With the technology choices available today, we rarely need to write SQL code or SQL mapping code. Today we have NHibernate, Entity Framework, Linq2SQL, RavenDb etc.

Nhibernate

Consider NHibernate - the session you open acts like an in-memory collection of domain objects, where you can query, fetch and add domain objects without needing to do any SQL specific stuff. The data-mapping and SQL stuff is encapsulated in your NHibernate mappings code. The NHibernate session IS your repository. It also acts as a Unit of Work, but that is a different pattern.

RavenDb

The same thing happens with RavenDb's client. It provides a "session" which encapsulates all the persistence details like http calls, (de)serialization, caching, concurrency, a lot of safety checks, while acting like an in-memory collection of domain objects. When you do session.Load() or session.Query(), you do this in a very similar way to what you would do when dealing with an in-memory collection of domain objects. The RavenDB's session IS your repository.

Complete Abstractions?

Do HNibernate or RavenDb sessions hide ALL the details of the persistence engine? No, but you don't want them to be hidden from you. Except for the most trivial applications, you can't ignore the way your data is persisted. You are going to need to handle relationships between the objects in your persistent store, you are going to need to handle concurrency problems and you are going to have to write efficient queries. All these concerns are application specific and no abstraction is going to be good enough to handle all your application's use cases. But the concept of session from NHibernate or Raven does provide an abstraction over the persistence details, while exposing to the developer ways in which efficient operations can be performed.

There is also a very strong resemblance between the session from RavenDb and the session from NHibernate, but that is probably because Ayende is (was?) one of the most active and knowledgeable developers and "evangelist" for NHibernate, and also because the session is a good model for a unit of work. Ayende also has some good posts about the Repository pattern.

Testing

One question that always pops up in the context of the Repository pattern is Unit Testing. How do you mock the ISession from RavenDb or NHibernate. Well ... you don't. It's very hard to mock a non-trivial repository. Instead you "use the real thing", just in an in-memory form. RavenDb's ecosystem provides an embedded, in-memory store that is perfect for running unit tests against the real store. NHibernate provides a SQLite back-end which, while not perfect, can be used for fast tests against a real sql implementation.

Conclusion

In conclusion, before you start defining the IRepository which i know sounds like a cool "good practice", stop for a minute and check if you are not already using an abstraction over your persistence.

Clean Coders .COM

2012-08-31T00:00:00+00:00

Lately I've started watching episodes from CleanCoders.com, a podcast created by Uncle Bob ( Robert C. Martin ) and i must say they are excellent.

I must confess that when I've purchased the "Names" episode I was a bit skeptical about how much can be said about how to name the things you use in your code. I have already been paying much attention to names and overall "cleanness" of my code so I was not expecting to learn a lot of new things, but as usual, Mr. Martin as blown my mind with his way of explaining the "hows" and "whys" of doing things.

I can say almost the same about the "Functions" episode. Very important small details that even if I was applying previously in my code, I can now actively think about them when writing code.

Also for me one important gain from watching the videos was that now I can better explain to others the importance and details of clean code using the arguments and reasoning of Uncle Bob.

I highly recommend following all the videos on cleancoders.com. No matter if you are a junior dev or have lots of years of experience I'm sure you will gain a lot of knowledge from them. The videos are created in a non-conventional and funny way which makes them pretty entertaining to watch.

Thank you Uncle Bob for the great videos.

Continue to Clean Coders.

I'm becoming a fan boy

2012-02-29T00:00:00+00:00

I could not resist anymore and i've bought a mac. I've been contemplating with the idea of getting a laptop lately and the mac book air was always coming at the top of the search results.

Finally i could not delay it anymore and one morning i wake up, went to the apple store, paid 2000$ and got the all new shiny 13 inch Mac Book Air.

My expectations were to get some good and reliable hardware, with a toy OS where i can use bootcamp to run windows 7 for everything. Boy was i in for a surprise.

The hardware is not good - its 10 levels above good, it extraordinary, everything just works great, from wake up time which is practically zero to install mono develop and create my first asp.net app with mono in less than 1 minute. I don't care what anybody says apple is light years ahead the competition when is comes to hardware quality and especially design. Everything just fits together and leaves you the feeling that somebody has invested a good amount of thought in every little aspect of the product.

Now comes the true surprise: Mac OS X 10.7.3 Lion - my first OS X system.

I don't even know where to begin - it took me less than a minute to make a screen shot of the above window ( command-shift-4, space, click window ) and drag it in the MarsEdit that i'm using to write this. I was writing an email in italian and i'm terrible at writing italian and i notice that iMail starts correcting my words in proper italian - without ever configuring anything related to italian. I was able to setup email accounts without any frustration, setup all IM accounts, twitter, vpn, RSS reader and everything i use daily without having to spend a lot of time google for it or trying to guess what it is that i'm supposed to do.

I've completely renounced at the idea of using bootcamp to run windows. Sure i'll install parallels or vmware to run a VM with windows mostly for Visual Studio, but for now i'm in no hurry to do that.

Final word: If you tried it already you have most probably switched to OS X, if you have not tried it and you have the $$ go for it and you will enjoy a new way of using a computer.

I'll just enjoy the honeymoon with my new powerful "toy".

*UPDATE: *and then i found parallels with coherence ... and i still don't believe this it true ... i run visual studio like any other mac app. Somehow i've been in this field for more than 15 years and i just discover this now... and i don't think there is any going back. There is absolutely no reason for to use anything else. And this comes from someone who has used FreeBSD as a main OS for years. Again - if you have the money, it will blow your mind, don't hesitate another minute.

Hopefully i'll keep the posts coming.

Test Data Generator

2011-11-17T00:00:00+00:00

I've pushed to github a new project, TestDataGenerator that should help with filling random objects with data. I felt the need for a tool like this when testing various serialization techniques and persistence strategies.

Basically this utility should construct the instance of an object using a public constructor, and fill all it's public, writable properties with random data.

Sample usage:

 1 class Sample
 2 {
 3  private readonly int intValue;
 4  private readonly string stringValue;
 5 
 6  private Sample()
 7  {
 8      this.intValue = -1;
 9      this.stringValue = null;
10      this.StringProp = null;
11      this.DateProp = DateTime.MinValue;
12  }
13 
14  public Sample(int intVal, string stringVal)
15      :this()
16  {
17      this.intValue = intVal;
18      this.stringValue = stringVal;
19  }
20 
21  public int PrivateInt { get { return intValue; } }
22  public string PrivateString { get { return stringValue; } }
23 
24  public string StringProp { get; set; }
25  public DateTime DateProp { get; set; }
26 }
27 
28 [Test]
29 public void Catalog_Can_Create_Using_Consutrctor()
30 {
31  Catalog catalog = new Catalog();
32  object instance = catalog.CreateInstance(typeof(Sample));
33 
34  Assert.IsInstanceOfType(typeof(Sample), instance);
35  Sample sample = instance as Sample;
36 
37  Assert.AreNotEqual(-1, sample.PrivateInt);
38  Assert.IsNotNull(sample.PrivateString);
39  Assert.IsNotNull(sample.StringProp);
40  Assert.AreNotEqual(DateTime.MinValue, sample.DateProp);
41 }

For more samples see unit tests.
I'll try to add more cases ( abstract class, interface implementation discovery ) in the future. You are welcome to submit sample classes, preferably as unit tests.

RavenDb EnsureDatabaseExists extension method

2011-11-03T00:00:00+00:00

Just a quick hint for others like me, who spent some minutes searching for the EnsureDatabaseExists method when trying to use RavenDb with Multi-Databases.

EnsureDatabaseExists is an extension method on IDatabaseCommands defined in the Raven.Client.Extensions namespace.

To make it work you need to add a using statement for this namespace.

 1 using Raven.Client;
 2 using Raven.Client.Extensions;
 3 
 4 using (DocumentStore store = new DocumentStore()
 5 {
 6     Url = "http://localhost:8080/" 
 7 })
 8 {
 9     store.Initialize();
10     store.DatabaseCommands.EnsureDatabaseExists("SomeDatabase");
11 }

Cassette - Asset management for .NET web apps

2011-08-31T00:00:00+00:00

A few weeks ago i've started using Cassette, a very nice asset management tool for .net web applications. It will basically take your assets ( javascript, coffescript, css, images ) and do the right thing with them ( combine, minify etc).

While working with it i had a few issues which I've managed to solve and i would like to share them.

File not found in module.txt

The first problem was related to a .css file for the jquery-ui theme that was present in the ~Content/themes/redmond/jquery-ui.medsentry.css folders. My module.txt looked like this:

themes/redmond/jquery-ui-1.8.16.custom.css
Site.css
# other css files

After compiling cassette locally and using the debug symbols to step trough the code I've noticed that cassette was storing a list of files ( assets ) it found in the Content folder but the path separator was "\" instead of "/". After changing the slashes in the included paths in my module.txt everything works as expected. Now the module.txt looks like this:

themes\redmond\jquery-ui-1.8.16.custom.css
Site.css
# other css files

Using Cassette modules with IIS6

Due to some external constaints i had use IIS6 on a test instance of the application. After the deployment I've noticed that the images in the .css files are not working when using cassette modules to combine the files. This was due to cassette rewriting the paths and using Routes to manage them. In IIS6 you have to do one additional step for this to work. Open IIS Manager, locate your application and open the properties window. On the Virtual Directory tab open the Configuration dialog, located in the Application settings section of the tab. In the Wildcard application maps section click Insert and add "C:\WINDOWS\Microsoft.NET\Framework64\v4.0.30319\aspnet_isapi.dll" or whatever the path to aspnet_isapi.dll is on your system. Click Ok and Apply and enjoy.

Another way to manage the images in the css files is to embed them as data links. Cassette supports this, BUT Internet Explorer <= 7 does not. Also IE 8 has some size limitations.

Image Optimizer for Visual Studio

Not related to cassette but i'd also like to recommend the Image Optimizer for Visual Studio Extension. Exactly what you would expect from an extension with this name :).

NHibernate Cache Provider for Membase

2011-05-25T00:00:00+00:00

I've written a cache provider for NHibernate which uses Membase as a cache engine.

You can find the souce code on github.

Example configuration can be found in the test project.

To use the cache provider add the following lines to the hnibernate configuration:

<property name='cache.provider_class'>Nhibernate.Caches.Membase.MembaseCacheProvider, Nhibernate.Caches.Membase</property>
<property name='cache.use_second_level_cache'>true</property></pre>

If you encounter any problems with it drop me a line.

.NET Serialization Choices - Raven.Json

2011-05-03T00:00:00+00:00

This post is an addition to .NET Serialization Choices post.

I've added to the SerializationTests Project the new serialization implementation in Raven.Json.

The Raven.Json Serializer is an extension to Newtonsoft Json.NET serializer and as expected the results are similar. Also the Pros & Cons are similar:

Raven.Json

Optimized DOM for Newtonsoft Json.NET

Pro: JSON Based, human readable, platform independent , fast, passes almost all the tests except the ones where the message has public readonly fields that have dirrerent names from the constructor parameters.

Cons: Without automatic testing, naming a constructor parameter different from the field can cause hard to notice bugs where certain fields are not deserialized. It should however be possible to write a deserializer based on the existing one which throws an exception if there are doubts when deserializing.

.NET Serialization Choices

2011-04-08T00:00:00+00:00

Introduction

Serialization is not a trivial problem in any language. In .NET there are quite some choices available for serializing/deserializing objects. Each available option has it's strengths and weaknesses.

I've started a project on git hub SerializationTests where i'm trying to determine what is supported by which library and draw a few conclusions that should be helpful when designing serializable objects and in general when dealing with serialization.

I've started this project because I've been bitten too many times by issues related to serialization, either trying to send a message with an System.Uri property in NServiceBus, or trying to deserialize an object from Json which had an ID public readonly field named different from the corresponding parameter in the constructor. In the end i'm trying to put together a few tips that i hope will be useful in the to myself and others.

Tested Implementations

The following serialization choices have been included in the SerializationTests project:

Update: Added Raven.Json Serializer - see post

Tested Messages

When designing an object that is meant to be used as a message, event or command there are a few approaches you can take:

public readonly fields and a constructor with appropriate parameters ( my favorite )
public properties with private setters and a constructor with appropriate parameters ( if you don't like public fields )
public fields, no constructor ( only generated default )
public properties, no constructor ( only generated default )
interfaces with getters and separate internal implementation (ex for events )
interfaces with getters and setters ( to help serialization )

I say i prefer the first choice since i find it the one that expresses my intentions the best. I want immutable data transfer objects. Also i have to mention that the first time i've seen this approach was in a video by Greg Young about event sourcing and CQRS architecture. To get the idea an event it's defined like this:

 1 public class PersonCreated
 2 {
 3     public readonly Guid AggregateId;
 4     public readonly string Name;
 5     public readonly string Street;
 6     public readonly string StreetNumber;
 7 
 8     public PersonCreated(Guid id, string name, string street, string streetNumber)
 9     {
10         this.AggregateId = id;
11         this.Name = name;
12         this.Street = street;
13         this.StreetNumber = streetNumber;
14     }
15 }

Serializers Conclusions

Disclaimer: I have not used all this serializers in real projects and i'm by no means an expert in any of them. Before adopting one of them do you research and try to see if they have any other drawbacks that might not be acceptable in your projects. If i don't mention Cons for one, it only means that i have not been interested in it so much to research it deeper.

BinaryFormatter

Pro: The BinaryFormatter included in the .NET Framework passes all tests except the DataContractOnly test which is expected since it relies on the presence of the [Serializable] Attribute.

Cons: Very Platform dependent. Assembly version dependent. It's complicated to handle different versions of the same class. Requires the [Serializable] attribute which you might not always be able to add.

DataContractSerializer & NetDataContractSerializer

Pro: Passes all the tests, XML Based, used in WCF

Cons: Requires attributes on the class ( [DataContract] ) and all members ([DataMember])

Newtonsoft Json.Net

One of the most common choices when doing serialization in an AJAX Call where the result is deserialized in JavaScript.

Pro:JSON Based, human readable, platform independent , fast, passes almost all the tests except the ones where the message has public readonly fields that have dirrerent names from the constructor parameters.

NServiceBus XMLSerialize

Integrated in NServiceBus. Has caused me a lot of problems when it was silently ignoring some properties.

Pro: XML Based, integrated with NSB, leaving aside the problems i've had with it has performed quite well in a few projects. Even if i don't personally love it, it's actively maintained by the NSB group and can be reliably used if you know it's limitations.

Cons: You might end up with properties being silently not serialized/deserialized

ProtocolBufers.NET Serializer

To quote the authors:

Protocol Buffers are a way of encoding structured data in an efficient yet extensible format. Google uses Protocol Buffers for almost all of its internal RPC protocols and file formats.

Pro: Fast, platform independent, compact, throws exceptions when unable to serialize/deserialive, supports versioning

Cons: Needs custom attributes on the class and members that need to be serialized.

Overall a good choice when size and performance really matter.

ServiceStack JsonSerializer

Pro: it's said to be the fastest json serializer.

Cons: Fails a lot of tests - you have to make sure you write your objects in ways supported by the serializer

SoapFormatter

Pro: Xml Based, Passes all the tests except the one without the Serializable attribute

Cons: The outputed xml is big

XmlSerializer

Pro: widely known?

Cons: I must be doing something wrong since it fails a lot of tests.

Conclusion

My first conclusion is that when approaching an architecture that relies on serialization like EventSourcing and CQRS you should carefully plan the way you write and persist the events in your system.

At the moment my choice is Json.NET with a Gzip filter. The main reason is that it has a minimal impact on how i must write events and is efficient in therms of speed and size yet is still human readable.

I'm hoping after i let this sink in a little i'll come back an update the conclusions.

Also i would be glad if others share their opinions & experiences related to serialization.

Verify that the current user is granted access in the appropriate allowAccounts section of SMSvcHost.exe.config

2011-03-07T00:00:00+00:00

Quick tip that maybe will save others time:

In Windows 7 ( and maybe Vista) when hosting a WCF Service in a console application AS A USER and not as administrator, you might get an exception telling you that you don't have access to register with the net.tcp port sharing service.

To resolve this problem:

Download PsGetSid ( this will give you the Security Identifier - SID for your user or user group )
Find and open SMSvcHost.exe.config ( usualy in c:\Windows\Microsoft.NET\Framework64\v4.0.30319\SMSvcHost.exe.config or c:\Windows\Microsoft.NET\Framework\v4.0.30319\SMSvcHost.exe.config )
Run psgetsid.exe to get the SID ( should look something like this S-1-5-21-1754548885-2506776180-2303324228-4659 )
By default the SMSvcHost.exe.config contains a section which has a child which has a child. The IMPORTANT part if you are using an editor that does not do syntax highlight for XML - THE ABOVE SECTION IS COMMENTED by DEFAULT so adding child has not effect.
YOU NEED TO ADD A NEW SECTION with it's children sections to make it work ( or uncomment the existing one and removing the // comments which are not valid xml ).
Restart the net.tcp portsharing service and you are good to go.

In the end you should have something like this:

 1 < ?xml version="1.0" encoding="utf-8"?>
 2 <!-- The configuration file for SMSvcHost.exe -->
 3 <configuration>
 4     <runtime>
 5         <gcconcurrent enabled="false" />
 6     </runtime>
 7     <system.serviceModel>
 8         <diagnostics performanceCounters="Off" etwProviderId="{f18839f5-27ff-4e66-bd2d-639b768cf18b}"/>
 9     </system.serviceModel>
10  <system.serviceModel.activation>
11         <net.tcp listenBacklog="10" maxPendingConnections="100" maxPendingAccepts="2" receiveTimeout="00:00:10" teredoEnabled="false">
12             <allowaccounts>
13                 <add securityIdentifier="S-1-5-21-1754548885-2506776180-2303324228-4659"/>
14             </allowaccounts>
15         </net.tcp>
16     </system.serviceModel.activation>
17 </configuration>

Also you might need to grant access to the user to register as a listener for an url:

netsh http add urlacl url=http://+: <port>/MyUri user=DOMAIN\user

I have to admin the most of the time i've spent on this issue was because i was modifying the commented section in the the config xml.

New Github account

2011-02-25T00:00:00+00:00

I’ve opened a Github account where i keep some stuff I've been playing with recently. At the moment is mostly related to doing DDD with CQRS and Event Sourcing.

I’ve created a sample project to demonstrate the concepts. Here is a short description of the sample:

Sample project demonstrating CQRS & Event Sourcing

The sample uses the following projects from Jonathan Oliver

EventStore
NanoServiceBus
CommonDomain
StorageAccess

For more information on CQRS and Event Sourcing http://cqrsinfo.com/

A short description of the projects in the sample:

a. Write Side

1. Sample.DomainModel   
    - the domain model for the sample   
    - this model is persisted using event sourcing and does not need to handle reads ( queries )   
    since they are done on the read side.   
    - the resulting event stream is the "source of truth" data. 

2. Sample.AppService   
    - the handlers for the commands our domain knows to execute 

3. Sample.AppServiceHost   
    - infrastructure for wiring up the command handlers to NanoServiceBus   
    - this is the actual instance of the service that needs to be running for the write side to process   
    commands.

b. Read Side

1. Sample.ReadModel   
    - the read model on witch queries are executed   
    - this model should be mapped as close as possible to the views   
    - this model can be regenerated from the event stream 

2. Sample.Denormalizer   
    - the event handlers for the domain events that are published by the DomainModel on the Write Side   
    - this handlers keep the read model in sync with the event stream   
    - a better name for it is welcome 

3. Sample.DenormalizerHost   
    - infrastructure for wiring up the event handlers to NanoServiceBus   
    - this process needs to run for the read side to be updated.

c. Infrastructure

1. Sample.Messages   
    - definitions for commands and events that are handled or published by the Domain Model   
2. Sample.Client.Web   
    - ASP.NET MVC3 application that demonstrates how to integrate the Write Side & the Read Side in an application.

TODO

Add more complex objects to the domain
Review transaction management on integration points
use ConfOrm for mapping the read model

I’d be glad if other shared their comments on the code there.

Hardware-based brute force attacks

2011-02-09T00:00:00+00:00

I've found this image in a PDF presentation document and i feel like sharing:

Source: scrypt-slides by Colin Percival

Log4Net not logging errors to db

2010-11-04T00:00:00+00:00

This is a short post about logging errors to db with log4net.

I've recently been bitten by a bug or better said miss configuration of log4net. I'm using log4net with database logging and file logging in a project and i was wandering why there are no error messages logged in the db when they are logged in the log files configured.

At one point it hit me: Transactions. Good ol` transactions. Distributed ones.

In this project i use a mix of NServiceBus based windows services and WCF services. They all use a variation of the Unit Of Work pattern to manage the atomicity of the operations. Since we use MSMQ messaging and NHibernate sessions they all fall under a distributed transaction. By default log4net AdoNet appender also uses a transaction to insert the log message in the db. And this transaction was enlisting in the ambient distributed transaction. Now in case of an error the unit of work get's aborted and the transactions get rolled back. This also includes log4net's transaction.

The solution is simple after you figure out the problem. Just add in the configuration section for the ADONetAppender. Ex:

1 <appender name="MySQLAppender" type="log4net.Appender.ADONetAppender">
2     <usetransactions value="false"/>
3     <!-- rest of the configuration -->
4 </appender>

Context Matters

2010-06-19T00:00:00+00:00

The next paragraph made me remember of a lot of hours of discussions with various developers:

If you’re arguing with me that NHibernate is the wrong tool because you’re writing reporting applications, just say “I write reporting applications.” And when I say “I think NHibernate is the best tool for persisting domain models,” I should probably talk about that specific context. And definitely, when you say “Linq to Sql rocks!” you better follow that up with “for applications with simplistic domain models.” (source: Context Matters)

If more developers would think like this we would probably have better tools with better acceptance in a lot less time.

NServiceBus with NHibernate and MySQL

2010-03-07T00:00:00+00:00

In the last project I have been working i finally got a chance to design and implement a solution based on NServiceBus and NHibernate, two tools I’ve been watching for a while but never got a chance to play with in more than sample applications. For some external reasons I've been forced to use MySQL as a database server in this project.

So basically I’m using NServiceBus to provide reliable communication between the involved components and NHibernate to do the persistence of the domain objects used by the components. Up to this point the whole design of the solution looks good, with minimal effort i have reliable, fault tolerant services that are ready to do their job.

Now i start implementing the details and get to the point where MySQL comes into play. I must say, it has surprised me... both ways.

The good thing is that after careful tuning, where my previous UNIX experience had a very important role, the database is able to handle the amounts of data that i plan to throw at it. Also it surprised me that some pretty complex queries run a lot faster than expected.

Then the bad things started to show up. The hardest to debug was that updating an indexed column from multiple parallel transactions causes deadlocks witch cause transactions to be aborted. Of course this only happens at high loads. It was not hard to avoid this after i found out what the problem was ... but still after this i had a feeling of working with something that might not be as reliable as expected.

After that the MySQL .NET connector dropped the bomb on me: Distributed Transactions are not supported. Ok, they are not supported but why the hell does the connector throws an exception when used in a distributed transaction? I can understand that i can’t rely on the MySQL transaction being enlisted in the Distributed Transaction (DT) and that i have to handle that myself but not being able to use the connector AT ALL under a DT was unexpected. At this point i see only one solution: grab the source for the connector and modify the part that checks if a DT is present and just ignore it. Turns out this was very easy to do. If anyone is interested in this change in the connector i can provide more details.

Now i need to find a way of having a NHibernate ISession and a ITransaction per NServiceBus message handler.

The first approach was something similar to what Andreas Öhlund describes in this blog post. The only problem is that IMessageModule implementations in NServiceBus 2.0 are singletons and that was a problem because i need to store the ITransaction to commit or rollback at the end of the message handler. If in the next version of the NServiceBus there will be a way to have some message handler “wrapper” that it could work.

My solution was to use a base abstract class for the massage handlers. So instead of just implementing IMessageHandler now i derive from this base class. The code below should speak for itself:

 1 /// <summary>
 2 /// Base class for message handlers.
 3 /// Manages the unit of work required for handling the message.
 4 /// </summary>
 5 public abstract class MessageHandler<T> : IMessageHandler<T>
 6           where T : IMessage
 7 {
 8     /// <summary>
 9     /// The injected unit of work implementation.
10     /// </summary>
11     private readonly IUnitOfWork unitOfWork;
12 
13     public MessageHandler(IUnitOfWork unitOfWork)
14     {
15         this.unitOfWork = unitOfWork;
16     }
17 
18     /// <summary>
19     /// Concrete handlers must implement this method.
20     /// </summary>
21     public abstract void HandleMessage(T message);
22 
23     public void Handle(T message)
24     {
25         try
26         {
27             HandleMessage(message);
28             unitOfWork.Complete();
29         }
30         catch
31         {
32             unitOfWork.Abort();
33             throw;
34         }
35         finally
36         {
37             unitOfWork.Dispose();
38         }
39     }
40 }

The implementation for the IUnitOfWork is in this case very simple, providing only the creation of the session and the transaction and the required operations. Since the unit of work is created per handler and the handlers don’t use other threads to do the work I don’t need to worry about making it thread safe.

 1 public class MessageUnitOfWork: IUnitOfWork
 2 {
 3     private readonly ISessionFactory factory;
 4     private readonly ITransaction transaction;
 5     private readonly ISession session;
 6 
 7     public MessageUnitOfWork(ISessionFactory factory)
 8     {
 9         this.factory = factory;
10         session = factory.OpenSession();
11         CurrentSessionContext.Bind(session);
12         transaction = session.BeginTransaction();
13     }
14 
15     public void Complete()
16     {
17         transaction.Commit();
18     }
19 
20     public void Abort()
21     {
22         transaction.Rollback();
23     }
24 
25     public void Dispose()
26     {
27         transaction.Dispose();
28         CurrentSessionContext.Unbind(factory);
29         session.Dispose();
30         GC.SuppressFinalize(this);
31     }
32 }

There is still one small problem. If the handler finishes it’s work without and exception and the mysql transaction is committed BUT an exception is thrown by the bus when committing the distributed transaction the MySQL transaction is not rolled back. But i realized that this only means that the same message might be sent again to the handler and that the handlers in general should handle this logical case since whoever sent the message is free to send it multiple times.

Since I’ve got this solution working it has handled a few millions of messages and there have been crashes and transaction that got rolled back occasionally but after all the system is designed to be fault tolerant and it has proven it is. Also in all the cases the database remained in a consistent state, witch in the beginning i was not sure it will.

In the end i would like to thank the NServiceBus team ( mainly Udi Dahan and Andreas Öhlund ) witch was very responsive and helpful on the support mailing list. I can only hope to find the time to contribute a few ideas to the next release of NSB. Also i would like to thank the NHibernate team for the great product they created ( can’t wait for the 3.0 ).

C# HTTP Request

2009-11-21T00:00:00+00:00

A few years ago I've written an article on how to perform a synchronous HTTP request using Qt 4.2. I I've seen this article today and since now I'm mostly working with C# i was wandering how fast can i do the same thing in c#. After very few minutes I've managed to came up with this:

 1 using System;
 2 using System.Net;
 3 using System.IO;
 4 
 5 class Program
 6 {
 7     static void Main(string[] args)
 8     {
 9         WebRequest request = HttpWebRequest.Create("http://www.google.com");
10         WebResponse response = request.GetResponse();
11         using(StreamReader reader = new StreamReader(response.GetResponseStream()))
12             Console.WriteLine(reader.ReadToEnd());
13         Console.ReadKey();
14     }
15 }

And of course the WebReqest class allows you to specify all the properties you would ever need for any request, and also allows you to perform request in an asynchronous maner. Now i don't think you can be more expressive than this. I like qt a lot... but i love c# now.

Update...

Or you can do this:

 1 using System;
 2 using System.Net;
 3 using System.IO;
 4 
 5 class Program
 6 {
 7     static void Main(string[] args)
 8     {
 9         var content = new WebClient().DownloadString("http://www.google.com");
10         Console.WriteLine(content);
11     }
12 }

A few things from C++ that i miss in C#

2009-07-11T00:00:00+00:00

The last 6-7 months i've been working mostly with C# and to be honest I've grown found to it. It has really nice features like lambda expressions, auto properties, extension methods, LINQ witch can really speed up things in terms development time.

Still ... there are some things from C++ that i now miss: Templates ( as in true generics ) , const correctness, deterministic destruction, static initialization and most of all the control over what is a value and what is a reference.

Generics ( Templates )

Yes, C# has generics witch give you a level of freedom and ability to write generic code but they are nothing compared to the power of c++ templates. In C++ templates are compiled when instantiated and you don't need to specify any constraints on the generic type and as long as the instantiated type is syntactically correct your code will compile. There are ways to define concepts witch can impose requirements on the types but the concepts are also verified when the template is instantiated. Also C# does not allow specialization or partial specialization of generic types.

Const Correctness

There are a lot of articles on the web about why C# does not implement const correctness. In the end you learn to leave without it but its a nice thing to have and as long as you use it consistently it can help prevent a lot of bugs mostly design bugs. I guess that there are ways in C# to prevent the bugs that const correctness prevents ( immutable objects, ReadOnly interfaces ) but they are harder to impose on a team of developers.

Deterministic Destruction

I'm quite sure that I only miss this feature because i come with a C++ background. You can easily have deterministic destruction in C# by using IDisposable and using(...) features, still after years of C++ i kinda like to know when an object gets destructed.

Static Initialization

To be honest I don't miss this feature too much. The only thing i find it useful for is implementing the factory pattern where objects register themselves in the factory using static initialization. The downside of this is that it can relay get messy ( hint Static Initialization Fiasco )

Control over what is a value and what is a reference

Now this i miss the most. I have to admit it I like pointers. Well i actually like smart pointers and i like being in control of what and how i pass as arguments to methods. In C# it's written in stone: structs are values and classes are references (ok, with boxing you can have references to structs but that's not the point ). One of the problems is that there is no way of declaring a method that accepts a CONST reference to an instance. When you pass an instance of a class to a method (you actually pass the reference to that instance) , the method has full control over the instance. This might not be a problem when you write all the code, but in a team with developers of different skill bad things can happen. Add some multi-threading to this and things can get relay messy. Another problem is when you have a value object implemented as a struct that is holding a lot of data and you need to pass this object to a Method that only needs to read something from this object. If you pass it as a value object it will probably get copied ( i'm not sure if the compiler does not implement some copy-on-write here but i would not relay on that ) and if you pass it as a ref than the method might change the data.

In conclusion I like working with C# but there are some things that i hope will get better in time. Other important "bonuses" you get with C# are the .NET Framework and Visual Studio IDE that speed up substantially the development. The major downsite witch in some cases is a showstopper: Platform Dependent. I don't know where MONO is right now but if you develop with C# and .NET you better make sure your main target platform is Windows.

I would be glad to hear what others think about this.

The power of SUDO

2009-06-26T00:00:00+00:00

C# Extension Methods

2009-06-24T00:00:00+00:00

After a few months I've enjoyed the newly ( or not ) added extension methods in 3.0 yesterday i discovered that one of the assumptions I've made about them was false. My assumption was that if the instance on witch you call an extension method is NULL you would get a NullReferenceException. Turns out it's not the case and you can call the extension method on a null reference.

Thinking about it later, it makes sense. In reality extension methods are nothing more than syntax sugar for calling static methods. And since too much sugar can hurt extension methods might do the same thing if not properly used. The code speaks pretty much for itself.

The class used as example:

1 public class TestClass
2 {
3     //existing method defined on the class
4     public string Hello()
5     {
6         return "Hello";
7     }
8 }

Static class defining extension method

1 public static class ExtensionContainer
2 {
3     //the extension method defined as static
4     public static string NewHello(this TestClass instance)
5     {
6         return "World";
7     }
8 }

Test code:

 1 class Program
 2 {
 3     static void Main(string[] args)
 4     {
 5         // declare and create a test instance
 6         TestClass first = new TestClass();
 7 
 8         Console.WriteLine(first.Hello()); //outputs: "Hello"
 9         //call the extension method as a normal method
10         Console.WriteLine(first.NewHello()); // outputs: "World"
11 
12         // declare a null reference
13         TestClass second = null;
14 
15         try
16         {
17             //calling a method on a null reference throws
18             Console.WriteLine(second.Hello()); //will throw Null Reference exception
19         }
20         catch (NullReferenceException)
21         {
22             Console.WriteLine("NullReference calling Hello");
23         }
24 
25         try
26         {
27             //calling an extension method does not throw
28             Console.WriteLine(second.NewHello()); //will not throw
29         }
30         catch (NullReferenceException)
31         {
32             Console.WriteLine("NullReference calling NewHello");
33         }
34 
35         //the next two calls are exactly the same
36         Console.WriteLine(second.NewHello());
37         Console.WriteLine(ExtensionContainer.NewHello(second));
38 
39         Console.ReadKey();
40     }
41 }

Even now is still consider extension methods as a very valuable addition to the language, but you just need to understand exactly what they are. The wrong assumption about calling them on a null reference can affect some design choices you may take where in case of a null reference you want an exception to be thrown and no other method called.

Also using them when appropriate the ability to call extension methods on null references might help you write more readable code by moving the null check in the extension method. This can be particularly useful on writing extension methods that are supposed to be used in Views in ASP.NET MVC where moving the null check in the method leaves the the view's html code easier to read an understand.

One conclusion that I've drown from this is that in an extension method you should either check for null and throw NullReferenceException if null is not a valid value or handle null appropriately.

PS: Is it visible that the last 6+ months I've been working with C# and not C++ ? :)

ASP.NET MVC or ASP.NET just got plesent

2009-06-15T00:00:00+00:00

I've been really busy the last months with the new job where we use c#. I've grown found to it ... it has a lot of nice features ... lambda expressions, anonymous types, generic types. Sure there are a lot of thing that can be improved like real generic types ( aka C++ templates ) but it's a really reliable tool.

In the past I've worked on a few ASP.NET applications and I've not enjoyed it very much ( actually every time I've used it i promised myself not to use it again ) ... until now. I've heard of ASP.NET MVC witch is supposed to be a different way of developing web applications in .NET and after a few months of using it I can only say ... ya baby ... finally a nice way to use c# and .NET to do web apps.

If you get the chance give it a go... it's worthed.

WordPress Exploit Scanner Plugin

2009-03-03T00:00:00+00:00

I've found a few days ago this plugin and i would like to share with everybody since it may help secure the web. This plugin scans the wordpress installation folder for paterns usualy found in exploits/throjans/malware web scripts. The plugin produces quite a few false positive alerts but also it's very easy to spot suspicious code in suspicious files.

It would be nice if wordpress.org published an api to get the md5 of the core files in a specific version since the plugin right now is hardcoded with wordpress 6.3 md5s and will warn you about core files modifications if you use different versions. Also it would be nice if the plugins on wordpress.org would provide a way to get their md5 and use that to look for modifications and file additions.

In conclusion there are ways to improve this plugin but as it is it's a very useful security tool. Also, if you have wordpress installed in the root of your domain, it will scan every file under it. This plugin is actually what made me write the Coppermine article since it found a few uploaded scripts.

Stay safe

WordPress Exploit Scanner

I gave up on Coppermine Gallery

2009-02-27T00:00:00+00:00

I used to use Coppermine Photo Gallery for http://gallery.erata.net but today i've decided to close it down. The number of vulnerabilities ( most of them critical ) discovered over the years got me thinking. You can see what i mean here http://www.milw0rm.com/search.php?dong=coppermine. 3 vulnerabilities found in the first two months of 2009. 4 in 2008. This remembers me of sendmail and bind ( witch I've heard got better but who cares about them anymore ).

I know it's not written in stone but from the number of vulnerabilities i deduce the quality of a pice of software. I also understand open-source ... and open-source brings a lot more peer reviews ... and i don't mind (that much) vulnerabilities in pre 1.0 versions. But after a few years of development and after a few critical vulnerabilities i'd expect a product to get better and safer.

Oh well... good luck to the developers of coppermine keeping their users safe.

Alternatives i consider using for web gallery needs:

Picassa - if it's enough for the job works very well, provides desktop application for management and until proven wrong i belive it's safe ( and if it's not i don't worry to much since it does not run on my server )
WordPress with NextGen Gallery plugin - can be greatly customized and works well and also allows other content posting.

Nokia to Add LGPL to Qt Licensing Model

2009-02-11T00:00:00+00:00

This is incredible news. No point in making this post longer ... who knows what this means understands why it's incredible. Who does not know what this means ... won't care anyhow :)

http://www.qtsoftware.com/about/news/lgpl-license-option-added-to-qt

And when i first heard about nokia buying trolltech i thought i'll have to forget about Qt.

Life with g++

2008-05-28T00:00:00+00:00

The intention of this article is to show a few useful ( i hope ) tips about using g++ in the every day development cycle. Also this article describes three build configurations that should help the developer catch bugs in the early stages of development.

Debug Build

Most often you will use this configuration. With this you will have a bigger and slower application but it will be easier to debug. The application will contain no optimizations made by the compiler and will also contain debug symbols witch are needed to run you application inside gdb ( or other debugger ) or if you want to analyze a core dump. For the debug build i suggest you use the following flags to g++:

g++ -W -Wall -Werror -g -fno-inline

The -W and -Wall parameters will enable all the warnings in g++ and the -Werror will make g++ treat the warnings as errors. If g++ is warning you about something it's either a compiler bug ( witch is not very probable if you are using a recent version of g++ ) either something you need to fix. You might thing that the compiler is wrong but in 99.99% of the cases the bastard is right and the warning needs to be fixed. And by fixed i don't mean adding something to quiet down the compiler but really fixed as in correcting the code. The -g option will make g++ add debugging symbols to the binary and the -fno-inline will prevent g++ from inlining any method or function. In some rare cases no-inline will make it easy to debug functions that otherwise would be inline.

Developer Release Build

The purpose of this build is to catch more warnings and errors that might not occur in the debug build. You should run all the test cases ( if you don't have test cases stop reading and go write them ) against this version. Your application will be compiled with optimizations and also with debug code disabled but with debug symbols witch will only make the binary bigger. My recommendation for the flags is:

g++ -W -Wall -Werror -g -O4 -DNDEBUG

As with the debug build the -W -Wall and -Werror will add warnings and the -g will add the debug symbols. The -O4 will enable the optimizer inside g++. The use of the optimizer in this build is very important because it will enable g++ to find even more warnings than the debug build. Because of this you should do this build at least once a few hours of development or at least before an important commit to the repository. The -DNEBUG will disable all the debug code ( like asserts and debug messages ). You may need to add other defines if your code does not relay on NDEBUG for debug only code.

Customer Release Build

This is the build that will produce what is given to the customer. This should be an optimized build without debug symbols. This build should contain a build number ( a static string or something similar ) and in the repository you should add a tag or create a branch for the build. If the product is in an alfa/beta stage you might let the debug symbols in. The flags for the build are:

g++ -W -Wall -Werror -O2 -DNDEBUG

The only difference from above is the absence of -g because we don't need debug symbols and the use of -O2 instead of -O4. In the developer build we use O4 to try to catch as many warnings high optimization might reveal but in practice the speed difference between -O4 and O2 is negligible. You should have a good reason for using more than -O2 in this kind of build. Usually computation intensive software ( science , math ) might benefit from using -O3 but for a desktop application -O2 should be more than enough.

Other Tips

After changing something that affects ABI ( application binary interface ), like adding a function/member, if you receive strange errors you should always make clean and rebuild your application. To speed up the step of make clean and make you may use ccache witch caches preprocessed c++ files and saves a lot of time parsing included files. Also the build typed described above can be used with other compilers ( C or C++ compilers ), by using the equivalent options.

In conclusion you should listen to your compiler and remember that most of the time you are wrong and he is right.

Wt, C++ Web Toolkit

2008-05-27T00:00:00+00:00

Wt (pronounced 'witty') is a C++ library and application server for developing and deploying web applications. It is not a 'framework', which enforces a way of programming, but a library.

I've found myself a new toy :). No more php or C# or ASP or other things i don't like to use but i have to use. I haven't yet started to use Wt, but looks nice. Like it or not the web-app is competing really well with the native-app, at least from the desktop user's point of view. It would be nice to have a way of writing web apps with things like type safety, generic code and most important reusing libraries like boost.

From the first look at the Wt code i looks somehow familiar, maybe because I've used Qt a lot and the Wt's slot/signal mechanism seems similar.

A C++ library that allows creation of a web graphical interface by using widgets rather than hand written html. Sounds wonderful. I can only hope it will raise up to the expectations.

More after a play with it.

details at: http://www.webtoolkit.eu/wt/

Flash on FreeBSD - Gnash

2008-05-27T00:00:00+00:00

Watching flash content on FreeBSD was and still is a problem. It's sad Adobe won't publish a native version of flash for FreeBSD, or even better open the sources for the flash player. It's strange they don't, unless the sources are a real mess I don't see any strong reason for not doing so. Now that rumors are found about new technologies for web graphics like svg, html 5, flash might not be so popular in the future.

About the linux plugin for native firefox i don't have much to say. I does not work reliable. Yes you can watch some movies in some conditions with linux-flash7 with nspluginwrapper but still with a lot of problems depending on arch/site/browser/movie version. With linux-flash9 even more problems. I realized that in the end it's more frustration than satisfaction and the feeling was that I'm fighting the wrong battle in trying to make the linux-flash work on FreeBSD.

Now I'm living with gnash ( graphics/gnash ). Well, it's not perfect but at least it's open source and does play a lot of movies. And the best thing about it was that with the release of a new version ( 0.8.2 ) movies that did not work with 0.8.1 are now working so it's progressing. Youtube is working pretty well also.

In conclusion if you really need flash on FreeBSD try supporting gnash. As an open source alternative to the flash plugin it's starting to look more than promising.

WordPress WassUp Error

2008-04-07T00:00:00+00:00

I've upgraded the wassup plugin to the latest version ( 1.5 ) and an error started to show in the pages:

[ wp_wassup_tmp' doesn't exist ]

The solution is to deactivate the plugin and then reactivate it... witch should be the right way to upgrade your plugins.

Update:

I also get a syntax error from the wassup plugin, on the dashboard:

***WordPress database error:*** [You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'AND 1207656596' at line 1]

SELECT COUNT(DISTINCT wassup_id) AS itemstot FROM wp_wassup WHERE wassup_id IS NOT NULL  AND timestamp BETWEEN  AND 1207656596

It seems that the errore comes from a debug query that was left in the wassupDashChart() function. To fix the error you need to comment two lines (1631,1632) of code in wassup.php:

 1 // This is the function to print out a chart's preview in the dashboard
 2 function wassupDashChart() {
 3   global $wpdb;
 4   $table_name = $wpdb->prefix . "wassup";
 5   $to_date = wassup_get_time();
 6   $Chart = New MainItems;
 7   $Chart->tableName = $table_name;
 8   $Chart->to_date = $to_date;
 9 
10   //$itemstot = $Chart->calc_tot("count", $search, null, "DISTINCT");
11   //print $itemstot;
12 
13   echo '<h3>WassUp Stats <cite><a href="admin.php?page=wassup">More »</a></cite></h3>';
14   echo $Chart->TheChart(1, "400", "125", "", $wassup_options->wassup_chart_type, "bg,s,00000000", "dashboard", "left");
15 } //end function wassupDashChart

myWeather widget

2007-12-06T00:00:00+00:00

To create a quick and dirty widget from myWeather wordpress plugin just add the following to the end of myweather.php

 1 function widget_weather_init(){
 2   if ( !function_exists('register_sidebar_widget') )
 3     return;
 4   register_sidebar_widget(array('Weather', 'widgets'), 'widget_weather');
 5 }
 6 
 7 function widget_weather($args){
 8   extract($args);
 9   echo $before_widget . $before_title . $title . $after_title;
10   wp_myweather();
11   echo $after_widget;
12 }
13 
14 add_action('widgets_init', 'widget_weather_init');

Also the part where you search for the city could be added as a control for the widget but i didn't relay need it to be .

MyWeather plugin's homepage.

Update

If you have multiple users with different credentials on your blog and you are the only administrator you can restrict the access to myWeather plugin administration interface by changing the function myweathertoadmin() to look like this (use 10 instead of 1 for required level) :

1 function myweather_to_admin() {
2   if (function_exists('add_submenu_page')) {
3     add_submenu_page('options-general.php', 'myWeather',
4     'myWeather', 10, basename(__FILE__),'myweather_admin');
5   }
6 }

qmake error: sible braces mismatch %s:%d

2007-07-27T00:00:00+00:00

I've just received this error with qmake on windows. Googling for it showed no results so after i've solved the problem i decided to put it here.

The error was near a conditional block like:

1 unix{
2 }
3 win32{
4   QMAKE_UIC+ = -L ../bin/plugin
5 }

...after searching for unclosed/missing braces for a while without any result and comparing svn versions i've finally found the problem. On the QMAKE_UIC linke instead of "+=" somehow ( probably kdevelop's qmake parser/generator ) got saved as "+ =" (notice the space). The problem would have been much easier to solve if the error message wasn't so misleading.

1 #bad version
2 QMAKE_UIC + = -L ../bin/plugin
3 #good version
4 QMAKE_UIC += -L ../bin/plugin

So if you see this error look for operators that might have a space inserted in the middle. Hope this will save somebody's time.

Qmake Error Sible Braces Mismatch Sd

2007-07-27T00:00:00+00:00

ASP.NET 2.0 and SQLServer 2000 Transactions

2007-06-27T00:00:00+00:00

The new ADO.NET 2.0 and it's typed datasets and table adapters are very useful tools for creating a DAL in you application. Still when working with SQLServer 2000 implementing transactions is a bit tricky.

You can use TransactionScope but is seems to be designed for SQLServer 2005 as it will promote all transactions to distributed on SQLServer 2000 witch is a performance problem but also a bigger problem if your hosting provider does not provide distributed transactions. I've found a few solutions on the net some using reflection some using partial classes to add a method to the table adapter. Since i don't like using reflection unless i absolutely have to I've come up with a solution using partial classes but trying to minimize the code you have to write for each table adapter and also trying to make is less error prone.

First we define an interface witch the table adapters will implement:

1 public interface ITransactionable
2 {
3   SqlConnection SqlConnection {get;set;}
4   SqlCommand[] Commands {get;}
5 }

Now define a helper class that will manage all transaction operations.

 1 public class TransactionHelper
 2 {
 3   public static SqlTransaction BeginTransaction(ITransactionable adapter)
 4   {
 5     return BeginTransaction(adapter,IsolationLevel.ReadUncommitted);
 6   }
 7 
 8   public static SqlTransaction BeginTransaction(ITransactionable adapter, IsolationLevel lvl)
 9   {
10     SqlConnection con = adapter.SqlConnection;
11     if (con.State == ConnectionState.Closed)
12     con.Open();
13     SqlTransaction trans = con.BeginTransaction(lvl);
14     SetTransaction(adapter, trans);
15     return trans;
16   }
17   
18   public static void SetTransaction(ITransactionable adapter, SqlTransaction trans)
19   {
20     foreach (SqlCommand com in adapter.Commands)
21     com.Transaction = trans;
22     adapter.SqlConnection = trans.Connection;
23   }
24 }

Now for each table adapter that we need to use transactions:

1 public partial class DemoTableAdapter : ITransactionable
2 {
3   public SqlConnection SqlConnection { get { return Connection; } set { Connection = value; } }
4   public SqlCommand[] Commands { get { return CommandCollection; } }
5 }

As you can see the code needed for each table adapter is minimal and only requires copy-pasting and changing the name of the class. If c# would support c-style macros this could be reduced to a single line of code. To use the TransactionHelper class:

 1 ....
 2 SqlTransaction transaction=null;
 3 try{
 4   using(DemoTableAdapter adapter = new DemoTableAdapter())
 5   {
 6     transaction = TransactionHelper.BeginTransaction(adapter);
 7   .... operations with adapter ....
 8   }
 9   using(OtherTableAdapter adapter = new OtherTableAdapter ())
10   {
11     TransactionHelper.SetTransaction(adapter,transaction);
12   .... operations with adapter ....
13   }
14   transaction.Commit();
15 }
16 catch(Exception x)
17 {
18   transaction.Rollback();
19   ...handle error....
20 }
21 finally 
22 {
23   transaction.Dispose();
24 }

Hope it helps.

Windows Integration

2007-05-18T00:00:00+00:00

Here is a preview of what you can do if you have a windows machine somewhere.

Flash7 with native FreeBSD browsers

2007-03-18T00:00:00+00:00

Finally the linux-flash7 plugin works with native firefox. If it's not working for you this may help.

Update

With the nspluginwrapper port this
post may be more usefull. The if you still have problems you may try the tips below.

Step By Step HOWTO

Use cvsup or portsnap to update your ports tree
install www/linuxpluginwrapper
copy libflashplayer.so and flashplayer.xpt from distfiles/flashplugin/7.0r68/installflashplayer7linux.tar.gz to /usr/local/lib/browser_plugins
Fireup firefox or opera and test

If in the future the dependency on the linux emulation layer is removed from linux-flash7 port it will be ok to install the port as you would normally do, but until then i prefer to manually unpack the two files instead of installing the hole linux-base port(s) witch is not needed anyway for the plugin to work.

If it works congratulations, if firefox crashes and opera displays just an empty box ... read on.

Fixing Plugin Crash

Here there are two things to try. (the first one is recommended)

edit /etc/X11/xorg.conf and disable the Composite extension and set the ColorDepth to 24
edit /usr/local/bin/firefox and add export XLIB_SKIP_ARGB_VISUALS=1 in the beginning of the file

I've noticed that opera still has some trouble working with the plugin and as a fix I've done a small change to the linuxpluginwrapper port. In compat_linux/linux_ioctl.c I've changed the default switch branch from:

1 default:
2     errno = ENOSYS;
3     ret   = -1;
4     dprintf("ioctl(fd=%d, request=%08X(on Linux), ...) = %d / ERRNO = %d", d, l_request, ret, errno);
5     break;

1 default:
2      ret= _ioctl(d,l_request,va_arg(ap,int*)); 
3      break;

After this change the plugin works fine but sometimes you need to reload the page for opera to display the flash content.

References

Information on fixing the crash was gathered from:

To be continued

Now that I've learned this i really have a good feeling about linux-flash9 and native browsers. I haven't tested yet but i guess the fix for the crashes works also for linux-flash9 with linu-firefox and linux-opera.

KDevelop - KDevTrollProject Problem

2007-03-17T00:00:00+00:00

After updating kde to 3.5.6 and kdevelop to 3.4.0_1 ( around 15.03.2007 ) kdevelop failed to open qmake based projects. The error was "Error creating project management plugin KDevTrollProject".

Update

On 23.03.2007 a fix has been committed to the kdevelop port so if you encounter this problem it's better to update the ports tree and recompile kdevelop.

Finding why

It took me a few houers to find out that the problem was an undefined reference to a symbol in /usr/local/lib/libkdevqmakeparser.so. The symbol was QMake::Lexer. Grepping to kdevelop sources i found that the libkdevqmakeparser is build in buildtools/lib/parsers/qmake. There i found that qmake_lex.cpp was not compiled and linked to the library.

Quick Fix

Below are the command lines to build qmake_lex.cpp and relink libkdevqmakeparser.so

The fix_lex.sh file:

#!/bin/sh
if /bin/sh /usr/local/bin/libtool --silent --tag=CXX --mode=compile c++ -DHAVE_CONFIG_H  -I. -I. -I../../../.. -I../../../../lib/interfaces  -I../../../../lib/interfaces/extensions -I../../../../lib/util -I../../../../lib/widgets/propeditor  -I/usr/local/include -I/usr/X11R6/include  -I/usr/local/include  -D_THREAD_SAFE -pthread -DQT_THREAD_SUPPORT   -I/usr/local/include -I/usr/local/include  -I/usr/X11R6/include -D_GETOPT_H -D_THREAD_SAFE   -Wno-long-long -Wundef -Wall -W -Wpointer-arith -DNDEBUG -DNO_DEBUG -O2 -O2 -fno-strict-aliasing -pipe -Wno-non-virtual-dtor -fno-exceptions -fno-check-new -fno-common -DQT_CLEAN_NAMESPACE -DQT_NO_ASCII_CAST -DQT_NO_STL -DQT_NO_COMPAT -DQT_NO_TRANSLATION  -MT qmake_lex.lo -MD -MP -MF ".deps/qmake_lex.Tpo" -c -o qmake_lex.lo qmake_lex.cpp;  then mv -f ".deps/qmake_lex.Tpo" ".deps/qmake_lex.Plo"; else rm -f ".deps/qmake_lex.Tpo"; exit 1; fi
/bin/sh /usr/local/bin/libtool --silent --tag=CXX --mode=link c++   -Wno-long-long -Wundef -Wall -W -Wpointer-arith -DNDEBUG -DNO_DEBUG -O2 -O2 -fno-strict-aliasing -pipe -Wno-non-virtual-dtor -fno-exceptions -fno-check-new -fno-common -DQT_CLEAN_NAMESPACE -DQT_NO_ASCII_CAST -DQT_NO_STL -DQT_NO_COMPAT -DQT_NO_TRANSLATION    -o libkdevqmakeparser.la -rpath /usr/local/lib -no-undefined  -L/usr/local/lib -L/usr/X11R6/lib   -D_THREAD_SAFE -pthread  -L/usr/local/lib -lkio qmakeast.lo qmakedriver.lo  qmake_yacc.lo qmake_lex.lo qmakeastvisitor.lo  -Wl,-export-dynamic -L/usr/local/lib -L/usr/X11R6/lib -ljpeg  -L/usr/X11R6/lib

cd /usr/ports/devel/kdevelop
make extract patch configure
cd work/kdevelop-3.4.0/buildtools/lib/parsers/qmake
sh fix_lex.sh
cp .libs/libkdevqmakeparser.so.0 /usr/local/lib/

Conclusion

I somehow have the impression that I'm the only one having this problem since i haven't seen any reports on the mailing lists. It may be some bad combination of autotools/libtools. If I'm the only one having this problem then it's not worth investing more time in a proper fix. If other users will report this problem then probably a patch to add the qmake_lex.cpp file to the Makefile with appropriate rules may be needed. My experience with autotools and handwritten Makefiles is close to zero (and i intend to keep it that way) but i guess the kde folks will have no problem fixing this once the problem i known.

If someone else experiences this problem please leave a comment or a post a message on the freebsd mailing list

Virtual Destructors

2006-11-18T00:00:00+00:00

When do you need to declare a destructor virtual?

Recently I’ve been involved in a thread on qt-interest mailing list about virtual destructors. A few people there suggested that you should always make the destructor of your class virtual. I strongly disagree.

The argument was that if you use the virtual keyword on your destructor you will always be on the safe side. If someone inherits from your class and then deletes one of his instances of the class by a pointer to your base class you are sure his destructor gets called. So you are designing a class with value semantics (can be copied, can be assigned to) and you fear that someone will eventually inherit from your class so you make the destructor virtual. Well then why not make all the methods virtual so that if someone will inherit your class will have the possibility to customize its behavior. Is this starting to sound wrong? Well it should.

Just to be clear, I’m not trying to say that declaring a destructor virtual on a class WILL cause trouble. I’ll try to explain why it’s BAD CODING STYLE to always declare you destructor virtual. A good coding style will most probably keep you away from dangers like inheriting a class that has not been meant to be used as a base class.

As Herb Sutter and Andrei Alexandrescu state in “C++ Coding Standards: 101 Rules, Guidelines, and Best Practices” item 32:

Be clear what kind of class you're writing

Be clear to yourself and to the users of your class. The most used types of classes are value classes and polymorphic classes. As a rule of thumb:

Never try to write a class that is of both types.

These types of classes are complementary. Value types are concrete types that mimic the built-in types, which are usually instantiated on the stack and are passed around by reference or by value. Polymorphic classes are meant to be used as base classes, are usually instantiated on the heap and used via (smart) pointer. In C++ the keyword virtual means “this will be a polymorphic class” and a copy constructor and assignment operator mean “this will be a value class”.

Now when the destructor should be declared virtual?

You MUST declare the destructor virtual ONLY if you are creating a polymorphic class witch has a PUBLIC destructor. That’s it. That’s the only case when you must declare the destructor virtual. If you don’t declare it virtual BAD THINGS will happen (destructors of child classes will not be called when deleting by base class).

 1 struct A { 
 2   ~A(){
 3     cout << " calling ~A()"; 
 4   }
 5 };
 6 
 7 struct B: public A { 
 8   ~B(){
 9     cout << "~B()";
10   }
11 };
12 
13 int main(){
14   A* a = new B;
15   delete a; ///In this case ~B() will not be called.
16 }

You SHULD (read MUST) NOT declare your destructor virtual in any other case. If you declare it virtual the biggest problem is someone using your class might think that your class is a polymorphic class that can be inherited from. If this happens the code will probably work but will be awkward and difficult to write and maintain. There is also a performance hit that comes with virtual destructors but unless you are writing something where every nanosecond counts you will not care about this hit.

Now what do you do if you find a nice class that is a value class (even if the author made the destructor virtual) but you want to add some custom functionality to it?

Use FREE FUNCTIONS. That’s it. Having free functions will not make your coding less Object Oriented. Free function will give you the same access level as inheriting from a value class as value classes don’t usually have protected methods. To avoid problems place the free functions in the same namespace as the class. As an example if you have a string class and want to have a method that will return the word count from the string:

1 int wordCount( const string& str ){
2   ///count the words
3   return count;
4 }
5 
6 void doSomething(const string& str){
7   /// instead of str.wordCount() you will now have:
8   int count = wordCount(str);
9 }

There are lots of things to be said about class design. You should search for books written on the subject and learn about good ways of creating classes. The sad part is that until you are bitten by a bad design that you have to live with you most probably will not understand the importance of good design and coding style in general.

In the end if you are an employer and want to quickly see if a candidate knows about virtual functions ask him what “static virtual functions” and “virtual constructors” have in common. (if you don’t know the answer contact me).

Updates

A very good article about virtuality.

And another great article about virtual functions.

"Virtual functions should be treated very much like data members — make them private, until design needs indicate a less restricted approach is indicated. It is much easier to promote them to a more accessible level, than it is to demote them to a more private level."

Synchronous Http Request

2006-11-08T00:00:00+00:00

Qt4 Synchronous HTTP Request

2006-11-08T00:00:00+00:00

Problem

Creating a wrapper over QHttp that can perform GET and POST requests in a synchronous way.

Solution

In Qt4 QHttp can perform HTTP requests, but the API only allows asynchronous requests. This means that you need to specify a set of slots to handle the signals that can be emitted while the request if performed. Now don't get me wrong, this is event driven programming and most of the times is the right way to go especially when writing a Qt based application. The cases when event driven operations prove difficult to use are rare and usually can be programmed using events.

Recently I’ve developed a small application that had to do a number of requests in a specified order and show a message if an error was to occur. The algorithm was simple:

1 foreach ( Request r , requestList )
2   if(! runRequest(r) ) return error;

For writing something like this I would like to not split the algorithm across a set of slots. So I would like to have a possibility to run a request and wait for it to finish.

Implementation

We start by creating a SyncHTTP class that is derived from QHttp:

 1 class SyncHTTP: public QHttp{
 2   Q_OBJECT
 3   /// id of current request
 4   int requestID;
 5   /// error status of current request
 6   bool status;
 7   /// event loop used to block until request finished
 8   QEventLoop loop; 
 9 
10 public:
11   /// structors
12   SyncHTTP( QObject * parent = 0 )
13   :QHttp(parent),requestID(-1),status(false){}
14 
15   SyncHTTP( const QString & hostName,quint16 port = 80, QObject * parent = 0 )
16   :QHttp(hostName,port,parent),requestID(-1),status(false){}
17 
18   virtual ~SyncHTTP(){}
19 }

Now we need to implement the methods for syncGet and syncPost to mimic the get and post methods from QHttp:

 1 /// send GET request and wait until finished
 2 bool syncGet ( const QString & path, QIODevice * to = 0 )
 3 {
 4   ///connect the requestFinished signal to our finished slot
 5   connect(this,SIGNAL(requestFinished(int,bool)),
 6   SLOT(finished(int,bool)));
 7   /// start the request and store the requestID
 8   requestID = get(path, to );
 9   /// block until the request is finished
10   loop.exec();
11   /// return the request status
12   return status;
13 }
14 /// send POST request and wait until finished
15 bool syncPost ( const QString & path, QIODevice * data, QIODevice * to = 0 )
16 {
17   ///connect the requestFinished signal to our finished slot
18   connect(this,SIGNAL(requestFinished(int,bool)),
19   SLOT(finished(int,bool)));
20   /// start the request and store the requestID
21   requestID = post(path, data , to );
22   /// block until the request is finished
23   loop.exec();
24   /// return the request status
25   return status;
26 }
27 
28 bool syncPost ( const QString & path, const QByteArray& data, QIODevice * to = 0 )
29 {
30   /// create io device from QByteArray
31   QBuffer buffer;
32   buffer.setData(data);
33   return syncPost(path,&buffer,to);
34 }

The syncGet and syncPost functions both use the same strategy. They connect the requestFinished signal from QHttp to a finished slot, they ask the QHttp object to perform the request. The QHttp's get/post method returns immediately with the request id that has been assigned to the current request. Now the nice part: After asking the QHttp object to perform the request we need to wait for it to finish, so we create an event loop and start it. The main event loop is now blocked and we are nor risking being called aging while executing the request. The blocking code is not using tricks like sleep or infinite loops so we don't end up using 100% CPU or wasting time sleeping. Now we just need to implement the finished slot to save the result and exit the loop:

 1 protected slots:
 2 virtual void finished(int idx, bool err)
 3 {
 4   /// check to see if it's the request we made
 5   if(idx!=requestID) return;
 6   /// set status of the request
 7   status = !err;
 8   /// end the loop
 9   loop.exit();
10 }

Now this slot first checks to see if it's being called for our request and returns if not, after that saves the error status returned by the QHttp object and exits the loop. The execution now will return in the syncGet/syncPost function that will return the status to the caller.

Testing

Ok, let's write a small test application. Writing real automated test cases for this kind of class is possible but not trivial so will just write a small test app.

 1 #include
 2 #include
 3 #include "synchttp.h"
 4 
 5 int main(int argc, char *argv[])
 6 {
 7   QApplication a(argc,argv);
 8   /// create object
 9   SyncHTTP h("www.google.com");
10 
11   ///prepare output buffer
12   QBuffer getOutput;
13   h.syncGet( "/search?q=erata.net",&getOutput );
14   qDebug() << getOutput.data();
15 
16   QByteArray data(QString("q=data").toLatin1());
17   QBuffer postOutput;
18 
19   h.syncPost( "/search",data, &postOutput);
20 
21   qDebug() << postOutput.data();
22 
23   return 0;
24 }

Now in conclusion i would like to say that i recommend using this class only when you simply need to perform a request and know if it was successful or not. On any other case, where you need to do more complicated things learn to use Qt's event driven programming model.

Download synchttp.h

PS: this is my first programming HOWTO so be merciful :)

The code and ideas from this article can be used without any license restrictions. You can consider it under BSD,MIT,BSL or whatever suites you best. Providing my name or a link to this website would be nice but is not required.

Modern Websites

2006-10-25T00:00:00+00:00

This is not really a project, just my opinion on how modern websites should be done.

Doing Erata.NET website I've learned the advantages of using strict XHTML ( 1.1 ) and CSS. Web design is getting closer to programing this way. You have a modular structure, an easy to follow document layout on witch you can apply styles with CSS. I'm not much of a designer and I'm happy with this style of web design because it allows me to create the structure of the website and use the style-sheets that a real designer can provide.

This site was mostly tested using firefox, but should look the same in any standards compliant browser. From what I've seen in Internet Explorer there are a few thing that don't look exactly the same, but i consider it a waste of time to try to fight browser stupidity. As long as you are using a bad browser you will be able to use this website, just don't complain if some things don't look that good.

This is it for now, until I'll find the time and resources to write an "how to start with XHTML & CSS" but I'd like to recommend to anyone to take the time to understand and learn how this technologies work.

Flash Plugin On Freebsd

2006-10-16T00:00:00+00:00

Flash Plugin on FreeBSD

2006-10-16T00:00:00+00:00

Problem to solve

Make a recent flash player plugin work in firefox on FreeBSD 6.

Update

There is a new article on enabling the linux-flash7 plugin with native browsers.

With the release of FreeBSD 6.2 and the update on linuxpluginwrapper this tutorial has become obsolete. It's only recommended if you need to stick with an older version of freebsd and/or flashplugin.

Solution

First thing to do is get this patch for /usr/src/libexec/rtld-elf/rtld.c, copy it in /usr/src and do the following: (you must have the source tree in /usr/src and the version of the sources must be the same with the version of the installed world, if they are different or you are not sure it's probably best to rebuild your world ( FreeBSD world that is :) ) and kernel )

1 root#cd /usr/src
2 root#patch -p0 < rtld_dlsym_hack.diff
3 root#cd libexec/rtld-elf/
4 root#make && make install

After successfully installing the patch you need to fetch and unpack the linuxflashplayer7:

1 root#cd /usr/ports/www/linux-flashplugin7/
2 root#make fetch
3 root#cd /usr/local/lib/firefox/plugins/
4 root#tar zxvf /usr/ports/distfiles/flashplugin/7.0r68/install_flash_player_7_linux.tar.gz
5 root#cp install_flash_player_7_linux/libflashplayer.so libflashplayer.so
6 root#cp install_flash_player_7_linux/flashplayer.xpt flashplayer.xpt

Now make sure you have the plugin files ( libflashplayer.so and flashplayer.xpt ) in /usr/local/lib/firefox/plugins.You now need to install the linuxpluginwrapper:

1 root#cd /usr/ports/www/linuxpluginwrapper/ && make install clean

Finally you have to change /etc/libmap.conf:

1 [/usr/local/lib/firefox/plugins/libflashplayer.so]
2 libpthread.so.0 libpthread.so.2
3 libdl.so.2 pluginwrapper/flash7.so
4 libz.so.1 libz.so.3
5 libm.so.6 libm.so.4
6 libc.so.6 pluginwrapper/flash7.so

Now open your browser and go to about:plugins. Flash 7 plugins is active.

Web application security flows

2006-10-12T00:00:00+00:00

Here goes... my first security related post :)

Intro

First i must say that this will not be a way to prove that your application is secure, it will only be a quick & dirty way of finding common bugs in web applications. The following will be a few techniques that i tend to use the first time i see an web application ( and generally a database application ). I will use PHP for the few code examples but the vulnerabilities presented are not limited to PHP, in fact are independent of the server-side programming language used.

Unexpected Input

The first and most common bug is unsanitized user input. The easy way to check for this error is to start replacing variables passed on the URL with characters that need to be escaped when used in an SQL query. Let's say the after submitting some data or clicking a link in the application you end up in a page like

http://www.example.com/products.php?category=1

In an vulnerable application this most probably generates a query like

$query='SELECT * FROM products WHERE category_id='.$_GET['category']

Now changing the value of the category to a special SQL character is straightforward:

http://www.example.com/products.php?category='

And if the assumption about how the variable is used is correct you will probably end up in a page containing an error message from the database server telling you that there was an error executing the query. Now the executed query looks like this:

SELECT * FROM products WHERE category_id='

Hmm... what next? Server said error, it was an error so what's the big deal? Very big deal. Now let's change the variable to something that actually does something.

http://www.example.com/products.php?category=1;drop table products;

witch will generate a query like

SELECT * FROM products WHERE category_id=1;DROP TABLE products;

Is this big deal enough ? Or maybe something like:

http://www.example.com/products.php?category=1;insert into users values( null , 'x',password('x'),true );

assuming that there is a user system in the application and a table users ( userid , username , userpassword , isadmin ) the query will become:

SELECT * FROM products WHERE category_id=1;insert into users values( null , 'x',password('x'),true );

which, if the assumptions are correct, will insert a new user x witch is admin.

This example is only supposed to give you an idea of this attack vector on an application. Evan if this example is very simplistic I'm amazed of how many application i found ( and find every day ) that are vulnerable to this attack.

As a habit when i see an URL that contains something like variable=value i usually try changing value to an ' and see what happens. This takes only a few seconds. If i end up in an empty page most probably the attack works but errors are not displayed. If the application keeps working but says an error has occurred again most probably this attack works. If no error is displayed and the application keeps working as expected it only means that the particular variable that I've changed is probably checked and escaped before building the query string.

Again i have to say that if you test your application like this and nothing happens it DOES NOT mean that your application is secure, only that the particular variables that you tried are probably escaped.

I'll stop here for now, but just remember to change the URL every time you see a variable and do report the problem to the author or owner of the application so that maybe the web will become a safer place.

... more to come soon ...