QBRANCH:CODE

Load testing WCF services with Apache Bench

Fredrik Lindström — Sat, 17 Jul 2010 17:31:45 GMT

Apache Bench is a load testing tool that ships with Apache HTTP Server but it can also be run against WCF services with basicHttp or wsHttp bindings. The tool itself consists of one EXE file which makes deploying it to multiple client machines a trivial affair.

Notes

We used Apache Bench against BizTalk WCF (wsHttp, two-way) receive locations to identify bottlenecks throughout our entire solution
Security was disabled on the WCF service that Apache Bench was hitting
There is no ramp-up. Apache Bench will hit your service with full force right from the start.
Use PerfMon to see how your service and back-end systems are doing
Apache bench is very light weight on the client, hardly CPU utilization at all

Apache Bench command – 200 requests with 50 concurrent calls

ab -T “application/soap+xml; charset=UTF-8″ -p data.xml -n 200 -c 50 http://biztalk/OrderService.svc

data.xml

This file contains the message that Apache Bench will send to the service. Use Fiddler to see and copy what is going over the wire when a normal WCF client is sending messages to your service

Apache Bench output

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking biztalk (be patient)
Completed 100 requests
Completed 200 requests
Finished 200 requests

Server Software: Microsoft-IIS/7.5
Server Hostname: biztalk
Server Port: 80

Document Path: /OrderService.svc
Document Length: 1766 bytes

Concurrency Level: 50
Time taken for tests: 3.767 seconds
Complete requests: 200
Failed requests: 0
Write errors: 0
Total transferred: 416000 bytes
Total POSTed: 188200
HTML transferred: 353200 bytes
Requests per second: 53.09 [#/sec] (mean)
Time per request: 941.844 [ms] (mean)
Time per request: 18.837 [ms] (mean, across all concurrent requests)
Transfer rate: 107.83 [Kbytes/sec] received
48.78 kb/s sent
156.62 kb/s total

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 0.6 1 5
Processing: 217 851 312.2 795 1937
Waiting: 216 850 312.2 791 1937
Total: 218 852 312.3 795 1938

Percentage of the requests served within a certain time (ms)
50% 795
66% 922
75% 1026
80% 1079
90% 1255
95% 1551
98% 1781
99% 1879
100% 1938 (longest request)

Qbranch Syd söker utvecklare

Per Önnemyr — Tue, 06 Apr 2010 11:35:00 GMT

Vi på Qbranch Syd söker en systemutvecklare med fokus på integration till Malmö. Är du sugen eller känner någon som är så gå in på vår sida Qbranch.se och läs mer under Karriär.

Hoppas vi ses!

Recap from GTUG Sthlm 4/3: Groovy, MIDI and some Closure...

Leonard Axelsson — Tue, 09 Mar 2010 08:33:00 GMT

Did a presentation about Groovy on Google's App Engine at GTUG Stockholm last thursday. Had a great evening as expected.

Johan Burell (@Burre83) demoed his api and client implementation of a MIDI api for the android. The live "DJ on the Android" demo was cool even though it was impossible to actually see what he did as he used his own phone instead of the emulator during the demo.

Mikael Kindborg (@divineprog) presented on Google Closure, a new library for JS development which comes with what seems to be a really effective JavaScript Optimizer. There was even some more Android to be had when he demoed his JS Android Client that receives JavaScript over a socket and runs it on the device. Was kinda cool when a guy on the first row held up the Mikaels Android and we could se how the ui on the phone changed as Mikael recoded it live on his computer. I have a hard time seeing the practical applications for this but it's a cool demo non the less.

I focused on how to leverage Groovy on App Engine using Gaelyk as Grails IMHO isn't really worth the effort right now (on GAE). The environment I used was SpringSource Tool Suite (Groovy support) and the Google App Engine Plugin for Eclipse.

Presentation and examples: on Github as well as on GTUG Stockholm:s page

DSCT 2.0 Beta 3

Fredrik Lindström — Mon, 15 Feb 2010 18:06:46 GMT

I’ve been meaning to get a final 2.0 version out the door for quite some time but work has taken up most of my free time and will continue to do so for a while longer. I hope you find this new version useful but please keep in mind that it is a beta version.

Special Thanks to Daniel Brad for some real world testing.

Download

DSCT 2.0 Beta 3 x64

DSCT 2.0 Beta 3 x86

Scalability

DSCT now supports ADs containing a large amount of objects (several 100k)

Performance

Big speed increase when restoring groups with many members (10k and up)

Big speed increase when comparing AD:s containing an extreme amount of objects to snapshots

Snapshot state is now saved to disk (1kb file containing a DirSync cookie) when DSCT connects to a snapshot for the first time. Subsequent connections to the same snapshot utilize the saved cookie. Big performance gain.

New features

View who was added/removed from a specific group since the snapshot was taken. Selectively undo/restore changes to the membership

Tree view

Misc

Support for running DSCT 2.0 on Windows XP has been dropped

Known Issues

Full membership restore of very large groups (20k+) fails on a DC running Windows Server 2008. I have not been able to reproduce this error on Windows Server 2008 R2. Needs some more investigation

Let me know
Feel free to leave a comment if you run into any issues

Sharepoint 2010, ADFS 2.0 and Roles

Fredrik Lindström — Sat, 06 Feb 2010 01:39:32 GMT

I’ve been tinkering quite a bit with Sharepoint 2010 and ADFS 2.0 lately and figured that this was worth sharing.

I followed the steps outlined in Travis Nielsen’s blog post to configure a federated identity provider in Sharepoint 2010 and configured ADFS 2.0 in my own way since our setup involves quite a few partner organizations. One thing that is not mentioned in the step by step guide is how to configure Sharepoint to accept role claims and assign access rights based on those claims.

The following Powershell snippet will do the trick

$issuer = Get-SPTrustedIdentityTokenIssuer
$issuer.ClaimTypes.Add(”http://schemas.microsoft.com/ws/2008/06/identity/claims/role“)
$map = New-SPClaimTypeMapping “http://schemas.microsoft.com/ws/2008/06/identity/claims/role” -IncomingClaimTypeDisplayName “Role” -SameAsIncoming
$issuer.AddClaimTypeInformation($map)
$issuer.Update()

After this the “Role” entry will show up in the people picker and you will be able to assign role claims to Sharepoint groups.

New Mailing List for Qanban

Leonard Axelsson — Sat, 23 Jan 2010 12:20:00 GMT

Got any feedback or questions regarding Qanban? Post at qanban-users and we'll get back to you.

Looking forward to hearing from you!

/The Qanban Team

The Qanban Installer

Mattias Mirhagen — Wed, 13 Jan 2010 09:45:00 GMT

Since the development of the Qanban standalone, I started to take a look on taking it to it's next step; a Qanban Installer. Using IzPack I hacked an installation script, installing Qanban on your computer, with the posibility to add items to the start menu. Now all you have to do to run your local standalone Qanban is by going to Qanban in the start menu and click on Qanban->Qanban.

IzPack

As stated on the IzPack website, "IzPack is a one-stop solution for packaging, distributing and deploying applications. It's fully cross-platform and generates a single installer." It's fairly simple to use, and I can really recommend it.

Easier Qanban Install

When running the installer.jar a wizard will take you through the installation process. In this installation you will be able to set standard port and the amount of memory you wish to give to Qanban (port 8080 and 512mb memory is standard).

This will give you a menu item to launch Qanban and Uninstall Qanban. Now it will be even easier to install and run Qanban!

The Installer isn't complete, and therefore there is no release, but quite soon I hope for a first release.

Qanban 0.1 Released!

Leonard Axelsson — Fri, 18 Dec 2009 15:19:00 GMT

Introduction (updated)

Qanban is a digital, multi-user Kanban-board. It's a tool to aid you when using the Agile Process Kanban.

The team here at Qanban Headquarters^TM are happy to present Qanban 0.1 to you! It might be a small step for the world but a big step for us.

Lately we've been focusing on squashing bugs, testing, squashing more bugs and one final feature: standalone Qanban. The 0.1 version can be run directly from commandline and uses embedded Jetty and HSQLDB.

Kanban

This section takes for granted that you have some understanding of how Scrum works.

Kanban is used to visualize your workflow with the help of cards that are moved trough workflow states on a board, very much like in Scrum. Your work in progress is limited by workflow state which means you agree on how many tasks can be in WIP at any one time. In Kanban there are no sprints, instead you use the state limits to control the flow and minimalize lead time. Finally in Kanban your Backlog order is used to set priority and the engineers pull cards form the top of the Backlog when they are ready.

Kanban gives you the tools to visualize your workflow and minimize lead time and that's why we at Qbranch use it. Read more about our reasons for using Kanban here.

That was a very short introduction to Kanban for more information, check out this excellent guide.

Features

Creating and managing cards and phases
User and Admin Roles (in 0.1 all users are created as Admin)
Drag-n-drop for pulling cards through the workflow
Change log
Archive for finished cards
Gravatar integration for user portraits
Rearrangement of phases
Standalone mode
Autorefresh
Offline/session timeout warning

It's early days for Qanban but the featureset is broad enough that we hope for some external usage. Most basic features are there, what we miss are things like statistics, user management and an option if you'd like to have hard or soft rules (currently they are hard). We'd love to hear from you about how you use Kanban!

Running Qanban Standalone

The only thing you need to run Qanban is Java 1.6. Run the command below on the commandline and point your browser to http://localhost:8080/ and you should be greeted by a login screen. After that you'll have to create a user and then you're good to go.

java -Xmx512m -jar qanban-0.1.war

Running Qanban standalone creates files for the database and log in the folder where you ran the command.

How does it work?

We use an embedded Jetty and HSQLDB to get the standalone version working. It's recommended to give the JVM at least 512mb of ram, Grails is hungry. Ofcourse qanban-0.1.war is a normal war file and you can use it with your servlet container of choice as you would with any war.

Open Source

Qanban has been open from the get go but a while back we got round to adding the license terms (Apache V2). Feel free to browse the sourcecode and file bugs on our github repo. In closing, thanks to Mattias and Patrik for doing a great job.

Download: qanban-0.1.war

SweGUG 26th of November - Recapped!

Leonard Axelsson — Wed, 02 Dec 2009 14:59:00 GMT

In my humble opinion the meeting last week was a great success. Göran and Hjalmar did a great job with their presentations and the attendee number went from 8 to 15, lots of good discussions too. It feels like we’ve reached some kind of take-off stage and I’m really looking forward to the meetings to come.

We’re keeping an eye out for interesting presentations, give us a shout on swegug [at] gmail [dot] com if you’re interested in speaking!

Presentations

Working with Excel the easy way

Göran Ehrsson talked about the creation and inner workings of a dirt-simple DSL for importing Excel data, something he’s had to do many times over for his clients. I highly recommend checking out the code, it’s a great demo of a real use case for Groovy DSLs.

// Possible scenario when using the builder in a Grails app
new ExcelBuilder("customers.xls").eachLine([labels:true]) {
  new Person(name:fullname,
    address:address, telephone:phone).save()
}

FSM for text parsing in Groovy

Hjalmar Ekengren showed how to create a Finitie State Machine for parsing text files in groovy and explained when and how it might be useful to implement your own FSM instead of importing Lex. You find the code on our google group.

#3 in sight

In other words, we’re getting ready for our third meeting sometime in the beginning of 2010. We haven’t decided the focus yet but one thing we’ve discussed is the possibility of doing a workshop/hackathlon-style day focusing on Grails Plugin Development, something many of our atteendes were interested in. Keep an eye out for more information here on the blog and on our list.

Photos

Thanks to Johan Moberg for taking pictures during the meeting.

Four weeks of QANBAN

Leonard Axelsson — Mon, 23 Nov 2009 19:25:00 GMT

In the mirror

We’re heading into the fifth week of development and if all goes well we’ll start using QANBAN internally next week. Things have gone smoothly most of the time, got stuck on a couple of features and spent a lot more time than expected on UI/AJAX (github claims our project is a JavaScript project).

New Features

Workflow when moving a card

One of the new features since the last report is a workflow where you get a dialog when moving a card to a new phase. You can change assignee or verify that you wanna be set as assignee for the card (by hitting enter or Ok). The assignee shows in the dialog as a profile picture from Gravatar. We’re planning to add commenting to this dialog as well.

Move card

Split into user & admin actions

As it stands right now you need the admin role to do the following:

Add cards
Add Phases
Modify a phase

Add Cards is something I see the "users" doing at some companies, but in that case maybe all of them should be admins? This might be configurable if the need arises.

Hard rules

When digitalizing a business process there is two extremes when it comes to how you implement business rules. At one end all rules are set in stone and tool enforce them to 100%, at the other end you can do whatever you like and are expected to know how to use the process. When designing QANBAN we took a road that leans towards the “set in stone”-direction, something I’ve come to question the last couple of days. In practice it means that you can only move cards forward, not backward and you can’t pull a card into a phase where the limit is reached. I did a presentation on Kanban last friday (at FindOut Technologies) and one of the things we discussed was that it would probably be more useful to clearly mark a phase where the limit has been reached, but allow it the move itself. If you have any thoughts about the implementation of rules I’d appreciate comments and/or emails (@ leonard dot axelsson at qbranch dot se).

Tracking changes

We spent a lot of time discussing how to implement change tracking in a way that allows for good analysis later. We've choosen to implement changes as change events (see: Event Sourcing) which means that we keep a record of all changes (roughly like a VCS do with diffs). In the long term that means we should be able to implement good workflow analysis and even playback functionality so that you could see the history of the application played out before your eyes. Right now only card moves are tracked.

Changelog

Live for internal usage next week

Integrating with Active Directory is really important for our internal usage and that is what our developers will be focusing on this week. As long as we get that feature down we’ll start using it instead of our analog Kanban wall. Other than that they’ll be adding small things such as:

editing cards
changing the order of phases
restyling of cards with description visible on the board
clear indication of the logged in users cards

We're getting closer and closer to a usable product and would appreciate your suggestions and feedback. Send me an email (leonard dot axelsson at qbranch dot se) if you're interested in testing it and I'll see about packaging an alpha-version.

Inbjudan till SweGUG den 26:e november

Leonard Axelsson — Wed, 18 Nov 2009 13:20:00 GMT

Nu är det dags igen! Vi träffas på Primusgatan 18 nästa torsdag (26:e november) för att snacka utveckling, knyta kontakter och lära oss mer om DSL:er och textparsning i Groovy.

Mer information hittar du på vår google grupp.

A few new features in the next release of DSCT

Fredrik Lindström — Mon, 16 Nov 2009 23:54:52 GMT

Treeview
DSCT displays added/modified/deleted objects in a familiar tree structure. Objects are color coded depending on the type of operation (the color part still needs some work, my respect to anyone who designs GUIs for a living)

Differences in the membership of a group
DSCT can now show you which objects were added and removed from a group since the snapshot was taken. The selected group in the screenshot has over 11000 members but only the changes are displayed.

New features in DSCT 2

Performance & scalability
Early tests have shown that the new version of DSCT works on Active Directories and snapshots containing ~600k objects, older versions of DSCT will flake out well before 100k objects due to limitations within System.DirectoryServices. The majority of DSCT has been rewritten to make use of System.DirectoryServices.Protocols instead. As an added bonus performance has improved dramatically in certain scenarios.

S.DS.P, DirSync & IncrementalValues

Fredrik Lindström — Sat, 14 Nov 2009 16:28:46 GMT

The ability to query Active Directory for changes within multi value attributes was introduced with Windows Server 2003. Before that all the values for a multi-value attribute were returned, even if only one row out of thousands had been modified.

My scenario was the ability to see which objects had been added or removed from the membership of a group, in this case very large groups containing thousands of members. Using System.DirectoryServices.Protocols (S.DS.P) we can achieve this behaviour by passing DirectorySynchronizationOptions.IncrementalValues to our DirSyncRequestControl.

Once you have your initial DirSync cookie only the changes made to the membership of the group will be returned instead of the entire member attribute. The SearchResultEntry representing the group will contain two DirectoryAttributes:

The DirectoryAttribute “member;range=0-0″ contains all delete operations
The DirectoryAttribute “member;range=1-1″ contains all add operations

Now we know which member was affected and what type of operation it was enabling us to write some neat code to act upon these changes.

Data Detective

Daniel Sundqvist — Thu, 12 Nov 2009 09:54:00 GMT

When implementing ILM, you are often challenged with finding information in gooey databases of varying quality. Worst case scenarios include thousands of tables with creative names that not even in the slightest way imply what they may (or may not) contain. Since the SQL-Manager does not support free text search we had to take matters in our own hands. Thanks to our SQL-ninja colleague Pål Edman we got started on developing our own tool for this cause. We started out with a couple of methods and some unit tests which eventually resulted in a serious amount of code that could perform the tasks we wanted it to. After some testing we got self impressed by the performance qualities of our code - it managed to search through 1700 tables within 4 minutes!

the DataDetective in action

The next step was to wrap it all up with a GUI. Although none of us had worked with WPF before this came as an opportunity to try it out and we decided to explore its possibilities for our application. The result was astonishing! Thanks to the BackgroundWorker class avaliable in .Net we managed to scramble together an awesome progressbar, an ETA and an indicator which very pedagogically indicates whether there is time to get a cup of coffee or not. Apart from these vital features it also keeps the application to automatically run a set of threads which prevents it from tangeling itself up when querying the database. Also it lets the user cancel an ongoing search. To prevent us from being sued by multinational companies holding the right for the theme song from "Inspector Gadget" we won´t mention that the song is played as a search is performed. As for now the DataDetective only works with MS SQL.

//Daniel Sundqvist and Olga Stern

Joachim vs Joachim

Fredrik Lindström — Wed, 11 Nov 2009 18:14:35 GMT

Two of my best friends, Joachim Nässlander of Knowledge Factory and fellow Qbrancher Joachim Sörensen, are nominated for IT Consultant of the year here in Sweden. There are three finalists in total so I have a two in three chance of getting treated to lots of free beer (I did not in any way invent that tradition while writing this congratulatory post).