#.think.in

#.think.in infoDose #36 (4th July - 11th July)

brodie — Sat, 11 Jul 2009 07:40:15 +0900

Announcements

SubSonic 3.0 is Released
Will Code for Green! Developer Contest - any proxy americans out there?
Google Set to Take On Microsoft With Desktop OS
MSDN Up North Downloadable NDC2009 videos
See the Light
Jesse Liberty - Silverlight Geek
Silverlight 3 Relase Download Links and Installation Instructions
The MossyBlog Times. Australian Silverlight & Expression 3 Tour.

Architecture

Developer

UI

Code

Utilities

Singularity Watch

Other

How does the Universe work and What is Reality

#.think.in infoDose #35 (25th June - 2nd July)

brodie — Sat, 04 Jul 2009 09:06:35 +0900

Announcements

Architecture

Developer

How to Make Efficiency Infectious When Working With a Team
The 7 Software Development Wastes - Part 1 Agile Software Development
A Lifestreaming Workflow - tools for lifestreaming
Top 200 Blogs for Developers (Q2 2009)
CouchDB and Me - watch this if you ever considered breaking away from the corporate shackles and go it alone
The HaaHa Show visits .NET Rocks
Did you know... You can get the name of the calling method
Understanding expression trees
Given When Then (GWT) extension for xUnit.net

Silverlight

Utilities

Singularity Watch

The Coming Artilect War - Hugo de Garis reckons gigadeath is imminent
TransAlchemy
Singularity Watch - hey, that's my idea!
Congratulations Human, You've Been Accepted to Singularity University - funny
Msg to our decendants File 0001 – umm, … yea??
TransAlchemy The Age of Transitions - brainwashed, Darwinism, eugenics, big brother matrix styled post human society, worth a view
Scientists create first electronic quantum processor

Books

Compare book & DVD prices in Australia with Booko – next time you’re thinking of buying a book (in Australia) go to this site first
Fishpond.com.au – this book ecommerce site seems to be pretty good for Aussies

Other

Introduction to Nanotechnology • Nanotechnology 101 • Nanotechnology Project
Andrew Brodie, Institute of Fundamental Sciences, Massey University - yea, didn't you know, I study nanotech in my spare time ;-)
Book Lists 100 Books Your Child Should Hear before Kindergarten at the Library
Microsoft misses the Outlook point
StackExchange™—The Stack Overflow Knowledge Exchange Platform

#.think.in infoDose #34 (18th June - 24th June)

brodie — Wed, 24 Jun 2009 21:52:42 +0900

Announcements

Remix Australia Videos - if you didn't get to Sydney, like me :(, check out the presentations.
Microsoft Robotics Developer Studio 2008 R2 Released - maybe
Microsoft Tech.Ed Australia 2009 - Register for the event - early bird registration - $300 off ... now I just need another $1699

Architecture - data access, languages, tech choices

The Only Pattern for Data Access is - There Are No Patterns for Data Access
NHibnerate Handling the Special Cases
Getting Started First Project in Fluent NHibernate
NHibernate - natural-id
Data Access Stored Procedures vs. ORM (ad hoc) Queries
Windows Mobile Developers Australia Codemasons' Guild
Focus on the Extension of WCF Behavior
Installing Windows SharePoint Services Application Template Core and All Templates
Ruby at ThoughtWorks - Fowler discusses experiences with Ruby based projects
dnrTV - F# screncast
Getting the Scoop About Olso and M

Developer - processes, learning

Who's the owner Shared code vs. code ownership Agile Software Development
Effective software development - Mentoring the value of pairing
Patterns and Practices of Lean Software Development
Values
Corporate Information - Our Philosophy - Google Philosophy
Lean Concepts for IT Professionals
Stop and Refactor
SQL Server Reporting Services and Working with Overlay Data
Adopting Agile Practices
Microsoft Learning Manager - start here if iinterested in getting certifications
9 Important XML Syntax Rules You Must Know
Episode 1 The Code Room Channel 9 – gold geekiness
ThoughtWorks - Audio Series

Silverlight/WPF

Utilities

Singularity Watch

Are Commercial Buildings Ready for Open-Source Energy Management
Kurzweil in the Lyons’ Den - Ray cops a beating for being the glass is half full guy
H+ Cafe - Upload Your Thoughts
'Lightbulb' molecule has a bright future - teeny-tiny lights!
Brain could adapt well to cyborg enhancements - I want my terminator hand!

Other

Wookah - I was wondering if there was an aggregated search engine out there
Geeky Innovations - Geekdom - Kingdom of Geeks - this seem to be a v.cool company
Revision3 Current Shows - if you are a geek, you don't need ever need foxtel ... or a tv for that matter!
Warning over 'superbug' risk from pets - anyone want two cats ... cheap!
The Real Lessons From Twitter

#.think.in infoDose #33 (10th June - 17th June)

brodie — Wed, 17 Jun 2009 22:04:39 +0900

Development

Architecture

Singularity Watch

Other

#.think.in infoDose #32 (2nd June - 9th June)

brodie — Sat, 13 Jun 2009 07:22:23 +0900

Announcements

Server Fault – stackoverflow for NOCs :)

Development

Architecture

Training – software architecture training
The role of a hands-on software architect
Domain-Driven Design Two basic premises - DDD
I don’t believe in ORM object as your domain model
To ORM or Not to ORM.

Unit Testing

A Look at CreateMock, DynamicMock and PartialMock Methods in RhinoMocks

.Net

Books

Singularity Watch

Other

Quotes

" Live as if you were to die tomorrow. Learn as if you were to live forever." - Gandhi

NGourd BDD for WPF Applications

tarn — Mon, 08 Jun 2009 10:40:46 +0900

I checked out NGourd, a project to build a BDD tool based on Cucumber which Michael Minutillo has started. I’d never previously used or even heard of Cucumber, but as I read on I liked a lot about it. It’s an agile development tool and it has a colourful console!

The concept is illustrated beautifully at http://cukes.info/, but here is my take anyway. Behavior is described in the form of a series of plain text steps. The steps are then implemented as parameterized test methods. Using some reflection and regular expressions the plain text steps are mapped to corresponding implementations and run as red/green light tests.

I started wondering if WPF applications could be tested with NGourd. There’s a great IronPython sample in which a WPF application is started in a new thread and built dynamically from the IronPython interactive console.

Could something like this could be done in NGourd steps?

	
Scenario: Can Perform basic arithmetic

Given I start the application
When I enter 2 + 2
And I click GO
Then I should see 4

By using a thread to run the application and a dispatcher to invoke methods on it I had a crude framework to write the steps to make a System.Windows.Application dance. The steps class library has references to the application (to run and test it) and to the NGourd.Core (which has the Attributes I need to decorate the step classes and methods). I’ve exposed the controls of application publically to make the test easier to write.

[Steps]
public class Behaviour
{
    public Thread _thread;
    public Dispatcher _dispatcher;
    public App _app;

    [Step("start the application")]
    public void StartApplication()
    {
        AutoResetEvent are = new AutoResetEvent(false);
        _thread = new Thread(() =>
        {
            _app = new App();
            _app.Startup += (sender, eventArgs) =>
                    {
                        _dispatcher =
                            Dispatcher.FromThread(_thread);
                        are.Set();
                    };
            _app.Run();
        });
        _thread.SetApartmentState(ApartmentState.STA);
        _thread.Start();
        are.WaitOne();
    }

    [Step(@"enter (.*)")]
    public void EnterScript(string script)
    {
        _dispatcher.Invoke(
            new Action(() =>
            {
                _app.MainWindow.UserScript.Text = script;
            }));
    }

    ...

    public void After()
    {
        _dispatcher.Invoke(
            new Action(() =>
            {
                _app.MainWindow.Close();
            }
            ));
        _thread.Join();
    }
}

Writing the application

The demo application is a trivial calculator that executes Python statements.

The method below is the model for this application. It’s worth mentioning it’s very easy to include C# objects to the scope of the ScriptSource, but only for this post, I’ve restrained myself to the more simple behavior I specified earlier (but I did add it to the demo which can be downloaded here)

public string EvaluateExpression(string expression)
{
    try
    {
        ScriptEngine engine = Python.CreateEngine();
        ScriptRuntime runtime = engine.Runtime;

        ScriptSource source =
          engine.CreateScriptSourceFromString(
                       expression,
                       SourceCodeKind.Expression);

        return (source.Execute() ?? "").ToString();
    }
    catch(Exception ex)
    {
        return ex.Message;
    }
}

Of course this could be unit tested itself.

[TestMethod]
public void ModelCanDoBasicArithmetic()
{
    Model model = new Model();
    string output = model.EvaluateExpression("2+2");
    Assert.AreEqual("4", output);
}

So what are we testing? I guess its this

public void Execute_Click(object sender, EventArgs e)
{
    output.Text = _model.EvaluateExpression(input.Text);
}

The only thing I had to do the application specifically to get it working in test environment was delete an attribute in App.xaml root element

StartupUri="Window1.xaml"

This has consequences. No window loads on start up! By added a constructor in my App class I was able to remedy this. This may have wider reaching consequences I’m not aware of, and I’m also not sure if using the OnLoad event is better practice. Anyway it’s working fine for now.

public partial class App : Application
{
    public App()
    {
        MainWindow w = new MainWindow();
        w.Show();
    }
}

It was fun implementing the steps in my scenario then writing the application and watching the behavior test turn green like a cucumber!

Isolating scenarios in separate application domains

Even though everything looked looks like it was working, I wanted to add another scenario (which won’t surprise readers of my recent posts)

	
Scenario: Can Execute Python statements

Given I start the application
When I enter “hello”.upper()
And I click GO
Then I should see HELLO

It turns out this extra test didn’t need any additional step implementations or changes to the WPF application itself, but did show a very real deficiency in what I was doing. I was reminded of the rule enforced by InvalidOperationException:

Cannot create more than one System.Windows.Application instance in the same AppDomain.

I knew of this rule, I still tried to get round it anyway. Can I close one somehow? Delete one? Delete one and de-reference its friends and family?

Creating a new AppDomain seems a clever way round the rules reasoning, but the wary fear this may be some type of pandora's box, at very least additional complexities. Just thinking of MarshalByRefObject proxies makes me feel I should be spending more time enjoying life with my friends and family.

I got some way using DoCallBack until I realized I couldn’t make the steps parameterized as DoCallBack doesn’t have any parameters! (I also stubbornly know it throws runtime errors if one tries to use the looser lambda scoping to cheekily pass data across application domains, luckily though it’s the same stubboness that stopped me giving up this post all together).

As I was thinking I should use WCF for communication betweens application domains, another way I could do to it clicked. I just needed to create my original steps class in a new application domain. By deriving the test class from MarshalByRefObject I could call its step methods from the control domain using it dynamic proxy. I then created a wrapper class which creates a new AppDomain, creates the previous steps in the new domain and uses the dynamic proxy to call the steps. This wrapper class now has the Step attributes for NGourd to pick up.

[Steps]
public class BehaviourWrapper
{
    AppDomain domain;
    Behaviour behaviourProxy;

    public void Before()
    {
        domain = AppDomain.CreateDomain("TestDomain");
        behaviourProxy = domain.CreateInstanceFromAndUnwrap
            ("DLRCalculator.BehaviourTests.dll",
             "DLRCalculator.BehaviourTests.Behaviour") 
                as Behaviour;
    }

    public void After()
    {
        behaviourProxy.After();
        AppDomain.Unload(domain);
    }

    [Step("start the application")]
    public void StartApplication()
    {
        behaviourProxy.StartApplication();
    }

    [Step(@"enter (.*)")]
    public void EnterScript(string script)
    {
        behaviourProxy.EnterScript(script);
        Thread.Sleep(5000);
    }

    [Step(@"click GO")]
    public void ClickGo()
    {
        behaviourProxy.ValidateOutput();
    }

    [Step(@"should see (.*)")]
    public void ValidateOutput(string expected)
    {
        behaviourProxy.ValidateOutput(expected);
    }
}

Running some tests with NGourd

I’m impressed with NGourd and have generally had a lot of fun and success playing with it. It’s a little disappointing that no results are displayed until all the tests are complete. Perhaps it’s because my tests take ages and it would be nice to see what they’re doing (starting an AppDomain and running an Application in it isn’t a lightweight task). Mainly though, I wanted to include an action shot of it firing up windows during the test.

Some thoughts and reflection

I didn’t write this to actually use it in a real project, this was a personal project deliberately devised to cover some some things I was interesting in learning about and trying out. To do automation on a real project I would look further into White an open source project released by ThoughtWorks, which I hear is really good.

The concept of Cucumber is pretty cool and I was nice to have a look at the NGourd source code. In writing this post I enjoyed putting in practice what I new could be done relatively easily (creating and managing multiple AppDomains, Applications and Threads in a process) and I was happy to find I had no problems at all with NGourd itself getting the UI tests working.

This is all I planned to do with NGourd, the UI steps and the Calculator, but I’d be interested to hear what you think of all this rambling. I know I got a lot out of it, did you? What of the other behavior driven development tools MSpec, NSpec or NBehave? What are your experiences?

I’ve upload the project from this post here.

#.think.in infoDose #31 (24th May - 1st June)

brodie — Mon, 01 Jun 2009 22:41:36 +0900

Google’s cupcakes from Google IO… – Google developer lovefest – got to check out Google Waves – awesome email/messaging/collaboration application – can transmit characters as they are typed!

Developer

Architecture

Silverlight/WPF

Utilities

TSql Unit Test Generator

Books

Singularity Watch

The Future of Artificial Intelligence
FOREVERISM
Create an AI on Your Computer ... this should help to to get singularity v0.1 out the door
The QB1 Computer No Mouse, No Keyboard. Just Gestures
6 in 1 Solar Robot Kit Standing on the Shoulders of Giant Memes - i will need this
The Emotiv Headset - Gaming With Thoughts Alone - i will definitely need this
The secret to a really long life. Be happy. Exercise. Eat right. And sign up now to get nanobots in 2030
The Future Brain
Will enhancement destroy the real you

#.think.in infoDose #30 (18th May - 22nd May)

brodie — Mon, 01 Jun 2009 21:43:32 +0900

Developer

Architecture

WCF

Anil Speaks WCF Intermediate Service between client and server

.Net

Utilities

Silverlight/WPF

Book

Singularity Watch

Other

Top 100 Aussie Web Startups - May 09

#.think.in infoDose #29 (11th May - 15th May)

brodie — Sun, 31 May 2009 07:44:23 +0900

Announcements/Events

Development

Web/Silverlight

WPF

Mark Wilson-Thomas WPF and Silverlight Development in Visual Studio 2010 The Knowledge Chamber Channel 9

WCF

Architecture

Utilities

bDule – nice looking twitter/facebook client
C# Script The Missing Puzzle Piece.
CS-Script

Other

The Human Network
Us Now watch the film
TeamCity.CodeBetter Project List
A 3 minute guide to embedding IronPython in a C# application
Office 2010 - countdown to the beginning of awesome ;-)

#.think.in infoDose #28 (29th Apr - 8th May)

brodie — Sun, 31 May 2009 07:40:48 +0900

Announcements

LIDNUG ScottGu talks shop with developers on LinkedIn Events
Victoria .Net Meeting 12th MAy
What's on at Remix 09 Sydney Australia

Development

WPF

WPF Practical Solutions

WCF

Utilities

Web/Silverlight

Architecture

Books

Code: The Hidden Language of Computer Hardware and Software, by Charles Petzold

Singularity Watch

Other

Google named best place to work in Australia
Dime Casts.Net -- Inform and Educate in ~10 Minutes or Less - Func and Action
Dime Casts.Net -- Inform and Educate in ~10 Minutes or Less - ReSharper 4.5 Navigation
F# Stuff I get confused about at Mark Needham

#.think.in infoDose #27 (20th Apr - 28th Apr)

brodie — Wed, 27 May 2009 20:41:33 +0900

I've changed jobs recently and my working habits have changed slightly, so bear with me as I get these link posts back on track :-)

Architecture

Videos on Oslo M = DSL Domain Specific Languages
Repository is the new Singleton
To mock or not to mock, that is the question – Part 1
Fluent NHibernate and Linq2NHibernate
.NET Rocks! - Jonas Follesø on UI Framework Choices
The Voidspace Techie Blog - Dynamic Languages and Architecture
Repository is Dead Long Live Repository
DDD Making implicit concepts explicit

Development

Web

I Spose I’ll Just Say It You Should Learn MVC « Rob Conery Gold!

Utilities

Singularity Watch

SmartFaucet By IHouse Recognizes Faces To Adjust Temperature And Flow - the singularity has to start somewhere, why not a tap?
The World's First Solar-Powered Waterproof Cell Phone
Deseret News Universities will be 'irrelevant' by 2020, Y. professor says

Books

IronPython in Action Front Page

Other

#.think.in infoDose #26 (14th Apr - 17th Apr)

brodie — Wed, 27 May 2009 20:38:58 +0900

Development

100 Interview Questions for Software Developers
Answers To 100 Interview Questions
Agile practices are not just about management.
The Pond - overcoming difficulties with remote working

Web

Using jQuery Grid With ASP.NET MVC

Silverlight/WPF

UI Design

50 Sites That Will Help You To Become a Better Designer

Utilities

Books

Other

Quotes

"People who cease to grow can't inspire others. Leadership begins with challenging oneself." ~Daisaku Ikeda

IronPython In Action

tarn — Mon, 25 May 2009 10:39:00 +0900

While twittering (or tweeting) about an IronPython presentation I was doing, @voidspace messaged me asking if I might mention his book. I’d contacted Michael Foord a while back about some code I was playing around with. I also knew of him from his blog, his prolific contribution to IronPythonURLs and as the co-author of the new book IronPython in Action. He’s so all over everything IronPython he even reads this blog.

Mainly because I was too cheap to buy myself a copy it, I hadn’t read his book. I was wrapped when he offered to get me a copy if I’d review it on this blog. I wanted to read it as I was learning IronPython and I knew it would be a good book, I’d read chapters 1 and 7 which are online. I happily recommended it in my presentation, here’s what I thought about the book.

IronPython in Action is a fantastic resource for anyone learning IronPython and for anyone wondering what this dynamic language IronPython is all about and whether they should learn it. It feels nicely balanced for both .NET users looking to learn about IronPython, and for Python users looking to learn about IronPython on .NET.

I prefer software books that have plenty of discussion and can be read from start to finish otherwise I end up skipping through chapters I never go back too. I found this book well written and fun to read with interesting insight all the way through. There is enough about the python language and .NET to get you through the book, and rather than being a reference book it instead provides information on how to learn more.

The book is divided into four main sections. The first section Getting Started with Iron Python contains a good introduction and history of IronPython on the .NET Framework.

The Core Development Techniques section is a walkthrough of building a simple Windows Forms application. I think this section is fantastic as the text places high value on Agile Development philosophy, using a Model-View-Controller pattern and Test Driven Development. I think these are exciting ways to develop software on the .NET framework. There is also a really good section discussing the difficulties, pitfalls and work-arounds using IronPython and dynamic programming languages generally on the .NET framework.

The third section IronPython and Advanced .NET contains chapters about using the major pillars of the .NET framework with IronPython. It has chapters on WPF, Powershell, ASP.NET, Databases, Webservices and Silverlight. It really is amazing how much of the framework can be used in IronPython.

The final section Reaching Out with IronPython has fantastic chapters on extending IronPython with C#/VB and embedding an IronPython Engine in C#/VB applications to provide extensibility.

I highly recommend this book, I enjoyed reading it and learned a lot from it. If you’re looking for a little more insight, the forward by Jim Hugunin frames the language, the technology and author well.

IronPython Presentation

tarn — Mon, 11 May 2009 10:25:00 +0900

I’m doing an introduction to IronPython presentation at the Victoria .Net user group tomorrow, I’ve been attending the user group for about a year and I’m pretty excited about presenting. I think python is a fantastic language and it’s great to be able to build software on the .Net framework with it.

The Slides

	Introduction I wanted to start with the second slide and have a photo of a pub on the contact slide but decided to get through this before trying out PowerPoint jokes.
	Why Python I just plan to discuss Why I started using Python. The cartoon is, of course, XKCD.
	Python Basic information about the language and its background. Would have liked to have added meta programming or at least “..” to the paradigms. I think it’s a fantastic language so I’ll mention it’s expressive, fun, productive or whatever adjectives I’ve got in my head at the time.
	People using Python I really want to demonstrate that Python is real language, being used by real people, to write real code. I probably should also have mentioned Google App Engine here which I’ve been tinkering with a bit recently and think is pretty cool.
	IronPython The Python language on .Net. Most of the demo is doing cool stuff in .Net with the Python language. I think it goes further than this and allows developers to use their own paradigms and patterns to do .NET development.
	C# and IronPython comparison I saw this example in the awesome Iron Python in Action book. I included it as it’s a great introduction IronPython code, as it clearly and simply shows differences on similarities with C#.
	Dynamic Language Runtime The Dynamic Language Runtime project is open source. Its quite well described here as: A dynamic type system Dynamic method dispatch Dynamic code generation IronLisp has been superseded by the IronScheme project.
	Dynamic Typing This slide is just a brief overview of what duck typing is. It is great for writing test objects, but I have a slide for the later.
	Interactive Interpreter I think it’s awesomely cool, and I use it through the entire demo so I thought it might be worth warning people.

The Demo

The demo is a series of scripts run into the IronPython Interactive Console. I’ve uploaded the very small collection of scripts for this demo here.

IronPython and the CLR

Create a string, show how its methods can be reflected. Import the CLR and show how the .Net String methods are now available.

# Introduction to IronPython and the CLR

# Standard first Python console command
2 + 2

# Create a string
s = 'Hello IronPython'

# Inspect its methods, note that its only Python methods.  
dir(s)

# Python has a method to make the text uppercase
s.upper()

# Import the CLR
import clr

# Now look at the same objects methods,
# We now see the .NET string methods!
dir(s)

# And can use them!
s.ToUpper()

Screen Shot

Using a .Net Classes

Create a WebClient object from the .Net framework to download an RSS feed. Use an xml2py module written by Harry Pierson to deserialise the RSS feed into an object graph. Then send the dates and titles to the Speech Synthesizer. I’ve seen this done before in an IronPython presentation, and just thought it was to much fun to leave out.

import clr

# Create a WebClient and download a string of XML
from System.Net import WebClient
w = WebClient()
xml = w.DownloadString('http://feeds.theage.com.au/rssheadlines/technology.xml')

# Lets have a look at first 60 characters of the xml..
print xml[0:60]

# If bad stuff happens in the presentation..
#xml = open('theage.rss').read()

# Deserialize to objects using xml2py
from devhawk import xml2py
rss = xml2py.parseString(xml)

# Print the results
for post in rss.channel.item:
  print post.pubDate, post.title 

# Import Speech  
clr.AddReference('System.Speech')
from System.Speech import *
ss = Synthesis.SpeechSynthesizer()

# Say titles and dates using the SpeechSynthesizer 
for post in rss.channel.item:
  s = "%s. %s." % (post.pubDate[:16], post.title)
  ss.SpeakAsync(s)

Screen Shot

SQL

I really don’t want to do this in the presentation. I did spend a fair amount of time playing round with asynchronous SQL queries for use in a WPF application, but it seemed too complicated for the demo so I’ve left it out. I use

# import a simple wrapper around System.Data.SqlClient  
# the wrapper is on my blog, I just wanted to show the AdventureWorks 
# database, it seems a bit of .NET presentation tradition. 
from demo import sql

# setup a query on the AdventureWords database
query = 'SELECT * FROM Production.Product'
sql.connection = 'Data Source=localhost\SQLEXPRESS;Initial Catalog=AdventureWorks;Integrated Security=SSPI;'

# execute the query
products = sql.SqlQuery(query)

# print some results
for p in products[:5]:
  print p.Name

WPF, Data Binding and Exec

I think there is some pretty cool stuff in this demo. Although it takes a lot more to write real applications it does open new ways to develop and test .Net applications.

# More pythonic way of getting the feed.. lets hope I have internet
from devhawk import xml2py
rss = xml2py.parse('http://feeds.theage.com.au/rssheadlines/technology.xml')

# Importing avalon starts a new thread to run the WPF application
# It sets up some dispatures so the console can talk to the GUI 
from samples import avalon

# Create a window, show it and load some XAML into it
w = avalon.Window()
w.Show()
xaml = avalon.LoadXaml('demo.xaml')
w.Content = xaml

# Loading names is really cool, it traverses the control tree 
# described in the XAML and creates a dictionary of names and 
# controls. We can add those dictionary items to the local scope,
# so we get the same effect as a generated designer files in
# Visual Studio. 
avalon.LoadNames(xaml,locals())

# We can bind a Python list of Python objects to the ListView
listView.ItemsSource = rss.channel.item

# Create a button click callback that will execute the script
# in the query TextBox.
def OnClick(*args):
  try:
    exec (query.Text)
  except:
    print "Script threw an exception"
  pass  

# Wire up the click event to our function  
execute.Click += OnClick
  
# We can now execute this on the client  
listView.ItemsSource = [p for p in rss.channel.item if p.title.find('to') >= 0]

Screen Shot

Debugging IronPython with my Excalibur

tarn — Sat, 11 Apr 2009 10:50:00 +0900

My interest in a light-weight command-line debugger developed a while back after I watched one used in a Ruby on Rails video. I suspect there are many .Net developers with more experience than me who have used command line debuggers and never want to go back. The more I code, the more I long to leave my mouse behind and communicate with my development tools using only my keyboard, Excalibur.

I have been following a series of blogs by the awesome Harry Pierson (aka DevHawk) on writing an IronPython debugger. When I decided I actually wanted to play with the debugger, I found myself going back over these blogs taking notes as I went. This post is an attempt to summarize all this information and provide the links for going deeper, for myself if no one else.

Writing an IronPython Debugger: Introduction

Explains why he built a debugger when there are other alternatives. The Visual Studios debugging was required to much mouse clicking, although he has posted about doing that too in Debugging IronPython Code in Visual Studio. MDbg which is a .Net Command Line Debugger doesn’t support Just My Code debugging. He also notes the reason for not using/porting pdb (The Python Debugger) is due to IronPython not yet implementing settrace.

Writing an IronPython Debugger: MDbg 101

Describes the basics of MDbg. I found it interesting there is also an IronPython extension for MDbg which isn’t for debugging IronPython code, but to interact with MDbg using IronPython. Sounds cool, I’ll have check it out some time.

Writing an IronPython Debugger: Hello, Debugger!

Goes over the worlds simplest debugger, and it really is pretty straight forward. It just passes the path of ipy.exe and the python code to debug as a parameter into CorDebug (provided by MDbg). CorDebug starts the process (it can be used to debug any .Net process) and provides some events to hook into; OnCreateAppDomain and OnProcessExit. The world simplest debugger basically just prints some text when the AppDonain is starts and stops.

Writing an IronPython Debugger: Setting a Breakpoint

Provides some background information:

ipy.exe produces debug information when the –D parameter is used.
IL generation is dynamic and in memory so the debugger API provides the Symbols (equivalent to .pdb files) as a Stream and has a callback when the Symbols change.
The .Net framework has an API to read and write these symbols a files and MDbg provides a wrapper to read them from a Stream.

Then covers the python code to load the symbols, translate document/line into a function/offset, set a breakpoint. Does not implement user defined breakpoints in the post.

Writing and IronPython Debugger: Adding Interactivity

Covers the basics of how the interactivity will happen, which I’ve just realized isn’t that complicated. The process being debugged just runs until a breakpoint event occurs (or the symbols change or the process completes). Once the process has stopped the the breakpoint the interactivity can occur. It would be nice to be able to set breakpoints while the process is running, but it does keep it more simple for now.

Writing an IronPython Debugger: Dynamic Stack Trace

Background info on MDbg:

Two stack chains in a typical managed app; unmanaged and managed. The debugger for managed code so only the managed chain used
Each contains a collection of stack frames which is the familiar call stack. There are three types of stack frame; IL, native and internal.
Explains Dynamic Methods are usually used created by IronPython but can’t be debugged and are implemented as non-dynamic methods when the –D option is set on ipy.exe.

MDbg has a wrapper around the unmanaged metadata API to get method information for displaying the call stack.

The command in ipydbg is used to view the trace from the interactive console is “T”.

Writing an IronPython Debugger: Refactoring

Mostly discussion about code changes and refactoring. Add code to automatically generate the MTA apartment state if the –X:MTA argument is not used. Explains the effect of this on the debugger design.

Writing an IronPython Debugger: Stepping Thru Code

Introduces console commands for step (S), step in (I), step out (O)

Writing an IronPython Debugger: Debugging Just My Code

Skipped over this one, and feel I may start skipping over a few where I’m not specially interested in the implementation. I can always refer back to them later if I really need to understand something.

Writing an IronPython Debugger: Showing Source Code

Describes an issue when stepping into a Python function the CLR breaks at some infrastructure code, presumably there to manage Pythons decorators and such. Hence there is no line of user code to display. To resolve this an additional automatic step is added when stepping into a function so it can be mapped to a line of user code.

To avoid multiple hits to the file system source files used to retrieve user code are cached.

Writing an IronPython Debugger: Colorful Console

Describes how he choose to implement colors in the console, while bemoaning the stateful nature of the console foreground colours (ie when you change the foreground colour it will stay that way until you change it back). I’m sure I’ll come back to this post when I want to add colours to the console but for now I skipped through it, it doesn’t have much to do with writing or using an IronPython debugger. I guess that’s why he discusses moving it out of the ipydbg and into its own module in his next post.

Writing an IronPython Debugger: A Little Hack…err…Cleanup

Discusses some issues regarding mapping to the debugger COM object instance. This didn’t sound like a whole log of fun and I hope I don’t have to come back and fully understand this one.

Writing an IronPython Debugger: Getting Local Variables

Uses the GetLocalVariable(int index) and GetLocalVariablesCount() methods from the MDbg CorFame class. This post finally made me look up what lexical scoping meant, its a term I’ve heard heaps usually in discussions about compilers that I never really understood. Its fair to say that I still not confident in my understanding of it.

Discusses matching up debug symbols with variable names from the user code. Doesn’t actually evaluate the variable in this post. Notes get_locals from the IronPython process emits some locals used internally, these are prefixed with a dollar sign.

Writing an IronPython Debugger: Displaying Values

This post covers some pretty tricky interfaces for dealing with all the different types, which requires a pretty good understand of how the CLR handles variables under the cover. I didn’t read into to much detail but will consider revisiting it later as the content is quite interesting and it’s good to understand core CLR stuff.

Writing an IronPython Debugger: Command Routing

Discusses how console commands are routed to functions. This really has nothing to do with debugging IronPython but it’s interesting the way it’s implemented. He starts by implementing a switch using a dictionary of input commands and functions (Python has no C# switch keyword equivalent). He then takes it further by making use of Python decorators to bind commands to functions.

Writing an IronPython Debugger: Getting Arguments

Discusses getting function arguments as locals.

Writing an IronPython Debugger: REPL Console

Discusses implementing a REPL console in the debugger with the IPyDebugProcess object available in the console scope. This is awesome for exploring the API using python, just by reflecting the methods of the process object I realised it would be trivial to add a command to list the source files currently being debugged as it’s just a property on the object. Explorative coding using reflection is a really powerful concept in Python.

In the current implementation is a new local scope is created and used by the by the REPL console, it does not yet support executing code in the process being debugged.

Writing an IronPython Debugger: Breakpoint Management

Implements console commands for setting Breakpoints, as the original post was about how it worked and only implemented breaking on the first line of the user code. Introduces multi-key commands for breakpoints. Adds a breakpoint function (B) with sub functions add (A), list (L), enable (E), disable (D).

Here’s a screenshot of me debugging a really simple app which might help explain what I’ve been talking about for this whole post.

I’m pretty amazed by the productivity and brilliance of Harry Pierson, I’ve learned heaps reading his blogs and I’m really impressed by the progress he’s making with IpyDbg. The debugger still has a long way to being a really useful tool, but its coming along very quickly. I think I will have to try using MDbg with the Python extensions, but I really hope to make some use of ipydbg and maybe even find something I can contribute to it.

If your interested in trying out the debugger I recommend checking his blog and the latest version of the project on GitHub.

	Testing in Python It’s really cool you can do true test driven development in that you can write unit tests for classes and methods that haven’t yet been written. Some cool GUI testing is also possible using some the stuff in this presentation.
	Tools / SDKs I’m not expecting to mention Python Studio or IpyDbg as I’ll probably be running out of time. I really should have included a link to python.org, I’ll make sure I mention it. You can get IronPython here and the Silverlight Dynamic Languages SDK
	Resources Dive Into Python – Free online, python for programmers. IronPython In Action – Python and everything .NET with TDD and MVC practices. IronPython Cookbook – Online collection of IronPython .Net scripts. IronPython Urls, MSDN, Devhawk – Harry Pierson, Program Manager at Microsoft. Voidspace – Michael Foord, Author IronPython in Action.
	Thanks By this stage I’ll almost certainly be running late, and probably be really worried about some minor disaster during the presentation. I hope it goes well and everyone takes something out of it, I look forward to hearing what you thought.