Bitter Coder

Architecture Chat #35

Alex Henderson — Sun, 12 Oct 2008 20:14:33 GMT

After a general catch-up on what everyone's doing and introductions from a graduate new comer (Nick Irvine) - we launched in talking about robotics including:

Automated Kiwifruit picking robots.
Fruit laser bar coding and it's lack of uptake so far.
The possibility of per-ordered fruit i.e. you identify how you want your fruit, and it's picked at the precise moment when it fits the consumers needs.
The user of heuristic markings for fingerprinting of individual fruit (ie. the Idea that nature, by design, provides unique identifier for each piece of fruit, or that we could mark a fruit for identification that wasn't detectable/displeasing to the human eye.
Smart cars, self navigating cars, convoy or drafting applications for self-navigating cars and how these systems would deal with disconnections or extraordinary circumstances.

As an offshoot of self driving cars peter talked a little bit about Scribe (or livescribe - pens that record what you write and say - in unison - and allow playback or online publishing ) - and the future of pen based capture devices and note taking ie. evernote etc.

We talked around Multi-dimensional separation of concerns, and the idea of having both distinct dimensions (that may not be based on a single physical AOP approach) and modules of concerns, and the challenges/opportunities/solutions these "hypermodules" could provide to every day business problems - this also hi lighted the pitfalls of existing AOP approaches which often let you assemble incompatible/incorrect sets of concerns, which modules could help prevent...

I talked briefly around REST and the concept of a generic RESTful application development platform that I've been prototyping lately (like dream, but far more resource and query oriented) rather then re-purposing an MVC framework or using WCF (which also feels like a bad fit) or ADO.Net Data Services.

Last of all we talked about Mass Transit just as we were breaking up - Jamie noted he'd been working on a similar project (but for java?) while at Auckland Uni. Perhaps I'll have a more in-depth report on it next time as I'm currently experimenting with it at the moment when I get time.

Thanks all for coming - write-up's of the previous chats can be found on the wiki.

Upcoming Architecture Chat

Alex Henderson — Wed, 08 Oct 2008 08:44:12 GMT

Hi All,

The Architecture Chat is Tomorrow - or probably "today" by the time your read this - Thursday 9^th October, 11:30am, Garrisons, Sylvia Park.

Some topics that have caught my eye since last time include:

VS 2010 previews/announcments and it's modelling focus, manual testing, test impact analysis etc. - I wonder what opportunities this opens up for 3rd party visual studio tool developers.
JQuery shipping with Visual Studio .Net (only interesting because of the political implications - at least for those of us doing Monorail development - but a very positive sign none the less).
Mono gets a spot at PDC 2008 -about time I say too, it's good to see Mono getting the respect it deserves :)
NHibernate ProxyGenerators 1.0.0 Beta Released - great for those dealing with medium trust (see the intro for details).
Mass transit - definitely a "watch this space" project - they just hit milestone 0.4.
This post from hammet I found very interesting - Multi-dimensional separation of conerns - not something I've come across before , also check out HyperJ which hammet linked in his post. Food for thought.

If anyone else has some additional topics they'd like to discuss (or raise in absence) then just leave a comment on this post or send me an email.

See you all tomorrow. And remember newcomers are always welcome - see the Wiki for details on location and write-ups from previous sessions.

Moving to Orcon LLU - bumpy ride so far

Alex Henderson — Mon, 06 Oct 2008 23:27:33 GMT

As some may have noticed, this blog has been up and down like a yo-yo for the last two weeks.

The issues are caused by the fact that I host it off the home office connection - which by and large has been solid as a rock for the last 2 years, considering the amount of traffic the site gets is fairly minimal.

I opted to be one of the first guinea pigs to try the LLU (Local Loop Unbundled) offering from Orcon on the home office - alas it has not been a pain-free experience. I think google analytics tells the story best:

The connection got switched to LLU on around the 21st / 22nd of September - at which point my connection became a bit erratic - then it started working well enough - before going wonky again, and then completely falling over altogether on the following Saturday - at that point there was no sync.

Around Wednesday sync was restored, then Auth as well on thursday (so 5 days without any access) - the connection was looking ok at that point, but then a day or so later I started noticing that it was disconnecting every 5 to 10 minutes, then taking another 20 or so seconds to re-establish the connection, that's still happening as we speak - though they helpdesk has now raised the priority of the ticket to urgent in the hope that it might be resolved soon.

So far I've been a pretty disappointed with the alacrity of Orcons resolution process and the lack of direct contact that can be made with the LLU team. Normally with Orcon issues get logged with the help desk, a ticket is raised, and after a couple of days you end up dealing directly with corporate support (i.e. people who know what's wrong) - this process works well, and seems to filter out those people who actually have problems related to their own hardware or lack of knowledge.

Not so with LLU, you raise a ticket, but the LLU team can't be contacted directly, and as happened with me - I had no connectivity for 5 days, and also nothing done about the issue, it just sat in the queue - and the helpdesk did not seem to know any more about the issue then I did - I end up feeling bad having to pester the helpdesk every day to find out the progress on the issue - and they keep feeding me the same line of "the LLU tech will call you once the issue has started being worked on / resolved, I'm sure they'll do a card reset 1am tomorrow" - So far nobody from Orcon has ever called me (and I'm not actually sure they ever did do a card reset, but I suspect that's probably not the issue either).

Now I'm not blogging this out of Venom - I mean all said 'n done I am guinea pigging the service to some extent and I know if I'd just stuck with my previous telecom/Orcon mix none of this would have happened - and the @Orcon folks on twitter have offered to help push the ticket through quicker which was nice (incidentally ISP available on twitter == good) - but still I just hope the issue resolution process becomes a little more robust, and I thought this might be interesting to anyone else out there considering jumping on the LLU band wagon.

So my peeves so far are:

Orcon don't actually seem to be doing any follow-up - I had to do all the calling (and sitting in the queue) incidentally, 3pm is the perfect time to call their helpdesk, it's never busy.
The LLU team can't be contacted - that bugs me - pestering the help desk is counter-productive for days on end, when the issue is sitting with the LLU team. Being able to check the ticket status (with notes etc.) on-line would be have been nice as well (and saved some phone calls).
The helpdesk doesn't seem to even know when the LLU team will look at an issue. I got the distinct feeling the help desk guys knew as little as I did.
No weekend support - I lost connectivity early Saturday, but the LLU team doesn't work on the weekend (or so the helpdesk told me) - this sucks - telecom have engineers working weekends, especially as Orcon are responsible for phone issues as well, I could end up with no internet or voice for an entire weekend - something I've always found Telecom very quick to respond to. I would hope this only applies to home customers!

Hopefully these (and the technical issues I'm having) are all just teething problems and will go away as the LLU roll out continues - in the mean time forgive this blog for being partially unavailable, and my apathy in not bothering to move the site to dedicated hosting (which I do plan to do sometime this year, probably along with a change in blogging software - but I'm too busy with other things at the moment).

Arhictecture Chat #34

Alex Henderson — Mon, 06 Oct 2008 23:21:14 GMT

We talked about a couple of announcments, including the Prism V2 Drop 1, and it's multi-targeting support for Silverlight/WPF (Something we've talked about for a while since WPF & WPF/E first emerged) - we did cover some of the other announcments in this post as well.

After that we ended up with quite a general discussion on what makes adopting unit testing (and test driven) practices difficult and how mocking/stubbing/AAA etc. fits in. Interesting side-topics included the idea of teaching test driven development as the very first part of a first-year programming paper at university, behaviour driven development / specs etc.

We had another new comer this week - Guy - who gave us some perspective on how shops still focusing on C/C++ work - as always it's great to have newcomers coming along and contributing to the conversation.

Guy was Albany based, which raised the question again of if we need to have the odd Architecture chat over on the shore - my only concern is that we'd probably loose half of our existing attendees, so unless I got interest for a good 4 or 5 people on the shore It's probably not worth bothering with - and we'd need a location that's not too noisy (we abandoned our original location in Ellerslie for that very reason).

Apologies for the late post - I have been experiencing a lot of issues since moving to Orcons LLU (that'll teach me for hosting this blog and wiki on-site :)

Write-ups of previous Architecture Chats can be found on the wiki here.

Upcoming Architecture Chat - Thursday 25th September 2008 @ 11:30am

Alex Henderson — Wed, 24 Sep 2008 08:48:56 GMT

Architecture Chat Tomorrow (Thursday) 25^th September 2008 at 11:30am, Garrisons, Sylvia Park, Auckland, New Zealand.

Some ideas for topics:

Gallio - interoperable testing tools.
What are the barriers to unit testing - why is still "so hard" for so many?
Stack overflow public beta and impressions after having used it for 3 or so weeks.
Prism V2 Drop 1.
BDUF & DUF (Big Design Up Front and Design Up Front)
Iron Python Beta 5
ASP.Net Dynamic Language support
Arrays considered somewhat harmful
Android 1.0 SDK Released.
C# Eval in Mono (see Miguel's posts here as well as the CharpRepl page here.

All are welcome - and for new comers directions can be found on the wiki.

If anyone has any other topics please leave a comment on this post or flick me an email, or of course just bring along your questions/topics on the day - otherwise I'll see you all there tomorrow.

Architecture Chat #33

Alex Henderson — Tue, 23 Sep 2008 10:36:27 GMT

Short writeup for the last architecture chat - we had 7 (or was it 8) attending the 11th September 2008 chat - including a couple of intergenites who made the arduous journey over the bridge (always nice to see some new faces!).

Topics of discussion included:

Teched 2008 NZ
Lack of developers in New Zealand, and problems/opportunities with current graduates emerging from various institutes.
Working for Microsoft, the buy-up of blogging talent.
The push to improve the profile of bloggers in NZ.
Bloggers Dinner, Scott Hanselman etc.
Google chrome.
PDC 2008.
Managed Extensibility Framework, what it is/is not - though I don't think of any us are quite sure yet - perhaps Hammet needs to do some videos? ;o)
Generic Natural Language DSL's and interesting ideas like Intellisense for a Natural Language DSL's (I found this interesting, not sure anyone else did...)

It was a bit rambling so by the end of it I think most of us were not really sure just what we'd talked about.

As per usual previous write-ups can be found here on the wiki. There's also a group on LinkedIn for those who have attended in the past (or plan to attend in the future? perhaps) so you can connect with other participants.

The next chat is this Thursday - 25th September - I'll post a reminder tomorrow - and if anyone has any topic suggestions please email or leave a comment on this post.

Writing OpenSocial applications in ASP.Net

Alex Henderson — Sun, 21 Sep 2008 22:33:18 GMT

I've had some requests from different ASP.Net developers writing applications for the OpenSocial platform lately wanting to use DevDefined.OAuth to validate signed requests with a public certificate - so I've put together a little example to help get people started.

What are we trying to do?

So first of let's review how the process works and what we're trying to achieve.

Basically as a developer for the OpenSocial platform you will be writing gadgets - these gadgets are generally defined in an XML document providing information about your application and a chunk of html content which will be rendered into an IFrame to kick the application off (including javascript functions etc. to fetch remote content) - to work around spoofing issues and to avoid cross-scripting issues your gadget can use pre-canned functions in the open-social java-script library to make requests to your own services, which to the developer appear to be a direct request to their site.

Under the hood the the requests are instead made to the opensocial host platform, which acts as a proxy passing the request onto your application - at this point the platform employes OAuth (though without any 3-pronged delegate authority being established) to sign the request using a x509 Certificate (i.e. the RSA-SHA1 signature method) at which point it calls your application with the signed request and passed on response back to the calling gadget.

The main goals in signing the request (in so far as I can tell) are to ensure it hasn't been tampered with, spoofed and provide a Nonce and timestamp so you can void replay attacks - something that using SSL alone would not prevent.

Because a lot of the mechanics of OAuth itself is being circumnavigated, it's much simpler to verify the signature is valid.

Gadget XML

Right - onto the code - Let's take a look at the Gadget XML first:

<?xml version="1.0" encoding="UTF-8" ?>

<Module>

  <ModulePrefs title="OAuth
TestBed" description="Testbed
to test the OAuth library" 

    title_url="http://www.devdefined.com/" author="Alex
Henderson" author_email="bittercoder@gmail.com" author_affiliation="friendster"

    author_location="Auckland,
New Zealand" thumbnail="" icon="">

    <Icon></Icon>

    <Locale lang="en" />

    <Require feature="opensocial-0.7" />

    <Require feature="dynamic-height"/>

    <Require feature="views"/>

  </ModulePrefs>

  <Content type="html">

    <![CDATA[

    <script>

      function makeRequest(server_url,fn){

          var params = {};

          params[gadgets.io.RequestParameters.METHOD]
= gadgets.io.MethodType.GET;

          params[gadgets.io.RequestParameters.CONTENT_TYPE]
= gadgets.io.ContentType.TEXT;

          params[gadgets.io.RequestParameters.AUTHORIZATION]
= gadgets.io.AuthorizationType.SIGNED;

          gadgets.io.makeRequest(server_url,
fn, params);

      }

      function renderPage(){

        var url = 'http://demo.devdefined.com/Friendster/SocialService.aspx';

        makeRequest(url, pageload_Callback);

      }

      function pageload_Callback(data) { 

        var response;

        if(data.text){response = data.text;}else
if(data.body){response = data.body;}else{response = data;}

        if(response.indexOf){

          document.getElementById('container').innerHTML
= response;

        }

      }

      gadgets.util.registerOnLoadHandler(function()
{        

          renderPage();  

      });

    </script>

    <div id="container">

    </div>

  ]]>

  </Content>

</Module>

Basically the XML for the gadget is a chunk of metadata and finally some actual content for the gadget itself - notice that it regsters a handler to run on load of the gadget, and that it's then making a request to the url http://demo.devdefined.com/Friendster/SocialService.aspx and render the response in the inner HTML of the container div DOM element.

The key to this being an OAuth signed request is the 5^th line of the makeRequest function:

params[gadgets.io.RequestParameters.AUTHORIZATION] = gadgets.io.AuthorizationType.SIGNED;

So far so good - uploading your application is pretty easy - but is generally platform specific - if you wanted to try this out on Friendster then just head to http://www.friendster.com/developer/ click on the OpenSocial tab and follow the instructions - the approach is to save your gadget definition as an XML file somewhere, then publish that XML document on the web and then create an application in your open social host of choice and direct th it to the XML files location. It takes all of a minute to do.

ASP.Net Application (Provider)

Now for the application itself - first off you need to locate and download the provided public key (certificate) for the social platform host and either store it on the filesystem or in the case of this demo we'll just store it as a string, like so:

public class OpenSocialCertificates

{

    private const string _friendsterCertificate
=

@"-----BEGIN CERTIFICATE-----

MIIB2TCCAYOgAwIBAgIBADANBgkqhkiG9w0BAQUFADAvMQswCQYDVQQGEwJVUzEL

MAkGA1UECBMCQ0ExEzARBgNVBAoTCkZyaWVuZHN0ZXIwHhcNMDgwODEzMTgwMzQ5

WhcNMTQwMjAzMTgwMzQ5WjAvMQswCQYDVQQGEwJVUzELMAkGA1UECBMCQ0ExEzAR

BgNVBAoTCkZyaWVuZHN0ZXIwXDANBgkqhkiG9w0BAQEFAANLADBIAkEAyVjnX2Hr

SLTyAuh2f2/OSRWkLFo3+q+l0Czb48v24Me6CsoexkPgwLOjXmPn/Pt8WtwlisQP

tZ9RX30iymg0owIDAQABo4GJMIGGMB0GA1UdDgQWBBQlDiW+HfExpSnvWqM5a1JD

C+IMyTBXBgNVHSMEUDBOgBQlDiW+HfExpSnvWqM5a1JDC+IMyaEzpDEwLzELMAkG

A1UEBhMCVVMxCzAJBgNVBAgTAkNBMRMwEQYDVQQKEwpGcmllbmRzdGVyggEAMAwG

A1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADQQCXFtEZswNcPcOTT78oeTuslgmu

0shaZB0PAjA3I89OJZBI7SknIwDxj56kNZpEo6Rhf3uilpj44gkJFecSYnG2

-----END CERTIFICATE-----";

    public static X509Certificate2 FriendsterCertificate

    {

      get { return new X509Certificate2(Encoding.ASCII.GetBytes(_friendsterCertificate));
}

    }

}

It's worth noting that the certificates can be changed at any time - but fair warning will be given (according to the document) - and that the location (url) of the certificate is actually included as a parameter in the requests the host makes to your application (xoauth_signature_publickey) - so it's fairly trivial to implement a caching scheme to automatically pull down and update the certificate if it has changed.

Now that we have the certificate we just need to validate the request, in this cause I'm just going to check it's been signed - I've left validating timestamps and nonces out of the example, as these tend to be propritary to the app your building, but both are easy to implement.

public partial class SocialService : Page

{

    protected void Page_Load(object sender, EventArgs e)

    {

        ValidateWithDevDefinedOAuth();

        // now handle the
request...

    }

    private void ValidateWithDevDefinedOAuth()

    {

        try

        {

            OAuthContext context
= new OAuthContextBuilder().FromHttpRequest(Request);

            var signer
= new OAuthContextSigner();

            var signingContext
= new SigningContext {Algorithm
= OpenSocialCertificates.FriendsterCertificate.PublicKey.Key};

            if (!signer.ValidateSignature(context,
signingContext))

            {

                throw new OAuthException(context, OAuthProblems.SignatureInvalid, "check
certificate is still valid");

            }

        }

        catch (OAuthException authEx)

        {

            Response.Clear();

            Response.Write(authEx.Report.ToString());

            Response.End();

        }

    }

}

The process is basically using the OAuthContextBuilder class to construct a context (which is used to calculate a signature base) from the current pages request. It then constructs a signer (used for signing requests and checking their signatures i.e. validation) and a signing context, which is used to provide additional information to the signer, such as the public key of the friendster certificate.

We then invoke ValidateSignature to ensure the signature is valid - if not we throw an exception.

Finally if an OAuthException is thrown we catch it and render it's report to response stream and close it - this provides diagnostic information as per the OAuth problem reporting extension.

Hope this has been useful to some people out there - BTW I'm no OpenSocial expert, so please direct general OpenSocial questions to the apropriate google groups, I can only help with OAuth library questions.

Problems with Friendster Requests

As a side note - for anyone who's tried writing an OpenSocial application using DevDefined.OAuth to validate request signatures they may have found it didn't work! This was a result of Friendster appending an ampersand (&) to the end of the query string on the Url, which was causing the signature base to be incorrect (I'm not sure if this is an issue for other OpenSocial implementations) - The code now checks for this and removes it if discovered - and signature validation works as expected.

Architecture chat tomorrow (Thursday)

Alex Henderson — Tue, 09 Sep 2008 20:50:20 GMT

Architecture chat tomorrow, Thursday - 11^th September @ 11:30am onwards.

Some possible topics / things that have caught my eye lately:

Google Chrome - V8's great, flash works but our "anywhere" platform silverlight wont, and apparently not until chrome gains market share... hmmm.
TechEd Roundup - including the infamous keynote.
Teched Bloggers Dinner report and related "NZ blogging push" for the next year.
C# Library for controlling EC2 Instances (via David)
Generic Natural Language DSL (via ayende)
Managed Extensibility Framework (MEF) Preview 2 vs. Mono.Addins.
Resharper 4.1 released.
Spec/BDD frameworks (MSpec, StoryQ etc.)
Monorail in the Cloud.

See you all there!

P.S. Information regarding the location and previous chat write-ups can be found here.

ADVANCED-DOTNET is being retired...

Alex Henderson — Mon, 08 Sep 2008 20:20:04 GMT

Unfortunately the old ADVANCED-DOTNET mailing list at DevelopMentor is being put down, like an old but still on the odd occasion faithful dog. This would be one of the first .Net mailing lists I ever really latched onto (about 6 years ago...) and there have been some pretty bright sparks on their since day one.

Hi all

We are currently undertaking an infrastructure review at DevelopMentor and
have realised that apart from this one list the listserv is not really used
any more. Unfortunately we cannot really justify maintaining the server for
this one list and so we have decided to retire the listserv.

I created this list as a moderated list six and a half years ago and so feel
a bit sad to see its passing, but the world has moved on from mail
reflectors to blogs, web forums and other discussion mediums.

We will be taking the server offline on the 30th September so you have a bit
of time to get anything you need from the archives

Thanks for your involvement over the last few years

Richard Blewett
CTO
DevelopMentor

I think what's more disappointing is that in turning it off they certainly haven't spent much time thinking about an exit plan, because there's no archive availability (the listserv is getting turned off completely) or alternative place identified for list participants to move to... I often think this is something that needs to be thought about up front (and budgeted for, almost in some separate trust fund) especially as communities and social platforms proliferate - it's a shame to see information evaporate.

That said it looks like some of the long time readers of the group are working to migrate the server away from develop mentor, or to move onto this google group - be interesting to see where this goes.

On a side note I wonder how much sites like stack overflow will errode the last remnants of mail-only technical mailing lists... personally (while I've been beta testing it) I've found the stack overflow concept doesn't really gel with me - but I can see it's appeal to others - personally I prefer discussion over problem "resolution" and the ability for a group to collectively choose not to answer a problem (because it's but 2 clicks away on a simple google search) - where as with the reputation system in Stack overflow the simple/dumb/obvious questions are like crack for the reputation kidies.

On the same note, how much will stack overflow and similar concepts errode organic blog discovery (which is normally while someone is trying to find an answer to a question).... hmmm

Choice is good - another OAuth library for .Net emerges

Alex Henderson — Mon, 08 Sep 2008 10:11:11 GMT

You may recall a few months back I released an OAuth library for .Net (which can be found here on google code) - at the time of writing that library the only other OAuth "solution" for .Net developers was a C# file containing some simple helper functions, that didn't even produce valid results in many cases, and the rather basic implementation as part of the restful chess/myspace example.

Well the guys at Madgex in the UK have now released their own OAuth.Net library.. for more details you can:

Check out Bruce Boughton's post on their initial release.
Get some more details on the Madgex lab page here.
And of course grab the code from here.

As for differences between the two implementations... from a 50K foot glance.

There's a dependency on the Windsor container when using the Madgex library (if you want to use it's out the box implementation / configuration examples etc.)
The Madgex library includes a http module for making a provider implementation easier when building web apps.
The madgex library appears to take a harder dependency on HttpRequest then the DevDefined library (but that's just a hunch, saw it in a few interface definitions, rather then some wrapper equivalent).
The madgex library looks a little more polished (i.e. xml docs) and has a configuration section.
The Madgex library features some niceities like Sliding window validation support (i.e. they only allow messages to be supplied where the timestamp falls within a certain time window of the server, and keep the list of previous nonces only for the period of that window - flushing nonces that are not longer required via a background thread - saving on the volume of nonces to record i.e. for replay attacks). Of course it's an in-memory solution, so not suited to farms, but still cool.
Slightly less fluent consumer implementation, I prefer my libraries consumer implementation, but of course that's just me :P
Both libraries are under the MIT license.

All said and done it's a very sound effort, great work guys! And it's awesome to have some choice emerging in this space.