Later I revived my interest at a second college in London, it was very hands on affair with electronics. We built stuff like bistable and astables from scratch and programmed a microprocessor (6502) motherboard using a hex keypad and 7 segment LED displays. After this I attended university, focusing on Electronics Systems Engineering which was electronics with computing (analog and digital). We did things like analog control systems using analog computing blocks to create PID based control systems. One example I remember fondly machine having a dial that controlled via a motor another replica dial, the object was to get the replica dial to follow the movements you initiated on the control dial. This kind of problem sounds simple until you actually try to do it – Thats when you learn about dynamics, thats when engineering control and real world reveal themselves via experimentation. You take those tools that you learnt in Maths class and actually apply them to physically controlling something, its a real eye opener. Once you have been there its difficult to get the same satisfaction from an theory or algorithm. Doing stuff that has physical results is deeply satisfying and incredibly rewarding. Fast forward to today with the rise of making and hacking, I think this is why folks like to tinker and to hack, its simultaneously a creative, physical and educational exercise using 3 crucial parts of the brain. Adding opensource to the hardware equation engages a 4th part of the brain – social and community interaction, these in turn feedback,amplify and reinforce those creative juices.

In the earlier days of computing we tinkered and hacked a great deal both with the software and the hardware innards of computers. Eventually Apple and motorola, Intel and Microsoft came along and began their long journey of unification of computing singularity. As these companies pushed forward in their singular directions the other ideas and tinkering of computing fell away replaced instead by prescribed de-facto standardisation and convenient monoculture. I think we lost a great deal on that journey, the train was moving so fast that alternative concepts were simply passed like passenger-less stations on an express line. so keen were the drivers of these corporations to increase their revenue, that little thought was given to the question of whether it was the right direction or not, we were launched headlong into obscurity, conform or be dammed.

I think now however we are coming to the pinnacle of that ignorance, the train is slowing and the wheels are starting to rattle loose, the energy providing the locomotion is no longer sufficient to provide acceleration, rather we are now experiencing deceleration. But this isn’t a train wreck its more a darwinian, a fork in computing evolution that leads to a thousand dead ends, some of which will take years maybe decades to fizzle out. The latest greatest example of this is the iPad a minor fork of a struggling branch, one of the latter attempts to reinvent this evolutionary cul-de-sac.

At the same time the iPad reveals how this once firm evolutionary trunk is beginning to bifurcate, take at look at its innards to reveal ARM based processing from what was an embedded microprocessor IP company, very different from the end to end design and manufacture of Intel and motorola. yet the iPad is the ultimate in this same trunk of evolution, an inevitable device hatched to be used as an instrument of consumption. This is not a computer with which you hack or tinker, it is not a device for which you are expected to create, this it a consumption gateway to its vendor, the pinnacle of our computing evolution. This is what our best minds can come up with, its a hands off, design for single purpose, its actually a just very pretty cash register for its manufacturer Apple. Pretty soon the rest of the herd will follow in their footsteps placing the final nails in the coffin of this popular computing evolution.

Obviously I am little sad to see it decay in this manner, but I am not surprised, it was inevitable, driven by a selfish 20th century industrial strategy, the result of a economic foundation whose days are numbered. But surely I hear you say “it has been huge success, look a the proliferation of personal computers, look at how little they cost, look at the bang for buck you get from these prolific PCs.” It is true, if population is a measure of success, then they are very successful. However throughout its assent I have often wondered about those ideas that have fallen by the wayside, all of those missed opportunities, the many diverse possibilities that could have taken us down different roads. Maybe just maybe things may have turned out much more different even more successful and much more diverse. For example if we had taken a more concurrent approach to computing from both a hardware and software perspective rather than being blindly led by Intel’s clock race we might have ended up with a more evolutionary successful class of computing. Something much more robust than the current crop of high end PCs and servers, something more creative and enabling than the iPad.

But these are just whimsical what ifs, we cannot change the past, what is done is done. But we can invent our own future. This is a good time to reinvent a better more flexible computing, one not tied to clock races, corporate profits and monocultures, rather something more radical, computing that can effectively reinvent, reconfigure and replicate itself on the fly. Computing that is designed with real world concurrency, responsive,reactive and parallel like the real world, something not hampered by 20th century sequential thinking, a new computing of real and the physical rather than a virtual one. More importantly I want computing I can tinker and experiment with, something not confined to repetitive recipes, but one with infinitely combinational components with which I can assemble new ideas. Computing that connects to physical as well as virtual, computing which enables the internet of things, computing that enables me to make new things. Most of all I want creative, intelligent computing not stupid consumptive computing, I want 21st century computing for everybody and everything and all the things I cannot even envisage yet and I want it open completely open from the bottom up this time.

Who is with me..

So what I want to share today is the challenges of OpenSource Hardware Production (OSHP) and the work I have been researching in order to overcome some of these obstacles I (and many others) have experienced.

In order to understand the problems of OSH I will compare it to OpenSource Software (OSS) a more familiar OpenSource model. The biggest main difference is that OSH requires ‘Hardware’ or more importantly components and connectivity. Software has no such requirements. OpenSource Software production is merely the digits these can be produced and transported and duplicated at almost zero cost. Hardware on the other hand has very real physical costs :- components, PCBs, tools, assembly, transportation etc..

It is therefore no surprise to find that hot areas in OSH are ones in which these costs and complexities are reduced, a good example is the burgeoning Arduino OpenSource Hardware community. The way that Arduino helps mitigate some of the challenges is down to the nature of the device and its implementation.

First Arduino is a microcontroller based device, this is like a microprocessor (CPU) with built in peripheral pins to make connecting to simple actuators, indicators and measurement/detection devices straightforward using just a breadboard and some wires. Secondly it relies heavily on a familiar concept ‘software’ to make it useful. Although this software is much lower level and hardware device oriented it can benefit from OpenSource software models that we are already familiar with. Production and distribution are therefore relatively simple using the familiar OSS model, albeit with the proviso that you need to have an Arduino dongle. In fact in the OSS world we also need a dongle that dongle is called a PC, which is now so familiar and common place that we don’t actually see it as such. For the Arduino community you actually need 2 dongles a PC and an Arduino, in fact you might also need a few other bits to make it interesting such as a Breadboard and some components, a market for Arduino kits has thus emerged around the community to satisfy the need. Even the Arduino itself is available in many different formats from different supplies in addition to the ‘official’ founder organisation who started Arduino.

In trying to build OSH projects one comes across common issues with supply, production and assembly because of the way in which the embeded/device industry has grown up. This industry is based around around the head of the production curve, focused on producing popular items in extremely large quantities. For those playing in the tail of the production curve manufacture costs rise steeply as they lean on prototype scales rather than mass production. Having priced a number of projects we are working on I would commonly find that producing 100 unit vs a couple of units results in significant lower cost per unit. These costs are not just the raw components but the connectivity (PCB) and assembly. In some cases this can be alleviated by producing ones own PCBs and assembling at home. There are however some pitfalls to this approach. The industry has rapidly moved to Surface Mount production over the last decade and many components are only available in this format. Home surface mount PCB production and assembly is not for the faint hearted and requires special tools and processes. Thus home production at this time (apart from certain categories of through hole production) is out of reach for many.

So how can we overcome such obstacles for production? well lets take a leaf out of the Arduino book and take it a little further, abstract it and expand it’s community range. Even though we love our Arduinos we can’t do everything we would like to do project wise. It was initially designed as an educational device for teaching about embedded microcontrollers and as such took on certain constraints in its initial design. For example it is designed for experiment and prototype rather than production or deliver, secondly it is confined to a narrow 8 bit microcontroller range from a single vendor (Atmels ATmega subset). Often I needed to use 16 or 32 bit controllers from the Arm family provided by many competitive vendors. Thirdly Arduino’s expansion and customisation routes using shields (daughter boards) does not go far enough in solving the production and assembly challenges outlined above. It became clear to me whilst trying to get around these limitations the challenges could be resolved using a more abstract and general design by taking a birds eye view of the processes. below are some guideline requirements to solve some of the challenges experienced and grease the production wheels for OpenSource Hardware Production:

1. Software as a major component for projects which means microcontroller based. However this should not be confined to a single vendor or family.
2. Standards based on OpenSource implementations and documentation using low cost components.
3. Modularity, like the linux kernel, build modules that conform to the standard and that solve common requirements, let the creative process snap together modules to resolve bigger design challenges. Also leverage the cost benefits of volume production for popular modules.

Modularity is key to the abstraction and there are several important features which will amplify its usefulness:

1. Module composition will be constrained to certain physical dimensions and connectivity. These constraints are designed to optimise Both PCB/component construction costs and macro-assembly into standard physical compartments.
2. Module functionality should be designed such that the largest possible audience for their usage should be considered. This should mean design scratches the biggest itches at the lowest production cost where possible.
3. Avoid overlapping or duplicating of functionality, each module should be designed to co-exists with one or more other modules in order to solve problems via composition.

These requirements lead me to a number of projects moving forward, the first an most obvious would be that of an open hardware device bus. This will allow modules (small boards) to be assembled at low cost which slot into a simple hardware bus managed by a microcontroller. The bus itself is very simple and requires little additional component support in order to make building peripherals a snap, modules and the bus itself as well as controller implementations will be opensourced to provide reference standards.

Please let me know your thoughts, does this make sense, would you like to get involved, would you like more info, should we setup a wiki or user group to get it rolling? let me know.

Curiously I didn’t start of in computer science like many of my colleagues, I actually studied systems engineering which was a mixture of analogue + digital electronics, control systems and programming. Thus recently when I started construction around 3fficient I realised I would probably need to start wielding my soldering iron once more. I was actually quite keen to do so as had gotten re-interested both in industrial and embedded systems.

Whilst doing the 3fficient research and getting back into hardware I discovered some great work being performed around opensource hardware, imagine my joy – OpenSource and Hardware hacking how cool is that. So it is no suprise that I have dived deep into OpenSource hardware there are some great projects out there, here are couple of the things I’m hacking around with :-

• Arduino – This little Microcontroller board is big and getting bigger all of the time, it has spurred so many other projects its created it’s own eco system.
• BeagleBoard – Low powered ARM Cortex 8 board, This thing is the start of something big, SI’s OMAP series are definitely going places

There are many projects out there to many too enumerate here, but check out Opencircuits for more clues. My point is that there is a great deal of energy being contributed to open hardware and hacking hardware and I can see this having the same kind of influence in physical electronics as OpenSource software has in I.T. With this in mind I thought it might be a good idea to get together with others having similar interests, and helping those who may be new to hardware hacking. That is why I have set up the Open Hardware Hackers group (OHH). Its not just a Google group we actually plan on meeting regularly (in London initially) to get together to tinker/discuss/share etc..

The Point of OHH is to help each other, to collaborate on projects, to share ideas, help others enter into hardware hacking. As such we would also invite members to contribute energy towards building a set of tools to help new entrants get into the digital-playground as well as for our own mutual benefit. That way we can give back to the community as well as take adavantage of it.

So what are the OHH members up to well Thats a question I’m about to kick off on the group, but I already have some clues, they are hacking utility meters, wheather stations, virtual/physical computing, low powered servers and controlling household appliances as well as working on gadgets and tooling, its pretty diverse. So if your interested in electronics, hardware hacking or just tinkering join the group and maybe even meetup with us. We will be getting together to setup the physical meeting agendas at the ICA Cafe on Tueseday at 2pm, your welcome to come along and contribute at an early stage if your interested. If you can’t make it, leave any suggestion on the group and well make sure to cover it.

Anyhow that just a quick update to let you know about OHH, hopefully I might even post a little more now that the work has let up a little.

More recently I have been involved with the 3fficient projects (partly the reason for slower posts here!). This has taken me from my decades work in the PC and web software industry back into the hardware and embedded industries. It’s beeen a fascinating journey over the last few months and I see radical changes afoot as both of these markets collide along with mobile technologies.

It’s like the coming together of 3 great continents : PC desktop/server meets embedded meets mobile and mixed into the whole boiling pot is the magic ingredient of OpenSource, shearing off complete business models and opening a multitude on new opportunities. To be honest I haven’t been this excited since the late 90′s when I discovered linux and the internet, given the changes OpenSource and open standards brought to bear on those emerging markets, the scope for embedded and mobile space innovations are absolutely huge. At this early stage there are great deal of opportunities for change and invention, I’m not just talking about iPhones, I am talking about the wider picture, complete re-invetion of ubiquitous computing itself.

There are already key signs of things changing, a good example is the Netbook, I love it’s emergence and adore it’s underlying philosophy of ‘just enough’. But it isn’t just netbooks that emerge from this thinking as we will see ‘just enough’ changing many niches and even entire markets. Take servers for example, although many see these just getting bigger and more powerfull, at 3fficient we are building ‘just good enough’, adequate, sufficient and requisite and turning a few heads. In the enterprise we are also seeing OpenSource as being ‘good enough‘ and I don’t mean that in a bad way, good enough is really important and is a definite requirement for commodity and rapidly expanding value driven markets.

But there are many other great things that cross over from these different markets, embedded’s efficiency brings a welcome breeze of fresh air to an industry obese with it’s own wastefulness, yes I’m talking about the PC industry, it’s fat margins and greedy consumption of resources and power are a folly waiting to be exposed by an efficient army of adequates. The artificial walls erected by the greedy vendors are about to be kicked in by a band of decent,fair and capable new innovators and value driven green entrepreneurial anarchists. I’m not joking, these fat bastards are going to see their knees shattered by the crippling depression unfolding beneath them, they will not survive unscathed.

The Age of endless consumption’s collapses beneath us is revealing it’s true lack of value creation. But built upon the consumerist rubble from its collapse will come a new efficient age. Instead of consumption, innovation, dedication and collaboration will bring efficiency and value, business itself will be reborn to construct and deploy value back into the global eco system. Contrast this to the worlds resource strip mining of the 19th and 20th centuries, its about bloody time and I am glad to be instrumental in it, what about you lot what are you doing to help?

Anyhow it’s been a while since I blogged here and had a few things to get of my chest sorry and hope you don’t mind my occasional rants

My suspicion is that virtualization is really just a transitionary technology that gets us over the open & concurrency hump rearing up in the technology road ahead of us.

It’s not that this hump poses a problem to everyone, many developers are well ahead and have been riding it’s crest for some time, building concurrent and or multi-tenant solutions that run natively on the internet.

But organizations of all sorts have a great deal invested in pre-hump technologies, a transition to post-hump is going to take time. In the interim, cloud vendors, shower us with their virtualization rainbows to lead us to the open native internet pot of gold on the other side.

So what is at the end of the rainbow? Although its difficult to see (primarily due to the cloud marketing fog obscuring it) one would imagine it looks a lot like the internet does now only bigger and better. That is it will be built around open standards like TCP/IP, http, html, xml, javascript, ajax and perhaps some newer emergent ones. We should also expect to see REST rather than WS* hairball, view source rather than binaries and most likely OpenSource rather than proprietary. Whatever is there, it will be used to build the new native internet software and services world post-hump.

What is fascinating is what happens to the current diversity of native web production which emerged from CGI grew up through LAMP evolving into the popular state of the art frameworks like RAILs, DJango and cornucopia of similar solutions in other languages. We already see emergent post-hump candidates like Google’s App Engine and Microsoft’s Azure tools + platform, can these succeed or will diversity rein, perhaps running on more diverse plays like Amazon’s AWS or a Eucalyptus fork. I personally have a difficult time imagining any single one of these dominating, I would expect diversity to continue but with some consolidation. I also have an inkling that something else is likely to emerge soon that changes the game around concurrency, something that makes it central to its architecture. In many ways it will be an anti virtualization force, it will also likely be a highly efficient development environment and solution consuming less and delivering much more bucking the historical trend to date, it really would be development gold!

What do you think might emerge, have you seen the signs yet, anyone got some clues to throw us?

Microsoft have been talking about being more open and talking about things outside of their regular dev toolbox. The words I had heard were ‘PHP’ and ‘Eclipse’ amongst others. Not only that but for these two at least I had seen a great deal of confirmation from other sources, like Microsoft working with Zend and the Eclipse org. Thus I was expecting their compute on demand offerings to include at least PHP and possibly others outside of the visual studio regulars. It was a surprise to me that these outsiders were not mentioned in the keynote or subsequent analysis. You see unlike 5-10 years ago where Microsoft pretty much 0wn3d Developers, there are now many large parties of devs outside of their toolbox/language reach. It has therefore got to be a priority for Microsoft to some how leverage at least some of these out lying groups and to entice them into their new strategy. At the least I would expect Microsoft to play their familiar embrace and extend card, but there was no sign of it.

Luckily after following through on the commentary I came across a channel 9 video (silverlight less version) by Manuvir Das (Director in the Windows Azure team) that provided a much more in depth explanation of Azure. Not only does this video better explain Azure and it’s infrastructure but it also alludes to those missing pieces I expected to hear about. Just after halfway through the real elephant in the room is revealed, as the virtualisation is explained the presenter reveals (around 22 minutes in) that other languages do actually run within their new virtualised platform. Not only does he use the ‘PHP’ word and Python but he also drops the real anvil he mentions ‘RAILS’ ! not just Ruby but ‘Ruby on Rails’! That would mean they have gotten rails running on their virtual machine and will have ActiveRecord ported to their virtualised infinite storage – Bingo!! If this is true it could be big, it would mean Microsoft could attract PHP, Python and Rails devs into their cosy cloud play. It could be very attractive much like like Google’s AppEngine is for Pythonistas, infinite indestructible storage, instant scalability to as many systems you can shake a stick at, all in the releative safety of your favorite web language or DSL.

So what do you think about this huge cloud based hug to web developers, is this a master stroke from Microsoft, their worse kept secret, or are we being invited to put our heads inside the lions mouth ? Let me know your thoughts. Any links to further info or analysis about this would also be appreciated.

P.S. Yes I know this doesn’t mean it is a pure ‘Open’ play and I know that still hasn’t emerged from any player yet, but it is a still a very interesting story from a developers POV.

Here is a scenario, someone inside your organisation (Alice) has a great new idea for a widget, but doesn’t know how to implement the processes or systems to produce, package and deliver the widget. At the moment the widget is just an idea, not an actual product, so first Alice needs to talk to the early adopters support department (ASD) to refine the product idea. So who are the ASD folks? they are of course the elite task force members of the same department as I.T Alice’s customers. So Alice collaborates with ASD, establishes that it’s a go product and refines the widget requirements. She then hands over the widget reqs to the buying department (BD).

The members of BD actually have two roles, the first is to coordinate the just-in-time (JIT) manufacture and supply of widgets. Their second role is of course as the I.T department for a number of Widget manufacturers. Because Alice’s widget has never been made before a few of the BD folk are acting as the JIT supplier’s ASD task force. After collaborating with the JIT suppliers via your I.T systems a widget solution emerges to meet the costs and quality targets established by Alice’s ASD task force.

Alice is now happy to move forward with widget production and delivery, so she engages with the marketing and communications department (MCD). Who are the MCD? well the MCD are the same department as the I.T. department, her customers of course. What Alice needs to do is collaborate and engage with her customers in order to help them understand the new widget. She is hoping that they will like the new widget as much as the ASD folks. Of course the ASD folks are also members of the MCD themselves and collaborate with other members of the MCD by engaging in conversations around the new widget, assisted by Alice and here colleagues. But Alice has been here before and knows this is no time for sitting on your laurels, there is still much to do. So she listens to the MCD and tweeks the widgets to help them slip through the MCDs more quickly and in growing numbers, the widget becomes more and more refined and more and more slippy until one day Alice realises she has a hit. But she doesn’t go all Britney Spears about it, because she knows it wasn’t her achievement, she recognises she was just helping to Teflon coat the tubes, so she makes sure that all of the departments know how insanely great they were with this widget. Just as she is doing this one of her colleagues engaged with an MCD, notices they are doing something different with the widget and suddenly has this brilliant new idea….

Examples of customer I.T : Blogs, getsatisifaction, twitter, wiki’s, social networks, VRM and lots more coming…..

Second a little history is in order, last year as some of you are probably aware we built the first prototypes for Rel3. Rel3 was originally a framework that I was designing to build Folknology projects. Instead of just building a framework, I figured I should actually build something useful with the frame work concurrently. Doing things this way means you build a useful framework that does something concrete rather than just an endless series of components built without scope or constraint. Turns out this is a good way to build stuff because you end up scratching itches that exist. The downside is it seems to take forever, one of the reasons I called the project code name rel3 was because I had already built the thing twice so logically the next version would be release 3 AKA ‘Rel3’. We also gave a sneak preview of a rel3 app last year at wiki wednesday which some of you guys attended. In the interim we got some commercial interest from a couple of paying gigs and proceeded to build a prototype for one of them. That prototype based on rel3 is still in testing and has been going through several revisions.

In the meantime I was still not happy with rel3 the framework, it needed to be more robust, scalable and extensible. In addition I was getting requests for commercial projects that I wanted to build using the framework. Thus over the last few months I have completely rebuild the framework to meet it’s architectural goals and commercial requirements. The new framework is built using Erlang/OTP and I (with help from Phil) have named it Reactor, to be precise the project is actually called Reactored and the framework engine is called reactor. here is a potted history

1. 2005 Rel1 – Java/Xml based framework – To complex and unwieldy
2. 2006 Rel2 – Java + Groovy/Grails based framework – Simpler but messy
3. 2007 Rel2.5 – Ruby/Rail based framework – Failed, Rails got in the way
4. 2007 ReL3 – Erlang/ErlyWeb – Success but messy (wouldn’t dare show code in public!)
5. 2008 Reactor – Erlang/OTP – Everything we ever wanted and more, it just Rocks!

Reactor is the core of OTP components that do the heavy lifting. Reactored expands the scope to combine the Reactor engine with the following components in future:

1. Web server Yaws and ErlyWeb,
2. Messaging ejabbered and RabbitMQ
3. Non native CGI/FW Python 1st then maybe Php ,Ruby..
4. Whatever anyone wants and is willing to contribute

Some advantages we are already benefitting from:

1. Yaws is native in the framework language unlike Apache
2. Highly perform-ant, scalable, redundant and eventually reliable!
3. Component, modular and simpler to extend to requirements.

So what the hell is Reactor and what is Reactored for?
I will take the second first, Reactored will be the community that I hope we can build around Reactor and it is of course OpenSource (GPL’d). As for Reactor, well that will probably take a number of posts and examples to explain, but here is a brief description in the meantime.

Reactor is a new angle on building web based projects. It focuses on packets, patterns and events. It’s a reactive framework built around REST based cached/distributed storage, queues/sinks and domain based pattern matching.
Perhaps the best way of visualizing it is comparing it to existing domain patterns where we store and then post process using techniques like search and query. With a reactive storage we do the queries in realtime (as data enters) using pattern matching. The framework also allows for common post storage queries using search, tags and identity avoiding the need for a separate query language.

Reactor is not a web framework, but is something that can be integrated into an existing web framework. I will however be adding Yaws and ErlyWeb support as a matter of personal preference. Next likely stop is a python framework integration, but this largely depends on some help with the Python, that also goes for any other language frameworks such as PHP, Ruby etc.. You could think of it a new type of backend for frameworks, except if you did you would miss out how frontend it can be!

It goes without saying of course that if any of this is mildly interesting to you and you would like to find out more or contribute in some way please tweet/comment/email me and let me know, here are some indications of skills that could really help:

1. Erlang geeks – anyone looking to join an OpenSource erlang project, I don’t just mean experts if you already have Python/Javascript and are learning Erlang count yourself in.
2. Javascript,AJAX,Air geeks that want to play with interesting new frameworks
3. Testers, documentors, evangelists, bloggers and general participants if your interested your in!

In terms of code and project status, Reactor will be functionally complete at version 0.1 on Sunday June 1 2008 (probably midnight!), if you are impatient,just request it and I can send a tarball. For everyone else if you could wait a few days/weeks whilst I get the code out of my private GIT repository and into a public repository (GitHub/GoogleCode?). I will also be setting up the project website/blog/tracking stuff as and when I can over the next few months using the reserved reactored.org domain.

I guess you can consider this the unofficial announcement of Reactored!

* Update It is now available from the public Git repository at Reactored on GitHub now