The Sources Mouth

Querying the future

2011-04-05T00:00:00+01:00

Querying the future

CLOSED: 2011-04-06 Wed 01:05

Since learning Clojure I've long felt there was a connection between lazy sequences and event streams. After all, streams and lazy-sequences are both sources of un-realised values. Well, unsurprisingly it turns out that many people have had these thoughts too; and have spent significantly more time than me in rationalising their differences.

One such framework built on the formal rationalisation of these differences is a .Net framework called Rx, or the Reactive eXtensions. It's fascinating on many levels from its strong foundations as the mathematical dual of Enumerables, to its foundations in Functional Programming and Monadic Types.

The primary difference between either a lazy-sequence and a stream is that sequences pull their values; for example when you walk a database result-set you are continually asking the database driver for the next value for your query; the thread of control here lies with you. With stream processing however, the opposite is true; the control lies elsewhere and data gets pushed to you, with the arrival of new data typically calling a handler function or callback.

"There is a becoming of continuity, not a continuity of becoming." – Alfred North Whitehead

The primitives of the Rx world are Events, objects or values occuring at discrete points on a timeline or stream. In and of themselves neither the events nor the stream have a measure, i.e. events are atomic and occur as blips (with no length) on a stream with no concept of distance between events. The only notion of time at this level is the sequence of one event 'happening before' another.

This is not to say that a measured notion of time cannot be modelled, through either storing timestamp metadata inside the events themselves or composing a stream of clock tick events paired with events. Indeed you can get Rx to emit events onto a stream at an interval or even throttle it by sampling at specified rate; however the point still stands, the stream sees only the order of events.

In Rx the unit of composition is the stream, not the event and it is through composing event streams with functional operators, or combinators that Rx builds a compositional query language across disparate events from any number of sources.

This declarative language for querying event streams, hide the complex imperative details concerned with managing state and concurrency, and leave you to solve your problems at a level significantly closer to the problems of your domain.

Once you have a query language for events such as Rx, the similarities and differences between databases and event streams become even clearer. With databases you typically query the past, in a store of data you already have; where as with event composition you query the future!

Event Stream Processing is nothing new, but as a side-effect of the multi-core revolution, we're seeing a move out of the tarpit towards functional languages and the adoption of functional features in imperative languages. As a result of this we can hope to see more API's fashioned in the form of declarative languages that help us better manage time, order, and asynchronicity.

Personally, I'm looking forward to playing with Clojure's adoption of these ideas in the forms of Lamina and Cljque; which hope to complement Clojure's already comprehensive time management features.

From Fallow Fields, a new harvest

2010-02-09T00:00:00+00:00

From Fallow Fields, a new harvest

After a long hiatus, I've decided to return and restart my blogging activities. It's hard to be sure of my commitment, but my hope is that after leaving this pasture fallow for two years, it'll now yield a plentiful crop of posts.

Those who visited my blog before might be surprised to find that Epwna my blogs trusty steed has been put out to grass. Due to a change of blogging engine and theme, he can now be found munching on printer paper in the paddock. Perhaps he will one day return.

So whats changed to make me start blogging again? In short a change of blogging engine from Blorg, which had started failing to properly export my blog.

So for those who care, my revamped blog is still written in Emacs and the ever powerful Org-mode. This time using a combination of Jekyll, the excellent ruby static blog generator and secret sauce behind github pages, and the new org-jekyll, that acts as the glue between org-mode markup, and Jekyll.

So what's so cool about this combination? Well firstly it means I can blog like a hacker, keeping my posts backed up and managed inside git. And secondly it means I can write my posts in org-mode's familiar and powerful syntax. The benefits of using org-mode stretch way beyond those of traditional lightweight markup languages, as it'll let me merge my blogging activities with my existing note taking and planning workflows, whilst giving me access to powerful features such as org-mode's headline folding, its agenda, its Emacs based syntax highlighting, and org-babel's ability to execute and output the results of embedded code snippets directly into my blog!

Here's a small snippet of this blogs org-mode file:

* From Fallow Fields, a new harvest
   :PROPERTIES:
   :on:   [2010-02-06 Sat]
   :END:

After a long hiatus, I've decided to return and restart my blogging
activities.  It's hard to be sure of my commitment, but my hope is
that after leaving this pasture fallow for two years, it'll now yield
a plentiful crop of posts.

...
* Forget the Linux Desktop, it's the Linux Laptop that matters!
  :PROPERTIES:
  :on: <2007-11-27 Tue 22:25>
  :END:
       
The best innovations tend to be cheap and disruptive.  Hand in hand as
they're usually found, these characteristics go some way to explaining
why I like the EeePC (Asus's new laptop) so much.  The other reasons
are obvious, it's small, it's light, it has WiFi, Firefox and
 Open Office, and judging by the reactions of those who saw Paul and I
with them at Bar Camp Leeds, it's cool enough for everyone to want
one!

As you can see, org-mode can even export its own markup with htmlized syntax highlighting!

This new combination, is a huge improvement, and has many features that I hope to blog about in a future post. The first is that as it's built on top of Org-mode's excellent publishing and export features, which mean that I get easily publish code with full syntax highlighting.

For example below is the Emacs-lisp I'm using to publish this blog, isn't it pretty?

(require 'org-publish)
(require 'org-jekyll)

(setq sourcesmouth-org "~/git/sourcesmouth")
(setq sourcesmouth-org-publish (concat sourcesmouth-org "/publish/"))

(add-to-list 'org-publish-project-alist
             `("sourcesmouth-org"
               :base-directory ,sourcesmouth-org
               :recursive t
               :base-extension "org"
               :publishing-directory ,sourcesmouth-org-publish
               :publishing-function org-publish-org-to-html
               :section-numbers nil
               :headline-levels 4
               :table-of-contents nil
               :auto-index nil
               :inline-images t
               :todo-keywords t
               :auto-preamble nil
               :body-only t
               :auto-postamble nil))

(add-to-list 'org-publish-project-alist
             `("sourcesmouth-img"
               :base-directory ,sourcesmouth-org
               :recursive t
               :exclude "^publish"
               :base-extension "jpg\\|gif\\|png"
               :publishing-directory ,sourcesmouth-org-publish
               :publishing-function org-publish-attachment))

(add-to-list 'org-publish-project-alist
             '("sourcesmouth" :components ("sourcesmouth-org"
                                           "sourcesmouth-img")))

Cooler still is that with org-babel these embedded code snippets, can be made executable, capturing their output, and outputting their results directly into the blog. E.g. the following Emacs lisp reduce statement:

(reduce '+ '(1 2 3 4 5 6 7 8 9 10))

Yields the following result, directly into this document.

No other blogging engine I've seen can do anything like this!

So now I have a blog running again, what am I going to blog about? Well, I guess you'll have to stay tuned to find out.

Forget the Linux Desktop, it's the Linux Laptop that matters!

2007-11-27T00:00:00+00:00

Forget the Linux Desktop, it's the Linux Laptop that matters!

The best innovations tend to be cheap and disruptive. Hand in hand as they're usually found, these characteristics go some way to explaining why I like the EeePC (Asus's new laptop) so much. The other reasons are obvious, it's small, it's light, it has WiFi, Firefox and Open Office, and judging by the reactions of those who saw Paul and I with them at Bar Camp Leeds, it's cool enough for everyone to want one!

Laptops have (until now) tended to follow an unwritten law of small gadgets; the smaller it is the more you'll pay. However the Eee seems to mark a reversal of the trend. We're now seeing the law of mobility combined with commodity hardware, meaning one of the smallest laptops available is also the cheapest (and most valuable in terms of it's utility).

The real disruption however; isn't in how it's defining a new market for cheap, super small, capable laptops, but it's in how it nicely packages free and open alternatives to the two biggest monopolies in the technology industry, Microsoft Windows and Office. This strategy accompanied by their recent announcement to release an SDK and support the community should only serve to foster a strong community and potential competitor.

As hardware costs have fallen Microsoft customers have been paying ever greater percentages of the total device cost to Microsoft. Though it seems with the release of the EEE PC we have reached the threshhold where manufacturers are beginning to produce devices so cheap that the cost of Windows is by far the most expensive part of the device.

Hence ASUS have persued the most cost effective and profitable solution, to tailor an existing Linux distribution to the device constraints (7" screen, 4GB SSD disk) whilst targeting the device at the mass market of people wanting cheap, portable, internet access.

Despite being careful not to divorce themselves of Windows completely by including Windows drivers for the hardware, Asus's strategy of unbundling Windows and shipping the device with Linux clearly caused a panic in Redmond. The result, a substantial discount on Windows XP for Eee users, but what about Office software? (Similarly the OLPC program has led to Microsoft discounting software to $3 in developing nations whilst allegations of dirty tricks surround both Microsoft and Intel and their attempts to sink Negroponte's project).

Even with the Eee discount persuading some users not to leave Windows behind, I expect the Eeepc and other devices of a similar form factor and cost to significantly further free software adoption particularly on the laptop.

Desktop computing is a complex landscape with a diversity of hardware, user requirements and expectations. Consequently users are locked into the Windows mindset as well as their software, making anything else appear foreign and unintuitive. Despite this, Desktop Linux is viable for most types of PC user, though there are many barriers to overcome.

In the super portable Laptop market however, user expectations are different. They want something small and simple to browse the web, check their email and perform basic office tasks. Further to this the windowing metaphor begins to breakdown as users are required to run most of their applications full screen. These constraints immediately put the Linux O/S with Asus's customisations at an advantage, as the interface has been tailored to suit the devices form factor and usage scenarios.

It's a classic Christensen disruptive technology which will soon cause both new market and low-end disruption. This has been on the cards for some time and led to the development of Windows XP Starter Edition and Windows Fundamentals. However this strategy of developing cut down Windows distributions can only go on so long before the game is no longer worth the candle.

Indeed, recent comments from Microsoft VP Will Poole indicate an engineering struggle in trying to port Windows onto the OLPC's XO, and it's by no means clear whether they'll manage to get it to work on the device. Yet this is seldom a problem for Linux which supports more architectures than any other O/S.

Ultimately however it's clear that Microsoft will long remain a prominent player in the industry, however some speculate they may copy Apple and build their platform ontop of another O/S. Given their 2004 $2bn peace treaty with Sun, Solaris might be a candidate. Either way, what's important is that we're beginning to see some much needed competition and innovation in this space.

Update 2008-01-20 Sun

After following the discussion that this post generated on the web, I thought I'd post some follow up points reiterating what others have said and addressing some common criticisms.

Predictably a variety of commenters felt that the EEE PC is nothing remarkable. A cheap laptop, with cheap hardware, and not the sort of thing a "real user" wants. However, it's my belief that these users typically already have their needs fulfilled by other devices and hardware. If they have the money they'll purchase their expensive "better" hardware, however what they should realise is that the higher-end (£550+) PC will become the niche market; and that UMPC's are not laptops or desktop replacements. Sadly for Microsoft this direction indicates that Vista might become a niche O/S rather than the dominant main stream O/S they intended.

A common comment was that installing your own software on Linux is a problem. This is certainly a common issue, and perhaps best left addressed in a post of its own. However, here the EEE PC wisely choses to install enough to please 99% of people, knowing fullwell that most people never install their own software. This follows a trend away from the general purpose computer, towards tailored network appliances, of which the EEE PC is just one example.

Several other readers mentioned how Linux is acting as a similar force in the mobile space with it powering mobile platforms on the likes of Symbian and Windows Mobile. Highlighting this disruptive force is a valid point, but it's unlikely to promote Linux to anyone except the software developer.

Asus EEE Hacks seemed to agree with my general assertion, adding that the EEE PC has "…succeeded in becoming a disruptive technology without using new technology - practically all of its parts are off-the-shelf components". This is certainly true, and indeed fairly typical of the long nose of innovation.

My favourite comments of all were however the ones taking a pop at my web design skills. And included: "Web 1.0 FTW!", and perhaps unsurprisingly, even an OMG!!! Ponies!!! discussion.

Freedom at the end of the (ZX) Spectrum

2007-10-13T00:00:00+01:00

Freedom at the end of the (ZX) Spectrum

CLOSED: 2007-10-13 Sat 22:31

It probably won't surprise anyone who actually knows me that the earliest memory I vividly remember is of my first encounter with a computer. Waking up late, well past my bed time at the tender age of 4 or 5, sneaking down stairs to find my dad crouching over a small unassuming Sinclair ZX Spectrum (48k) playing Ground Force Zero. A simple game involving only the timely pushes of a single [b]omb key. The glow of the TV screen in the dark, sitting on my dad's knee and staying up late to bomb a city and land our plane.

Though he probably didn't know it at the time, it's clear to me now that the purchase of that £129 box of tricks was the wisest investment in my education my dad ever made. Over the next 7 years I would tinker with BASIC programming, read Your Sinclair, and develop a keen interest in computers.

This education was partly made possible because the Spectrum and other computers of the era were essentially built for a hobbyist market, consequently shipping with comprehensive documentation, technical manuals and a BASIC programming environment. This meant that the machine itself was not just a platform for consuming games and other Software, but an environment for learning and experimentation. A refuge for a child tired of getting shouted at for dismantling (or breaking?) toys out of a desire to unlock the secrets of their mechanisms.

Years later at about the age of 15 I obtained a copy of FreeBSD and later Slackware Linux on some Tucows CDROMS which I'd installed mostly out of curiosity. Being able to install a Unix system on your own commodity PC which just a few years earlier could only be run on incredibly expensive servers had an instant appeal. At first it felt a little like exploring a Pharaoh's tomb, decyphering the hieroglyphics ls, pwd, grep, root, vi, su, sed, awk, sh, /etc… and then learning that there is a deep and fascinating history behind it.

Shortly after though other things began to appeal; a powerful commandline, a single rooted filesystem, having a multi-user system (when Windows could barely multi-task), the technical reasons it appealed were countless, yet somehow became dwarfed by ideas far greater and more noble than the Unix philosophy.

Buried away in a previously unexplored directory somewhere under usr/share/emacs/etc I'd discovered a small series of text files that explained everything. The ideas of freedom, community, collaboration and open knowledge outlined in the GNU Manifesto and enshrined in legalese by the GNU Public License instantly struck home. I not only had in my hands a free Unix like operating system, but also the blueprints to build it and the rights to change it. I knew then that Richard Stallman the Free Software Foundation and thousands of developers had sewn enough of the right seeds to change the software industry forever.

1998 saw the end of my A-Levels; the start of my studies at the University of Dundee and the formal beginings of the Open Source Movement; a faction who believed largely in the pragmatics of free software but rejected the political stance associated with it.

My thesis which I hope to discuss further in future posts, is that Free/Libre Open Source Software (FLOSS) is the most important innovation born in computing since the silicon chip. Its applicability is global and its methods and principles are deserving of adoption elsewhere in a diversity of fields.

I hope to unravel these ideas in future posts particularly in relation to:

The future evolution of the Internet
Education and the importance of being open
Innovation
Software Design
Freedom & Control
Business

First Post!

2007-09-06T00:00:00+01:00

First Post!

CLOSED: 2007-09-06 Thu 20:26

Over the past few years I've persuaded various people to get blogging; my friend Dave writes on education and the suffering in Lebanon, a small group of Malawian clinics write about health care in Malawi, whilst my bosses Paul and Chris write about technology and the work we do in multi-agent systems and intelligent context oriented communication at Calico Jack.

During this time I've installed more Wordpress instances than I care to remember, yet only now after too many opinionated discussions with the very much alive, "late Mr B", have I decided to try and structure my thoughts on software and the tech industry here.

Having had a long held, keen interest in the free and open source software movements I settled upon the name The Sources Mouth. For those who care about such things, I've chosen to start writing this blog with GNU Emacs, org-mode and the static blogging engine blorg. We'll see how I get on.