metajack.im

Servo Update: Upgrading Rust, GPU Rendering, and Automation

2013-04-12T16:40:00Z

I've been working on Servo for three weeks now. There's an enormous amount of work to do, and I want to capture what's going on and how it's progressing. This should be the first of many such updates on the project.

Day One

When I arrived, Servo no longer built at all, at least not on OS X. Servo often requires bleeding edge versions of Rust, and backwards incompatible changes to Rust are still happening on a regular basis. Since all of the contributors to Rust work on different platforms, when porting to a new Rust compiler, some platforms have gotten left behind. This was particularly acute this time because Rust 0.6 contained a lot of syntax changes, mostly things that got removed from the language, and many pieces of Servo were using syntax that was deprecated in Rust 0.5, and was finally deleted entirely in Rust 0.6.

Upgrading to Rust 0.6

Rust 0.6 removed a lot of keywords and syntax from the language. Porting Servo required modifying all the constants, many function declarations, many import statements, etc. These changes were largely mechanical. There were a few changes that weren't so easy.

Mutable fields are being removed from the language, and mutability will be controlled by the mutability of the struct itself. Not all of these had to be removed in Servo, but many of them did, and removing them often required slightly changing the data structures and their type signatures. In some cases this was trivial, but in a few cases these changes needed more care. In particular, lots of these changes bumped up against the Rust borrow checker, which ensures it's safe to hand out pointers to memory. There are still some bugs in the borrow check, and workarounds are not always straightforward.

It took me about a week and a half to work my way through all the dependent libraries and Servo itself at which point I had a build. By the end of that second week I had landed the language upgrade to servo as well as some Rust library changes that were needed. The end result is that Servo is now using Rust 0.6 syntax, but it requires a post-0.6 version of Rust due to the Rust changes not landing quite in time for the 0.6 release.

GPU Rendering

Servo uses many forms of parallelism, but one bit of low hanging fruit is to move to a fully GPU rendering path. Currently compositing is done on the GPU, but rendering to the various layers is done on the CPU. This is how most current browsers operate as well.

We're moving to rendering on the GPU as well which should speed up some things a bit. Instead of rendering in parallel to several layers, Servo will render directly into textures on the GPU which the compositor can use without doing CPU to GPU memory transfers.

This required upgrading the rendering stack to a newer version of Azure (Mozilla's drawing library) and a new version of Skia (the specific backend that Azure uses on OS X, Linux, and Android). Now that this part is done, we'll be adding texture layers to the renderer and switching drawing to those.

Automation

We're setting up build and testing automation for Servo now, which should help ensure Servo remains buildable on all platforms. Rust has an amazing set of tools for this already, which we are hoping to reuse fully. Buildbot machines run builds and tests, and a GitHub bot called Bors handles dispatching builds for patches that have been reviewed and merging pull requests that have passed tests.

For now this work will be on Linux, but we hope to expand it to cover OS X and Android as well in the near future. Once Servo is a little farther along, we plan to put up nightly snapshots so more people can follow along with our progress.

Other Work

There's tons of other work in progress on both Servo and Rust. The DOM bindings are getting improved, a new Rust scheduler that will make performance and I/O better is in progress, a more optimized C FFI in Rust should also land soon, and the rustpkg package manager is shaping up which we'll be switching to for more and more of Servo as it matures.

We need more help in lots of areas. Please join us in IRC in #servo or on the mailing list. We'll be trying to mark bugs and projects that are well suited for new contributors. If you want to work on Servo and write Rust code all the time, we're hiring.

Joining Mozilla

2013-03-22T10:52:00Z

On Monday, I join Mozilla to work on Servo, a new and experimental web browser engine built on Rust, a new systems programming language. I am perhaps the first professional Rust programmer.

This will also be the first time in over a decade that I'm not working in a small company or a startup (usually both). I've been thinking for a while that it would be nice to work for a company that has real resources to solve problems, as opposed to being at the mercy of venture capitalists or the whims of users. Mozilla's mission statement is one that is easy for me to get behind, and they are doing very interesting things at Mozilla Research.

I enjoy working on difficult and important projects, and it's hard for me to imagine much that is more difficult or important than web browsers. It's an added bonus to be working in and (hopefully) contributing to a new programming language. I also love working with smart people, and Mozilla seems to have those in abundance.

This is going to be awesome.

Digital Audio and Sampling Explained

2013-02-26T08:23:00Z

Xiph.org has just posted the second in its series of videos on digital media concepts and techniques. It's packed with information and demonstrations, and you're sure to learn a huge amount. As an added bonus, it's hosted by Monty, the creator of Ogg Vorbis (and many other amazing things). You couldn't ask for a more qualified teacher.

Watch below, or on Xiph.org.

There is also a detailed write up.

XEP Lookups on DuckDuckGo

2012-07-09T14:42:00Z

The search engine DuckDuckGo recently added the ability for developers to develop instant answer plugins for their search engine. This project is called DuckDuckHack.

Instant answers on DuckDuckGo are really nice, in that they highlight a specific result in a context sensitive way. For example, Stack Overflow questions that match will show up with the highest rated answer at the top of the page, and Wikipedia articles will be presented as a title an abstract.

I decided to play with DuckDuckHack and added a plugin for XMPP Extension Proposal (XEP) lookups. I do XEP lookups often when I'm answering people's XMPP-related questions, and this plugin makes the XEPs title and abstract appear as an instant answer. The plugin was recently merged into the tree, and it is now live on DuckDuckGo itself.

Try it by searching for XEP 45. View the code on GitHub.

The Numbers Behind the Twitter Data Silo

2012-01-30T20:47:00Z

The dark future of search is being foreshadowed by this Twitter vs. Google fight. The latest Twitter volley at Google is this quote (seen on GigaOm) from Twitter CEO Dick Costolo:

"Google crawls us at a rate of 1300 hits per second... They've indexed 3 billion of our pages," Costolo said. "They have all the data they need."

There's no doubt that 1,300 hits per second is a large number, but let's put that in perspective:

In February 2010, Twitter was at 50 million tweets per day. This is just under 600 tweets per second.
In June 2011, Twitter was at 200 million tweets per day. This is over 2,300 per second.
In October 2011, Twitter hit 250 million tweets per day or just under 3,000 per second.
They have spikes of over 7,000 tweets per second, with the largest (so far) being just over 25,000 tweets per second.

For part of 2010, Google was perhaps able to keep up with the stream at 1,300 requests per second. Somewhere between February and June, the average volume of tweets outpaced them.

Let's assume that they kept pace until June 2011, and that on June 1, Twitter went from somewhere in the range of 1,300 tweets per second to their reported 2,300 tweets per second. Google is 1,000 tweets behind per second.

By the end of the year, Google missed 15.5 billion tweets. They are two months behind if they didn't skip any, and the tweet volume did not increase. But it did increase by 25% or so by October, and surely it has grown more since then.

If Google has only indexed 3 billion pages so far, they have approximately 12 days of tweets at current volume. It's pretty hard to rationalize the 3 billion pages number against the 1,300 per second number. Was Google indexing at a much slower rate before? Did they not start until a few months ago?

Of course Google may be getting multiple tweets per request, perhaps by crawling the timelines of important users. But this means that they probably get a lot of requests that don't give them any new tweets, or else the timeliness of the data is poor.

No matter how you slice it, it appears Google would be unable to keep up. Even if they were keeping up now, Twitter's growth probably sets a time limit for which keeping up remains possible.

Perhaps Google is super clever, and can index only the right tweets. I think that it's more probable they have "enough" data to surface results for the super popular topics, and miss nearly everything in the long tail of the distribution. I expect that this adversely affects search quality, which one suspects is a high priority for the world's best search engine.

Google is no saint. They are just as guilty of the same data hoarding. If you ran these numbers for YouTube indexing, I think you will find the situation is much worse. I imagine that most of these data silo companies purposefully set their crawl rates too low for anyone to achieve high quality search results.

In the case of Twitter, the end result for users is even worse because Twitter's own attempts at search are terrible and are getting worse over time. At least Google makes a decent YouTube search, even if no one else can.

Even if Google could get all the tweets, they still would have very little to no Facebook data. I still think the best strategy in this situation for them is to create their own social data and use that instead. It's a tough road, but they seem to have little choice.

In the end, it's not about Google or Twitter or Facebook, but the stifling of innovation and competition around data. We can only hope that some federated solution or some data-liberal company wins out in the end.

The More Things Change: A Review of The Soul of a New Machine

2012-01-20T10:20:00Z

Already in my career I've experienced enormous passion, burnout, extraordinary dedication to my team and projects, and depression. I'm sure many others have as well. Has it always been this way with technology? I often wonder if this rollercoaster is necessary, healthy, or normal.

I recently saw a recommendation for Soul of a New Machine, which tells the story of a team of engineers at Data General who built a new 32-bit computer in the late 1970s. The book is fascinating. Thirty year later, many of its descriptions of the project and the way the team worked and was treated could apply to any modern project.

The plot summary will no doubt sound familiar to you: A team of mostly young, mostly male engineers works grueling hours to build something amazing in too short an amount of time. They succeed, albeit a bit over their original schedule. Despite the project's commercial success, the team is denied both recognition and financial rewards and many end up leaving the company. Almost all of them ultimately enjoyed it and would (and did) do it again.

There were many pieces of this story that resonated with me.

Work is a Drug

On overworking Tom West, the manager of the team in the book, says:

That's the bear trap, the greatest vice. Your job. You can justify just about any behavior with it. Maybe that's why you do it, so you don't have to deal with all those other problems.

Why deal with the unpredictable world, when the controllable world of creation is available? It's code as escapist drug, and I love to get high on it. Mundane things like cleaning my house, and more serious ones like taking care of my health, are all easy to avoid while fixing bugs or starting a new project.

It's both possible and important to find a balance.

The team's secretary, who was much more than her title suggests, suffered and succeeded with the rest of the team. Even she says:

I would do it again. I would be very grateful to do it again. I think I would take a cut in pay to do it again.

Even as I recover from projects that burned me out, I am constantly thinking about how to do new ones. In fact, while I'm doing any project, I'm already thinking about doing another. This sounds like drugs again. But they are good drugs.

Harassment and Treatment of Women

The book describes how some team members tormented the lone female engineer. This is something that still happens today, and it's terrible. And people then wonder why there are so few women in our industry.

In addition to that, at the end when they hand out the peer awards, their award to the woman was for putting up with them, not for any of her actual accomplishments.

Betty Shanahan was that lone woman, and it looks to me that she deserved more than just an award for thick skin. She's the CEO of the Society of Women Engineers, and she was "a member of the design team for the first parallel processing minicomputer and manager of hardware design for subsequent systems." She later moved to the business side of technology, and I wonder if that had anything to do with her having to put up with the Eagle team's harassment.

How Something is Done is Important Too

Often we judge things by their properties, but one can also rightly judge something by how it is made. Shoes made from child labor are less good than those made in other ways.

Kidder, the book's author, discusses this:

In The Nature of the Gothic John Ruskin decries the tendency of the industrial age to fragment work into tasks so trivial that they are fit to be performed ony by the equivalent of slave labor. Writing in the nineteeth century, Ruskin was one of the first, with Marx, to have raised this now-familiar complaint. In the Gothic cathedrals of Europe, Ruskin believed, you can see the glorious fruits of free labor given freely. What is usually meant by the term craftsmanship is the production of things of high quality; Ruskin makes the crucial point that a thing may also be judged according to the conditions under which it was built.

By this kind of measure, is the work many teams do good? Is the Eagle computer that Tom West's team built really a success since the team worked much overtime, suffered divorces and other problems, and in the end received little to no reward?

I think it's time for entrepreneurs and workers in our industry to demand better. Our outputs will be better if they are made sustainably, and not just by the measure above. In retrospect, maybe the reviewers of LA Noire should have taken into the account the trials of its developers; it certainly would not have fared well.

Freedom of Expression

I want to hire resourceful people. I want to describe a general outline of a design and not have to describe it in intricate detail in order for them to build it.

It turns out that this is critical for happiness. If we're told exactly how to do something, it takes much of the creativity and fun out of the work.

Engineers are supposed to stand among the privileged members of industrial enterprises, but several studies suggest that a fairly large percentage of engineers in America are not content with their jobs. Among the reasons cited are the nature of the jobs themselves and the restrictive way sin which they are managed. Among the terms used to describe their malaise are *declining technical challenge; misutilization; limited freedom of action; tight control of working conditions*.

You must trust those you work with to be resourceful. If you don't trust them, you will end up micromanaging them into unhappiness, and you will also remove their valuable creative input from your product.

There is a balance to be struck with feedback. The Eagle engineers thought that the managers didn't appreciate their efforts, but in reality, some of this was them trying to stay out of the way. Kidder asked the Tom West's boss:

Had the Eagle project always interested him or had it grown in importance gradually?

"From the start it was a very important project."

Was he pleased with the work of the Eclipse group?

"Absolutely!" His voice falls. "They did a hell of a job."

But some members of the team felt that they had been rather neglected by the company.

"That doesn't surprise me," he says. "That's frequently the case. There's often a conflict in people's minds. How much direction do they want?"

I've had this same issue with investors as well. You don't want them to meddle with your company or your product, but you also want their advice and guidance. It's possible to go too far in either direction, but mostly you hear about stories where investors meddle too much. I personally think it's probably better to err on the side of too little help than to end up with too much meddling.

The Venture Capitalists

Even thirty years ago, the VCs had a bad rap. Tom West was asked in a Wired article years after the book's publishing why he stayed at Data General until he retired:

"You could do new products and companies within the company, rather than shag some venture capitalist and kill yourself for five years." To be an entrepreneur, he says, "you have to be interested in networking, even with fools."

This is another reason why I would prefer to bootstrap companies if at all possible.

Tom West ended up working on many interesting projects at Data General, but ultimately, none of them got the support or recognition they deserved. The other members of the Eagle team spread out and started or worked for new companies, and in general seemed much happier.

Final Thoughts

In the end, it's both a fascinating tale of heroism and creativity and a saddening tale of undervalued and underpaid engineers. I am both emboldened to keep following my passions and more mindful of its dangers. My troubles are not unique - not even modern. Thirty years after this book was written, I feel like it could have been written yesterday.

The Potentially Dark Future of Search

2012-01-12T10:12:00Z

Twitter sees Google's latest Google+ feature, integration into Google search, as anti-competitive, and it probably is. However, it brings to the surface some real issues with the future of search and of data.

Twitter's argument:

We're concerned that as a result of Google's changes, finding this information will be much harder for everyone. We think that's bad for people, publishers, news organizations and Twitter users.

Google's response was:

We are a bit surprised by Twitter's comments about Search plus Your World, because they chose not to renew their agreement with us last summer (http://goo.gl/chKwi), and since then we have observed their rel=nofollow instructions.

People have been digging into the semantics of nofollow (see Danny Sullivan and Luigi Montanez), but there is a much bigger issue.

Google and other established and up-and-coming search engines have no real way to include lots of data in their index. It's easy to imagine that the lack of access to Twitter and Facebook data was a motivator for Google+ in the first place.

Lots of sites now generate enough data that it is unrealistic to crawl them. For example, Youtube has more new content every day than they allow anyone to crawl. Twitter is essentially the same. This means there is no way to index this data without special arrangements with the provider. Twitter has closely guarded their firehose of data, but at least they have some mechanism to obtain it. Youtube, as far as I am aware, has no such mechanism.

My team and I ran into this problem head on trying to build Collecta, a real-time search engine. Access to the data was a primary blocker for many features and product ideas, and over the too short life of that company, access became significantly more difficult, not easier.

Google can build an effective search, even a real-time one, for Youtube, but no one else can. Twitter can build search for their data, but few others can, and their data access policies can and do change on a whim.

If Google believes that microblogging data will improve their search product, then a reasonable strategy to obtain that data is to try and build their own microblogging service to generate it. I can't fault Google for trying. If I thought Collecta could have effectively competed against Twitter for their audience, I would certainly have attempted that as well.

Google, Twitter, Facebook and others are hoarding silos of otherwise public data. Not only is this artificially limiting the features of their products, but it squashes the potential for new and exciting search applications. The search services that have sprung up are limited to your own data, aggregate results from service-specific search APIs, exist at the mercy of data providers, or make do with a tiny subset of the data. I don't think Google could have built their own search engine if the Web were similarly hostile.

One could argue for requiring these bits of data to be openly available, but unlike the data of the past, this data is expensive to publish and consume. Most of these services may not even have a mechanism to publish the data, even internally. Simply receiving the Youtube or Twitter firehoses (and not counting video or image media) would require significant engineering effort, and the rate of data generation is only accelerating.

I think we must push for open access to data, even if it is costly. These data wars benefit very few. If things don't change, the future of search is dark.

Strophe.js 1.0.2 Released

2011-06-19T10:26:00Z

I've just tagged and released Strophe.js 1.0.2. You can find it on the new Strophe.js site.

Please consider upgrading as soon as possible, as a security problem was found in Strophe.js 1.0.1. The DIGEST-MD5 SASL method used a constant client nonce due to a bug in Strophe's use of the underlying MD5 library. I don't know of any exploits for this bug, but it could compromise your site's security.

Much of the credit for this release goes to the many contributions and pull requests that people have sent in the last year. The community's effort continues to make Strophe.js better and better.

XMPP Case Study Interview: Drakontas

2011-03-29T11:07:00Z

XMPP is everywhere. Did you have any idea that your local SWAT team might be using XMPP for tactical coordination, or that the same technology might be used to allow friends and family to track and share location with each other? It always amazes me where XMPP will turn up, but Ilya and his team at Drakontas have used the technology to great effect with their two products, Smyle and DragonForce.

I often meet people doing amazing things with XMPP technologies, and through these case study interviews, I want to highlight the great work that these people and companies are doing. This is the second case study interview (see the first one on Cabulous), and I really hope you are enjoying these as much as I am.

Ilya Braude is our guest for this interview, and he is the lead software engineer for Drakontas, working on both their Smyle and DragonForce products.

Drakontas

Jack: What is Smyle?

Ilya: Smyle is a group-based social collaboration application that allows you and your friends to interact with each other to plan or coordinate around specific events.

It does this by allowing group members to chat, share location, and pictures. All pictures and maps are also instantly turned into shared whiteboard sessions, allowing annotations and icon placement.

Jack: Can you give some examples of how people are starting to use Smyle?

Ilya: Smyle in being used in various ways. First, it's being used as a real-time location tracking service. People are using it to track each other as they are arriving at the same destination, without having to call or text each other to get status updates.

Then, they can use the whiteboarding features to enhance that experience. For example, I might draw a path for you to take on the map, to show you exactly where I am and how to get to me and use icons to mark destinations and places of interest.

The chat feature is also heavily utilized, probably because people are accustomed to it. However, the other collaborative features were designed to minimize overt chat and allow users to collaborate naturally.

Jack: You currently have a version for the Web and for Android phones. Are other implementations planned?

Ilya: Yes, we are constantly being asked if an iPhone version is available, and we are working on it. There's no planned release date yet, but it's definitely coming.

Jack: It seems like the friends, chat, and collaboration are all great places for XMPP. How much of the system is powered by XMPP?

Ilya: Because the application must be real-time and collaborative, XMPP is a natural choice that has worked very well for us. Almost all of the communication is over XMPP, with the exception of out-of-band image downloads, which use HTTP.

In fact our protocols are based on the PubSub specification. We have implemented custom components on the server side which masquerade as PubSub services, but perform business logic operations on the payloads.

The whole XMPP ecosystem has been great for this: everything from libraries for the browser (we use Strophe), to the excellent work that has been done with XMPP and Twisted, to stable XMPP servers that are at the core of the messaging bus.

Jack: It's great that you were able to map most of the functionality onto well known XMPP extensions. Did all of the functionality map this way, or did you use some custom extensions as well?

Ilya: We have been trying to stick to existing specifications as much as possible for several reasons. Client (and server) development is easier if we reuse existing library support for PubSub and Data Forms, and it would be easier for third party integration if we decide to open an API in the future. So we have been able to use existing specifications in creative ways. For example, a whiteboard is a PubSub node, where each item in the node is an annotation or icon. We use Data Forms for structured data and configuration. The customization comes from how clients and servers interpret these existing protocols.

Jack: Did you find that using existing protocols, even if library support wasn't available, made you more productive?

Ilya: A lot of smart people have spent time designing the protocols, so it's great to be able to rely on well-written and thorough specifications. We don't have library support on all platforms. In many cases, we've had to implement our own, and have contributed the code back to the open source community. This allowed us to concentrate on what was important for our application rather than the lower-level protocols. We did face some challenges with this approach and have had to come up with interesting ways to model information. However, in the end, I believe reusing existing work was the right approach.

Jack: You've given some great reasons for choosing XMPP. Were there other technologies that you evaluated as well?

Ilya: The first consideration was, of course, to implement our own. And having done so in a previous application, we decided that it wasn't the best choice. We've also briefly looked at other messaging platforms such as JMS, but it was quickly obvious that they weren't a good fit. XMPP was simple enough to understand, and develop with, especially because we could see wire traffic and compare it with existing working implementations. So it was a natural choice.

I should also note that XMPP servers offer a lot out of the box, especially Multi-User Chat and PubSub. Our initial implementations used built-in server implementations, which we replaced with custom components as our requirements and feature set grew.

Jack: Was this your first time using XMPP?

Ilya: Prior to this application and another application that we have also built with XMPP, I've had no experience with it at all. Needless to say, there was a lot of time spent reading specs!

Jack: Did it take long to get up to speed with XMPP? Which parts gave you the most trouble?

Ilya: One of the first things that I did with XMPP was implement a J2ME client from the ground up. That didn't take too much time. One of the initial challenges was determining what server features were available and how we could use them correctly. And also we've had to figure out what our server of choice actually implemented and any quirks it had.

Jack: When I first met you, you showed me another project that your company created. It seems like something you'd see on TV shows like 24. Can you tell me a little bit about DragonForce?

Ilya: DragonForce is an application for law enforcement, first responders, and tactical teams. It allows for rich real-time information exchange between the units in the field and command centers. We've worked a lot with SWAT teams, and routinely participate in tactical exercises with them. The state of the art in communications is usually a voice radio. DragonForce allows teams and individuals to track each other on a map, send text messages, and share annotatable pictures, floor plans, and maps.

There are numerous advantages to this type of system. For example, sending a picture of a suspect augmented with a text description immediately to everyone in the field is far better than sending a description over the radio. The guys in the field can take pictures using their mobile devices and send them to the commander, providing valuable visibility into a situation.

We have a mobile application that runs on rugged handhelds, and a command center application that is a web-based application. In fact, Smyle is our adaptation of that technology to the consumer space.

Jack: All the same features from Smyle are there as well right? Real-time location updates on a map, whiteboarding on floor plans, etc?

Ilya: Yes, all of these features are in DragonForce as well. Floorplans turn into whiteboards so that, for example, as rooms are cleared in a building they can be marked off silently and quickly without radio chatter.

Jack: Will you be using XMPP in future projects? Do you recommend it to others?

Ilya: I would absolutely recommend it, and we'll keep building collaboration software using XMPP. Besides the technology, there's a great and active community that has helped us a lot, and I'm sure will help others as well.

Technical Details

Smyle and DragonForce are both custom XMPP clients. The websites both use the Strophe.js library for XMPP communication over BOSH. The mobile clients use native connections.

Ilya has made many of his own plugins for Strophe.js as well, including ones for Data Forms, PubSub, Multi-user Chat, Service Discovery, and roster management.

The XMPP server used for DragonForce is Openfire, but the Drakontas team have switched to Prosody for Smyle.

They have written a number of custom components in Python using Wokkel and Idavoll. Having the components being external to the server makes them portable across the two different servers.

My Thoughts

I met Ilya at the XMPP Summit in Brussels in 2010. He completely floored me with his demo of DragonForce, and he followed up later in the summit with some great ideas in his Strophe.js plugins. He and his team have done some amazing things, both with web technology in general and with XMPP specifically.

I can hardly believe that the same technology platform they've built powers both such a niche use case and a consumer application, but once you use the products, it makes a lot of sense.

You can try out Smyle yourself right now on the web or with their Android client.

If you have some questions for Ilya, please leave a comment. I'll make sure he sees them.

Introducing Snack Words

2011-03-28T09:47:00Z

The earliest things I remember about learning to write code are all related to trying to create games. Whether it was learning basic linear algebra to do 3D rotations or the basics of language parsing to make text adventures, most of my early learning was a quest to implement something silly. Over 20 years later, I've finally shipped something silly.

Snack Words is a simple word game. You are given six tiles, and you must make as many words from those letters as possible before time runs out. The twist is that you aren't playing alone; you are playing head-to-head against another person with the same tiles. Each game is short and fun, like a snack.

I've been working on Snack Words in my spare time over the last several months. I haven't yet written about it here, but I left Collecta in early October and have been working on a number of side projects since then. My wife and I did all the game design in a few weeks of lunch conversations and then I wrote the code between other projects. A small group of friends and family helped beta test everything along the way. Now I'm playing a game I created with people from all over the world.

The inspiration for the game came from a couple of places. My wife and I spent some time addicted to similar games, although we always wished for various features that were missing. When we decided to create Snack Words, we already had a healthy list of ideas. Another source came from my colleagues at Chesspark. We all wanted to make a simplified version of the game that focused on fast games where you didn't spend a lot of time matchmaking.

Please give it a try, and let me know what you think. I learned a lot of great stuff along the way, which I'll be writing about here. For example, I recently gave a talk at Erlang Factory where I showed how Erlang made the game server a simple endeavor; in fact, it's just slightly over 2,400 lines of code, including white space and comments. You might also be surprised to learn that I did not use XMPP for the protocol, and I have a post planned to explain that decision as well.

Learn Ejabberd With Me At Erlang Factory

2011-01-24T16:58:00Z

I'll be teaching a three day course on ejabberd and XMPP applications at the next Erlang Factory in San Francisco. I'll also be speaking at the conference about how Erlang gives me superpowers. The course will be March 21st through March 23rd, and the Erlang Factory conference will be on the 24th and 25th.

I'll be covering everything from XMPP basics and installing ejabberd to writing your own extensions, web services, and clients. See the full course description for more information.

You can register for Erlang Factory, including the course, or get more information about the event at Erlang Factory SF 2011.

I highly recommend this conference if you are interested at all in Erlang. There are some amazing talks each year, and the conference is always well run (this will be my third year attending).

I hope to see you there!

Strophe.js Community Plugins Project

2010-11-29T15:12:00Z

The community has been doing great work on Strophe.js plugins, but for the most part, this work has remained invisible. I want to change that.

I have created the strophejs-plugins repository on GitHub and moved everything but the flXHR plugin into this new repository. My plan is for this to be the clearing house for community plugins.

If you want to add a new plugin or contribute to an existing one, please fork this repository and send me a pull request. I will not be nearly as picky about commits to strophejs-plugins as I am with the main Strophe.js code, but I expect as plugins mature, a higher bar will be created for those specific plugins.

Within a few hours, the first community plugin was added to the repository, the serverdate plugin, which detects clock skew between the client and server. I know there are some excellent plugins out there, so don't be shy - fork and hack!

Bite-sized XMPP News #4

2010-02-22T00:30:00Z

Here's a collection of interesting XMPP-related updates from the microblogging world this week:

malditogeek: http://github.com/malditogeek/Pocho takes the stage! #weekendhack #xmpp #redis #ruby #

Pocho looks to be a neat little XMPP bot that stores hashtagged messages to XMPP group chat rooms. It also exposes these stored messages via a web interface as well. It sounds like an interesting bot!

kocio: #Jingle Nodes on Talkr.IM - simpler #XMPP #VoIP for the people http://www.process-one.net/en/blogs/article/jinglenodeson_talkr.im/ #osnews #

Thiago talked about Jingle Nodes at the XMPP summit, and they look interesting. I unfortunately missed his talk at the summit, but from what I understand, it brings Skype like peer-to-peer relaying to XMPP Jingle.

It will be interesting to see this develop further. Jingle is really starting to become part of the normal XMPP landscape. For example, at the summit, the N900s that Nokia kindly let us borrow all had support for Jingle calls.

tofu: 'BOSH warming' ejabberd module. cool stuff! !xmpp http://github.com/theoooo/modwarmbindings #

This is another implementation of the HTTP pre-binding technique. See mod_prebind and ModHttpPre_Bind for two other example implementations.

There is not yet a standard for this technique, so each of these implementations does it differently. I think it's time to write a XEP.

stpeter: it's somewhat amusing that both the SIP-Communicator client and onsip PBX support XMPP these days... #

Amusing, but not surprising. After all, Cisco is a huge VoIP company, and they themselves purchased Jabber.com. The writing has been on the wall for a while for SIMPLE.

arc: #jabiru is a horribly broken !xmpp client for !android, is this really our only option? #

stevegibson: @arc: a quick google found the JabbIM !xmpp client for !android. #

thaluser: I found a GPLv2 !xmpp client for !android that works with #facebook called Yaxim. Unfortunately, it's not in the official market. #

If you're looking for an XMPP client for your favorite Android device, these three might be worth checking out.

danbri: experimenting with QR Codes as TV Test Cards as !XMPP service discovery hack - http://www.flickr.com/photos/danbri/4359291206/ #

danbri: Scanning our TV with 3g iphone; it reads !xmpp jid from qrcode at 3 paces http://www.flickr.com/photos/danbri/4360377558/in/photostream/ #

I met Dan Brickley at the summit, and he is doing some really interesting things with remote controlling media experiences that incorporates XMPP. I highly recommend paying attention to what he's doing as I suspect you will be seeing more great stuff.

tdfischer: Dear tokbox: Thanks for using XMPP. You make reverse engineering easy. <3, tdfischer #

Thankfully, since XMPP is an open standard, reverse engineering is unnecessary. That's the whole point!

mathemonkey: Wow! @elance uses #xmpp http-bind for "workroom" collaboration service… but it's not accessible using ordinary XMPP client or GTalk :-( #

I wonder if this is because they use a custom authentication mechanism or if it's just a case of not knowing the credentials to provide. In any case, it's great to see XMPP in more places. I'm getting a little tired of seeing so many duplicate implementations of chat services.

ssethi: XMPP and One Social Web http://j.mp/cRykDk #

The One Social Web presentation at the summit was one of the most talked about, but unfortunately I showed up only at the end. I expect we'll see some good things from this team.

h4nnes: Any recommendations for a xmpp client for iphone? im+ keeps crashing here :/ #

I use OneTeam for the iPhone, which has pretty good support for group chat as well as normal IM. The recently launched Meebo client also supports XMPP, but it does not support group chat.

vmlemon: Just configured XMPP-based FaceBook Chat in Pidgin, and pleased to see that it works as advertised. Now if only it supported federation... #

I think Facebook will get there. Server to server communication is one of the trickiest parts of XMPP, and it makes since that they would save this part for last. It took Google a while to get federation working as well.

dbanes: Anyone implementing XEP-0277: Microblogging over XMPP ? #

I just heard about such an implementation from a comment on my blog. The site is called BuddyMob, and apparently is an extended implemenation of XEP-0277. Hopefully the extensions will find their way into the specification as well.

jprieur: Archipel, xmpp based orchestrator for virtualization http://www.archipelproject.org/ #

This looks like an interesting start to virtualization infrastructure management.

drunknbass: I'm thinking you could totally power a game like words with friends with XMPP (openfire) #

I think that would be a great weekend project, and XMPP would be a perfect transport. With Strophe.js, HTML5 Canvas, and some JavaScript, this game could be made fairly easily.

xmppjingle: @pedrosorren Do you know Microsoft released an XMPP Gateway for its OCS closed services, due high customer demand? #theworldisopen #

One day, maybe we'll see MSN and AIM get XMPP gateways just like Facebook. Sometimes it takes a decade or so for a project to fulfill its original goals.

xmppjingle: Facebook announced that their are building Mobile APIs, why not just expose its services over XMPP? The Chat works extremely well so far #

I think it makes a lot of sense to have a native XMPP interface to these APIs. For example, Gmail has shown that even though its mail interface doesn't map cleanly to IMAP, there is enormous value in leveraging IMAP clients. Sure, it takes a little thought and planning, but the end result is well worth it.

rektide: what do #buzz, #xmpp, and to a degree #wave have in common? failure to abide the most basic/essential axiomatics of the #web; the uri. #

URIs are great and are used in XMPP where they make sense. I think most people would agree that e-mail style addresses are easier to use for humans than most URIs, though. XMPP is not a solution to all problems, and neither is HTTP.

That said, I've seen several efforts now to bridge some of the gap between HTTP and XMPP - sending stanzas through REST APIs, exposing chat rooms as web pages, etc.

gashcrumb: just a little bit closer to having a proper mock XMPP server to run unit tests against. nice! #

I hope this will be open sourced. There is unfortunately little in the way of specific XMPP testing tools. This is often brought up at the XMPP summits as something most of us want and need, but so far it has proved more complicated that it looked.

XMPP 101: A Video From the XMPP Summit

2010-02-16T21:04:00Z

The 8th XMPP Summit has come and gone, and it was a blast. I tried to record all the excellent talks from the FOSDEM XMPP dev room, but I only managed to record the very first one. Watch below as Remko gives a good overview of all things XMPP.

This video requires a browser that supports HTML5 video in Theora format. Support open, royalty-free multimedia codecs!

If you'd like to learn more about XMPP from Remko, he and some others wrote a book as well.

I'll bring some better equipment to the next XMPP Summit so that more of these videos can be published.

asmack: XMPP Library for Android Developers

2010-02-10T23:30:00Z

Mobile XMPP development came up over and over at the recent XMPP Summit. With several Java libraries to choose from for desktop XMPP development, one would imagine that these would work easily on Android, which is also Java based. Unfortunately, for a variety of reasons, the currently Java libraries don't seem to work. Thankfully, the BuddyCloud team has released asmack, which is a fork of the SMACK library customized for the Android platform.

If you're thinking of getting into mobile XMPP development on Android, you should definitely check this out.