Bocoup Blog

Open Peer and the Frontiers of WebRTC

Mike Pennisi — Thu, 2 May 2013 12:26:14 +0000

At Bocoup, we love working on the latest problems in front-end development such as parallel processing, best practices in web application development, and media-rich experiences. This is why we consider the folks at HookFlash kindred spirits; they have made browser-based peer-to-peer communication their “thing”. We recently had the privilege of working with the HookFlash team, and (as can only be expected when collaborating with passionate, talented individuals) we learned quite a bit.

For Those Late to the Party

So much of today’s web is about communicating with one’s peers, the term “peer-to-peer” (or “P2P” for short) can seem like old news. When you write a Direct Message to someone on Twitter, isn’t the text delivered from one peer to another? Really though, the term is less about who is communicating and more about how they are communicating. A true “peer-to-peer” channel does not rely on any sort of specialized central service (i.e. twitter.com in the previous example); all the messages travel directly from the sender to the recipient.* This communication model is highly sought-after because it enhances privacy, security, robustness, and accessibility far beyond what is possible within “mediated” channels.

WebRTC is a developing standard to facilitate streaming data transfer between web browsers. It defines a JavaScript API for generating the information that clients need to communicate in real-time (this is called the Session Description Protocol, or SDP for short). Creating such rich communication channels is no small feat (check out the architecture reference to get an idea of the complexity), but it isn’t the end of the story.

This Space Intentionally Left Blank

If we take the extraordinary service rendered by the WebRTC API for granted (which we do only temporarily), we can more clearly see its limitations. Sure, peers are able to connect with each other directly, but how do they negotiate connections? What if the peer does not want to immediately accept a call? What if she is away from her device? How do the peers end the call? These questions fall under the purview of “signalling”, and the WebRTC specification makes no attempt to address them.

This is no oversight, however. Defining a fully-featured signalling protocol is a challenge unto itself, but it is tangential to the technical challenges that the WebRTC specification set out to address. As it stands, it has taken the standards body years to define a solution for streaming data transfer (and it still isn’t complete). By intentionally limiting the scope of the specification, the WebRTC working group is able to arrive at consensus far more quickly.

WebRTC defines all the functionality that developers around the world need to begin experimenting with their own protocol implementations. Any website can use WebRTC to implement fully-featured peer-to-peer applications (and indeed, many already have).

Unfortunately, without a standard, each of these web applications is a silo unto itself–there can be no communication across web sites. Thanks to WebRTC, web developers have all the tools they need to prototype open standards for signalling over WebRTC.

Enter Open Peer

Open Peer is Hookflash’s open-source signalling protocol. (Its open nature is important because it means anyone is able to create compatible applications for any platform, free of charge.) First and foremost, it addresses the needs described above. But since peer-to-peer communication has so much more potential than simple actions like “hang up” and “reject call”, Open Peer implements far more powerful abstractions.

For one, the concept of “identity” has been built into the protocol. This enables users to find each other by some persistent, human-readable name (think “Mike Pennisi” instead of 2001:0db8:8ba3:4043:1020:812e:b3d0:7aa4). But instead of relying on some centralized authority to provide this name, identities in Open Peer are federated (think e-mail addresses instead of Twitter handles). It also recognizes the fact that people already have identities with “legacy” systems (e.g. phone numbers and e-mail addresses) and provides hooks for them.

Support for identities necessarily requires the addition of some infrastructure into the protocol. If you want to be able to find your friend by name, some service must exist that knows how to locate him. After you’re connected, true peer-to-peer communication can begin, but the reliance on third-party services remains something of a necessary evil for the niceties of identities.

Open Peer attempts to mitigate the security and privacy risks by making the infrastructure as distributed and stateless as possible. The servers have no sensitive data, and they are completely interchangeable. This de-incentivizes the servers as a target for hackers and severely limits the abilities of malicious administrators.

By making a distinction between the concepts of identity and location, Open Peer enables a particularly interesting use case. Consider the following hypothetical situation:

You’ve just won a contest! The prize? A ten-minute phone call with your favorite performer, Englebert Humperdinck! Tonight, he will be calling you online using the Open Peer protocol. Now, the Dinck is a popular guy–not anyone can just call him at will. Despite this, when you receive a call at 6:00PM, you can be 100% positive that the caller is, in fact, the legendary British hearthrob. Once your ten minutes is up, Englebert ends the call, and you are left with a sense of awe, but (and this is the important part) no means to call him back. There’s no *-6-9 by default. Of course, if you guys had really hit it off, there would be nothing to prevent the crooner from sharing his contact details, but this is completely optional.

Users are only able to locate a given peer if they know their “name” and their “find secret”. By allowing peer-to-peer communication without the exchange of this information, Open Peer enables scenarios like the one described above. You’ll find more about this concept in the documentation of the Public Peer File.

The Future of P2P

While we find it easy to get excited about the promise of the Open Peer protocol, it’s important to remember that there is still a lot of work to be done, and nothing is set in stone.

WebRTC remains unfinished. The specification itself is still being written; you can get involved through the WebRTC working group’s public mailing list and the getUserMedia API mailing list. The WebRTC Google Group and webrtc.is offer coverage on the latest developments. And don’t forget that both Firefox and Chromium are open-source, so you can get in on the implementation as well. You can find a guide for contributing to Firefox here and a guide for contributing to Chromium here.

Open Peer is under heavy development as a collection of open-source projects. You can find SDK’s for a number of environments on GitHub: iOS, BlackBerry, C++, and the browser (Android support is in the works). Protocol nuts will no doubt be interested in reviewing the full specification.

Of course, the space for signalling protocols is wide open. Peer.js is another project that attempts to address signalling concerns for WebRTC. Though if you’re particularly opinionated, you might choose to propose a brand new open standard of your own design. If this is the case, all we ask is that you try to learn from the current proposals and those that have come before (e.g. Jingle, SIP, and Chord).

From a developer’s perspective, this may be the best aspect of the open web: if you’re passionate about the problem, opportunity abounds.

* - Sure, there will always be machines like routers and switches between the peers, but these devices have no privileged information and are completely interchangeable.

Welcome Kacie Cabral

Jory Burson — Tue, 30 Apr 2013 11:30:00 +0000

The next time you stop by the Bocoup Loft, there will be a new face greeting you at the door - Kacie Cabral. As Bocoup’s newest team member and Office Assistant, Kacie will be helping keep the Bocoup Loft in top-form for meetups and community events in addition to organizing travel and inventory for her fellow Bocoupers.

Most recently, Kacie has been attending college at Bristol Community College where she is studying Computer Information Systems. When the time comes, Kacie plans to join Bocoup’s engineering team and build apps for the Open Web. Kacie is a very fast learner - there’s no doubt that time will come sooner rather than later!

At Bocoup, we realize great programmers come from all manner of educational backgrounds, and we’re thrilled to be part of Kacie’s engineering growth. Join us in welcoming her to the team!

The Dimensions of Web Design

Greg Smith — Mon, 22 Apr 2013 12:00:00 +0000

The 5-dimensional Web

Today’s web has at least five dimensions:

Width
Height
Time - Site content is constantly changing and being added.
Hardware - Content needs to be usable on a variety of devices.
Software - Different users have different browser capabilities and settings.

Good websites embrace this complexity as a competitive advantage. Bad websites try to do things the old way and it doesn’t work.

The best part is that this mindset actually makes it easier to design websites. Taking advantage of the web’s best qualities means cutting with the grain*. Going with the flow. If you’re fighting to put a square peg in a round hole, you’re probably making your user experience worse anyway. Learn to let go and do things the web way.

Welcome to the future. The web is 5-dimensional. Here is your hypercube. Now let’s get to work.

*I mean like wood, not steak. Cut against the grain of steak so it’s more tender.

Content is God-Emperor

Let’s start with a goal. Every website has a different purpose, but it can always boil down to providing content. Even web applications like Gmail provide content: the added complexity is that you can interact with and create content as well.

Since all websites present content, all websites should do it well. Content should be easy to find and easy to understand. Content is King. In fact, Content is God-Emperor of Dune.

#1 goal of web design: Make it easy for people to find and understand your content.

Design is not just “make it look pretty”. Pretty things are good because they work better, but that’s just one way design can help you meet your goals. Good web design can make information easier to understand. It can make content easier to find. It can make it clear what you’re expected to do. It can inspire trust.

There is a lot that goes into designing good websites. The Elements of User Experience is a great book to read if you want to start at the beginning.

The Width and Height Dimensions

Web sites that stretch to fit different browser sizes have been around forever. We’ve called them fluid or liquid layouts for over 10 years. It was always a good idea. The primary thing that has changed is the explosion of mobile. Now fluid layouts are more appealing than ever.

In that case, what’s actually new here? Responsive Design is the new buzzword, but media queries are the new tech, since they let you serve CSS specific to the situation. Besides that, most of the layout is still being cobbled together using floats, percent-width columns, max-width images, and so on.

Media queries let you do a lot of things. But to be honest, the most important part is checking the viewport width. For example, you can use media queries to show a different number of columns for different screen sizes. It’s actually super easy. Well, it’s easy as long as your design process is willing to accommodate it. Think about this from the start: it’s much easier to design a layout where the number of columns can change than it is to take a static layout and figure out how to compress it onto a mobile phone. This is one reason why designing your mobile experience first is such a good idea.

The Time Dimension

Content changes. Even if you’re making a static site for a coffee shop, it’s likely their hours will change, or they’ll start serving breakfast, or they’ll get sued and have to change the “McMuffin” to the “Independently-operated Non-franchise Muffin”.

Content tends to expand. The web has the convention of vertically-scrolling sites for this reason. There is infinite room to expand vertically when new content is added, just as the site can stretch horizontally when the browser is resized.

It’s not just about adding new posts to a blog index, though. Your navigation was carefully designed with the whole of your content in mind, I assume? Every time your content changes, you need to make sure your navigation still makes sense. If you already have “Cocktails” and you’re adding “Beers”, maybe you should consolidate both into “Drinks”. Don’t just add new things wherever there’s space or eventually you’ll get a cluttered mess.

Part of this is doing the right thing when your content changes. Another part is making a design that can easily accommodate change. For example, lets say you have a menu that is 10em wide. You have to add a new menu item, but the text is longer than 10em. How easy is it to make this work? Do you have to change the CSS in 20 places to make the columns different sizes? Do you end up making the font size smaller to make it fit, despite the fact that the font size was already perfect? A good design is ready to evolve. A stylesheet language like Less can help you avoid redundancy, which makes the design easier to maintain.

The Hardware Dimension

We’re used to varying screen resolutions, but displays are increasingly variable in even more ways. With different pixel densities, aspect ratios, and physical sizes, it has become impractical to design for specific configurations. It’s more important to focus on adaptability in general. Make sure the font sizes are big enough on small screens. Use the viewport meta tag. Consider scalable graphics with icon fonts and SVG: these will not pixelate on high density displays, such as Apple’s Retina Display.

Touch input is a major consideration for mobile, and even some non-mobile displays. It’s important to use large enough touch targets: Microsoft suggests at least 38x38, Apple suggests 44x44. You don’t have to think of this as chore that’s just for mobile users: big targets are nice when using a mouse too. In the end, making sure your UI elements are properly sized will be helpful everywhere.

The Software Dimension

Different browsers, especially different versions of browsers, have different features. The nice thing, though, is that most of these new features were designed to not break the page if the browser doesn’t support them, or if the user turned them off. Progressive enhancement is the strategy of starting with the basic content and functionality of a page that works anywhere, then layering on enhanced features when supported. This works for fancy HTML5 application features like Geolocation and WebSockets, but it also works for design. CSS3 contains a lot of cool new properties for fancy styles, but you should always make sure your site works without them.

Modernizr is great for writing CSS that is aware of browser support. It adds classes to the html element that explain what features are and are not natively supported. You can use these classes in your CSS to write styles specific to different situations.

The ultimate test of Progressive Enhancement is to view your site with a screen reader, or at least with JavaScript and CSS disabled. If it’s still useable, you’re probably on the right track.

Things that get in the way

By learning to let go, web design can be easier than ever. It’s very Zen, don’t you think? Sure, we’re dealing with more variables than ever, but the technology and tools have improved so much! Let’s look at some things that just get in the way and waste your time.

It must work on old browsers!

Supporting old browsers can make a project take several times as long as it would otherwise. It’s easy to say “We have to support IE7, no matter what!” It’s hard to say “We have to support IE7, even if it takes 8 extra weeks of development time!” Be realistic about what browser support does to your schedule and costs. I’m not saying you should never bother supporting old browsers. I’m saying it’s a tradeoff that requires careful consideration.

It must look the same on all platforms!

You’ll never get your site to look the same on all platforms. And that’s fine. Remember goal #1: make it easy for people to find and understand your content. Users don’t care if your site looks slightly different in someone else’s browser. If you’re worried about consistency, make sure you can tie that worry to a real objective. How much does it matter? Is it worth the additional development cost?

Look at platform inconsistency as an opportunity to look and behave in a familiar way. Some input controls look different on different browsers. This reflects the user’s choice to use a certain browser over another, so you should embrace it. In OSX, using touch input, you can scroll the site in a way that shows a gray background “behind” the site. I’ve actually seen people try to prevent this in their web designs. This is part of how the user’s operating system works, it’s what they’re used to, so you should go with the flow.

Pixel Perfection

Pixel perfection doesn’t matter, and is harder than ever. On a 1024x768 display there are 786,432 pixels. On every possible display between 240x320 (HTC Tattoo) and 2880x1800 (Apple 15” MacBook Pro) there are 6,455,828,464,800. There are too many pixels. If that menu item is 2 pixels to the left in Firefox, will users have a harder time finding it? If you just focus on the content, you may find that some little tweaks just aren’t that important.

Magic Ratios

Did you know the Google homepage is perfect because it uses the golden ratio to achieve ultimate Feng shui? Actually I just made that up. Relying on magical ratios and proportions is superstition. Your web design is a way to display your content, and your content probably doesn’t adhere to a mystical formula. Maybe the number 1.61899 is particularly beautiful, but don’t prioritize it over your content. If a column needs to be wider to fit the content, just do it.

Conclusion

Design for the web is tricky because there are a lot of variables, and a lot of organizations just aren’t used to the new way of doing things yet. We’re all evolving though, because the web is evolving, and if we keep up, we’re rewarded with an incredible new way to access any information and do anything, anywhere. Even in the fifth dimension.

Random Number Generation in JavaScript

Adam Hyland — Wed, 17 Apr 2013 11:00:00 +0000

Paul Irish recently gave a talk at Bocoup with a dire warning: “The Mobile Web is in Trouble.” The general theme of the talk, somewhat apparent from the title, revolved around developers moving to native apps for performance, capabilities and discoverability. Near the end of the talk he offered a challenge–developers working on the Open Web should focus on expanding the capabilities of our apps and the Open Web in general.

One of the core preserves of native applications is numerical simulation and modeling. Numerical simulation is commonly used to model and predict the behavior of systems which cannot be modeled analytically or which have a significant random component. Let’s get the core objection out of the way: it’s crazy to perform numerical simulation in JavaScript. There are a half-dozen languages with full featured simulation and modeling capabilities built into the language or well maintained libraries¹.

But let’s be crazy. Over the next few weeks I’ll be covering numerical simulation techniques in JavaScript, starting today with an overview of random number generation, then generating normally distributed random variables (along with other distributions) and finally adding in a robust resampling method to the Miso Project’s Dataset library.

What’s the matter with Math.random()?

JavaScript already has a facility to generate uniformly distributed random variables: Math.random(). It seems to cover all our needs, but let’s look at what ES5 has to say about the expectations for output:

Returns a Number value with positive sign, greater than or equal to 0 but less than 1, chosen randomly or pseudo randomly with approximately uniform distribution over that range, using an implementation-dependent algorithm or strategy. This function takes no arguments.

Let’s not (yet) get hung up on the distinction between random and pseudo-random². The real problematic area is “using an implementation-dependent algorithm or strategy.” Application designers can rely on Math.random() to provide a black-box of entropy. In that regard it’s pretty good. All of the modern browsers with open source engines use fairly decent random number generators and they’re all pretty fast. Why isn’t that enough?

Pseudo-random number generators output a long, non-recurring sequence of numbers. The exact value of that sequence is determined by an initial starting value referred to as the “seed”. For any given seed the sequence is apparently random but known and recoverable so long as we know the position and the seed. Each implementation of Math.random() requires a seed and most seed from the current time or a source of OS randomness (if available). As the spec indicates, Math.random() does not accept a seed value from the user and even if it could, the actual generator varies across browsers; Chrome, Firefox, Safari each use different random number generators³.

Chrome(v8): Uses a version of the Multiply-with-carry generator.
Firefox (SpiderMonkey): Uses a linear feedback shift generator similar to Java’s. There was discussion to move to the Mersenne Twister but the decision was made to wait until the Crypto API was implemented and simply use that.
Safari (JavaScriptCore): Used the Mersenne Twister until recently, then switched to a newer generator described here.

We can imagine two broad uses for random numbers where an implementation like Math.random() would be insufficient: simulation and procedurally generated content. Both of these require a generator which is both apparently random and perfectly repeatable.

A trustworthy simulation is one where the exact output can be reproduced deterministically. Scientists increasingly depend upon numerical simulation to prove theorems, test systems and inform policy decisions. A seedable random number generator is crucial to reproducibility of those results. But we don’t need to be simulating collisions in the Large Hadron Collider in order to want a means to reproduce our results! If we wrote a physics simulation and wanted users to be able to share their results, it would be physically impossible to do so using Math.random().

Most random number generation in games is perfectly suited to Math.random(). In general games want a fast source of entropy and don’t need client-side cryptographic strength RNGs. If we want to compute ricochets in a first person shooter, we care a lot about speed and apparent randomness–less so about the period of the generator or our ability to recover the exact numbers generated.

However if we’re working with procedurally generated content Math.random() puts us in a bind. We can generate content from the entropy provided but we can’t repeat or share the seed. If we wanted to build a side scroller where players explore procedurally generated landscape and let people play in the same landscape asynchronously, we want to be able to generate identical output for multiple players while storing a minimal amount of state. A seedable random number generator could allow players to share worlds in a web game just easily as they share Minecraft seeds.

One thing I haven’t mentioned is cryptography. The W3C has a draft Crypto API and there are a few good libraries which provide nominally cryptographic strength random number generators (namely CryptoJS and the Stanford JavaScript Crypto Library). But in general we do not want to do crypto in the browser. No amount of cleverness on the part of a developer can stop a client machine from simply reassigning our random number generator to reallyRandomYouGuys() { return 0.9;} and the client is always in the hands of the enemy.

How a random number generator works

There are a host of different random number generators, each with their own advantages and (occasionally fatal) flaws. One of the simplest is a linear congruential generator (LCG). It’s predecessor, the Lehmer generator, is even simpler but we can explore the LCG a bit to get a feel for how a computer generates random numbers.

A LCG works by exploiting a relationship between a (the multiplier), c (a shifter) and m (the modulus). The output z is an integer and serves to seed the next round of the generator. The apparently random numbers are generated by the modulus operation wrapping the value of a * z + c around m⁴. Each output z will range from 0 to m. A good generator will produce a very long sequence of apparently random numbers. The length of this sequence is referred to as the period of the generator (less commonly, the cycle). A properly configured LCG should have a period equal to its modulus, in this case, 25.

For longer streams with the same seed this pattern will repeat perpetually. At first this doesn’t seem too useful but if the period of our generator was something like 2^50 (perfectly achievable for some generators) we could generate a million numbers every second from now until 2038 and not run out. Most LCGs of practical values use a very large modulus in order to establish a long period. Unfortunately for us, the period isn’t the only thing which characterizes a random number generator.

Let’s make some random numbers!

Don’t worry, they’re not all like this.

I’ve written a number of implementations of standard random number generators in JavaScript. Each are written in a different style either to match the implementation in the browser (e.g. Chrome’s MWC generator or WebKit2’s invertible mapping generator) or to expose the basic function of the generator as with the linear congruential generator or the linear feedback shift generator.

Performance of these generators in the barest sense is actually pretty good. Comparisons with Math.random() across a number of browsers indicate that most basic generator are relatively competitive. Also included is David Bau’s ARC4 implementation as an example of a full featured, robust generator. If your primary goal is not speed and you’re balking at writing a strong random number generator from the ground up, Bau’s implementation is well documented, updated and generally very good. You can see the full source code here.

Broadly speaking, the lesson is that we don’t lose too much in performance by avoiding native code. So long as we choose our random number generator with a specific purpose in mind, generating robust, reliable sequences in JavaScript should be fast enough for most needs⁵.

Where do we go from here?

Over the next few weeks we’ll dig into generating numbers from distributions other than the uniform distribution by a number of methods. If you want a rough preview of what that might involve, you can take a look at one method for generating normally distributed variables. We’ll explain how that method works as well as a few others. As we step through creating the building blocks for numerical simulation in JavaScript we’ll also illustrate a few tests for randomness and methods to visualize the performance of a generator.

Basically what R is built for.↩
Pseudo-random versus random discussion has probably spilled more pointless bits than any debate which didn’t involve a semi-colon.↩
We don’t know about IE or Opera, as their engines aren’t open source.↩
Knuth Volume III offers a rather intensive introduction to the theory behind linear congruential generators as does Hull and Dobell.↩
Even a native implementation is only about 6000 times faster than a Lehmer RNG written for the IBM/360 45 years ago!↩

Announcing OpenGenderTracker

Cassie Irwin and Irene Ros — Tue, 9 Apr 2013 11:30:00 +0000

Computationally Tracking Gender Balance in News Content with Open Web Technologies

Last fall, the Knight Foundation granted us from the Prototype Fund to build OpenGenderTracker, a suite of open source tools for analyzing and visualizing gender representation in journalism through the examination of pronouns in bylines and news content. Bocoupers Irene Ros and Adam Hyland worked with Nathan Matias from the The MIT Center for Civic Media to create the tools.

With this grant, we’ve created an easy-to-deploy, easy-to-integrate service that also provides objective and meaningful metrics. With OpenGenderTracker, publications will be able to efficiently evaluate the voice and gender of content and use this data to inform strategic decision making. Real-time data allow newsrooms to set benchmarks specific to their desks and beats rather than relying on an external organization. Being able to discuss these issues accurately rather than anecdotally will empower modern news organizations to create more compelling stories for their readers.

The Knight Foundation’s support allowed us to invest in the critical data discovery phase, which is often overlooked in the data visualization process. Dedicating time to thoughtful data discovery builds a strong foundation for more meaningful data visualizations, and the tools the team has built in this phase can now be used by newsrooms to build compelling data visualizations.

At Bocoup, it is our mission to move the Open Web forward. OpenGenderTracker is an exciting project because it brings more Open Web technology to newsrooms. OpenGenderTracker pushes the diversity issue forward, and diversity on the Open Web makes it more viable. In adopting OpenGenderTracker, newsrooms are further committing to Open Web technologies.

The newsrooms we are working with praise OpenGenderTracker for providing diversity metrics with actionable tools. Over the next several weeks, we’ll be writing more here about our data discovery phase, our case studies with the Boston Globe and Global Voices, and our future plans and vision for this project.

Welcome Corey Frang

Cassie Irwin — Thu, 4 Apr 2013 11:30:00 +0000

We’re excited to announce that our friend and colleague Corey Frang has joined us at Bocoup.

Corey is a talented programmer and computer scientist who has long been a contributor to open source technologies. As a jQuery core committer, Corey led the jQuery Effects rewrite, and currently serves on the jQuery Foundation Board of Directors. We deeply respect Corey’s experience and passion for engineering, and look forward to working with him to bring his talents to our projects.

Be sure to follow Corey on Twitter and on GitHub, or say hello to gnarf in the #bocoup channel on the freenode irc network.

Welcome Brendan McLoughlin

Cassie Irwin — Wed, 3 Apr 2013 12:20:00 +0000

We’re happy to announce that Brendan McLoughlin has joined Bocoup as a member of our consulting team.

Brendan is an Open Web programmer with full-stack web development experience, who impressed us with his contribitions to open source projects including knockout.js and audioPromise. Commitment to open technologies is paramount at Bocoup - we’re excited that Brendan shares this interest and can’t wait to see where he’ll contribute next!

We’re eager to move the Open Web forward with Brendan. Follow him on Twitter and on GitHub, and be sure to stop by our next JavaScript meetup to talk with Brendan about his knockout.js work or have him explain just exactly what skimboarding is!

Learn CSS Layout Meets Latin America

Isaac Durazo — Mon, 25 Mar 2013 14:09:08 +0000

Last week I translated Greg Smith’s Learn CSS Layout into Spanish in an effort to bring this amazing guide to more people around the world.

When I was in college, Spanish-language resources for learning about new Web Design technologies were very limited. Not only was finding them difficult, but the resources were also frequently out of date. Often, the best option was to read English-language tutorials with a dictionary at hand, all that while shaking my fist in frustration. Forcing myself to read and watch videos in a foreign language is how I learned most of what I now know about the world of Web Development - and English.

When I first saw Learnlayout.com, I was struck by how useful a site like this would have been for me when I started to learn HTML and CSS. This gave me the idea of translating the site into Spanish. I wanted to share this with people from Latin America who want to learn, but simply because of the language barrier, can’t find the right place to do it.

Learn CSS Layout is one of the best guides out there, and now it has a Spanish version. And not only that, someone landed a pull request for a Farsi version last week, and even more people are working on Portuguese, French, Chinese, German and Russian translations. Isn’t that awesome?

There are instructions in the readme for translations. If you’d like to work on one, let us know in the project issue tracker.

Part of the Open Web mission is open access. Making resources like Learn CSS Layout more accessible to a wide and diverse audience, including non-English speakers, is very on mission for Bocoup and important for me in my work here. We can only move the Open Web forward if we move it forward for everyone.

Welcome K.Adam White

Cassie Irwin — Tue, 19 Mar 2013 11:30:00 +0000

A few weeks ago, Bocoup announced a casting call for our Open Web stand-up comedy troupe. K.Adam White heard our call, impressed us with his impeccable comedic delivery, and has joined our team this week!

K.Adam (pronounced “KAdam”) comes to Bocoup by way of Grand Circle Corporation, where he was the lead front end developer. He brings extensive experience in UI design and implementation to Bocoup. KAdam has also been active in the local WordPress community since 2008, and recently became one of the co-organizers of the local meetup group.

We are so excited to have KAdam on the team, and look forward to creating amazing things with him. Follow KAdam on Twitter at @kadamwhite and on GitHub. When you’re at our next meetup, be sure to get his favorite taco recipe (hint: it involves pickled cabbage!)

Getting Started with Mozilla Gaia

Dan Heberden — Tue, 5 Mar 2013 01:00:00 +0000

Mozilla recently announced FirefoxOS, an innovative project for mobile devices that uses Open Web standards to write UI and applications in HTML, CSS, and JavaScript. The team at Mozilla has done an incredible amount of work connecting a version of Gecko made specifically for the mobile operating system, to Gaia, the web technology-driven UI of the phone. They’ve also been working to create a full suite of applications to drive the capabilities of the phone.

The Mozilla Gaia project makes the Open Web a first-class citizen on mobile phones - an important endeavor considering that mobile is the only option in many parts of the world. We’re incredibly excited to join this project, and to augment the Mozilla team’s work with our expertise in solid application architecture, unit tests, and developer workflow. Our early work focused on getting acquainted with the project, improving the test suite, and digging into bug fixes and code review processes. As a set of fresh eyes on the project, this time allowed us to see possible pain points for new developers who want to build or work on Gaia, and create good documentation for developers new to working in Gaia.

In our gaia-notes repo, you can find documentation for getting started, debugging Gaia on desktop B2G, and building your first app. Mike Pennisi has also created a screencast to walk developers through getting started with the Gaia project. These resources have already proven very helpful to those involved with the project, and we’re looking forward to adding to the documentation and helping this project grow.

Learn CSS Layout

Greg Smith — Mon, 4 Mar 2013 12:00:00 +0000

Learn CSS Layout is a new learning and reference site I made in my open source time here at Bocoup. Although there are lots of resources on the web for learning CSS layout techniques, we’ve had trouble finding ones we’re comfortable recommending. Learnlayout.com is an enlightening and accessible look at modern techniques for beginners, while also being a useful reference for more advanced developers. The site also features:

Functional and flexible examples
A start-to-finish narrative
Pages that are useful as stand-alone references

Take a look, and if you have any feedback or suggestions, let me know on Github!

Open Source Everything: Working with the jQuery Foundation

Adam J. Sontag — Mon, 25 Feb 2013 16:20:00 +0000

One of the awesome things about being a Bocouper is being able to use a quarter of your time to explore and work on open source. Whether it’s just trying out new tools or regularly contributing to a large project, it’s a great way for us to improve and give back to the technologies we use and teach. Sometimes, we find that a project’s goals align so closely with our own that we go above and beyond that threshold to provide even more support.

Over the past few months, I’ve been able to spend a great deal of time working on the jQuery Foundation’s initiative to open source the content and design of all its sites. A major reason for this effort is to dramatically lower the barrier for community involvement and therefore open up entirely new avenues for people who’d like to begin helping out, but haven’t been sure how.

Throughout our experiences teaching classes and working with clients on consulting projects, many of us have noticed that newcomers face a double challenge when trying to get started with open source. This is because of the combination of needing to learn new tools while also searching to find bugs that they’re actually capable of solving (that haven’t already been swept up by the project’s maintainers). This lack of fodder can limit the opportunities to gain experience, and ultimately many end up demoralized and commit-less.

That’s why I’m so excited about the work we’ve done at the jQuery Foundation to apply the same workflows that we use when developing JavaScript libraries to documentation and design. There have always been a lot of ways to contribute to jQuery beyond writing code, but many of them required special permission to access and offered nothing but a textarea when you got there. With the new system we have in place, anyone can report an issue or propose a fix (from the comfort of their favorite text editor) simply by sending a pull request, and updating our sites is as simple as pushing a commit to master.

It’s good news for people who are already good at using git and GitHub, but it’s also good news for folks who have been hesitant to try them out. You don’t have to “reproduce” a typo, nor do you have to write unit tests for a new paragraph of documentation. So if you “just use a little bit” of jQuery but always thought you’d never have anything you’d be able to contribute back, it’s definitely time to think again! Or if you have been searching for an open source project where you can have a significant impact while you still learn to flex your git muscles, consider yourself at home!

To learn more about the technologies involved and find out how it all works, take a look at the jQuery Foundation’s web site contribution guide. The new jQuery Learning Center is especially ripe territory for anyone who likes helping other people out and has good ideas to share!

I’m thrilled to have had Bocoup’s support implementing this new stack, and I’m proud to have had a part in bringing things to where they are today. It’s been an amazing ride thus far, and I’m looking forward to watching the sites evolve as more developers join us in improving themselves, each other, and the entire Open Web.

Ember.js Training from Tilde at Bocoup

Jory Burson — Mon, 25 Feb 2013 01:00:00 +0000

We’re so excited that our friends Yehuda Katz and Tom Dale from Tilde will be stopping by the Bocoup Loft to deliver Introduction to Ember training April 22-24.

At Bocoup, we support a wide variety of Open Web technologies. We believe in the right tool for the job, so we’re actively exploring different MV* libraries and frameworks. That’s why we’re happy to support Ember.js training in addition to hosting Backbone.js trainings and meetups, and the newly-formed Angular.js meetup for the Boston development community. If you’re looking to learn more about Ember.js, we hope to see you there!

Making a RapBot with JavaScript

Darius Kazemi — Wed, 20 Feb 2013 15:33:00 +0000

This post talks about the development of RapBot, my freestyle 80s battle rap generator. You might want to see it in action before reading on, and you can check out the source code here.

For the past year I’ve been using the Wordnik API in my projects to generate random words. I’ve made extensive use of their randomWords feature in projects like All the Things, Metaphor-a-Minute!, and even Amazon Random Shopper but for every project I’d copy/paste code for interfacing with Wordnik. I’d been meaning to write my own interface to the API, but I’d been putting it off, because, well, I’m lazy. But a couple of weeks ago I took our Building Web Applications with Backbone class, and I was finally inspired to build a promises-based interface to Wordnik using Backbone, which I called wordnik-bb.

It was while building my library that I noticed that Wordnik had recently added rhyming to their API! That is, you can send an English word to their relatedWords API call and it’ll send you back an array of words that rhyme with it. I had never really played with the power of their powerful semantic “word graph” before and now I finally had an excuse to do so.

Experimenting with creative code

Whenever I start a creative coding exercise, I do some simple prototyping to see if the core idea I have posesses any potential to be interesting. Initially I just brute forced a few API calls using the excellent interactive tools over at the Wordnik API documentation. I wrote the first thing that came to mind:

I was pretty happy with the output of this. For example:

I once knew someone who was Iraqi
Their manners were very wacky

My rule of thumb for creative coding is: if it makes me laugh, it’s probably worth pursuing.

At any rate, I showed this to my friend Cameron Kunzelman and he immediately responded, “So basically you’re telling me this rhyming machine is going to be able to make late 1980s rap pretty consistently.” This reminded me that I’d wanted to make a “bad freestyle rapper” bot for ages, and now I had the means to do so.

One equation for making a cool textual generator is: randomness + formulaic writing = hilarity. So I pretty quickly realized that having my program generate a battle rap was the way to go. A good resource for this is Adam Bradley’s Book of Rhymes: The Poetics of Hip Hop, which lays out some of the formula behind battle rhymes.

Battles are better with promises

The basic algorithm for RapBot has remained the same from the start:

Get a random word and its part of speech. Get a ‘line’ for that word based on the part of speech. For example if it’s a noun, it’ll pick from a bunch of noun lines, like “I’m the illest MC to ever rock the [noun]”
Once I have that first word, get a list of words that rhyme with it. (Wordnik is sort of inconsistent. Some words just lack any rhyme data.)
If I get back any rhyming words, then I pick one at random and also get its part of speech.
Much like I did with the first word, I get a line to match its part of speech.
Now I have a couplet! I repeat this 12 times, which will get us anywhere from 0 (if it couldn’t find rhymes for any words at all, a rare case) to 12 couplets.

This algorithm would have been a lot harder to write if I weren’t using a promises implementation. If you’re not familiar with promises, here’s an introduction to jQuery’s implementation of promises. The short version is: if you’re making asynchronous calls to a service (like I’m doing with Wordnik), you can’t be guaranteed when you’ll get the data back. It could be in 200ms, or 2 seconds, or 20 seconds. So instead of writing a function that returns a value like the part of speech for a word, you write a function that returns a promise for the part of speech of a word. Then you can define callbacks on the promise telling it what to do if it’s resolved as a success, rejected as a failure, etc.

So now that you’re an expert on promises (cough), you can now see that an algorithm where it says “make this async call, then when that’s done make that async call” and chains it a bunch of times is a great place for us to use promises.

Here’s an example of promises in RapBot, when we ask the Wordnik API for a part of speech of a word:

Compare this to the structure of the getLine, which is a good ol’ synchronous function:

Reducing API calls

I did a “soft launch” of RapBot on a Saturday night to see how it would perform. The answer: not terribly well. Wordnik’s API caps the number of calls per hour at 5000 calls. That might seem like a lot, but at the time of my soft launch, every page load made somewhere close to 50 API calls! That’s because each couplet would:

Get a random word (1 API call)
Get the part of speech for it (1 API call)
Get a list of words that rhyme with it (1 API call)
Pick a word from that list (not an API call)
Get the part of speech for it (1 API call)

So each couplet is 4 API calls, and for 12 couplets that could be as many as 48 API calls.

Fortunately, I was able to get the number of API calls down to about 29 from 48. In addition to returning individual random words, Wordnik can also return an array of up to 1000 random words in a single API call. You can also ask for an array of random words of a particular part of speech. Looking at the first two steps of creating a couplet, you can see that I’m getting a word and getting its part of speech. The better approach would be: on page load, grab an array of random words for each part of speech that I need. Since I support 5 parts of speech, that’s 5 API calls. Then in each couplet I say: pick one of our parts of speech we support, then access the array of words for it. This means that the couplet algorithm now looks like this:

Pick one of our five parts of seech (not an API call)
Get a word from the array of random words from that part of speech (not an API call)
Get a list of words that rhyme with it (1 API call)
Pick a word from that list (not an API call)
Get the part of speech for it (1 API call)

So now each couplet is only 2 API calls, which means that the 12 couplets make 24 API calls total. Add in the 5 API calls on page load and we’re now down to 29 calls from 48. (As often happens with refactoring for performance, my code became more modular and suddenly I was able to tweak the frequency of appearance of each part of speech. So now I can say: give me a noun 40% of the time, an adverb 10% of the time, etc.)

There’s further work that could be done. Instead of making the 5 API calls on page load, the application could make the 5 API calls on startup and store it globally on the server. But that would reduce the variety of results on each page load. I could increase the randomness by updating those random word arrays silently in the background, making the API calls once every half hour, and appending the results to the existing arrays.

Beyond that I could cache a whole bunch of couplets and just refer to the cached results on every page load, but again that would further reduce the randomness of the results and make the whole exercise less fun.

I do have to give a shout-out to the team at Wordnik for increasing my API call limit, too!

Looking ahead

With any creative coding project there will always be a long list of ways to improve. Potential improvements include:

More complicated rhyme schemes
Bots that respond to each other (1: “You’re nothing but a [noun1]!” 2: “You call me a [noun1]? You’re just a [noun2]!”)
Take meter into account, so the rhymes scan right

All of these things will result in more API calls, so before I do anything else, the next step would involve caching results to reduce API calls by an order of magnitude to something like 2 or 3 per page.

In closing, I leave you with some cryptic words of wisdom about the Open Source ecosystem, courtesy of RapBot:

If you're gonna battle me, then you gotta be noncommercial 
When I rock a mic you know I rock it real uncontroversial

Tearing Grunt Apart

Tyler Kellen and Ben Alman — Mon, 18 Feb 2013 15:26:14 +0000

This is a guest post by Tyler Kellen, who co-maintains Grunt with Ben Alman, and is also the project lead for the officially curated contrib series of grunt plugins.

Introducing Grunt v0.4

For the last nine months, I’ve been working with Ben and our team of fantastic contributors to modularize the Grunt codebase. Today, we’re happy to announce that Grunt v0.4 has been published to npm. Hold on to your hats, a lot has changed!

Architectural Improvements

The primary focus of this version can be summarized by the venerable words of Doug McIlroy, “Do one thing and do it well.” We’re proud to say that this release is a major milestone on the road to fully embracing the Unix philosophy.

Built In Tasks Removed

If you’re coming from Grunt v0.3.x, the most immediate difference you’ll notice is that we’ve removed all of the built-in tasks. Don’t worry, they aren’t gone—we’ve extracted them into officially maintained, standalone plugins. If you visit the plugin listing on our website, look for the entries with stars next to their names.

One of the biggest benefits of this change is faster iteration. It’s no longer necessary to ship a new version of Grunt for every minor plugin change! If jshint gets updated, for example, we can just release a new version of grunt-contrib-jshint.

Finally, with our core plugins separated, the barrier to entry for collaboration is lower than ever. If you’d like to get involved, please join us in #grunt on irc.freenode.net, or check out the Contributing guide and submit a pull request to add the feature you want.

A note about the “init” task

This scaffolding task was always an odd-duck, and it really deserves it’s own project. So, we’ve broken it into a separate npm module, grunt-init, which can be installed globally with npm install -g grunt-init. In the coming weeks, the Yeoman team will be replacing this task entirely, using their rails-inspired generator system. If you’ve authored templates for the init task, get in touch with them for details on how to make the conversion.

Global Installation

Another departure for those coming from Grunt v0.3.x is that Grunt itself no longer ships with a binary. Instead of installing Grunt globally, you now specify it as a development dependency in your project’s package.json file, along with any Grunt plugins you might be using.

In order to get the grunt command, we’ve provided the grunt-cli tool, which should be installed globally. The sole purpose of this tool is to load and run the locally installed version of Grunt, and good news—it will work with every version from 0.3x and higher.

There has been considerable confusion in the community about global vs local installation of packages. We recommend reading this excellent article by @isaacs for more information about why it is desirable to install things locally when using node.

Supercharging Grunt’s Declarative Power

Gruntfiles are now more powerful than ever. We’ve been hard at work implementing community requests, and thanks to everyone’s feedback, the results are fantastic. Here are a few of the major additions:

Options Merging

Need to run the same task on more than one set of files? Say goodbye to duplicative configuration, options can now be defined at the task level—they’ll automatically apply for all of your target file groups. If needed, you can override them individually. Check out our documentation for more details.

Mapping individual files from one location to another

We have a new syntax for this common request (e.g. compiling coffeescript files from one directory to javascript files in another, while maintaining the folder structure), it works with any v0.4 compatible plugin, and it eliminates reams of boilerplate for plugin authors. Check out our documentation for more details.

Templates everywhere

Need data from package.json for your build process? How about a YAML file? Not a problem, just load it into your configuration and apply it anywhere you like. Check out our documentation for more details.

New APIs

Just to whet your whistle, here are a few handy methods we’ve added to make writing tasks a cinch: file.isPathInCwd, file.doesPathContain & file.arePathsEquivalent. There are far too many additions to list in this introduction, please see our migration documentation for a full rundown.

New Website

Grunt 0.4 ships with a flashy new website. We’ve got reams of documentation, and helpful tools for discovering the perfect plugin for your project. Check it out at gruntjs.com!

Roadmap

If you think Grunt is great now, just wait until you see what we have in store for the next major version. Here’s a glimpse of what you can expect next:

Glob Expansion Library

Grunt has made it easier than ever to select, negate and map your project files from one location to another—complicated globbing patterns need not apply! We’re so happy with the power this system affords that we’re going to extract it into a standalone library so anyone can use it.

Configuration Library

Sharing values across multiple configuration files is a breeze with Grunt’s templating system. Soon, you’ll be able to unleash Grunt’s declarative power anywhere you need it—we’re turning this system to a standalone npm package as well.

Grunt Middleware

Don’t like Grunt’s configuration syntax? Go crazy and write a middleware to convert your format to ours. Maybe we’ll love it and decide to make it the default.

Escaping Dependency Hell

When you upgrade to v0.4, chances are strong you’ll run into some tasks that don’t support it yet. We know this sucks, and we’re working with plugin authors to get the ecosystem up to speed as quickly as possible. Long term, we’re going to solve this problem entirely. Here’s how:

Grunt v0.5 will ship with support for a new plugin format called node-task. It defines a stateless, promise-based, event emitting API that doesn’t depend on Grunt. It has a real-live spec, and the Grunt team is working with the front-end developer community in the hopes that compliant modules will be compatible with every task runner under the sun. When it’s done, it will ship with a test suite that anyone can use to build their own tasks.

Hangout with Bocoup

Jory Burson — Mon, 18 Feb 2013 01:00:00 +0000

The Bocoup Loft is a fun and inspiring place to be. Between the technical debates and conversations amongst Bocoupers, regularly scheduled classes and meetups, and the occasional banana-themed event, 355 Congress St. is a very lively place!

For the past few months, we’ve been experimenting with bringing that liveliness online via our Bocoup on Air series with Google+. We’re inviting you, the open web developers of the world, to share ideas, questions, and discussions with one another in order to (hopefully!) learn something new and get to know more people in the community. So far, we’ve talked about JavaScript and robots, HTML5 game development, mobile development on the open web, and contributing to Open Source.

Ben Alman, Boaz Sender, and others will be talking about JavaScript build tools and workflows at our next hangout this Thursday, Feb. 21 at 2:30 EST. RSVP or post a question for the discussion on Bocoup’s Google+ page, and join us in the #bocoup channel on the freenode IRC network for real-time conversation. If you can’t make it at 2:30, you can follow our YouTube page to watch the recording later.

We hope you’ll hangout with us at the Loft or online soon!

Bring Out Your Bugs!

Irene Ros — Wed, 13 Feb 2013 15:26:14 +0000

At Bocoup, we’ve long been advocates of bringing open source practices to new communities wherever we go. That’s why we’re excited to join OpenHatch and Harvard Women in CS at Harvard University this weekend to introduce the student community to the open source process. OpenHatch is a non-profit dedicated to matching prospective free software contributors with communities, tools, and education.

We began our work with students at Seneca College’s Center for the Development of Open Technologywhile on Popcorn.js, and are excited to bring that experience to OpenHatch this Saturday, Feb. 16. While students commonly use Open Source projects in their coursework, the concept of ‘open source’ is rarely a topic of discussion in the classroom. Nonetheless, we at Bocoup believe that using and contributing to open source come hand-in-hand. And helping out is easier than you think! Conveniently, the easiest way to get started is through working on bugs and other project needs, which is where you come in.

How can you help?!

We’re reaching out to you, the community of open source developers, to submit links to bugs or projects on github that need help. We can’t guarantee that we’ll address all your issues, but we’ll do our best to identify issues the students can tackle, and hopefully submit a few pull requests in the process.

Please submit your bugs and other requests here.

Welcome Adam Hyland

Cassie Irwin — Mon, 11 Feb 2013 01:00:00 +0000

We are pleased to announce that Adam Hyland has joined the Bocoup team as a data visualization intern.

Adam brings expertise in statistical software development and data modeling to Bocoup’s data visualization projects. Through his background as an economist and R programmer, Adam has a wealth of experience in data handling and visualization, as well as building and assessing statistical models.

We are excited to have Adam on the team, and looking forward to working with him to grow as an Open Web developer!

You can follow Adam on Twitter and GitHub, or pop by Bocoup to talk with him more about his work on Wikipedia.

Sessions: The vim Feature You Probably Aren't Using

Adam J. Sontag — Fri, 8 Feb 2013 12:00:00 +0000

I may as well come out and say it. In 2010, I leapt directly from Dreamweaver to vim as my primary editor. (I’ve been told that I may be the only person ever to have made this jump, but have no concrete evidence to substantiate this claim.) I had grown tired of Dreamweaver’s engorged appetite for RAM, and had grappled with vim enough in the context of my dealings with my VPS and git that I had already needed to at least kinda learn how it worked. One of the major things that pushed me over the edge was catching a glimpse – I’m not sure where, maybe in MacVim’s menu – of the word “session.”

Part of the reason Dreamweaver was so gluttonous was because I kept every file from every project that I was working on at a time open at once. I had begun to dabble with TextMate and e, and found it incredibly convenient (read: sane) to be able to keep separate projects grouped together. Surrounded as I was, however, by a bunch of vocal vim partisans, its allure was irresistible, espcially once I realised it could provide me this very same convenience.

After a couple of hours (or even days) of working on a project in vim, it starts to have its own character. You get a bunch of tabs open, and then each one ends up with its own set of splits, and sure enough you know to go over here when you need to take a look at the config, and to go over there when you need to iterate on the new component you’ve been working on that week. You’ve built yourself a place to live, but you’re just renting. vim’s session features will let you make that house a home.

When you’ve been working for awhile and you’re all moved in to your new digs – maybe it’s the end of the day, maybe it’s been 71 days and you finally have to give in and restart your computer, you just save the session file to anywhere you’d like (I like ~/.vim/sessions/) using :mksession (or :mks).

:mks ~/.vim/sessions/rooster.vim

The next time you’re ready to start working on that project, source that session file within vim:

:source ~/.vim/sessions/rooster.vim

Or open it when you run vim using the -S flag:

vim -S ~/.vim/sessions/rooster.vim

Boom. Everything will be exactly as you left it: the working directory, your windows, splits, and buffers, and any options you’ve :set. When the cycle repeats as you rearrange the furniture, just overwite the old session by using :mks!.

Sessions don’t save marks, registers, or command line history (which are stored in viminfo), but this hasn’t been a hindrance so far, and should you want to persist it, you can. If you want to further bake the use of sessions into your vim experience, Peter Odding’s session.vim plugin adds a number of helpers for working with session files, and integrates session saving into the default vim workflow. I’ve only just discovered it in the process of researching this post, but I’m certainly looking forward to giving it a shot; it’s an epic pain when you forget to save a long-running session and you’re left traveling back in time the next time you resume your project.

Having my editor become a comfortable and familiar space, on a project-by-project basis, has been a huge boon. It’s immensely useful when you’re neck-deep in a project, and it’s a downright pleasure when stepping back into something you worked on a year earlier feels like returning to an old friend’s pad.

While this may be old hat to some, I’m sure it’s news to others. I’m not exactly the Gandalf of vim, but I couldn’t help the feeling that a lot of people are missing out on the magic here. What do you think? Excited to go cozy up in the comfort of your new digs? Dying to point out some subtlety that I’ve missed? Let’s continue the discussion in the comments!

A Bientot, Rebecca

Boaz Sender — Fri, 18 Jan 2013 01:00:00 +0000

Today, we’re saying goodbye to Rebecca Murphey. While we will miss her regular presence at Bocoup, we’re excited to continue working together on community projects as we have in the past.

I met Rebecca four years ago when we were both green jQuery developers. I have learned so much from her over the years, and so it has been a special honor for me to work with her this past year. Rebecca has an incredible intuition for community-driven software design, as well as understanding and articulating the nuances of tomorrow’s problems. Rebecca’s influence on the community is huge, and I feel lucky that she touched Bocoup.

We are so thrilled with the work she did while part of our team. Over the past year, Rebecca introduced two new classes to Bocoup Training, built and released the new interactive jQuery Fundamentals, developed our coaching product, started our screencasts program, and kicked off Bocoup on Air.

Rebecca is leaving Bocoup better than she found it. We are all eager to see the ways in which she continues to be a steward of the Open Web, and we’re looking forward to collaborating with Rebecca outside of Bocoup.