dshaw's shared items in Google Reader

SiteMesh 2.4 released

2009-01-22T20:52:12Z

After a long wait, SiteMesh 2.4 has just been released.

This mostly consists of minor bug fixes and improvements.

Website: http://www.opensymphony.com/sitemesh/
Download: http://www.opensymphony.com/sitemesh/download.action
Changes: http://jira.opensymphony.com/secure/IssueNavigator.jspa?reset=true&pid=10000&fixfor=21683

Stop Using Internet Explorer 7 (For Now), Or Deploy Workarounds

2008-12-28T02:29:28Z

Shared by dshaw
Just stop using IE. Please. It's a security architecture failure.

There is an unpatched vulnerability for Internet Explorer 7 being actively exploited in the wild. The details are public, so any bad guy can take advantage of this. It’s a heap overflow in the XML parser, for you geeks out there. It affects all current versions of Windows.

Microsoft issued an advisory with workarounds that prevent exploitation:

Set Internet and Local intranet security zone settings to “High” to prompt before running ActiveX Controls and Active Scripting in these zones.
Configure Internet Explorer to prompt before running Active Scripting or to disable Active Scripting in the Internet and Local intranet security zone.
Enable DEP for Internet Explorer 7.
Use ACL to disable OLEDB32.DLL.
Unregister OLEDB32.DLL.
Disable Data Binding support in Internet Explorer 8.

What Blogger Should Do

2008-12-28T02:27:36Z

Shared by dshaw
+1 Better templates. I've stopped short numerous times from doing a project with Blogger solely for this reason.

I was recently asked about the "death of blogging" for this article in The Economist. I didn't get back to the reporter in time, though, so my comments ended up, ironically, on his blog. (Conclusion: I don't believe blogging is dying, but...it's complicated. Like in most healthy ecosystems, new species are breeding. Whether or not they're called "blogging" is a question perhaps best left for scientists, but there are many new forms that are undeniably part of the blogging genus.)

Last night at the Churchill Club, I was quoted as saying that Twitter "will dwarf Blogger." I do believe that, but it will be no easy task and will not be soon. Blogger is big. Really big. That chart was from six months ago. Is it losing traction? I don't know. It doesn't look like it was then. And since then, the team over there seems to be kicking ass. A glance at Blogger Buzz show's they've been launching feature after feature the last few months. Launching any features when you're that big is usually a daunting task. Shows that a lot of years building a solid platform have paid off.

So, the question is: Where do they go from here? Part of that, I suppose, will be determined by where the Google powers-that-be decide Blogger lands on their priority list, given the leaner times. Clearly it's not one their cash cows, but it's also not a side project they're dabbling in. I've heard it makes money (from AdSense on blogs they host), but I really don't know. In fact, I know so little about Blogger these days, I feel like I can actually write about it as an outsider.

From a product perspective, I do feel like they could get more out of the capabilities and incredible usage they already have. Here's an unordered list of some of the ways I'd look to do that if I were in charge:

Build a Network of Networks: Building more interconnection between users and blogs is clearly part of the focus now with "followers." It's something we realized we were remiss in not doing more of way back when we built the (not very good) profile pages in 2004, when Friendster was the big thing and Orkut was launching down the hall. There are a ton of mechanisms to do this, but one thing to consider: Don't try to make one big network. Perhaps enable anybody to create a blog network/community thingy. (There might be a doc around there about "Blogger Hubs" -- not sure if it's still relevant.)

Point People to Good Content: When it comes to interconnectedness, don't just try to make it more "social." Social is important, but pure socialness can be achieved elsewhere. One unique thing about Blogger -- vs. say Facebook or MySpace -- is the content. How can you make the content more interlinked and use the network to get more attention to the good stuff, thus rewarding the creating of more good stuff? You know what the most-viewed/commented/linked-to post on Blogger was today. Show it to me! I bet it's interesting! (Even better: Show me what's most popular within my blog network.) BTW, if revenue, not just usage, is a priority, this plays to that, because it's the content focused blogs that can make the money.

Get More Out of the Navbar: The toolbar you have at the top of millions of blogs could do so much more. This is where you can put the feedback mechanisms, interlinking mechanisms, etc. NextBlog could be a whole thing! Make that so I never wanna stop clicking because it always shows me something awesome. (Think StumbleUpon within the Blogger network.)

Prettier Templates: When it comes down to it, many people just want a page that looks good. This a large part of Tumblr's appeal, in my opinion. You gotta upgrade those templates. I know prettiness is not a focus in your culture, but bring them into 2007 at least web design, if not 2009.

Help People use the Layout Engine: The new templating system does everything anyone would every want. But you kinda gotta be a programmer who likes programming in production, in a textarea, in a language you've never used, to tap into it. Yes, there's point-and-click design and widget goodness, but it seems...hard. Can you make it seem fun? Can you make it so pseudo-developers can figure it out and others can leverage that? Layout sharing perhaps (kill two birds)?

Make it Fast: You've made some progress on slimming down the posting form page (at least in Draft). But I don't know if we've fully embraced the Google mantra of speed is every. I predict you'd see a 30% increase in posts if you made posting twice as fast. (That goes for the whole workflow, not just the posting form.)

Become the Aggregator: One possible answer to the question to what role does the stand-alone blog live in the age of a million-and-one generalized and specialized participatory web experiences is as a personal aggregator that reflects back the other stuff one does on the web. Yes, I'll load all that stuff into FriendFeed, but that's not my "online presence" as we used to say back in the day. Everybody (or at least a lot of people) needs an URL -- and one without a ? in it. I want my tweets, and my photos, and my whatevers to show up on evhead.com (hosted by Blogger) in an attractive way.

Put Ads in the App Interface: For example, the published landing page alone must get millions of views a day. And it's the perfect point for someone to go elsewhere. See if you can target it off what they wrote about. No one will mind (much). And strengthening your revenue story will strengthen your position in the company.

Do Something Radical: I almost feel like this list is way too conservative. Not that I think Blogger's in trouble. But I suspect there something potentially more awesome that you could pull off by leveraging what you already have. You probably have those ideas. When there are so many great things to do that you know will work, it's hard to not focus on them. But it might be time to try something wacky.

Sorry, I know you've already thought of these things. When you're working on a product and people on the outside tell you what you should do, acting like they're all smart, it's annoying. The hard part is building stuff, not thinking it up. Carry on. (But seriously, the prettier templates thing.)

Encoding for streaming

2008-11-06T15:56:00Z

This is Neil Hunt, Netflix chief product officer, with some technical background on the Netflix encodes used for streaming. We get a lot of questions about this, so I'm going to go deep.

First Generation Encoding

Our first set of encodes are based on WMV3 and WMA in ASF with WMDRM10 (Janus). We chose these standards because the Janus components have been widely adopted by our CE partners such as Roku, LG Electronics, Samsung, TiVo, and of course Microsoft Xbox.

We encode most content at 500, 1000, 1600, and 2200kbps VBR, but some titles whose source quality merits it have also been encoded at 3400kbps. The highest bitrate encodes are fit into 720x480 non-square pixels (the usual 1.2 PAR for widescreen content, 0.9 PAR for 4:3), but optimum encoding at lower bitrates is achieved with fewer pixels. Encoded films are normally at 24fps to match the source, while shot-to-video and mixed material is de-interlaced to 30fps (or 25fps for PAL content).

Second Generation Encoding

The new Silverlight player (that some users are helping us test as I write) uses VC1 Advanced Profile encoding with PlayReady DRM. A key property is that each GOP header includes frame size and resolution, which allows us to assemble a stream on the fly from different bitrate encodes as your broadband bandwidth fluctuates. (Another key feature is more coverage, including Intel Macs and Firefox users.) We expect to switch completely to the new player later this year.

The VC1 encoders are more efficient than the WMV3 encoders, so we are currently encoding VC1AP at slightly lower birates: 375, 500, 1000, and 1500kbps, all square pixel. At some point we are likely to add a couple more resolutions of non-square pixel encodes capturing the original pixel-aspect-ratio of the source.

We are also re-wrapping the VC1AP encodes in WMDRM10 for CE devices, which will gradually switch to the more efficient encodes in future firmware upgrades.

High Definition Encodes

Today we have rights to deliver about 400 streams in HD (720p). More titles will be added over time. We experimented with first-generation WMV3 encodes at 4000kbps and 5500kbps, but settled on second-generation HD encodes with VC1AP at 2600kbps and 3800kbps, which extends their accessibility down to lower home broadband connections. As with SD, encodes of film material are at 24fps, and encodes of shot-to-video material are at 30fps (or 25fps for PAL), rather than the 60fps that would come from a Blu-ray disc - we judged the 60fps content as too expensive of bandwidth for now. In general, these encodes are definitively better than SD, but won't challenge well-executed Blu-ray encodes - that would require a bitrate out of reach for most domestic broadband today. We believe Moore's law will drive home broadband higher and higher enabling full 1080p60 encodes in a few years.

Stereo Audio

Today, we cannot use WMDRM to deliver AC3 or DD+ audio, which means that only stereo (delivered via WMA) is available. PCs and Macs decode the WMA, and CE players also transcode to PCM for digital connections to receivers. We could technically include multichannel audio using WMAPro, but essentially no receivers are actually capable of decoding that. We are working on solutions to deliver multichannel audio for all the streams where we have suitable source, but this won't happen in 2008 for sure.

Subtitles, Closed Captions, and Alternate Soundtracks

All these features are desired for future releases. Delivering closed-captions via the Silverlight player is probably closest, but it won't be 2008 either.

Sources

Our best sources are electronically delivered mezzanine files, or high quality D5 tapes, and the highest bitrate encodes of these sources really look as good or better than DVDs. Digibeta tape sources can also generate good encodes, but some sources just are not as good, regardless of the bitrate used for encoding. We also encode from DV tape and even on occasion from DVDs. We get HD sources for many titles, even if we only have the rights to stream SD. The HD sources permit a better SD encode than working from SD soures.

One class of sources has been derived from 24fps film, interlaced to i60 for TV broadcast, and then decimated to p30, and comes with restrictions on reprocessing. This results in frames that are even-odd interlaces of adjacent film frames, and a 4/5 cadence motion jerkiness. We are actively working to re-acquire these sources in better form.

Delivered Quality

Our first-gen PC streaming player uses 1-4 bars to represent the delivered quality, representing 500, 1000, 1600, and 2200 kbps. The 3400kbps encodes are represented as 4 bars. The player measures bandwidth once at the start of the title, and chooses a bitrate for delivery that has at least 40% headroom from the measured speed.

The Roku, LG, and Samsung, players use four dots during buffering in the same way, and Xbox has 4 bars just like the PC player. The TiVo player has a similar display, but with 10 thin bars.

The Silverlight player is currently more opaque, since it picks the stream to deliver dynamically. If your connection slows, as the buffer empties, the player starts buffering a lower bitrate stream and switches seamlessly across. Conversely, if the buffer fills rapidly again, the player can pick a higher bitrate stream. (Note that if Outlook (or some other large application) decides to wake up and refresh your email in the middle of a movie, Silverlight might be starved of CPU and drop some frames; this may cause the player to conclude that it should switch to a lower bitrate stream that won't overload the CPU. Today, we haven't figured a reliable way to determine that the CPU is again underutilized and permit switching back up again, so my advice is to close Outlook and similar periodically expensive applications prior to playing the movie!)

Other Notes

We strive to deliver great encodes, and the encoding recipe evolves as we learn new tricks and engage new encoding technology. Since we have a variety of software and CE players in the field, we have to continue to support existing encodes for a transition period if new versions incompatible with existing players are added, until we can work with partners to upgrade firmware to support the new encodes.
One new feature that I want to add is a post-play screen that appears when a playback is stopped, so that users can easily flag when encodes fall below par, so that we can prioritize identifying and fixing issues.

The W3C Core Working group has broken faith with the XML community by publishing an XML 1.0, fifth edition spec that is incompatible with all previous versions.

2008-12-11T19:18:16Z

Shared by Paul R Brown
And over the shark we go. Good riddance.

The W3C Core Working group has broken faith with the XML community by publishing an XML 1.0, fifth edition spec that is incompatible with all previous versions. The grammar has changed and previously malformed documents are suddenly well-formed. Existing parsers cannot handle the syntax defined by this edition. XML 1.1 has failed so now the W3C is trying to rewrite history and pretend that this is what they meant all along. (If that were true, why did we waste so much time on XML 1.1?) Apparently stability of standards is no longer a virtue at the W3C. This is even worse than the XML 1.1 debacle. At least there, the W3C admitted they were pushing a new, incompatible version; and gave users a hook to tell which version they were receiving. Now we don't even have that. As if XML weren't already confusing enough for people who don't spend 60 hours a week thinking about this stuff. Now we have to explain that the well-formedness of a document depends on which version of which parser is being used, and which edition of XML the parser implements, and no, there's nothing in the document to tell you which version you should be using. More...

Dependency injection with Guice

2008-12-11T19:17:01Z

Shared by crazybob
Nick Lesiecki on Guice

Guice is Google's open source dependency injection framework for Java development. It enables better testing and modularity by taking away the pain of writing your own factories. Nicholas Lesiecki offers a tour of the most important Guice concepts that will leave you ready to Guice up your applications.

Sleet

2008-12-01T05:00:00Z

Meyer and Resig on CSS3

2008-12-10T10:01:17Z

Six Revisions put six questions to the estimable Eric Meyer on the subject of CSS3, and we get some nice link love.

Also (and I’m slightly late with this one) John Resig, creator of the jQuery library, runs an approving eye over the Advanced Layout Module. Webmonkey provide further context. We looked at the module back in 2006 (part one, part two), and my tongue-in-cheek prediction of a ten year wait to use it is now down to only eight years…

Extreme Kite Snowboarding

2008-11-25T18:13:36Z

Shared by dshaw
Oh man, that looks sketchy and scary.

Scala?

2008-11-21T17:53:00Z

I received a free copy of the Scala eBook by Martin Odersky, Lex Spoon, and Bill Venners. I tried to read it on my CalTrain commute, but ended up only skimming -- at 700+ pages it's a hefty tome. I'm also not in the primary target audience, as its focus seems to bounce between trying to entice Java programmers to try a better language, and an introduction to programming that happens to use Scala. I ended up going to the Scala website where I downloaded various other documents, such as a tutorial, a more technical overview paper, and the language reference.

Sad to say, I'm quite disappointed (in the language -- can't really say much about the quality of the eBook except that it's as big as the language). One of the first times I heard Scala mentioned it was a talk by Steve Yegge at Google I/O last May. Steve characterized Scala as "static typic's revenge" (I'm paraphrasing -- too lazy to watch the video to find that section) and went on to make fun of the incredible complexity of Scala's type system, which contains such esoteric concepts as type erasures, variance annotations, existential types, polymorphic method types, and many more (just scan the ToC of the language reference).

I have to agree with Steve -- if this is what it takes to have compile-time type-safety in a language, I'll take dynamic typing any day. There's got to be a better way -- perhaps Haskell? Haskell is a pure functional language with a fast implementation that seems to have solved the I/O problem of functional languages well -- while its Monads are truly deep and take real work to create, their use seems quite straightforeward.

Perhaps my biggest disappointment in Scala is that they have so many rules to make it possible to write code that looks straightforward, while being anything but -- these rules all seem to have innumerable exceptions, making it hard to know when writing simple code will work and when not. For example, the Scala book explains that you can use multiple 'if' qualifiers in its for-statement, but points out that you have to insert a semicolon in a position where you'd expect it to be inferred by the parser. To make matters even more mysterious, if you were to use curly braces instead of parentheses, you can omit the semicolon. All this is presented without any explanation. I found the explanation by combining two seemingly unrelated rules in the reference manual: first, the for-statement (technically called for-comprehension) allows both braces or parentheses with exactly the same semantics, but the rules for inferring semicolons are different inside braces than they are inside parentheses. Obvious, eh! :-)

IntelliJ IDEA 8 Refactorings: Introduce Parameter Object

2008-11-12T14:31:02Z

As you might already have noticed from the Features list, IntelliJ IDEA 8 brings you 7 new refactorings to ease the daily coding routine. One of these refactorings is Introduce Parameter Object. Its name speaks for itself: when you have a large group of parameters which go a long way through a chain of delegating method calls, it lets you create a wrapper class that you can then use instead. Another case is when you have a group of method parameters semantically tied one to another â€“ a wrapper class comes handy as well.

Letâ€™s have a look at an example. Say, we have this code:

Look at how many parameters are passed to drawEdge method â€“ six! Imagine how lengthy the calls to this method are and how it all swamps the code. No worries â€“ IntelliJ IDEA can help us. As we can see, the X and Y coordinates and the edgeWidth naturally go together, so we can move them to their own object.

Right-click the method name and select Refactor | Introduce Parameter Object from the context menu.In the Introduce Parameter Object dialog, we can select whether we want to create a new class, inner class, or use an existing one to wrap the parameters. For this case, we opt to the first choice. Then, select the parameters to be extracted. As we decided, these are edgeWidth, x1, x2, y1, and y2. Click OK, and voila â€“ the things are a lot easier now!

Skiers now have an application they can use during runs

2008-11-13T21:06:00Z

Shared by dshaw
This is awesome!

Lately it seems like the iPhone is getting a lot of sports related applications. The latest by DropZone, HangTimer, has been launched in the App Store for both the iPhone and iPod Touch. As you might have guessed by the title, this application was created for skiers.

HangTimer utilizes both the built in accelerometer and technology used by NASA to determine the split second that you become airborne. This application records your airtime and top ten jumps. HangTimer will even automatically detect your ski resort from anywhere in the world and automatically download the area’s snow reports, lift stats and trail maps. HangTimer also accesses webcams in select resorts.

For iPhone 3G users, HangTimer has a bonus feature that utilizes the built in GPS to find and plot your exact location using Google Maps. Using the GPS you can also display your speed as you cruise along as well as how fast you are going. You can download HangTimer in the App Store today for only $9.99.

Pixar's Burn-E

2008-11-13T14:50:02Z

Pixar presents the adventures of Burn-E, a robot contemporary of Wall-E.

The events in Burn-E's short film take place concurrent with those in the feature film.

Update: YouTube just took the video down at Pixar's request. If you missed it, check it out here. (thx, jose)

(link)

Scala as a Scripting Language?

2008-11-03T08:00:31Z

I know, the title seems a bit…bizarre. I don’t know about you, but when I think of Scala, I think of many of the same uses to which I apply Java. Scala is firmly entrenched in my mind as a static, mid-level language highly applicable to things like large-scale applications and non-trivial architectures, but less so for tasks like file processing and system maintenance. However, as I have been discovering, Scala is also extremely well suited to basic scripting tasks, things that I would normally solve in a dynamic language like Ruby.

One particular task which I came across quite recently was the parsing of language files into bloom filters, which were then stored on disk. To me, this sounds like a perfect application for a scripting language. It’s fairly simple, self-contained, involves a moderate degree of file processing, and should be designed, coded and then discarded as quickly as possible. Dynamic languages have a tendency to produce working designs much faster than static ones, and given the fact that the use-case required access to a library written in Scala, JRuby seemed like the obvious choice (Groovy would have been a fine choice as well, but I’m more familiar with Ruby). The result looked something like this:

require 'scala'
 
import com.codecommit.collection.BloomSet
 
import java.io.BufferedOutputStream
import java.io.FileOutputStream
 
WIDTH = 2000000
 
def compute_k(lines, width)
  # ...
end
 
def compute_m(lines)
  #...
end
 
Dir.foreach 'wordlists' do |fname|
  unless File.directory? fname
    count = 0
    File.new "wordlists/#{fname}" do |file|
      file.each { |line| count += 1 }
    end
 
    optimal_m = compute_m(count)
    optimal_k = compute_k(count, WIDTH)
 
    set = BloomSet.new(optimal_m, optimal_k)
 
    File.new "wordlists/#{fname}" do |fname|
      file.each do |line|
        set += line.strip
      end
    end
 
    os = BufferedOutputStream.new FileOutputStream.new("gen/#{fname}")
    set.store os
    os.close
  end
end

As far as scripts go, this one isn’t too bad. I’ve written some real whoppers for things like video encoding and incremental backups. The main trick here is the fact that we need to make two separate passes over the same file in order to get the number of lines before constructing the set. We could load the file into an array buffer in a single pass, count its length and then iterate over the array, placing each element in the bloom filter. However, this really wouldn’t be too much faster than just hitting the file twice (we still need two separate passes) and it has the additional drawback of requiring a fair amount of memory.

All in all, this script is a fairly natural representation of my requirements. I needed to loop over a number of word lists, push the results into separate bloom filters and then freeze-dry the state. However, look at what we’ve actually done here. Remember earlier where we were considering which language to use? We wanted a language which could concisely and quickly express our intent. For that decision making process, we just assumed that a dynamic language would suffice better than one hampered by a static type system. However, at no point in the above script do we actually do anything truely dynamic. By that I mean: open classes, unfixed parameter types, method_missing, that sort of thing. In fact, we haven’t really done anything that we couldn’t do in Scala:

import com.codecommit.collection.BloomSet
import java.io.{BufferedOutputStream, File, FileOutputStream}
import scala.io.Source
 
val WIDTH = 2000000
 
def computeK(lines: Int, width: Int) = // ...
 
def computeM(lines: Double) = // ...
 
for (file <- new File("wordlists").listFiles) {
  if (!file.isDirectory) {
    val src = Source.fromFile(file)
    val count = src.getLines.foldLeft(0) { (i, line) => i + 1 }
 
    val optimalM = computeM(count)
    val optimalK = computeK(count, optimalM)
 
    val init = new BloomSet[String](optimalM, optimalK)
 
    val set = src.reset.getLines.foldLeft(init) { _ + _.trim }
 
    val os = new BufferedOutputStream(new FileOutputStream("gen/" + file.getName))
    set.store(os)
    os.close()
  }
}

This is actually runnable Scala. I’m not omitting boiler-plate or cheating in any similar respect. If you copy this code into a .scala file and make sure that BloomSet is on your CLASSPATH (which you would have needed anyway for JRuby), you would be able to run the script uncompiled using the scala command. Unlike Java, Scala actually includes an “interpreter” which can parse raw Scala sources and execute the representative program just as if it had been pre-compiled using scalac. One of the perquisites of this approach is the ability to simply omit any main method or Application class. In nearly every sense of the word, Scala is a scripting language…as well as an enterprise-ready Java-killer (let the flames begin).

Now that we’re fairly convinced that the above is valid Scala, let’s compare it with the original version of the script written using JRuby. If we just go off LoC (Lines of Code), Scala actually wins here. This was a more-than-slightly surprising discovery for me, given how often dynamic languages (and Ruby in particular) are touted as being more concise and expressive than static languages. But of course, sheer LoC-brevity isn’t everything: we also should consider things like readability. A few characters of Befunge can accomplish more than I can do in several lines of Scala, but that doesn’t mean I’ll be able to figure out what it means tomorrow morning.

On the readability score, I think Scala wins here too. The file processing and set creation is all done in a highly functional style (using foldLeft). At least to my eyes, this is a lot easier to follow than the imperative form in Ruby. More importantly, I think it’s a bit harder to make silly mistakes. When I wrote the Ruby version of the script, it took several tries before I solidly pinned down the exact incantation I was seeking. The Scala version literally required only one revision after the initial prototype. Granted, I had the Ruby version to go off of, but I think we would all agree that the scripts use some fairly different libraries and methodologies for accomplishing identical tasks.

So what is it that makes Scala so surprisingly well suited to the task of quick-and-dirty file processing and scripting? After all, isn’t is just a fancy syntax wrapping around the plain-old-Java standard library? While it is true that Scala has first-class access to Java libraries (as demonstrated in the script), that isn’t all that it offers. I believe that Scala has two important features which make it so suitable for these tasks:

Type inference
Powerful core libraries

The first feature is of course evident wherever you look in the script. With the exception of the two methods and the BloomSet constructor, we never actually declare a type anywhere in the script. This gives the whole thing a very “dynamic feel” without actually sacrificing static type safety. The first time you try this sort of language feature it is an almost euphoric experience (especially coming from highly-verbose languages like Java).

The second feature is a bit harder to see. It is most evident in the way in which we handle file IO. The directory listing is of course yet another application of the venerable java.io.File class, but the process of opening and reading the file line-by-line seems to be a lot easier than anything Java can muster. This is made possible by Scala’s Source API. Rather than fiddling with BufferedReader and the whole menagerie that goes along with it, we just get a new Source from a File instance and then use conventional Scala methods to iterate over its contents. In fact, we’re actually applying a functional idiom (fold) rather than a standard imperative iteration. Finally, when we’re done with our first pass, we don’t need to re-open the file from scratch (inviting initialization mistakes in our coding), we just reset the Source and start from the beginning once more.

Using Scala as a scripting language comes with some pretty hefty benefits. For one thing, you get immediate and idiomatic access to the mighty wealth of libraries which exist in Java. Even for scripting, this sort of interoperability is invaluable. JRuby does provide some excellent Java interop, but it simply can’t compare to what you get with Scala. Further, Scala has a static type system to check you (at runtime with a script) to ensure that you haven’t done anything obviously bone-headed. This too is nothing to sniff at.

Given the fact that Scala’s “scripting syntax” is just as concise as Ruby’s (sometimes more), it’s hard to see a reason not to employ it for around-the-server tasks. Amusingly, the most compelling reason not to use Scala for scripting just might be its comment syntax. Not having direct support for the magic “hash bang” (#!) incantation to define a file interpreter just means that Scala scripts have to go through some extra steps to be directly executable. However, if immediately-executable scripts aren’t an issue, you may want to consider Scala as your scripting language of choice for your next non-trivial outing. You may reap the rewards in ways you weren’t even expecting.

Adv. JavaScript and Processing.js

2008-09-22T18:53:14Z

Recently I gave two talks at the Web 2.0 Expo in New York City and one for the Boston IxDA.

Learning Advanced JavaScript

An advanced talk on the JavaScript language. Explored functions, closures, function prototypes, and inheritance. The entire presentation was given using an interactive site/presentation (tested in Firefox and Safari).

Feel free to browse through the presentation (I'm not sure how useful it will be without me talking about the particulars - but it may be nice).

There are a number of neat things that I like about the implementation of this talk:

It's interactive. Each code slide is executable (the user can see the output right away). Additionally each slide is editable - just double-click the code to go into an edit mode.
Code editing is simple. Basic IDE functions (auto-indentation, proper tabbing, and backspace-to-delete-tab) are included. It's not a ton but it's enough to get started.
All code slides include syntax hilighting.
All slides are bookmarkable.

The presentation includes a number fill-in-the-blank quizzes to help test your knowledge of what you just learned. In practice I may save this for situations in which more people have laptops/computers at the talk.

You can download the full presentation as a zip file.

Building a Visualization Language

I gave a talk on my work with Processing.js, together with covering how the Canvas element works and the Processing language itself.

During the talk I stepped through the construction of a visualization using Canvas:

jQuery for the Boston IxDA

An introductory presentation explaining how jQuery works.

The meat of the presentation was a series of interactive slides which could be run and played with in order to better understand how jQuery works.

You can download the runnable code as a zip file.

Upcoming

I'm going to be giving a number of talks this weekend at the jQuery Conference followed by The Ajax Experience, I'll be sure to post the slides and code from them, as well.

Bonus

Last week was the MIT Career Fair - I stopped by and worked the Mozilla booth with Boris, Brad, and Julie - a good time as usual:

Great Presentation From Saul Klein

2008-10-25T22:58:12Z

Thoughts on European Start Ups

View SlideShare presentation or Upload your own. (tags: startups vc)

Thanks to Andy at Betaworks for showcasing it on their blog.

Three ways to know when a MySQL slave is about to start lagging

2008-10-09T00:56:06Z

The trouble with slave lag is that you often can't see it coming. Especially if the slave's load is pretty uniform, a slave that's at 90% of its capacity to keep up with the master can be indistinguishable from one that's at 5% of its capacity.

So how can you tell when your slave is nearing its capacity to keep up with the master? Here are three ways:

One: watch for spikes of lag. If you have Cacti (and these Cacti templates for MySQL) you can see this in the graphs. If the graphs start to get a little bumpy, you can assume that the iceberg is floating higher and higher in the water, so to speak. (Hopefully that's not too strange a metaphor.) As the slave's routine work gets closer and closer to its capacity, you'll see these spikes get bigger and "wider". The front-side of the spike will always be less than a 45-degree angle in ordinary operation[1] but the back-side, when the slave is catching up after lagging behind, will become a gentler and gentler slope.

Two: deliberately make a slave fall behind, then see how fast it can catch up. This is sort of related to Method One. The goal here is to explicitly see how steep the backside of that slope is. If you stop a slave for an hour, then start it again and it catches up in one hour, it is running at 1/2 of its capacity. (In case that's confusing: if you stop it at noon and restart it at 1:00, and it's caught up again at 2:00, it played all statements from 12:00 to 2:00 in 1 hour, so it went at 2x speed.)

Three: measure it more scientifically. Use our patched server, which gives you a USER_STATISTICS table.

PLAIN TEXT

SQL:

mysql> SELECT * FROM INFORMATION_SCHEMA.USER_STATISTICS WHERE USER='#mysql_system#'\G
*************************** 1. row ***************************
USER: #mysql_system#
TOTAL_CONNECTIONS: 1
CONCURRENT_CONNECTIONS: 2
CONNECTED_TIME: 46188
BUSY_TIME: 719
ROWS_FETCHED: 0
ROWS_UPDATED: 1882292
SELECT_COMMANDS: 0
UPDATE_COMMANDS: 580431
OTHER_COMMANDS: 338857
COMMIT_TRANSACTIONS: 1016571
ROLLBACK_TRANSACTIONS: 0

You can compare the BUSY_TIME to one-half the CONNECTED_TIME (because there are two replication threads on the slave) to see how much of the time the slave thread was actively processing statements. If the slave threads are always running, you can just use the server's uptime instead.

[1] There are cases where this isn't true, especially if you're monitoring Seconds_behind_master instead of using mk-heartbeat, which is immune to this anomaly.

Entry posted by Baron Schwartz | 8 comments

Add to: | | | |

Percona’s patches spread to a wider audience

2008-10-14T03:50:00Z

Percona's patches are now available to a wider audience via OurDelta, a community effort to provide builds with features (Percona patches, Google patches, etc) and storage engines (PBXT, Sphinx, etc) that aren't in the main MySQL server. Arjen Lentz is really the brainchild behind this. Kudos Arjen!

What does this mean for the Percona patches? Well, now you can get them in more places. But it doesn't change our own commitment to keep innovating in ways our customers (and we ourselves) find useful. We're still building our own builds and hosting the downloads from our own website; it's important that you can get a build that you know authoritatively came from us and was built by us. The Percona patches and builds represent our best judgment about what's the state of the art in the MySQL server for real-world usage, and we have no plans to change that.

The OurDelta project is a great way to get pre-built versions of the server with cutting-edge features like PBXT. This should help lower the barriers to trying these things out. The Percona builds and patches are conservative (except for the -highperf builds, which are experimental) and won't include such features, at least not for the foreseeable future. We're really glad that others are adopting our patches and blending them with features our builds won't include. OurDelta also has a build infrastructure behind it, which is quite a nice thing to have, and we expect that bug reports and patches may even flow "upstream" back to us.

Of course the project is very new, and many changes will undoubtedly happen in the weeks and months to come. But you have to start somewhere, and it's really nice to see such an initiative.

Entry posted by Baron Schwartz | 7 comments

Add to: | | | |

● Google Chrome, Google Browser

2008-09-02T01:58:15Z

Google Browser, 2001.

What if Google built something that was very much like a browser but was mainly used for searching for information. What if they built a tool that was focused on searching for answers to your questions first, and looking at web pages second. Wrap your head around that. You have search needs. You also have unique search patterns. You have ways of looking for information that are very interesting and personal. Where are the tools that help you search? You are probably thinking of search engines, like Google. But search engines are server based. Why not bring the power of the server to the desktop? There are some tools out there like this, but they aren't complete. They also don't have the usability and brand recognition of Google.

Google Browser, 2003.

So, a Google browser, based on Mozilla. An easily-justified commitment to cross-platform support and outstanding user experience, based on Google's history of honoring those tenets and the Mozilla organization's inherent preference for them. Culturally, hiring the core members of the Mozilla dev team would be an extraordinarily easy fit. And, frankly, it'd probably require little more development resources, bandwidth, or staffing than the Pyra acquisition did.

Google Browser, 2004.

Google could use their JavaScript expertise (in the form of Gmail ubercoder Chris Wetherell) to build Mozilla applications. Built-in blogging tools. Built-in Gmail tools. Built-in search tools. A search pane that watches what you're browsing and suggests related pages and search queries or watches what you're blogging and suggests related pages, news items, or emails you've written. Google Toolbar++. You get the idea.

Google Browser, 2008.

All of us at Google spend much of our time working inside a browser. We search, chat, email and collaborate in a browser. And in our spare time, we shop, bank, read news and keep in touch with friends -- all using a browser. Because we spend so much time online, we began seriously thinking about what kind of browser could exist if we started from scratch and built on the best elements out there. We realized that the web had evolved from mainly simple text pages to rich, interactive applications and that we needed to completely rethink the browser. What we really needed was not just a browser, but also a modern platform for web pages and applications, and that's what we set out to build.

More on this tomorrow if motivation allows.

Welcome to Chromium

2008-09-02T20:02:43Z

Shared by Daniel Shaw
Chrome is going to be fantastic! I can't wait for it to run natively on a Mac.

Today, Google launched a new web browser called Google Chrome. At the same time, we are releasing all of the code as open source under a permissive BSD license. The open source project is called Chromium - after the metal used to make chrome.

Why did Google release the source code?

Primarily it's because one of the fundamental goals of the Chromium project is to help drive the web forward. Open source projects like Firefox and WebKit have led the way in defining the next generation of web technologies and standards, and we felt the best way we could help was to follow suit, and be as open as we could. To be clear, improving the web in this way also has some clear benefits for us as a company. With a richer set of APIs we can build more interesting apps allowing people to do more online. The more people do online, the more they can use our services. At any rate, we have worked on this project by ourselves for long enough - it's time for us to engage with the wider web community so that we can move on to the next set of challenges.

We believe that open source works not only because it allows people to join us and improve our products, but also (and more importantly) because it means other projects are able to use the code we've developed. Where we've developed innovative new technology, we hope that other projects can use it to make their products better, just as we've been able to adopt code from other open source projects to make our product better.

How will we be working with the open source community?

To begin with, we are engaging with the WebKit community to integrate our patches back into the main line of WebKit development. Because of Chromium's unique multi-process architecture, the integration of the V8 JavaScript engine, and other factors, we've built a fairly significant port of WebKit on Windows, and are developing the same for Mac OS X and Linux. We want to make sure that we can find a productive way to integrate and sync up with the WebKit community in this effort as we move forward.

Today, you can visit our project website at www.chromium.org, where you can get the latest source code or the freshest development build. If you're interested in keeping track of what's going on, you can join one of our discussion groups, where you can participate in development discussions and keep track of bugs as they're filed and fixed. Maybe you'll want to fix a few, too! You'll also find information on reporting bugs and all the various other aspects of the project. We hope you'll check it out!

This is the Chromium blog. The posts here will be of a mostly technical nature, discussing the design theory and implementation details of work we've done or are doing. Over the next few weeks there'll be a number of posts that give a high level tour of the most important aspects of the browser.

Finally, if you've not yet done so, take Google Chrome for a spin. You can download it from http://www.google.com/chrome/.

Posted by Ben Goodger, Software Engineer