The Web and all that Jazz

Practice coding faster

2009-12-23T01:17:00.001-08:00

map_with_index. Why is there no map_with_index in ruby? Ends up it's because you don't need it. You can simply do this:

Even now, I'm learning things about Ruby. The rabbit hole is deep.

Anyway, a post about learning how to code fast came across my desk. I knew that I wasn't quite as fast as other coders, but I had always thought that I thought deeper on the solution. But I think what he says makes sense. I know that violinists slowly ramp up their speed to a point where they're almost making mistakes, faster than they'd actually play the piece to practice playing it at the correct speed. Same with drawing. The more you practice drawing faster, the better you get at being economic with your strokes. So I figured I'd try the same with programming, since I've never done much of this type of exercise.

I decided to do the first Ruby Quiz, since I was most familiar with Ruby, and I should be able to do it quickly. It took me about three hours, including reading the instructions, going on bathroom breaks, etc. I think I should have been faster, and I noticed where I slowed down. I found myself trying things out in irb a lot because I didn't know the exact behavior of some array and string functions. Also, I spent some time in the beginning pondering how to structure it--should it be a class, or just a collection of functions, or should I extend the classes?

I'm not thrilled about how it's structured, but it works. Well, there's a small bug in there, but I'm going to refrain from fixing it. It'll tack on another 30 mins. The point of the exercise is that I can see what I need to work on. I'll try again next time with the next ruby quiz.

Posted via email from The Web and all that Jazz

On to the new old thing

2009-12-21T09:21:00.001-08:00

Ever since I quit my job at the lab 4 years ago to pursue a startup, I'd been fumbling around learning all sorts of things. And though I had determination and persistence, I simply didn't know a lot of things outside of coding. Who wants this? Why would they pay money for it? Where do you find them? How do you change your idea if it doesn't work? Going into it, I knew I didn't know anything and that I'd learn, but I also didn't know what I didn't know.

For the past year and a half, I've been with a startup that went into the YC program. So as much as I wanted to pursue my own ideas, I was advised to join up and watch other people and learn. And learn a lot I did. What to do, and what not to do. And just seeing founder from other startups helped. What their thought processes and attitudes about their line was work was. Meeting role models are easier than reading about them, I guess.

And yet, while I've been learning about things outside of code, I felt like I've been sailing downwind when it came to technical things. Sure, I'd mess around with chickens flocking (which embarrassingly, I haven't gotten back to), but for the most part, I was consumed by work. Getting the tickets done and getting better at communicating with other team members.

However, much of my creativity was sapped. It was hard to fire up the editor afterwards and explore something new. I have a whole list of things I wanted to dive into more deeply. Haskell's type system. Erlang servers. Spatial trees for Frock. Arc language. Potion language. Prof Strang's Linear Algebra lectures. Visualizations and info graphics. Mobile web apps. 3D printers. And though I've dabbled in all of the above, it's not yet been enough to satiate my cravings.

But I have learned that it's also a lesson in keeping things small and simple at first. Much kudos to those working a full time job and are able to get a side project up and running.

This blog also suffered as a result. I was afraid that the things I was learning outside of code might reveal too much about the internals of the startup I was with, so I just left nothing to chance. I simply fell out of the habit of blogging about what I've learned, and as a result I feel like my writing skills have deteriorated.

But now I've left the startup, moved to Mountain View, and I'm pursuing my own once again. It's not a secret what I'm working on, but I'd just rather talk about it in a separate post. I've also started exploring other technical topics as well, as I hinted in the last post about Potion. I'll start finishing up the backlogged technical posts.

So for those of you that still are subscribed, well, thanks for your faith. I'll be writing more, and I hope you'll be able to learn something from reading this blog as well. Have a great holiday!

Posted via web from The Web and all that Jazz

Playing with potion

2009-12-19T11:10:00.001-08:00

For fun this morning, I cloned _why's potion and started going through the tutorial. After cloning it, I was able to compile it on my mac, and then I tried to run:

"hello world" print

and it seg faulted. Boo. So in case any of you are looking for a work around, this worked:

1 times:
  "hello world!\n" print
.

And looking through the source, I found the about function's easter eggs. Humorous.

1 times:
  about("_why") print
  about("stage fright) print
.

Posted via email from The Web and all that Jazz

jQuery live events can only bind once

2009-11-13T17:33:00.001-08:00

I've been doing some more javascript on the side as of late, and I ran into a snag with jQuery. It was an odd case of live events that didn't seem to be taking hold. In addition, it only looked like it was happening on Chrome. As always, it helps to read the docs.

Unlike .bind(), only a single event can be bound in each call to the .live() method.

http://docs.jquery.com/Events/live#typefn

That means that my overlapping live binding were getting overwritten, and that Firefox and Chrome were merely adding the events in different orders. If it's not apparently in the docs, start looking in the code. As an aside, I've heard that the JQuery code is a good example of great javascript code. I don't understand a lot of what I saw. Well, tip. Been working on some other things on the side, and will reveal them in due time.

Posted via email from The Web and all that Jazz

Tarsnap docs as an example of confusing typography

2009-11-04T08:04:00.001-08:00

Because in 1960, the Bureau International des Poids et Mesures decided that the SI prefix G- meant 10^9.

But it means 2^30, really!

No it doesn't. Let's look at some examples:

via tarsnap.com

Just a quick thought. It's pretty basic, but this was the first time I had a front-row seat demonstration of basic design principles and why they're suggested.

You can't really see it in the quotes, but if you follow the link, you'll see that the headings and text are the same size. Not only that, but the spacing between paragraphs is the same between headings and paragraphs.

That confused me and I had thought that the bolded statements was part of the text, and hence it was something the author was saying, rather than as intentioned, something we are readers would be saying as headings of different sections.

We differentiate importance and grouping by weight, size, color, and spacing. It takes a combination of these to discern what we're reading.

Posted via web from The Web and all that Jazz

You probably don't need an OLAP

2009-10-29T18:54:00.001-07:00

It's a "well known" that relational databases are bad at multi-column "slice and dice" calculations. So when you have data that you'd like to represent as an aggregated trend, it's easy to reach for that OLAP. Chances are, you don't need it. Here's an example of something where you want a count of the number of comments from an author by day.

select DATE(created_at) as DateOnly, count(*)
from comments
where author_id = 877081418
group by DateOnly
order by DateOnly

The trick here is the DATE() function provided by various database vendors. This returns any datetime as simply a date that can be aggregated.

To be honest, I looked into this way too late and didn't contest the OLAP architectural decision until it was late. We ended up having the legacy of dragging a big fat OLAP with all its trappings of complicating our architecture. If you end up with a complex architecture, there's probably a simpler way you're not seeing. The simpler your setup, the easier it will be for you to hold it all in your head and understand it when things go wrong.

The only OLAPs we've found to be available was the open source Mondrian and Microsoft's Analysis Services. To be honest, I found both to be way harder to use than it should have been. If someone else wants to write another OLAP that's simpler to use without a lot of luggage to blow those two out of the water, the time is nigh.

Posted via email from The Web and all that Jazz

Scope in JavaScript is just from which door you entered

2009-10-08T16:35:00.001-07:00

Put simply, we entered BigComputer via new, so this meant “the new object.” On the other hand, we entered the_question via deep_thought, so while we’re executing that method, this means “whatever deep_thought refers to”. this is not read from the scope chain as other variables are, but instead is reset on a context by context basis.

via digital-web.com

Javascript's scoping has been one of most confusing things about it, just as Ruby's metaclass and object model is the most confusing things about it. If you're looking to expand the horizon of what you understand about programming languages, it's worth it to figure out javascript scoping.

The paragraph gave a good way to think about it: this changes based on the object that calls the method. It only gets confusing when you start passing around functions and using callbacks, which is most of the power of functional programming.

As an example, here, I was using an anonymous function as a callback in the request() method. But it doesn't work!

So that's just one way to solve it. If you're using Prototype, you can also try using the bind() method. jQuery doesn't have an equivalent bind method, as hard as I looked for it at one time. I was just about to write it myself (as it's not too hard), but according to the a list apart article on Getting out of binding situations in javascript:

jQuery does not provide such a binding facility. The library’s philosophy favors closures over binding and forces users to jump through hoops (that is, manually combine lexical closures and apply or call, much as other libraries do internally) when they actually need to pass along a piece of code referring to “instance members.”

So while I use closures extensively in Ruby, I haven't had to explicitly think about the scope until I was using closures in Javascript. Huzzah. Hopefully, it'll prompt you to take a deeper look at Javascript.

Posted via web from The Web and all that Jazz

Concurrency and integrity with validates_uniqueness_of

2009-10-07T08:54:00.001-07:00

Here's another tidbit that I hadn't noticed in the rails docs before. I was looking at validations for uniqueness and I saw this:

Using this validation method in conjunction with ActiveRecord::Base#save does not guarantee the absence of duplicate record insertions, because uniqueness checks on the application level are inherently prone to race conditions.

And the docs also offer some solutions:

This could even happen if you use transactions with the ‘serializable’ isolation level. There are several ways to get around this problem:

By locking the database table before validating, and unlocking it after saving. However, table locking is very expensive, and thus not recommended.

By locking a lock file before validating, and unlocking it after saving. This does not work if you‘ve scaled your Rails application across multiple web servers (because they cannot share lock files, or cannot do that efficiently), and thus not recommended.

Creating a unique index on the field, by using ActiveRecord::ConnectionAdapters::SchemaStatements#add_index. In the rare case that a race condition occurs, the database will guarantee the field‘s uniqueness.

This typically isn't something you'd need to worry about until you get to some traffic of scale and size. So don't worry about it too much until you get there, but be aware of the problem. Read the docs for more details and information. tip!

Posted via email from The Web and all that Jazz

Amazon S3 and Paperclip plugin

2009-10-06T12:50:00.001-07:00

Even after reading all the documentation, paperclip still has its quirks. I've been pretty busy, but here's a short tip to tide you over. When using paperclip with S3, make sure that you have the :path option set when using has_attached_file.

It didn't take too long to figure out, but just in case, make sure bucket option is set either in the has_attached_file declaration or your s3 config file pointed to by :s3_credentials option. Otherwise, you'll get a mysterious

"MethodNotAllowed: The specified method is not allowed against this resource." Error.

So head on over to Scott Mottes and learn it step by step. tip.

Posted via email from The Web and all that Jazz

If only

2009-09-22T06:41:00.001-07:00

"If only I had ____ I would succeed."

These simple words will kill your dreams faster than anything else you could say or think. There are so many self-defeating thoughts that an entrepreneur can have, and they often take this very simple form.

Posted 3 hours ago

via garry.posterous.com

While Garry takes it in the direction of getting your hands dirty and building, and the recent HN discussion talking about whether one should sell or not, reading these compels me to take it in a different direction this morning before work--I'd like to speak a little about mental blocks.

There were many reasons why you'd want to sell your company. Your business deals with fads and the market will go away. You're done with this thing and want to move on. But there is a bad reason I want to focus on: "it'll give me freedom to do what I want". I think when people say this, they mean two different things: 1) if I have lots of money to take care of life's annoyances like bills and college tuition, then my mind will be free to work on anything 2) if I have lots of money, I can fund whatever I want to work on. The latter, I find to be an unconvincing reason.

My dad is retired. He talks about starting a foundation to help education in Taiwan, and seems rather passionate about it. He spends a lot of time watching and reading Taiwanese news. Given a chance, he'll talk your ear off about it. However, he says, "if only I had a million dollars", he could start his foundation. And the way he usually thinks of getting the million dollars is through the lotto. Now, my dad is no fool. He knows the odds. And I don't know if it's a generation gap in the way jokes are told, but if he's serious, it's a mental block that I see in some friends also. It's an excuse to do nothing because of the preceived notion that the external world hasn't given you permission.

By contrast, a couple years back Oprah had some special on TV about a new school she was building in South Africa. Though she put in a hefty sum, I was surprised to find out that she didn't put in all the money herself. She had other people help her with donations. That's why she had Nelson Mendela, Maria Carey, and others visit the school--to help donate. Even when she could pay for it all herself, she enlisted other people to help. In a more recent example, Breadpig and xkcd joined forces to put a school in Laos. They're putting in the work, yes, but as far as I can tell, it's none of their personal money.

Just because something takes a million dollars to do, doesn't mean it has to be your million.

Perhaps this is obvious to some of you, but I was a little bit surprised when I realized this. Growing up, I never thought about it too much, because in movies like Batman, Bruce Wayne funded his own crazy toys. So I naturally assumed that if you want to do huge things, you do it all with your own money. As a kid, I thought: If I wanted to build a Mechwarrior, I'd have to do it with my own money. If I wanted to build a loop-de-loop highway, I'd have to do it with my own money. If I wanted to build a giant chicken slingshot, I'd have to do with with my own money.

Of course, this comes with some amount of responsibility and constraint. Pissing away other peoples' millions is a sure way to get your legs broken, especially with money from a loan shark (or its million dollar equivalent). But I believe constraint in business and philantropy, as constraint in design, is a good thing to focus your efforts. Sometimes, personal money projects fail because they're not as readily subjected to market forces. A bad idea is kept afloat because there's a huge chunk of personal money that keeps getting dumped into it.

In the end, I just want to say, you have a choice. Don't let a little thing like not having a couple million stop you from doing what you want to do, as there's always more than one way to skin a cat. But if you want to build a pyramid for your burial site, then yes, please do that with your own money.

Posted via web from The Web and all that Jazz

rake task with arguments - Ruby Forum

2009-09-13T07:24:00.001-07:00

Of course csh is evil! That's nothing new. http://ooblick.com/text/CshProgrammingConsideredHarmful.html This works just fine with bash:
rab://tmp $ cat Rakefile  namespace :foo do  desc 'lol'  task :bar, :num do |t, args|  puts "num = #{args.num}"  end  end  rab://tmp $ rake foo:bar[123]  (in /private/tmp)  num = 123  

via ruby-forum.com

Hey look. Arguments in Rake. I've been looking for this for a while now. No more using env variables.

Posted via web from The Web and all that Jazz

Another bad data visualization

2009-09-12T07:33:00.001-07:00

(click image to enlarge)

via permuto.com

This is one of the worst data visualizations I've seen. Problem is it looks pretty, so people send it around, but it's not very informational. Nor does it allow easy comparison of the data. First, it's not apparent that the light green and the dark green sections are the same thing until you realize it's an "O" from "Google", and actually adds no information. Second, what do the size of the circles represent? Is it combined daily spending or average daily spending per advertiser? It takes a while to find the circumferencial text, which you'd guess that it represents the amount of revenue from top N advertisers. Then the chart also mixes terminology. While spending by advertiser and revenue by google are the same thing, you need to do extra work to figure that out. Then, what the heck, the list of logos on the side is distracting. It's suppose to be the advertisers in the blue circle--the top 10 advertisers--but it sits firmly in the red section, which is the long tail of advertisers. Even more confusing, the $59,184,783 is red, but points to the blue list of logos. Lastly, the average daily spending is colored with the same position and weight as the combined daily spending, but it doesn't represent the size of the circles, which adds even more confusion.

The only thing they did right was to match the size of the circles with the amount of combined daily spending. Often times, people will draw these sorts of graphs using the diameter as the basis for comparison, which is misleading.

Posted via web from The Web and all that Jazz

Using HTML attributes as a mini-DSL for AJAX

2009-09-08T17:38:00.001-07:00

At first, I thought being able to return json as responses from the server was pretty neat, with rjs (now js.erb) files using render :update call in Rails. However, this often lead to some messy code by me and my colleagues. I would see lots of client side code in the application controller like:

Sometimes, this is ok, but when html elements change, the controller methods break, and the effect cascades from the views through the controllers. That's a code smell that our code is tightly coupled. Of course, we can refactor it to a separate .rjs template file, and justify it to ourselves that it's the same as having an .html.erb file. However, because .rjs files are often so short, the cognitive shift to find that other file is often distracting, and it still doesn't solve the problem of code coupling.

Consider the following: You want to have a link that makes an AJAX GET request to get a list of comments for the post when clicked. It gets the response as HTML and then dumps it in the target DOM. You also want to fade in an indicator feedback when the ajax is loading and fade it out when it's finished loading. Lastly, there's to be a slidedown effect on the target comments DOM element.

For the situation above, if we were to do it with render :update or with rjs files, you'd have code that is coupled with DOM elements as I said earlier. What if we can contain it all on the client side, and leave the server side for business logic?

One way is to use all the options that link_to_remote provides. This way, all the UI effects are contained on the client side when you have an AJAX GET request. We can now keep all our UI effects code in the views.

However, what if we can declare this effect in HTML? Would that work? What are the advantages and disadvantages?

I went to the local Ruby meetup, and remembered that Ben Johnson of Authlogic was mentioning something about "data-" attributes in HTML5 and its relation to the problem I described above. I wasn't entirely sure what he was talking about, until I looked around for the "data- attribute" and found good ole Resig blogging about it (last year, no less).

What this allows us to do is basically insert data into our HTML elements. And because jQuery events let us separate the "how" in javascript from the "what" in html, we can declaratively use it as a mini-DSL of sorts.

The basics are pretty easy to implement. I didn't do the data-indicator and the data-effect because I'm lazy and it's left to the reader "as an exercise".

Note that I'm proposing it to be a mini-DSL, so that very common AJAX idioms are covered, and you can simply declare things in HTML and not have to go into the javascript often, if at all. That way, you keep working in the same file, working on the same level of abstraction.

There are some advantages to doing it this way.

Server response can be faster, since we wouldn't have to rely on the server to generate the proper html link. The UI effects and behavior can be all done on the client side, where it should be.
I think it's a bit cleaner to be able to say things declaratively, more aligned with how html is declarative.
The DOM elements wouldn't couple the controller and the view. Everything that refers to DOM elements would be in the view and be easier to change without cascading effects.
You can work in the same level of abstraction while in the HTML views and won't have to jump between layers of abstraction.

There are few disadvantages I can see right now, other than not having the correct DSL, or having a method call that uses too many attributes, making the HTML hard to read. (If you have others, comment below)

Pretty neat. So why use the class attribute as the method "call"? Perhaps it's better to find some other attribute. I've seen facebox use the rel attribute instead. That lead me wonder if other people have thought to do this before. And of course, there's something similar called AHAH microformat, based on JAH from 2005, which is demoed here.

JAH does something notable, in that it uses the form:

to order to execute the javascript, instead of binding an event to the DOM. This removes the extraneous href="#" in the other way I showed you above (more succinct), but it breaks the declarative nature, and cannot be unbound and binded with something else easily--one would have to change all instances it's called, instead of re-adjusting the DOM selector element (as rare as that may be). I personally don't think it's as easy to read, especially when the method call has parameters, but the implementation would be shorter.

These techniques seem to be by no means widespread at the moment, but one of the contributors is DHH of the Rails fame, so I expect to see it in Rails soon. So be on the lookout for something similar in the future Rails. It seems like the technique was talked about back in 2005, but never fully incorporated or used. I have no idea why. In the mean time, I intend to incorporate it into new code I write.

(Apologies if there are typos above. It's late, and I haven't had my ramen)

Posted via email from The Web and all that Jazz

Getting all attributes of a DOM element in Javascript/jQuery

2009-09-07T20:33:00.001-07:00

Sometimes, you need to iterate over a number of jQuery elements. You pull something like this:

$.each($(".hello"), function() {
  alert(this);
});

In this context, the "this" variable is actually not the jQuery objects. According to the docs:

Whenever you use jQuery's each-method, the context of your callback is set to a DOM element. That is also the case for event handlers.

via docs.jquery.com

This is helpful when you want all the attributes of a particular DOM element. You can call attributes property on the "this" variable (this.attributes) inside of the each() method to get all attributes of each element with the class hello.

Lastly, if you have a jQuery element, you can get all attributes by:

$("#some_dom")[0].attributes

Posted via web from The Web and all that Jazz

Testing Named Routes in the Rails Console

2009-09-03T14:41:00.001-07:00

I finally found out how to do this, from the Rails Routing shortcut by David Black. In the Rails console, do this:
include ActionController::UrlWriter  default_url_options[:host] = 'whatever'  
Then you can call your named route methods directly from the console.

This entry was posted on Tuesday, January 8th, 2008 at 1:58 pm and is filed under Programming, Ruby.You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

via stuartsierra.com

I've always just worked around this by trying it out in the templates. Should keep looking things up on google. I learn much more that way. Anyway, thought the rest of you should know also.

Posted via web from The Web and all that Jazz

A tiny empty shortcut

2009-09-03T05:16:00.001-07:00

I think I've written about this before, by monkeypatching an empty method on the Array class that takes a block and executes it only if the array is empty. But anyway, for some of you, you might see something like this taking place often:

To clean it up a little, you can do:

Or else if each of your items is a partial, use the :collection method.

Posted via email from The Web and all that Jazz

Expected delayed_job.rb to define DelayedJob

2009-09-02T14:25:00.001-07:00

Recently, I needed to do some background processing. I just needed something simple, and looking at all various options, I decided to go with Bj. It was fairly simple to understand, and best of all, didn't require another daemon to be running by hand--it started one itself if you didn't configure it.

However, it's incompatible with SQLServer. I'm guessing no one's every used it with SQLServer before, since I didn't read anything about it in my research. Bj uses a column called "key" which SQLServer reserves as a future keyword, and thus, automatically changes the name of the column to "[key]" So consider yourself forewarned.

Thus, I decided to switch to something similarly simple, and I started using Delayed Jobs. It's all fine, except in some instances, I ran into the following error:

LoadError: Expected /[RAILS_ROOT]/vendor/plugins/delayed_job/lib/delayed_job.rb to define DelayedJob

After reading around on the web for a while, it seemed like any number of things could cause this error. I finally found way down in this rails ticket from a while back, had the lines that explained it.

The critical comment at that link is: "Prior to this revision, Rails would happily load files from Ruby’s standard lib via const_missing; you will now need to explicitly require such files."

The rest of the comments also talked about various causes. It's the case of different problems having the same symptom, and here, the backtrace isn't pointing to where the problem is.

In the case of Delayed Job, Rails expects delayed_job.rb to define a module or class named DelayedJob. However, that plugin doesn't have any such class or module. It defined Delayed::Job instead. So when loading up dependencies, It's looking for DelayedJob module or class when there is none.

A workaround is to simply define an empty DelayedJob module in your config/environment.rb file, or feel free to put it in a file under config/initializers if you don't want to pollute your environments file. Hopefully, that saves you some pain. tip.

Posted via email from The Web and all that Jazz

Sending HTML emails with attachments in Rails

2009-08-19T11:56:00.000-07:00

Here's something you might have missed in Rails.

From ActionMailer's Documentation:

Implicit template rendering is not performed if any attachments or parts have been added to the email. This means that you‘ll have to manually add each part to the email and set the content type of the email to multipart/alternative.

If you want to have pretty html templates and have an attached file, you can't just set the content_type to "text/html" and call attachment. You need to do it separately, like so:

There, now you can have html and your attachments too. Reference: http://am.rubyonrails.org/classes/ActionMailer/PartContainer.html#M000006

Building a culture of teaching and learning by Dr. Tae!

2009-07-10T23:27:00.001-07:00

via vimeo.com

Posted via web from The Web and all that Jazz

Regular Expression Matching and Postfix notation

2009-07-08T05:27:00.001-07:00

As the compiler scans the postfix expression, it maintains a stack of computed NFA fragments. Literals push new NFA fragments onto the stack, while operators pop fragments off the stack and then push a new fragment. For example, after compiling the abb in abb.+.a., the stack contains NFA fragments for a, b, and b. The compilation of the . that follows pops the two b NFA fragment from the stack and pushes an NFA fragment for the concatenation bb.. Each NFA fragment is defined by its start state and its outgoing arrows:

via swtch.com

The snippet doesn't make much sense unless you read the article, but this part, I thought was rather neat. Usually, when I wrote my crappy, one-off parsers, I just used regexes to pull out the tokens that I needed. Never thought too much about how it was implemented. But what's detailed here makes sense. Regexes are just state machines where you track whether the string you're matching against lets you traverse all the way through the state machine. And to do that, it pushes each fragment of the regex onto a stack until it reaches an operator, which then pops it off and works on it. While I've usually left post-fix notation is ass-backwards from a user perspective, I can see the elegance of the implementation. I suspect Forth and Factor are similar in this regard.

Posted via web from The Web and all that Jazz

Nerd time issue 16

2009-06-29T08:13:00.001-07:00

I use to work at a research lab and while it was cutting edge in some ways, it seemed unaware of things going on in software and web. I've since left the research lab, and Nerd Time started as a mailing list to tell my friends still at the lab about things that they might be able to use in their own projects.

------------

Hey all,

It's that time again. I never get feedback about whether you guys read this or not. But apparently, some of you do, since my last trip back to MD. This is just a collection of things I found interesting since the last nerd time. Obviously, there are other trends I'm missing.

If you want off, just lemme know.

This is rather long since it's been a good 6 months since the last nerd time. Work has kept me busy, and I don't read as much as I use to. Since there's no theme, but lots of trends, they're in no particular order this time. Skim through it and see if there's anything that catches your eye.

If you have questions about stuff, feel free to ask me (don't reply all!)

First, some stuff I did on the side:
Senate Majroity vs National Debt
I was talking with Ian about graphing public data, and this was what he wanted to know. This sort of thing should be so much easier. If you find the process of getting this data to graph, lemme know. I imagine it goes in line with a lot of the net-centric buzzwording that does on in DoD projects.
http://iamwil.posterous.com/senate-majority-vs-national-debt-getting-at-p

Frock, a chicken flocking simulator
I wanted to get to know the Lua programming language, so I chose this as a project. I'm getting it to support more chickens still.
http://webjazz.blogspot.com/2009/02/introducing-frock-flocking-chicken.html

And now, the other stuff.

Twitter
So I'm sure most of you by now have heard of twitter. Considering that Oprah did a show on it, it's crossed over to mainstream. A lot of you might not think of it as anything to pay attention to. However, it's one of those things where its value depends on whom you follow. Beyond the hype, it's mainly a messaging multicast system that has a dead-simple API, so that other people can build things on top of it. People have made things that twitter, such as plants that tell you when you need to water it, when bridges go up and down, when a meteor almost hits the earth, etc.
http://www.twitter.com
http://twollars.com/
http://www.botanicalls.com/kits/
http://twitter.com/riverthames

Wolfram Alpha and Google Squared and YQL
Wolfram Alpha and Google Squared had both been announced in the last month or so. Both are looking towards being able to query large amounts of structured data. However, wolfram curates this data with experts, and google squared attempts to make structured data from indexing tables of data on the web. In addition, Yahoo released YQL, which is a query you can use to scrap the web and treat it as just another database.
http://www.wolframalpha.com
http://www.google.com/squared/
http://tables.googlelabs.com/Home?pli=1
http://developer.yahoo.com/yql/

Real-time search
Real-time search seems to be the wrestling ground for the next generation of search right now. There are a number of competitors in this field, including giants and startups. It's evident with the death of Michael Jackson that news doesn't just travel through the old channels anymore.
http://venturebeat.com/2009/06/20/who-rules-real-time-search-a-look-at-9-contenders/
http://search.twitter.com
http://www.scoopler.com

Google Wave
If you haven't heard, google released a new communication tool called google wave. It's what email would be like if it was reinvented today. It's basically combining different aspects of our communication tools and merging them all together. It's best if you watch the video and play with the demo. If you want to play with wave yourself, you don't have to wait for an invite, but can sign up with a wave server that someone set up themselves. I recommend watching the video, as it breaks your presumptions of what's possible with HTML5 and the web.
http://wave.google.com/
http://wavety.com/pygo-wave-server/

Git and github
To me, this is really old news, but just in case you're still using SVN, you should checkout Git instead for your source control. It's ass-kicking good, though it has a slight learning curve. I won't say too much more about it, but you should really look into it. It'll expand your mind.
http://git-scm.com/
http://github.com/

Key-value stores
Lately, there's been a flurry of attention on key-value stores. I've mentioned one of them before, CouchDb. There are a bunch of others. Tokyo Cabinet (link #2) is used at Mixx.jp.co, a social network in Japan. Cassandra (link #3) is used at facebook. Amazon has SimpleDB and Dynamo. I've only played with tokyo cabinet and couchdb, so I can't really do a compare and contrast between them all. But to me, TC, couchdb, and redis seem to be the most interesting. This marks a shift away from relational dbs as the default data store. Not that they'll replace relational db, but we're finding there are a different class of constraints for the web not necessarily taken care of by relational dbs. In addition, they have properties not avail to relational dbs, such as being schema-less, an http server built in, replication, distributed, etc.
http://couchdb.apache.org/
http://tokyocabinet.sourceforge.net/index.html
http://incubator.apache.org/cassandra/
http://code.google.com/p/redis/
http://memcachedb.org/
http://github.com/cliffmoon/dynomite/tree/master
http://opensource.plurk.com/LightCloud/
http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html
http://www.oracle.com/technology/products/berkeley-db/index.html
http://code.google.com/p/thrudb/
http://code.google.com/p/scalaris/

The internet of things
It's something further out, but these first two talks from TED got me thinking about where the web was heading. I don't think that the semantic web, as we imagine it will come to fruition. However, having the things we own talk to each other over the internet is not unfathomable. They'll be able to negotiate with each other to perform a task, or they'll be able to keep a history of what they're doing or how you're interacting with them.

http://siftables.com/
http://www.readwriteweb.com/archives/5_companies_building_the_internet_of_things.php
www.makerbot.com
http://siftables.com/

Cheap hardware boards
Hardware is already cheap, but building hardware yourself has still been somewhat of a pain. I remember having to use Rabbit boards before. There are better ones now. I've mentioned arduino before. Beagle board is a full board that you can run Ubuntu on. Teensy is a small USB microcontroller.
http://www.arduino.cc/
http://beagleboard.org/
http://www.pjrc.com/teensy/?not_a_duplicate

Quake online
Gaming often is looked on as child's play, when in fact, it's some of the hardest programming around, and often drives innovation and progress in graphical techniques, AI, and hardware. Carmack, the guy that wrote Quake, wants to put Quake on the browser. For the longest time, people derided the web, saying it'll never match the performance of desktop apps, and never give the same user experience. If Carmack can run quake on a native browser, then I believe desktop will lose. If he's delivering quake as a video stream, then that's another matter altogether.
http://www.gamasutra.com/view/feature/3946/building_quake_live_carmack_speaks.php?print=1

Reverse HTTP
HTTP is by design a pull model, where the client requests resources from a server. If you wanted to push data to a browser client, you had to rely on a bit of javascript finangling called Comet (cousin to AJAX), where you open an http connection to the client, and leave it open until you want to push stuff to the client. This certainly puts a load on servers because you have to keep connections open. Alternatively, you can have the client keep polling the server. That sucks too. Reverse HTTP doesn't need to keep the connection open. It basically takes advantage of the upgrade field in the HTTP header normally used to find a more appropriate protocol, and instead to turn the connection around from the server to the client. It's still experimental, but it makes a lot more things simple instead of messing with javascript on clients to push data to browsers.
http://www.reversehttp.net/
http://wiki.secondlife.com/wiki/Reverse_HTTP
http://www.ietf.org/internet-drafts/draft-lentczner-rhttp-00.txt

Whiteboarding in real time
Many collaboration tools have come out. We've discovered that the web is essentially a communications medium. Anyway, this set of collaboration tools lets you whiteboard, compose text, and revise docs in real time as other people are editing them. The last link shows you a re-play of paul graham writing one of his essays. This allows people to see how they edit their text over time, and shows others how other people think as they write. It'd be useful as an educational tool.
http://www.twiddla.com/
http://www.etherpad.com/
http://www.wikirage.com/
http://www.revizr.com/
http://etherpad.com/ep/pad/slider/13sentences

Facebook's walled garden
Facebook is the AOL of today. It's basically a walled garden of data, where users live. There's a bunch of effort to break them open. Facebook also wants to open itself out as a fast follower to twitter. I won't say much more here, but there's an ongoing battle about where data gets to go on this front.
http://www.wired.com/techbiz/it/magazine/17-07/ff_facebookwall?currentPage=all
http://googledataapis.blogspot.com/2008/03/3-2-1-contact-api-has-landed.html
http://www.complang.org/dsnp/

Data.gov and the sunlight foundation
Since Obama took office, there's been a big push and initiative to open up the government to its citizens in the name of transparency. One of the things they're doing is data.gov and opening raw public data up to developers or anyone that wants to use it. The sunlight foundation is doing the same for legistator and voting data. I expect that we'll have more apps that will be able to take advantage of this data in the near future, not just to help the people govern their govenment, but also to lead more informed lives.
http://services.sunlightlabs.com/api/
http://www.data.gov/catalog/raw/category/0/agency/0/filter//type//sort//page/1/count/10

DNA engineering
A front that I don't know too much about, but is probably a bigger revolution than the information age and the internet are things that have to do with genetic and bio engineering. 23andme lets you submit cell samples of yourself, and they'll do genetic testing to tell you if you have genetic diseases, among other things. You can now submit gene sequences and get them built for a modest amount of money--not super expensive, but still out of reach for hobbists) As the cost goes down, you'll soon see designer pets and bacteria. The last post is about a guy that theoretically hacks a more potent variant of swine flu.
http://23andme.com
http://mrgene.com/desktopdefault.aspx/tabid-2/
http://www.bunniestudios.com/blog/?p=353

GWT, sproutcore, and Cappucino
Javascript is the most widely used language in the world. And while it has its merits as a functional language, people are trying to develop frameworks that compile to javascript. Javascript is not bad when you're using jQuery. Scriptaculous 2 just got released also.
http://code.google.com/webtoolkit/
http://www.sproutcore.com/
http://cappuccino.org/
http://jquery.com
http://scripty2.com/

Chat on couchdb, standalone web apps, and taking your data with you
This was curious. Couchdb is a key-value document orientated database with an http server as its frontend. They were able to demonstrate that you can fit an entire web app just in the database. Data is code, and code is data. Not only that, you can use the database's replication to port your data and sync it where-ever you go. It's an interesting head turn, even if it's just a demo.
http://damienkatz.net/2009/05/realtime_chat_on_couchdb.html
http://www.jasondavies.com/blog/2009/05/08/couchdb-on-wheels/
http://devlindaley.com/2009/2/14/couchdb-bloom-filters

Mozilla Ubiquity again, but this time hooks into webapps
I've mentioned ubiquity before, which is like a commandline interface for your browser. I use it myself, but only in limited amounts. What's interesting about the direction is that they leverage web services to complete tasks it can't complete for itself. I think that high level languages will eventually adopt the idea of being able to easily hook into web services as a natural part of the language, without extra libraries.
http://www.readwriteweb.com/archives/firefoxs_ubiqity_starts_thinking_for_itself.php

Mozilla Jetpack lets you write Firefox addons with the web
Traditionally, web developers have stayed out of the realm of desktop developers. This is one of the many indications I have that a lot of programming--especially those with user interfaces or a social aspect--will move towards web programming constructs. https://jetpack.mozillalabs.com/

Clojure, Scala, Haskell, Erlang
I'm not going to say too much about these programming languages, since I've mentioned them before, but just as a reminder, there's more than Java out there. These four to me, represent the edge of programming languages that have potential in the future. With the advent of multicores, it's likely that functional programming will lead the way in giving us adequate programming constructs to deal with multicores. If you're a programmer, it'd probably serve you well to learn at least one of these in the coming 4 years.
http://clojure.org/
http://www.scala-lang.org/
http://www.haskell.org/
http://erlang.org/

ParrotVM and mod_parrot
Admittedly, I don't know much about Parrot. But the claims it makes is big. With the rise and popularity of dynamic programming languages like Python and Ruby, we're struggling for a fast virtual machine. And to have to build a new virtual machine every time we have a new language is a pain. ParrotVM is suppose to take care of easing that pain. If that's the case, it might be easier to make languages catered to our problem domain.
http://www.linux-mag.com/cache/7373/1.html
http://www.smashing.org/talks/mod_parrot.ppt

Augmented reality and zombies
We've moved closer to having augmented reality. This is a far cry from the geeky headcam helmets and laptop backpacks that dorky MIT profs wore a decade ago. It still relies on a 2D barcode, and has limited uses, but now with the iPhone3GS out (it has a compass), we might see more augmented reality apps (as well as on android phones)
http://singularityhub.com/2009/06/19/augmented-reality-is-full-of-zombies/
http://www.thinkartificial.org/machine-interfaces/augmented-reality-iphone/
http://news.cnet.com/8301-17938_105-10266380-1.html

Sysadmin tips
Here are some good sysadmin tips. I'd like to think I know my way around linux, when in fact, I've just started.
http://www.ibm.com/developerworks/linux/library/l-10sysadtips/
http://blog.transmit.net/2008/10/my-best-unix-tricks.html

Probabilistic chips
I don't know anymore than what's written in the article. So read about it.
http://www.chron.com/disp/story.mpl/metropolitan/6252697.html

Google Moderator
Voting on websites is old hat since about 2005 with the advent of Digg.com and reddit.com. I found it curious that google has a moderator app, to help facilitate the asking of questions. If you want your own, you can create a white label voting site at slinkset.
http://moderator.appspot.com/
http://www.slinkset.com

The rest of these are related to software, but not about code. If you can only watch/read one, I'd recommend the poisonous people one. That applies to more than open source projects. In it, SVN core devs talk about how someone came in and told them they were all wrong. I have a feeling that was Linus Torvalds, as he rails on the SVN guys in his talk.

The Business and Politics of Software
http://blog.cusec.net/2009/01/05/zed-shaw-the-acl-is-dead-cusec-2008/

Pivoting, or knowing when to stop.
http://startuplessonslearned.blogspot.com/2009/06/pivot-dont-jump-to-new-vision.html

How open source projects survive poisonous people.

Linus Torvalds on Git

Build or buy?
http://sitecanary.com/blog/5

Posted via email from The Web and all that Jazz

Netflix Prize barrier of 10% has been broken

2009-06-26T17:10:00.001-07:00

via netflixprize.com

Well, looks like they did it. A bunch of teams came together and put their solutions together to do 10%. Congrats.

Posted via web from The Web and all that Jazz

Army Exoskeleton Suit Gives Man Superhuman Strength | Singularity Hub

2009-06-24T06:14:00.001-07:00

via singularityhub.com

"it is impressive enough to hear somebody say that they gave up on lifting a 200-pound weight after 500 repetitions not because they were tired but because they were bored."

I've always wanted one of these. I wonder if having mechs is too far off. Perhaps there's no tactical advantage to having a large humanoid robot.

In any case, this makes me wonder what other things DARPA funds.

Posted via web from The Web and all that Jazz

Returning the keys of all documents in CouchDb

2009-05-30T10:43:00.000-07:00

There's a bit of a learning curve when trying to use CouchDb's mapreduce. One of the harder parts is to write the reduce function, which can have two separate cases: called from the map functions, and called again from reduce functions.

When you emit data from map, the examples show you emitting the document, but you can emit any data structure you care to dream up in the key and value portion of the emit. I needed a mapreduce view that returned all the keys that were present in the all the documents. So if I had documents in the db in the form:

{"year": 2008, "birth_rate": 20.0 }
{"year": 2009, "birth_rate": 21.0 }
{"year": 2008, "death_rate": 20.0 }
{"year": 2009, "death_rate": 20.0 }

I wanted something that returned: ["year", "birth_rate", "death_rate"]

Here's one way to do it:

Tip!

How to add helpers, controllers, models, and views of your plugin into the Rails loadpath

2009-05-27T08:43:00.001-07:00

Sometimes, when you're writing a plugin, you end up writing models,
helpers, and controllers that the main app can use. However, you
don't want to copy it into the main app all the time. You'd like to
keep things separate between the plugin, but you'd like to be able to
include it in the path of the main app.

To do this, put the following in your init.rb file in the root
of your plugin. To add a new view path in your plugin that's at
PLUGIN_ROOT/lib/views (where PLUGIN_ROOT is the root directory of your
plugin):


   ActionController::Base.append_view_path(File.join(PLUGIN_ROOT, "lib", "views"))

Any template files (like html.erb) that you put in that path will be
seen in your app.

To add new helper, model, or controller directories in the rails load path:


 %w{ helpers model controller }.each do |dir|
   path = File.join(PLUGIN_ROOT, 'lib', dir)
   $LOAD_PATH << path
   Dependencies.load_paths << path
   Dependencies.load_once_paths.delete(path)
 end

And now, any models you put in lib/model, lib/controller, and
lib/helpers will be in the rails load path.

Of course, this might all be moot with the reintroduction of Rails
engines in 2.3. I haven't gotten around to using them or figuring it
out yet, but for now, this is how you do it with plugins. tip!

Posted via web from The Web and all that Jazz