There are several reasons for maintaining a blog, some of which I’m sure are relevant to you:

Reason 1 – A technical memento to your future self

I found out today how to do something cool. In a year, I won’t remember how to do it … but I might remember it enough to know what to look for. Once I’ve blogged about something, if I Google for it in the future I often find my own blog post, and save myself a considerable amount of time.

There are other venues for such mementos-to-self, but none as indexable and expressive as a blog post. Facebook content is poorly searchable … I searched for something I posted 2 days ago on Facebook and failed to find it except going over my stream/timeline. Twitter is very limiting … I guess it might be good for something, but not for keeping concrete knowledge about how to solve a problem, except that knowledge is just a link.

You could post a Stack Overflow or Quora question, and answer it, but that feels kind of akward. Actually, Quora has Boards which are rather similar to blogs for this purpose.

Reason 2 – Why not share the love?

An immediate followup to Reason#1 – if you worked hard and found out something, why not share it and save some time for other people who might run into the same problem? It’s just being a good citizen, and it doesn’t cost you more effort to share it rather than keep it in a private knowledge base e.g. Evernote / Google Docs. If you’ve ever Googled and found a blog post that solved your problem – now is your time to give something back.

Reason 3 – Professional Resume

Your blog tells the world about you. Whenever I interview for a job, the first thing on my resume is my blog. It shows that I am passionate about my profession. Even if your blog is just a collection of links, it shows what kind of technology stack you’re using or interested in, what your beliefs are, and what makes you tick. I’ll give a candidate with a personal blog a big +1 over a candidate without a blog any time. It takes courage to go out there and say “Even though I’m not worthy, I’m still out here, doing, writing, and sharing. I’m not the best software engineer / biologist / whatever in the world … but I’m trying my best, and I want to share it with you”.

Reason 4 - Alzheimer

This is an extension of reason 1. Reason 1 was about forgetting technical stuff … but as the years pass, you don’t just forget technical stuff, you forget who you were. Your life isn’t lived by a single personality, but is rather experienced by an endless series (or continuum) of personalities, each slightly different than the ones before it. I hardly remember what things we like twenty years ago … but I do know that 5 years ago, I felt the immense joy of leaving the army, didn’t like Google for a day, and had loads of fun playing Portal.

I recently started maintaining, in addition to this blog and social media accounts, a personal record of how each day worked out for me. My setup is simple – I setup a Google Calendar event to send me a daily reminder at 7 PM each day. I forward this reminder to Evernote, and prefix it with a number between 1 and 5 of “how much I enjoyed this day”, plus a one liner of key events that happened. I have a grand plan to one day go over these notes, chart out my happiness graph, and note any specific events … but for the time being I’m recording.

A blog can serve to record part of your history – the part you’re willing to be public about, of course. I picture my kids, ten or twenty years from now, reading through my blog (yeah right, all hundreds of entries , to learn who their father was back on 2012, when we didn’t have hovercrafts and teleportation. Maybe it will happen, maybe it won’t, but I know that I would have been happy if my parents had kept such a diary.

Reason 5 – Helping semi-distant friends keep in touch

I have 267 friends on Facebook … guess with how many I actually keep in touch in real life? I guess maybe ten-fifteen max, and the real number is probably more close to five. But, thanks to Facebook, I get to not lose my other friends completely. Even though I don’t spend enough time with them, I get occasional glimpses of their lives.

The problem with social networks is you can’t possibly keep up with everything … I even miss cool posts like this one because they’re swallowed up in a huge stream of noise.

A part of the fix is blogging. When you have a blog, I’ll follow it with my trusty Google Reader, and it won’t get swallowed up, because I do read or glance everything in my reader stream. So please, if you’re a friend of mine, do me a personal favor and blog – I want to keep in touch with you!

Summary

Please, don’t give me bullshit like “You don’t have anything worth writing about” or “You’re not a good enough writer”. Perfect is the enemy of good … just start by writing something, it’s better by any definition than not writing anything. If you care about it, work on your writing and save up interesting bits to write later (I was saving the idea for this post for a couple of months now, until I got the time and energy to write it). Stuff happens in your life, both personal & professional. Save up the good parts, and write … your future self + children will thank you later.

Our first Tools Night is this upcoming Sunday (29/04), in Sears Israeli (Hertzelia Pituah).

Please check out the schdeule on EventBrite, and subscribe if you’re interested. Also don’t forget to subscribe to the google group.

Read this bit on how UserVoice use Trello.

Takeways:

1. They use six+ dedicated board
2. Whenever a list gets to be so long that you’re scrolling down –> you’re doing it wrong.
3. Developer only works on 1-2 things at a time: One Major feature and sometimes one Minor feature. Don’t pre-assign ownership before you work on something … it’s flexbile.
4. Don’t have a separate system for bugs
5. They don’t try to do project estimates – it just does’t work
6. Celebrate deployments – each week/sprint, review what was achieved in that week, and celebrate the good stuff. Keeps the product team connected to R&D.

Why not Fiddler2?

Ron once asked me how to cause a few files on the server to load more slowly, in order to test many patterns of page loading clients might see and debug css races.

Any idea how to selectively slow down transfer of some but not all files from a web server?
I want to simulate a “css race” that happens when csses are delivered slowly.

Eventually, Ron found a fiddler extension doing just that. But it lead me to the question – how would one tweak with traffic to his servers in general?

The obvious answer is writing a Fiddler extension, however I find this solution lacking. The usual complaints apply here: Fiddler is a heavy GUI app, it doesn’t run on Linux, it’s not open source. The main technical point is that Fiddler is not designed to run as a real proxy. Fiddler is designed to listen on :8080 on my box, for looking at the traffic I’m generating myself. However I often want more than that, for instance I want my QA team use a proxy that emulates a flaky mobile connection. I want to record the traffic my QA team generated, so that I can replay it, and use real user input for load testing.

I was looking for a framework, a-la ruby on rails, that would allow me to write custom HTTP proxies, and I found none. So I decided to write one (open source on github, of course).

A taste of GoProxy

GoProxy, is a Go library allowing you to build a custom HTTP proxy. By default you can create a basic transparent proxy, that would simply deliver the request from the proxy client to the remote server, and write the response back

proxy := goproxy.NewProxyHttpServer()

proxy is a net/http handler, start it just as you start any other Go HTTP handler

http.ListenAndServe(":8080", proxy)

How do we customize it? We’ll add “handlers”, request handlers and response handlers. The request handler will receive the request after it is received from the proxy client, but before it is sent to the remote server, and they return another (maybe the same) request. Here is a simple request handler that simply adds a header:

proxy.OnRequest().DoFunc(func(r *http.Request, ctx *goproxy.ProxyCtx) (*http.Request,*http.Response) {
    r.Header.Set("X-Proxy","yxorP-X")
    return r,nil})

Note that the request handler can return an HTTP response. If the HTTP response is not nil, the proxy will never send the request to the remote client, and return the handler’s response to the client.

The case of response handlers is almost identical:

proxy.OnRequest().DoFunc(func(r *http.Response, ctx *goproxy.ProxyCtx) *http.Response {
    r.Header.Set("X-Proxy-Resp","pseR-yxorP-X")
    return r})

Here we’re adding a header to every response sent back to the client, leaving the request untouched.

Inside the OnRequest or OnResponse function, we can use a RequestCondition or ResponseCondition accordingly. Those conditions are interfaces that are able to Handle a request or a response and return a boolean – their decision whether or not to activate the handler given to the DoFunc method right afterwards.

proxy.OnRequest(goproxy.IsLocalHost).DoFunc(func(r *http.Response, ctx *goproxy.ProxyCtx) *http.Response {
    r.Header.Set("X-Proxy-Resp","pseR-yxorP-X")
    return r})

This will add the X-Proxy-Resp header only to requests sent to local host.

Another aspect of HTTP proxy is the HTTPS connections. Generally speaking, when a proxy client wants to use HTTPS over a proxy, it sends the proxy the http request “CONNECT remotehost.com:443\r\n\r\n” and the proxy then will forward all bytes sent to it by the client to remotehost.com:443. We allow you to eavesdrop the connection with the same mechanism of the OnRequest request handler.

proxy.OnRequest(goproxy.ReqHostIs("example.com:443")).HandleConnect(goproxy.AlwaysMitm)
proxy.OnRequest(goproxy.UrlIs("example.com/")).DoFunc(...) // now we will handle encrypted traffic to example.com

I tried to include a decent documentation and examples in the source code, please let me know if anything is unclear.

One important note, since the project is still in its infancy, in a pre-alpha stage, I reserve the right to change and break the API occasionally, until I’ll figure out what’s the best API. I hope to stabilize the code in a few months, and ensure no breaking changes are introduced, but to do that I need users input.

Do visit http://github.com/elazarl/goproxy and let me what you think about the code, documentation & usage examples.

Key Take-Aways from The Project

Go

Google’s Go is a very pleasant language, which was just released. I have no doubt this project would have taken much more time in other languages.

There are many merits to the language itself, but I think that what really made the difference was the absolutely excellent standard library for HTTP connections and data streams. I’ve never seen such a nicely layered abstraction, allowing you to use http.Get(“http://example.com”) as a quick replacement for wget on the one hand, and on the other hand allow you to read part of the response and then drop the connection.

The go tool is an excellent plus. It does an excellent job with managing dependencies, however it is not as strong as maven. So for example, you must have internet connection when building a package for the first time, you cannot rely on a local repository. Moreover, it cannot manage the versions of software packages, ensuring all developers are using the same version.

A weak point of the language is immature parts in the standard library. For example, JPEG encoding performance was not adequate to re-encode many jpegs on the fly (check out the 40 LOC image flipper proxy!), lack of documentation to some parts of the library, and unstable eco-system. A certain library I was using which was mentioned in the official golang blog had changed its API in the middle of  development (hey, I have no complaints for the author – he did an excellent job, but it’s important to know that such issues may exist). In another case a stray printf remained in a library causing weird printouts from time to time.

A peculiar benefit of Go, as I see it, is that it is a statically typed memory safe language, yet, it can produce a standalone executable, depending on nothing at all, and with fast startup time. There are some programming languages trying to do that, but which one of those have a profiler? They are compiled to C, Go compiles to assembly. (D might be an example for such a language, but I don’t know it enough to form an opinion).

Software design

The most important lesson I learned is, never ever design an API without writing real world useful software that uses it. I found most of the API gotchas while writing small examples.

The second important lesson is, as Joshua Bloch put it “when in doubt – leave it out”. Deleting code is the most effective way of improving it. Look at your API and try to see what you can delete without major consequences. Then delete it. Yes, you definitely have at least one function which is not needed, and is best left out.

The third lesson is, write documentation, try to cover a lot of the code. When explaining how to use your API, you’ll find out that the API doesn’t make a lot of sense.

The fourth lesson is, the API will never be perfect, you’d better release it and get feedback from real users, than endlessly “improve” it in the lab without getting input from real users.

(Heck, while writing it, I noticed I didn’t give any way to block CONNECT requests without MITM the https connection. Now it is fixed, but there are probably more problems to come).

So, if you’re in Israel (or just read Hebrew), and dig Bitcoin, drop by the site and subscribe. The site is of course non-profit, and its goal is to build and strengthen the Israeli Bitcoin Community, ranging from “Bitcoin Curious” through users to developers and investors.

P.S. We are looking for content writers – if you’d like to write articles about Bitcoin in Hebrew (and maybe even earn a few BTC in the process), be in touch.

duplicity is drop-dead simple backup system that does incremental/full backups, and supports a wide variety of backup targets from the box, including file, ftp, scp, s3, and a dozen others. The usage is simply

duplicity /folder/to/backup file:///path/to/target/folder

You can have the target folder be a local or S3-mounted folder (with the excellent s3fs), or use other supported URIs. The default is an incremental backup, by you can override by choosing a full backup. It even has support for “keep the last N full backups and delete the rest”.

The only feature that I required, and duplicity didn’t have, is automatically choosing when to do full vs automatic backups. I would like to tell it “every 100 runs, do a full backup”, and this is not supported out of the box … although easily fixed with a wrapper script. Put that in crontab and you’re set. Oh, by default it requires setting a PGP key, but if you’re lazy you can skip it with the –no-enc option.

Write your own jQuery plugins

The word “plugin” usually entails something complicated and with some non-trivial learning curve (e.g. how many of you ever wrote a Chrome of Firefox plugin?) Well, in jQuery, this is really not the case. Here is how you write a simple jQuery plugin:

$.fn.enable = function() {
  $(this).prop("disabled", false);
};
 
$.fn.disable = function() {
  $(this).prop("disabled", true);
};

I find it useful to wrap even simple one liners such as .prop(“disabled”, false) with a plugin, because the semantics of writing $(“#foo”).disable() is much nicer than playing with properties/attributes directly. I haven’t written a lot of plugins yet, but it’s something to keep in mind as a useful tool to wrap actions on specific DOM elements.

Know the commonly used plugins
There are a ton … I still know very few of them. Here are a bunch of useful ones (and the ones I personally know and use).

A lot of plugins are very easy to use, and have good documentation and demos, so not using them and rolling your own solution is usually just a result of ignorance. Take the time to educate yourself!

UI Queues
For a long time I’ve that you can do things like $(“#mydiv”).show(), $(“#mydiv”).hide() and even $(“#mydiv”).show(1000) for a simple animation. Only recently I discovered you can actually chain these using Event Queues:

$("#mydiv")
  .hide(1000)
  .delay(500)
  .show(200)
  .fadeOut();

Each call to an animation method gets queued up and executes after the previous one.

Deferred
jQuery Deferred is a little gem. It lets you write fluent code similar to the queue example above. Here is how you use it:

$.when($.post('http://api.com/request_1'),
       $.post('http://api.com/request_2'))
  .then(function(){alert("Got two responses")});

You can also use them with the event queue:

  $("#mydiv")
    .show(1000)
    .delay(500)
    .promise()
    .then(function(){/*do something */});

That’s all I have for now. Have any essential tips & tricks that I’m missing out on?

I am rather confident when I write “dirty code” in Java or C#, because I know that I can swiftly refactor it into beautiful code without too much trouble. Both aforementioned IDEs are so great at this, that it’s painless. You can take an ugly 300 line method and break it into several methods, break a long class into several classes, inline, move, and otherwise shape your code.

Enters javascript and web development.

Last year I’ve made the plunge and officially started working on front end web development, first at Google, and recently at Commerce Sciences. And I’ve recently discovered that … refactoring javascript and frontend code is hard.

The language is dynamically typed

Compilers and IDEs can help you less when the language is dynamic. They can make less assurances about your code, making reasoning and refactoring harder. Safely moving a method from class Foo to class Baz is an easy feat in a statically typed language, and a difficult or impossible one in a dynamic language.

Classes are not first-class citizens

While you can do OOP in javascript, classes are not first class citizens, but rather they’re implemented using functions, objects and prototypes. Automatically reasoning about “classes” without having an equivalent of keyword class is difficult.

Your code is not just Javascript, it’s HTML & CSS as well

Web development is not just about javascript, it’s a combination of JS, HTML & CSS (not to mention other potential technologies such as LESS/SCSS, HAML, JSON, and whatever language your backend is written in). Unless you’re already a web development ace and perfectly design your codebase … you will get these mixed up. Refactoring is about changing the design of your program, and when that design is split up between three or four different technology domains, design mistakes are harder to rectify.

You can’t (at least not yet) do an automatic refactoring to “move inline css into an external css file”, or to “convert static html snippet into a javascript DOM manipulation method”. The makers of IDEs and refactoring tools don’t have javascript completely figured out yet, no wonder they haven’t gotten around to building cross-domain refactoring!

Again, if you “know what you’re doing”, you can structure your program perfectly in the first place, and won’t ever have to do this kind of “cross domain refactoring”. Sadly, we’re not all born with this kind of experience.

It’s harder to know when you’ve broken something

Backend code is so much easier to test than frontend code. While frontend testing is important, and growing, it is by no means as well understood or practiced as classical “backend TDD, this is a calculator, assertEquals(30, calc.plus(10, 20))”.

So, since BE code has much wider test coverage than FE code, and is bloody compiled for Java & C#, when you refactor FE code it has a much greater chance of breaking down, often in subtle ways you’ll only notice on IE 8, or when the network is slow, or whatever edge case surfaces your particular bug.

So … how do we manage?

• Refactor less – Since we know refactoring is hard, we try to do less refactoring and more pre-factoring. Think a little more than usual before coding. While I’ve grown the habit of “code first, think about design later” over the years due to power of refactoring, it’s less useful in FE dev, so I need to take the time before coding and try harder to get it the design right on my first attempt.
• Still .. refactor – IntelliJ and Resharper both offer some refactoring capabilities. I’m most comfortable with “limited scope refactoring” – those that affect one function like Extract Variable. Use the tools you do have, instead of whining about the tools you don’t.
• Try to think harder about the different problem domains (JS/HTML/CSS), and to develop a better understanding of how to structure your program in a way that won’t force you do to refactoring across problem domains.

Please do share your own experience with refactoring in web dev!

So far it’s been just Eyal, Aviv and myself working here at our Herzliya office.

Well …

We are pleased to announce that we secured an investment of more than 1.5 million dollars, and are now ready to spend that money on you!

That is … if you’re a really top notch developer looking for adventure, learning, teaching, cutting edge tools and challenges, a terrific product (still stealth mode, but we already have some clients) that will change the way we do eCommerce, together with a superior team.

You will be our second or third employee … plenty of room to grow, influence the team, the code/architecture and the product.
I was actually just planning to write a post about “why I love startups so much” when the news of the investment round was made public .. look forward to it.

I can’t elaborate publicly about the product, but technology wise I can say we’re working with Java 7 (thinking about Scala), Groovy, Play! Framework, Javascript/HTML/LessCss (will integrate Coffee Script first chance we get) Amazon EC2, git, and Trello.

If you’re a Java/JVM developer, and live in the Tel Aviv (or Israel) area, please join.

I don’t like repeating myself, so please, if you fit the above criteria, read the linked post.