Shallow and Pedantic

The show downloading stack - part n+1

2011-12-15T14:08:00.000+02:00

I've already mentioned my show downloading stack on this blog. It's changed a bit since - I now use Transmission rather than rtorrent, as it has the excellent transmission-daemon package which has it acting exactly the way I like (without using screen). Also, it now E-mails me when a torrent is done downloading. So while this may be how TV works for you:

Notice that a new episode is out
Torrent it
Wait for the download to finish
Watch it

...this is how TV works for me now:

Receive E-mail notification of a new downloaded episode
Watch it

Here's how it's done: First, write /usr/local/bin/notify_torrent_done:

#!/bin/bash

cat_the_message() {
cat <<EOF
Subject: Torrent done: $TR_TORRENT_NAME

Hello from transmission $TR_APP_VERSION at $(hostname). The following torrent
has completed:

Name: $TR_TORRENT_NAME
Finished at: $TR_TIME_LOCALTIME
Downloaded to: $TR_TORRENT_DIR
Hash: $TR_TORRENT_HASH

Enjoy!
EOF
}

RETVAL=1

while [[ $RETVAL != 0 ]]; do
 cat_the_message | msmtp -C /etc/msmtprc.transmission --from default -t your@email.address
 RETVAL=$?
done

The various environment variables will be set by transmission when it calls this script. We rerun msmtp until it succeeds because you will often get a "connection timed out" response from gmail (...at least on my ISP...). Here's /etc/msmtprc.transmission which is relevant to gmail (this is a bit tricky and took a lot of fiddling around with):

defaults
tls on
tls_starttls on
tls_trust_file /etc/ssl/certs/ca-certificates.crt
timeout 60

account default
host smtp.gmail.com
port 587
auth on
user yourusername@gmail.com
password yourpasswordhere
from yourusername@gmail.com
syslog LOG_MAIL

As usual, caution is required when saving your password in plaintext. I highly recommend using Google's two-step authentication, which will have you creating a one-time password for each application - use one of those one-time passwords here.

Finally, in /etc/transmission-daemon/settings.json, add the following code:

"script-torrent-done-enabled": true,
"script-torrent-done-filename": "/usr/local/bin/notify_torrent_done",

Important: You need to run /etc/init.d/transmission-daemon reload at this point, not restart - that would cause settings.json to be rewritten from runtime configuration.

That's it. Enjoy!

Developing TransportDroidIL for Android

2011-10-08T21:43:00.000+02:00

Here are a few words on developing TransportDroidIL, a small utility to query Israeli public transportation sites more easily using an Android phone.

Source control is super important. Mistakes will be made, other coders will want to join in, and experimental features will want to be in their own branches. Git is awesome; it does source control right, gives me powerful tools, and isn't a hassle to set up - even for a small project like this. Github is also awesome; it makes collaboration with other coders - even just one, in my case - easy, organized, and fun.

One of my favorite git features is revert. It allows you to automatically inverse a previous change. Here's an example from TransportDroidIL: 0e194de. This commit reverts a previous "cleanup" commit in the autocompletion code - allegedly, I was keeping two copies of the autocompletion option list for no reason: One as a LinkedList<string>, and one internal to the AutoCompleteTextView which I can access via an ArrayAdapter<string>. Turns out that my LinkedList<string> copy is necessary, because the ArrayAdapter<string> always behaves as though it's empty, so it cannot truly be read from. Despite having performed a few commits since that bad "fix", git was very helpful in letting me revert that particular change, showing me the conflicts this operation causes, and allowing me to fix them. The lesson is an important one: Make small, manageable commits. Git is optimized for this, as commits are local (no need to contact the server until a push).

Developing for Android makes a lot of sense when using Eclipse. I'm a VIM junkie, and generally dislike IDEs. Eclipse is slow and heavy - but it gets the job done, and it does it very well. It's a bit weird that a plugin is required to manage color schemes - but it exists. It's very weird (and quite annoying) that it doesn't remove end-of-line white-space, and doesn't have an option to do this. This makes git complain. There is an option to add "clean-up" settings which are activated on every save, but this is far too cumbersome and might change code I didn't intend to change (which becomes confusing in the revision log). Still, the excellent debugging, JUnit and logcat support are worth it.

Logcat is another awesome feature of Android. Every logged line has both a "Tag" (usually defined per-class) and a severity. Logs can be filtered with a different severity for each tag, and still - one can use the same logcat to show messages from anything running on the Android device at the moment. It's basically Syslog done better.

One last point is about Hebrew. This has been a problem with Android for quite a while; for example, in a stock Android 2.3.3, numbers embedded in Hebrew string appear backwards. Fixes exist, and are implemented in most Israeli ROMs, especially the ones distributed by phone carriers - but they're different, and sometimes don't work for all applications. This causes numbers to appear backwards in TransportDroidIL, which in turn caused me to implement an ugly hack. I hope the upcoming Ice Cream Sandwich release fixes this.

Home network wiring

2011-09-04T21:53:00.000+03:00

I don't like wireless connections; they're always second-best, be it in terms of security, speed, or reliability. Here's how my apartment looks (very approximately):

.-P1----.  .-------------------P4-,---------,
|  PC   |  |       COMFY SOFA     .         |
|       |  |                      . Closed  |
|  .-,  |  |      Living room     . Porch   |
|  |B| P2  |                      .         |
|  |E|     |                      .         |
|  |D|  |      HUGE CUPBOARD   TV .  R PC   |
`-------'  `-P3-------------------'--P5-----'
  My room

The room on the left is mine, with my (constantly torrenting) PC in bed-viewing position. The router (R) is in the closed porch, connected to my roommate's PC. Wifi doesn't stand a chance through two walls and the porch's glass screen. P1 through P5 are power outlets.

First solution

Put a reverse DD-WRT router at P2 (with a cable going across the room from the PC). Slow connection, not very reliable. This worked well enough for several months.

Second solution

Get a pair of homeplugs. Stick one at P1 (connected through a power strip shared by the PC, speakers, screen and guitar amp). Stick another at P5 (connected through a power strip shared by the other PC, router, modem, TV, fan, printer, speakers and a lamp). It shouldn't work - but it does. It blinks red, is nowhere near the promised 200Mbps, but it's still faster (and more reliable) than my internet connection. These homeplug devices are fantastic - they require literally no configuration (unless you want to reconfigure the encryption keys) and work very well, I highly recommend them. The problem was when I got an Xtreamer - a cute device to watch my shows on my living room TV (see: comfy sofa). Once I plug it in, the homeplug connection dies on me, proverbially the last straw.

Third solution

Thankfully, I have my Big Box of Electronic Junk, which contained a Super Long (read: haven't measured it) network cable. Moved the homeplug to P3, hid the cable behind the Huge Cupboard. Problem solved. On a side note, I would have preferred a Boxee Box, but sadly it doesn't support lame old RCA connections.

Upcoming features in TransportDroidIL

2011-09-02T11:30:00.000+03:00

Hello everyone, TransportDroidIL 1.0 will be released this month. It has quite a few new features:

Separate "Source", "Destination" and "Time" fields. This is much better for auto-completion as well.
Automatic location-based detection of "Source". Hopefully you'll find the interface for this unobtrusive.
Quick reversal of "Source" and "Destination", for your return trip.
A new settings screen, with:

Provider selection (currently Egged or MOT)
A small attempt at right-to-left fixes for non-supporting devices.
A "clear history" button

Everyone loves screenshots:

For a sneak peek, go to

SMS and why it annoys me

2011-04-25T19:58:00.000+03:00

Don't get me wrong. I love being able to communicate textually with friends, coworkers and family. It's ideal for a noisy pub; a somewhat-private conversation on a crowded bus; telling something to someone who may be asleep, so they see it first thing when they wake up; making quick responses while in a meeting without being rude (well, at least at my workplace it's considered perfectly acceptable). It's also very handy when you want to tell someone something they ought to write down, such as a phone number or something they should remember to buy. My problem isn't with the concept of mobile textual messaging - it's with SMS, the "Short Message Service", as provided by Israeli carriers (and possibly worldwide).

The first problem I'd like to discuss is length. SMS messages are limited to 160 characters, or rather - 160 bytes. If your message includes any foreign characters, such as Hebrew letters, then UTF-16 mode takes effect, and your message is limited to 70 characters, which is ridiculously short. While the name of the service, SMS, implies that the messages should indeed be short, this is not the only common usage: If you want to have an actual conversation with someone (and this is a perfectly reasonable situation), messages will be longer than 160 characters, and certainly longer than 70. The problem becomes worse if u dont want to use shorthands n skip punctuations like many smsers do. (Ever found yourself about to send a 75-character message, going over it to find 5 characters to trim without looking like an idiot?)

There have been several attempts to overcome the short-message problem, all of them implemented as workarounds in cellphone software. The initial, primitive approach was to simply split a message up into appropriately-sized chunks, and send out those chunks as separate messages: A 150-character Hebrew message would be sent out as 3 separate messages (70, 70, 10 characters respectively). Later on, a mechanism was invented for the phones to detect the split messages, stitch them back up on the recipient's phone, and notify the recipient if the message is incomplete. Poor man's TCP.

The split-message solution is very problematic, especially when taking into account two other problems with SMS: Reliability and price. The reliability problem is subtle: SMS message usually make it across, but when they don't (and sometimes, in fact, they don't), no notification is given. Where E-mail servers notify you about delivery failure, where instant messaging services tell you that the other party has disconnected, SMS has... nothing. There's no way to know that your message hasn't been received, unless a little-known, not-supported-everywhere and highly annoying "read receipt" feature is enabled.

The price problem is completely absurd: I have a small 1GB plan at Orange, which costs 70₪/month. That's 70 NIS / 1GB, 1GB being 10⁹ bytes. That comes to 7 × 10^-6 Agorot per byte. Suppose that an SMS-style service would need a bloated 100-byte header, so an SMS message is 260 bytes. Therefore, an SMS message should cost 0.00182 Agorot. In reality, it costs 44 Agorot (inter-carrier average) without a plan, or 14 Agorot (at best, requiring a 1000-SMS plan) with a plan. That's between 7692× and 24176× as much.

One additional problem with SMS is the fact that they're locked into your device. Got a new phone? You have to go through a very complicated process to transfer your SMS messages over, and this isn't possible for every phone. Lost your old phone? Unless you were backing up manually, your messages are gone. Have a separate work-phone? You'll need to use each phone to see messages sent specifically to it. None of these problems occur with an online E-mail provider, why should they happen with mobile texting?

With modern phones, sending and receiving E-mail is just as easy as sending and receiving SMS messages. The main problem is that not everyone has a modern phone, and E-mail on older phones is quite cumbersome. Worse yet, older phones don't immediately notify you of E-mails. There could be some good money in creating an E-mail-to-SMS gateway - and it would be quite convenient to use if done properly. Imagine having your SMS messages united with your E-mails, organized and searchable by your favorite E-mail client. If you have a modern phone with proper E-mail notification and sending abilities, this could be very handy. Just sayin'.

Addendum: It seems that Google Voice already does exactly what I want, but is not available in Israel. Thanks, Shachar.

DD-WRT awesomeness

2010-11-20T17:19:00.000+02:00

Since I've last posted, I've moved to a new apartment. First order of business - get a working internet connection. This is extra-challenging when your primary machine doesn't even have a wireless network card.

My first hack used my trustee laptop - it has a properly working wireless card, and could connect to my roommate's router quite easily. It runs Ubuntu, and as it turns out, that means sharing the connection was dead-simple: Right-click on the network manager icon, add a new wired connection called "Shared", and under IPv4 settings, choose "Shared to other computers". That's it. Once I connected my desktop to my laptop, it automatically got all of its settings, and I was good to go.

However, this was kind of annoying - I had to leave my laptop on, the reception in my room isn't perfect so it would sometimes disconnect (requiring manual intervention), and my laptop wasn't free for ordinary use (if I wanted my torrents to keep going).

The solution: I grabbed my (horrible) old D-Link DIR-300 router, and installed DD-WRT on it. This gave it an awesome "client mode" feature, which allowed it to use it the same way I used my laptop to bridge my wireless connection. Flashing it worked quite well by following the guide (the updated version in the wiki, that is - it has proper instructions for connecting to RedBoot, the D-Link flashing interface, from Linux), and another guide helped me set up Client mode. All seems well, except for two issues:

First, port forwarding doesn't seem to work properly - it works well on the internal network (that is, I can SSH into my desktop using my laptop), but not on the internet (SSH port shows up as open, but I can't connect). I'm also guessing that UPnP/NAT-PMP won't work properly. Second, and this is an old problem - the router has a high-pitched whine. This may have something to do with the fact that the AC/DC adapter it came with is rated for 12V @ 1A, whereas the router is rated 5V @ 1.2A. Let's hope it doesn't fry (hasn't for the 3 years I used it).

Transport Droid IL

2010-09-04T19:21:00.000+03:00

After a couple of days' messing with it, I'm releasing it: Transport Droid IL! It's a handy little app for querying Egged's site, as well as the new Ministry of Transportation site, on transportation information. This is pretty beta, but seems to work well enough.

More info, source code: http://lutzky.github.com/TransportDroidIL

APK File: TransportDroidIL.APK

QR Code:

Setting up Hebrew Android fonts on your AVD emulator

2010-09-04T12:54:00.000+03:00

There are several good guides for installing Gilad Ben-Yossef's excellent Hebdroid fonts on physical Android devices, but those don't really work with the Android SDK's emulator - changes to the system directory aren't persistent. Here's how to get around that:

First, a few downloads. You'll need:

The android emulator (presumably you already have this, if not, you can get it at developer.android.com)
The hebdroid fonts
unyaffs, which will extract the system.img file
A snapshot of yaffs2, which will create our new system.img file. This is actually today's snapshot from the git repository, which worked for me. For later versions, take a look at the git repository.

Building unyaffs is simple enough, or you can use the prebuilt version from the site. Building mkyaffs2image is also quite easy - just untar the snapshot, and run make in the utils directory. Put both of these utilities somewhere in your $PATH for convenience.

Now we can get to work. First, locate your system.img file. It should be within your Android SDK directory, under platforms/android-3/images (or whatever version you're emulating). We'll extract that - create a temporary directory, say /tmp/system.img.hebdroid, and cd to it. Then run:

unyaffs [path-to-system.img]

With [path-to-system.img being the same one you just found. The whole /system filesystem should be extracted. Now extract the ttf files from hebdroid.zip into the fonts directory, replacing the original font files. To pack everything back up, run:

mkyaffs2image [your-temp-dir] system.img.hebdroid

Now, I recommend putting renaming your original system.img to system.img.orig, and using symlinking system.img.hebdroid as your new system.img (the emulator does indeed follow symlinks), but you can basically do whatever you like. You may have to recreate your AVD, but everything should work. Happy hacking!

Android

2010-09-04T12:26:00.000+03:00

I've recently "bought" a Samsung Galaxy Android phone (the older i7500, not the newer i9000 "S") model. It's a seriously serious upgrade from my old Nokia 6120 Classic, and as I broke the 6120's screen and reverted back to my trusty old Nokia 6070 (which I couldn't even get to run the GMail app), I was quite a happy camper switching to a modern phone.

The whole idea of Android has always been very appealing to me - Nokia's software has been declining in quality, and while Motorola and Samsung have always made excellent hardware, they could never get the knack of good software. Google has. Problem solved, right?

The Android software is miles ahead of anything I've ever seen, including iPhone. I now have connectivity everywhere, and apps to sync all of my favorite things - GMail and Google Calendar are included, GTasks is great for tasks, Paperdroid is great for Read It Later, NewsRob is great for Google Reader. The Facebook and Twidroyd applications are also quite nice, and the convenience has me using those networks more. I even have nifty stuff like Transdroid, a Transmission web client which is actually very good at adding torrents while I'm away, so they're done by the time I get home. I get my reading done with Aldiko and ACV (for comics). Even Israeli sites have some good applications up, such as YNet and Dapei Zahav have a nice Android app, though not as nice as their iPhone ones. And of course, there's the wonderful Waze, which is the most Israeli solution to the road congestion problem I've ever seen. I've even had the chance to do some on-foot navigation with Google Maps, which is also handy.

The phone itself has a beautiful AMOLED screen, great audio quality (comes with quite a good set of headphones as well), and looks sleek. The touchscreen is responsive enough (though not anywhere near new Android devices nor the iPhone/iPod touch), and it even comes with a free extra 8GB SD card, for a total of 16GB.

And now, to rant.

Battery life isn't what is should be - the phone is awesome, I want to use it, and it can barely get through the day - especially if I'm doing heavy stuff like Waze (GPS + Data + screen is always on + voice), but even if I'm just surfing casually. The unlock button is located inconveniently on the bottom part of the right side, just above the camera button. The home button is located between the Back and Send buttons, meaning it's very easy to hit by accident; it's not even labeled. The CPU isn't always fast enough to keep everything completely smooth, and while this is generally acceptable, it gets rather irksome in odd places: If you turn on screen auto-rotation (which uses more battery power), rotation takes a while and might be accidental. However, there is no manual screen rotation option as far as I can tell.

As a music player (relating to my previous post), the device works well enough - but I couldn't, for the life of me, get Hebrew support in ID3 tags (no matter what the encoding is), and only a limited subset of ID3 tag versions is supported, and everything works much better with Ogg files.

The flaws I've mentioned are relatively minor and nitpicky - for Waze, you can use the car charger (which comes with the phone, thankfully). The buttons are OK after some getting used to (and installing the excellent AnySoftKeyboard), and operation is generally smooth. However, there is a major issue I can't wrap my head around - the shipped firmware is 1.5 cupcake, and there are absolutely zero updates available from Samsung. There is a semi-official leaked 1.6 update which, as I've heard, is quite buggy. An awesome guy called drakaz has been working on a Froyo (2.2) port for the Galaxy, which I really should check out, but Samsung's behavior on this topic is inexcusable in my opinion.

All-in-all, however, I'm happy with my phone. It's a joy to use, and the price is hard to beat - free with my phone plan, provided I can rake up a bill of 150₪ (which I can, with some help from the 70₪ 1GB data plan, which comes in quite handy). My next phone will, however, probably not be a Samsung.

My music player has sunk

2010-03-13T18:35:00.000+02:00

My Meizu Mini M6 has died a tragic death as a result of being left in my shirt pocket, which in turn was - with the rest of my shirt - in the laundry. I've had it for three years, so this would be a good time to review.
My favorite thing about the Mini was the cost. When I bought it, it was far cheaper than comparable players, at ~400₪ for 8GB. For a player with good video support, it was a steal. Other pros it had include:

Great screen. I used to watch TV shows on it quite a bit
Great audio quality, especially in the lower range - very important for a bass guitarist
Perfect OS compatibility - shows up as a Mass Storage device, just drag your files over. This is also how firmware upgrades are performed
EBook reader, of sorts
Quite small and very thin

However, the player is full of quirks:

Slow startup time. This is compounded by the fact that to turn it on, you have to press the "Play" button for about 3 seconds - too much less or too much more, and it won't start up.

This is even worse when unplugging it from USB - it would rebuild the library every time, even if you were just charging it.

You can't turn the player on when it's connected to USB. When it's off, connecting it to USB is for charging only - to access the filesystem, you have to disconnect it, turn it on, and connect it again. (...not that the player is functional when in it's plugged in...)
A few podcast niceties would be very easy to implement and extremely helpful:

There's no way to separate podcasts from the rest of the music, so you don't get "This Week In Tech" when you put it on "All Music at random" mode.
The player doesn't keep track of positions within audio streams, other than the last played one.

There's no way to delete a song from within the player.
The interface is very inconsistent, with the video and audio sections having completely different key bindings. Long presses and short presses have very different meanings (but the length isn't all that different), so you have to be very careful.
Pressing various key during startup had interesting, non-obvious effects, such as rebuilding the music library or formatting the player without asking for confirmation. This isn't in the official documentation, and even if it were - that's a very bad misfeature.
The newer version of the player, which I have, did away with the bottom "play" button, and moved it to the bottom of the 4-way D-pad, instead of the "Enter" key. The "Enter" key was replaced with a tap on the d-pad. Unfortunately, such a tap is very similar to a volume change drag, and even if it weren't - is very easy to perform by accident.

One more feature that I would have liked is one I've only seen in the iPod (which I despise for a variety of reasons) - the iPod keeps track of which songs you've played all the way through, and remembers that you "like" them - this information is later used when building playlists, and is integrated into iTunes playlists as well.

Now, I might be going to a long course soon, one which will specifically mean plenty of bus time. So go ahead, guys - recommend a player. :)

Broken phone screen - data rescue

2009-10-31T04:32:00.001+02:00

Last weekend I broke my Nokia 6120's screen. I have a military phone, which is far cheaper, so I've decided to keep it offline. However, being the sentimental guy that I am, I did want to save all of my contacts and SMS messages (in addition to the photos, which presented less of a problem). This proved to be a bit of a challenge without the screen working.

Usually, when you connect the phone via USB, it asks if you want "PC Suite mode" or "Data Transfer mode". The "Data Transfer" mode has the phone show up as a standard USB storage device, which allows for easy transfer of MP3 files, photos and videos to and from the phone, without any nokia-specific software. However, it only works for the external SD card, so you can't use that to access SMS messages or contacts.

I usually only need "Data Transfer" mode, so I changed the default to that. Today I regret that decision, as it cost me a couple of hours' work. I called the Orange hotline, and they did their best to help me, including trying to blind-guide me through the menus, which failed because the menus are actually dynamic and I didn't have the default setup. They actually got me 90% of the way there - here's the solution I found: Hit the red (disconnect) button, and type the Soft Reset GSM code: *#7780#. Now press the "left menu" key (not the left key, nor the menu key - the left of the two "dynamic" keys) - this part was what the Orange hotline missed, because it seemed so obvious. Then hit 12345 (this is the default "secret code"), and the left menu key again. I found this by watching a demo of the soft reset on YouTube.

At this point I used VirtualBox and the Nokia PC suite (both are free-as-in-beer) to do the heavy lifting. I now have a text file with all of my contacts, a CSV file with all of my SMS messages, and all of my images saved both to my computer and a DR site. Now I just need to upgrade my military phone (Mirs)...

Loving the new Totem

2009-10-10T15:04:00.000+02:00

Totem is Gnome's built-in media player, and it really annoyed me in previous versions, and had me switching to VLC. However, the version included in the Ubuntu 9.10 release candidate has two features which are very important, in my opinion. The first feature is smooth graphical integration with compositing managers (such as compiz). In previous versions, as well as VLC, once you fullscreen the window, moving the mouse (which causes the cursor and the partial interface to appear) causes a very annoying flicker. This has been fixed (at least on my box, using an NVidia card).

The second, more important feature, is the exact one I've been missing and talked about in the previous post - hit Edit->Preferences->Start playing files from last position, and Totem will keep track of your last playback position when you close the video. Reading the implementation patch shows that there is a certain threshold for this - the position won't be saved if you're too close to the beginning or end of the video. So there, my show downloading stack now has every feature I'd want from Miro, without the downsides I've mentioned.

My show downloading stack

2009-09-05T16:42:00.000+03:00

I love watching TV, and hate it. Regular show schedules are horrible, commercial breaks are annoying, and the ability to rewind is very important. I love Hot's VOD service (and happily pay to watch the shows I enjoy), but my true favorite for getting my entertainment is everyone's favorite not-a-dumptruck, the internet. In this post, I will describe how I do it.
Everything I describe in this post can be done using miro. It's a neat piece of software, which lacked polish in version 2.4 (2.5 is out now though), but there are a few things I don't like about it:

You have to be graphically logged in for it to run. Among other things, this means that if someone reboots your computer, there's no way to get it to start automatically. (I'll be very happy to know if I'm wrong about this)
It doesn't give you as much control as I'd like over torrents.
Its BitTorrent client doesn't perform as well as rtorrent.

However, Miro does one thing which I haven't figured out how to do myself yet: It keeps track of your position within watched shows. That is, stop watching a show -and next time playback will resume from the same place.

The first thing you want to do is get a good RSS feed for your show. Unfortunately, Revision3's shows (many of which are quite good), are direct HTTP download links, as per the advertiser's request. For other shows, you can find torrent RSS feeds, which make much better use of everyone's bandwidth. Also, downloading will be handled by our trusty rtorrent, which we can configure for bandwidth limiting.

To download RSS feeds, I use flexget. It does its job well, but doesn't support bandwidth limiting. It accepts a simple YAML configuration file, and has good logging. I run it as a cron job - its locking mechanism prevents multiple instances from running simultaneously. For non-torrents, I set the output directory to ~/torrents/inbox. For torrents, I set the output directory to ~/torrents/from_rss.

For downloading torrents, I use rtorrent. It's a curses-based client which performs very well. My .rtorrent.rc file looks like this:

download_rate = 30
upload_rate = 2
directory = /home/ohad/torrents/in_progress
on_finished = move_complete,"execute=mv,-u,$d.get_base_path=,~/torrents/inbox/ ;d.set_directory=~/torrents/inbox/"
session = /home/ohad/torrents/.session
schedule = watch_directory,5,5,load_start=/home/ohad/torrents/from_rss/*.torrent
schedule = untied_directory,5,5,remove_untied=
schedule = throttle_1,23:00:00,24:00:00,download_rate=0
schedule = throttle_2,05:00:00,24:00:00,download_rate=30
port_range = 6881-6889
encryption = allow_incoming,enable_retry,prefer_plaintext
dht = auto
peer_exchange = yes
scgi_local = /tmp/rtorrent-scgi.socket

Interesting things to note here are:

Downloads live in one directory, but get moved to the inbox directory when they're done.
The session directory is important - this allows rtorrent to resume downloads if it's shut down.
The from_rss directory is watched for new torrent files. When the relevant downloads are stopped, remove_untied occurs and the torrent files are deleted.
Throttling is fully customizable.
The SCGI socket is useful for rtgui - we'll get to that.

I have a "watchdog"-style cron job which makes sure it's running if the computer is up. This is not as elegant as starting it from an RC-script, but keeps the whole setup confined to the limits of my own user. Again, rtorrent has a lock-file which prevents multiple instances from running.

#!/bin/bash

# A simple script to make sure I am running rtorrent in a screen

set -e

SCGI_SOCKET=/tmp/rtorrent-scgi.socket
SESSION_DIR=~/torrents/.session

screen -d -m -fn -S rtorrent -s /bin/bash -t rtorrent -m nice rtorrent

while [[ ! -S $SCGI_SOCKET ]]; do sleep 1; done

if [[ -S $SCGI_SOCKET ]]; then
chgrp www-data $SCGI_SOCKET
chmod g+rwx $SCGI_SOCKET
fi

RTGUI provides a nice web-based interface. It's a bit tricky to configure, and you'll need to use an HTTP server - preferably lighttpd, as it has support for SCGI UNIX sockets (as opposed to SCGI TCP sockets). This lets you keep the number of open network ports to a minimum. This is all well-documented on the RTGUI site.

Finally, I've written a little python script called check_shows which uses libnotify to show a pretty pop-up whenever downloads complete. It's a tiny little hack which uses inotify, which is lots of fun.

That's it. Any neat tricks are welcome :)

Timezones are fickle

2009-04-18T19:45:00.000+03:00

I've been trying to work out a system to be able to cleanly switch between IST (Israel Standard Time, GMT+2:00) and IDT (Israel Daylight savings Time, GMT+3:00) on command. The logical way to do this, in my opinion, is to have two separate files in /usr/share/zoneinfo, say IsraelIST and IsraelIDT, and copy (or link) the relevant one as /etc/localtime. The trick is creating the IsraelIDT file.

My first guess was the following zic source-file:

# Zone    NAME                GMTOFF  RULES/SAVE  FORMAT [UNTIL]
Zone      IsraelIDT           2:00    01:00       IDT

Now, this almost works. The problem is that both is_dst is set and timezone = -10800 (3 hours - should be 2, as it should represent local standard time), so some software double-compensates here for a grand total of GMT+4:00. After some research (walking through __tzfile_read gave the biggest hint), it turns out that timezone is set according to the minimal local time type which is transitioned into. So I came up with this file:

# Rule  NAME    FROM  TO    TYPE  IN   ON       AT    SAVE  LETTER/S
Rule    ZionIDT min   1939  -     Jan  1        00:00 1:00  D
Rule    ZionIDT 1939  only  -     Jan  1        00:00 0:00  S
Rule    ZionIDT 1940  max   -     Jan  1        00:00 1:00  D

# Zone    NAME                GMTOFF  RULES/SAVE  FORMAT  [UNTIL]
Zone      IsraelIDT           2:00    ZionIDT     I%sT

Sounds about right, nay? Even my handy little pyzdump confirms that it looks about how I want it to:

./pyzdump.py /usr/share/zoneinfo/IsraelIDT
Transitions: ['At Sat Dec 31 23:00:00 1938, switch to IST',
'At Sun Dec 31 22:00:00 1939, switch to IDT']
Types: [<tztype dst="True" idt:="" utc+10800="">,
<tztype dst="False" ist:="" utc+7200="">]

However, it still doesn't work. A test program:

int main() {
    tzset();
    time_t t = time(NULL);
    printf("Timezone name is %s, timezone=%ld\n",
            __tzname[1], timezone);
    printf("The time is %s", ctime(&t));
    printf("Timezone name is %s, timezone=%ld\n",
            __tzname[1], timezone);
    return 0;
}

And its results, as run at 14:42:17 UTC, which is 19:42:17 IDT:

Timezone name is IDT, timezone=-7200
The time is Sat Apr 18 14:42:17 2009
Timezone name is UTC, timezone=0

Or, as I described it to a friend:

Me: Hi computer, do you know what timezone are we in?
Computer: Yeah, it's Israel Daylight Savings time, GMT+2:00 for standard time.
Me: OK, and what time is it?
Computer: 14:42
Me: No, that's 3 hours late. What timezone are we in?
Computer: Umm... UTC?
Me: You just said IDT.
Computer: Nuh-uh.

I'll get to the bottom of this eventually :/
Addendum: It seems that the problem is even more complicated. For the following timezone file, C programs seem to work fine:

# Rule  NAME    FROM  TO    TYPE  IN   ON       AT    SAVE  LETTER/S
Rule    ZionIDT min   1939  -     Jan  1        00:00 1:00  D
Rule    ZionIDT 1939  only  -     Jan  1        00:00 0:00  S
Rule    ZionIDT 1940  2030  -     Jan  1        00:00 1:00  D
Rule    ZionIDT 2030  max   -     Jan  1        00:00 0:00  S
# Zone    NAME                GMTOFF  RULES/SAVE  FORMAT  [UNTIL]
Zone      IsraelIDT           2:00    ZionIDT     I%sT

However, Python programs still show timezone = -10800. Examining Python's code, I found this:

if( janzone < julyzone ) {
    /* DST is reversed in the southern hemisphere */
    PyModule_AddIntConstant(m, "timezone", julyzone);
    PyModule_AddIntConstant(m, "altzone", janzone);
    PyModule_AddIntConstant(m, "daylight",
        janzone != julyzone);
    PyModule_AddObject(m, "tzname",
        Py_BuildValue("(zz)",
        julyname, janname));
    } else {
        PyModule_AddIntConstant(m, "timezone", janzone);
        PyModule_AddIntConstant(m, "altzone", julyzone);
        PyModule_AddIntConstant(m, "daylight",
            janzone != julyzone);
        PyModule_AddObject(m, "tzname",
            Py_BuildValue("(zz)",
            janname, julyname));
}

And since June and July have the same timezone in our case, there's a good chance that this is what's going wrong.The moral of the story seems to be this - I should go with the first, simplest "always-DST" solution. Programs should ignore the timezone variable, as in our context it isn't reliable. In general, all internal time handling should be done in UTC; when reading times from the outside world, if they are in local time - use mktime. If they are in a specified timezone, use timegm and compensate manually. I'd love to hear better ideas in the comments.

Using git for code review

2009-04-04T15:23:00.000+03:00

At my workplace, I've recently been using git for code review purposes. I work on code in my own git clone, and ask a peer to review it. It works somewhat like this:

master branch is same code as currently in upstream.
Working to resolve issue #1234 pertaining to "Performance for gizmo", I work on a branch 1234-gizmo-performance.
I mail a peer, John, with this information, as well as my repository location.
John adds my repository as a remote, lutzky. Then he branches review1 (or review2 if that is taken, and so on) at lutzky/1234-gizmo-performance.
John adds comments with nice big "FIXME" tags, which are highlighted in any decent editor. He commits this, the commit-message stating that it was code review.
John tags his final review commit (or, if he had no comments - lutzky/1234-gizmo-performance) with a "reviewed1" (or reviewed2, etc.) annotated tag. Since the annotated tag includes all the necessary information (who tagged, when, and what), the number doesn't really matter.
I merge john/review1, incorporate the changes (or reject them) and remove the comments. If no further review is necessary, I submit this - and once submitted, I merge this back into master.

It's a nice system. I wonder what other methods there are of doing this.

Hardware doesn't like me

2009-02-27T00:15:00.000+02:00

I'm a software kind of guy. Here's proof.

Today I went to visit my grandparents, and it turned out their computer wouldn't boot. BIOS would load up fine, and I could browse the menus fine - but once it tried to go on from there, it would simply blink what looked like half a cursor (that is, half of a "_"-style cursor). I figured it might be the HDD - so I took it home, and decided to connect it to my own box. Upon disconnecting my DVD drive, I destroyed the SATA cord - it had an annoying little metal tab which had to be pushed in before it would release, and it just wouldn't give, and the connector just broke, exposing and bending the wires.

Checking if the computer still boots, the BIOS took much longer to display hard drive status, and while Ubuntu would start booting, it would fail in the process and tell me that my root hard drive (by UUID) isn't available. Looking at dmesg, the ata2 module was indeed reporting that the hard drive was too slow - but a few seconds later it would finally access the drive, and mount properly. This problem, however, disappeared once I connected my grandparents' drive! (Mounting it would fail, telling me that I either have a hardware error or need to connect it to a Windows machine, which I don't have, and run some diagnostic commands). Sure enough, when the HDD is connected by itself, it gets quite flaky, but once I connect a second drive (back to the DVD, eventually), everything works properly. This probably has to do with the fact that both drives are connected on continuations of the same power cord - but I've never experienced such a problem, where you must connect devices to both connectors on the power cord. A hardware guy I know says he's never heard of such a problem either.

Naturally, these things never happen when I mess with hardware at work, where there are plenty of spare parts...

Whatever happened to black & white LCDs?

2008-11-22T13:58:00.000+02:00

I had a Game Boy once. I could play it just about anywhere, and battery life - for the time - was great. I lost it at one point, and replaced it with a Game Gear, which sucked the life out of 6 AA batteries rather quickly. The Game Boy Color was actually decent on battery life, but since it didn't have a backlight, you had to play it at very specific angles.

For gaming, I can appreciate the need for a color screen. My point has to do with cellphones. True, most cellphones today come with cameras, are able to play video, and are rather capable mobile gaming platforms (when compared to the Game Boy, that is). All this does, in fact, require a color screen. However, I believe that there is a market for cellphones which do not support these features, but do support neat things like 3G internet connectivity (GMail and RSS on the phone is a major Win, in my opinion), and have a comfortable SMS interface. These features actually suffer from having a color screen: Battery life (for the powerful backlight), viewing angle, and screen resolution take a hit. While it's true that color LCDs have come a long way since the Game Gear, so have black & white display technologies (E-Paper, anyone?).

Of course, my wish for a modern B&W-screen cellphone will likely never come true. The simple reason is that they would be totally unmarketable. Even business-types like color screens nowadays. So I'll just keep holding out for a folding E-Paper mobile browser.

Another SSH trick

2008-11-11T01:01:00.000+02:00

Ever have a machine you can only ssh into through another machine? It's a very common situation in the Technion. Here's one way to get around it: Assume you can directly ssh into alpha, and from alpha you can ssh into beta. Have the following code in your ~/.ssh/config:


Host beta
	Hostname 1.2.3.4  # IP Address of beta
	ProxyCommand ssh alpha nc -w 1 %h %p

This requires you to have nc (netcat) installed on alpha. Once you do that, you can run ssh beta directly from your own box.

Automatically starting rtorrent within screen

2008-11-04T23:33:00.000+02:00

These days I don't stay at home often, but I do have an RSS/BitTorrent combo fetching me all kinds of neat stuff for me, so I can have it ready for me on the weekend. I love rtorrent, especially due to the fact that I can run it in screen, ssh home and see how things are doing (or add more torrent to the download). However, sometimes my net connection breaks down, computers gets shut off, or things like that. This week my router broke down, so I can't even ssh home to manually start up rtorrent. My solution: A small script, which checks whether rtorrent is already running, and if not - runs it in a detached screen session. Run this with your favorite cron software.

#!/bin/bash
# A simple script to make sure I am running rtorrent in a screen

if ! ps -o uname -C rtorrent | grep -q `whoami`; then
	screen -d -m rtorrent
fi

Quick time tracking hack

2008-10-26T20:57:00.000+02:00

Gnome 2.24 adds a new Time Tracking feature, which I would have found useful. I don't have Gnome 2.24 at work, but I do have a Unix-based operating system... Here's my new ~/bin/track:


#!/bin/bash
date >> ~/time_tracking
vim ~/time_tracking +

Now, if I could only get vim to automatically hit "A" and space for me afterwards... (I'm betting there's a way to do it, but AFAIK vim can only receive ex-mode commands as parameters).

Edit: ...and, of course it's possible. Here's the new version:


#!/bin/bash
echo "`date` " >> ~/time_tracking
vim ~/time_tracking + -c 'startinsert!'

Delegating methods in Ruby

2008-10-18T23:16:00.000+02:00

Sometimes, when constructing a compound object, we are interested in exporting functionality while retaining encapsulation. For example, suppose we have a Secretary class:


class Secretary
  def send_fax(destination, fax_contents)
    puts 'Sending fax "%s" to %s' % [fax_contents, destination]
  end

  def answer_call(call)
    ...
  end

  ...
end

Our Secretary provides a lot of useful functionality, that our Boss class would like to have. Boss would like to be able to say that he can send a fax, without having the user explicitly request his Secretary beforehand. The same goes for a lot of other methods Secretary provides. Instead of writing a stub function for each of these methods, it would be nice to do the following:


class Boss
  delegate_method :my_secretary, :send_fax, :answer_call

  def initialize
    @my_secretary = Secretary.new
  end
end

john = Boss.new
john.send_fax("Donald Trump", "YOU'RE fired")

Here's how we can get this to happen:


class Class
  def delegate_method(instance_var_name, *method_names)
    method_names.each do |method_name|
      define_method(method_name) do |*args|
        instance_var = instance_variable_get("@%s" % instance_var_name)
        instance_var.send(method_name, *args)
      end
    end
  end
end

This solution does have its drawbacks - it will not work for methods which are meant to accept blocks. I'm not sure how to get that to happen, short of using a string-based class_eval, which I'm not very fond of. (I find eval to be, well, evil...)

Tracking TTime

2008-07-30T21:03:00.000+03:00

I've added issue tracking for ttime using the fantastic ditz. I've also added ttime's rdoc documentation. (Note: Version 0.8.5 is out)

Three things I didn't know Ruby does

2008-07-25T20:55:00.000+03:00

Edit: I was misled!

Illustrated here. Hints below.

>> def inspect_x_and_y(x,y); puts "x: %p, y: %p" % [x,y]; end
=> nil
>> inspect_x_and_y(y={"hello" => "world"},x=[1,2,3])
x: {"hello"=>"world"}, y: [1, 2, 3]

The bits I didn't know about:

"Format strings using a %% sign, %s, %s!" % [ "just like in python", "but with arrays" ]

The %p formatting character is the same as inspect.

You can call methods with method_name(param2=val2, param1=val1), also like in python. No you can't! This code sets external variables called y and x.

How embarassing... :(

Security project

2008-07-23T19:19:00.000+03:00

This site gets indexed by the almighty google. This link is part of a security project I'm doing for my CS degree.

Gettext oddities with Ruby

2008-07-20T22:19:00.000+03:00

I was having a lot of trouble with gettext in Ruby, mostly due to lacking documentation. Here are some useful things I figured out while writing TTime. I ended up having a single gettext_settings.rb, included from every file which uses
gettext. Here it is (with some extra notes)

#!/usr/bin/ruby
begin
  require 'gettext'
  require 'pathname'

  include GetText

  # This fixes a swarm of problems on Windows
  GetText.locale.charset = "UTF-8"

  # Ruby's gettext acts in a sane
  # method - add a path to the set of paths
  # scanned.
  locale_in_data_path = Pathname.new($0).dirname + \
    "../data/locale/%{locale}/LC_MESSAGES/%{name}.mo"
  add_default_locale_path(locale_in_data_path.to_s)
  bound_text_domain = bindtextdomain("ttime")

  # For Glade, however, it only seems to
  # be possible to specify one path at a
  # time. Fortunately, gettext already
  # found it for us.
  my_current_mo = bound_text_domain.entries[0].current_mo
  if my_current_mo
    ENV["GETTEXT_PATH"] = my_current_mo.filename.gsub(
      /locale\/[^\/]+\/LC_MESSAGES.*/,
      "locale/")
  end
rescue LoadError
  def _ s #:nodoc:
    # No gettext? No problem.
    s
  end
end

One note for context: I use setup.rb (and ruby-pkg-tools) to package TTime. So my localizations go in data/locale.