Thoughts in Air

Upgrading SBCL

2009-01-01T21:31:00.003+08:00

Recently I upgraded SBCL from version 1.0.22 to 1.0.23. Then my window manager, Stumpwm refused to start. I have to solve the issue, otherwise I will start using console exclusively starting from 2009.

My first reaction was to reinstall stumpwm package. But when compiling, SBCL is complaining that there are version mismatch for stumpwm dependencies. I tried to solve the issue by reinstalling that package, but I met the same problem for its dependencies. I realized that I just stumbled upon dependency hell. I don't want to go through the same process of installing every CL package when I upgrade SBCL. So I need to find one simple solution. Indeed it exists: just delete all *.fasl files in SBCL's site directory (which is /usr/lib/sbcl/site under Arch Linux.), and then SBCL will rebuild all the installed packages. So the following command just solves the problem (assuming Arch Linux).

find /usr/lib/sbcl/site -name "*.fasl" -delete

Update (2009-2-11): do NOT run above command on the parent directory /usr/lib/sbcl, otherwise the fasl files bundled with SBCL will be deleted and many functionalities (e.g. ASDF) will break.

Adjusting Photos for Web Albums in Batch Mode

2008-12-24T20:37:00.001+08:00

I use Picasa Web Album to store my photos online. To save the precious storage at server, I have to change jpeg files before uploading them even for those taken by my old PowerShot S30 (which outputs photos with resolution 2048x1536, which is beyond the display capacity of most monitors today). To reduce file size, one normally need to reduce both resolution and image quality.

So how to adjust images in batch mode? There are various solutions of course. The one I take is to use the combination of find/xargs/mogrify (Note that mogrify modifies files in place. Therefore one may use convert instead of mogrify to preserve original files.) in Linux. Note that mogrify is one of ImageMagick utilities and may need separate installation depending on distribution. Following is my work flow when publishing images online.

Firstly, I select images with my preferred viewer gthumb. Images are rotated if necessary. During browsing, I copy the chosen files to another folder and run the following command in that folder:

find . -iname "*.jpg" | xargs mogrify -resize resize-option -quality quality-option

(I find the above command from here.)

Several notes regarding the above command:

resize-option could be either absolute (e.g. 800x600) or relative (e.g. 50%). For my S30, I use the latter which reduce the photo resolution to 1/4 of its original size.
quality-option is JPEG compression level. The larger the value, the better the quality, but the larger the file size. I often set the value as 60 to save space.

So for photos from my S30, I use the following command:

find . -iname "*.jpg" | xargs mogrify -resize 50% -quality 60

After running above command, photo file size can be reduced around 10 times. This means that I can put more images into my web album.

W(S)XGA+ Resolution in VirtualBox

2008-08-20T20:25:00.000+08:00

Note: this post is related to a workaround for one specific problem encountered by the author.

Recently I switched from coLinux to VirtualBox. VirtualBox provides guest addition which includes enhanced support for Video. Originally it worked fine for me. Due to unknown reason, the problem arises: I could not use resolution 1440x900 (WXGA+) or 1680x1050 (WSXGA+) any more. WXGA+ is the native resolution of my laptop, while WSXGA+ is the resolution of the external LCD when the laptop is placed in a dock. After tweaking for a long time, following is the solution I found so far (I installed Gentoo Linux as guest OS).

1. Revert video driver from VirtualBox to Vesa

This is done by editing /etc/X11/xorg.conf, change one line in Section Device from Driver "vboxvideo" to Driver "vesa".



2. Install support of 1440x900 and 1680x1050 to VESA BIOS

Add parameter vga=864 as Linux bootup parameter
(e.g. in /boot/grub/grub.conf) for resolution 1440x900,
or vga=865 for 1680x1050.

When Virtual machine is turned off, run the following two commands
at Windows command line prompt, assuming virtual machine is named
"Gentoo":

VBoxManage setextradata "Gentoo" "CustomeVideoMode1" "1440x900x24"
VBoxManage setextradata "Gentoo" "CustomeVideoMode2" "1680x1050x24"


Start virtual machine, select resolution 1440x900 or 1680x1050 for
console display. Note that the resolution for console is not
necessarily the same resolution for X, i.e. you can use 1440x900 for
console, and 1680x1050 for X.

3. Add modelines for resolution 1440x900 and 1680x1050.

I use gtf to generate needed modelines. For example for
resolution 1440x900 with 60 Hz refresh rate, I use the following
command:

$ gtf 1440 900 60


which give the following line:

  Modeline "1440x900_60.00"  106.47  1440 1520 1672 1904  900 901 904 932  -HSync +Vsync


I do not like the suffix _60.00, therefore what finally
end up in section Monitor of /etc/X11/xorg.conf is
(with addition of modeline for 1680x1050):

 modeline "1440x900" 106.47 1440 1520 1672 1904 900 901 904 932 -HSync +Vsync
 modeline  "1680x1050"  147.14  1680 1784 1968 2256  1050 1051 1054 1087  -HSync +Vsync


In addition, the two modes should be added in
section Screen of /etc/X11/xorg.conf as well, my
current setting is:

Section "Screen"
 Identifier "Screen0"
 Device     "Card0"
 Monitor    "Monitor0"
 DefaultDepth     24
 SubSection "Display"
  Viewport   0 0
  Depth     24
  Modes    "1680x1050" "1440x900"
 EndSubSection
EndSection


4. Select the resolution

Now the toughest part. You may have noticed that in
section Monitor, there are two lines about HorizSync
and VertRefresh. After experimenting, my conclusion is that
one should comment out them when using 1680x1050 while enable them for
1440x900. Weird.

Since I need to switch between these two resolutions frequently,
manually editing file /etc/X11/xorg.conf is boring. Therefore
I write one script to automate the task. I named the script
as xres and invoke it as sudo xres 1440 for resolution
1440x900 and sudo xres 1680 for 1680x1050. Following is its
content:

#!/bin/bash
case $1 in
    1440)
        sed -i 's/^#    HorizSync    31.5 - 64.3/       HorizSync    31.5 - 64.3/' /etc/X11/xorg.conf
        sed -i 's/^#    VertRefresh  50.0 - 70.0/       VertRefresh  50.0 - 70.0/' /etc/X11/xorg.conf
        ;;
    1680)
        sed -i 's/^     HorizSync    31.5 - 64.3/#      HorizSync    31.5 - 64.3/' /etc/X11/xorg.conf
        sed -i 's/^     VertRefresh  50.0 - 70.0/#      VertRefresh  50.0 - 70.0/' /etc/X11/xorg.conf
        ;;
    *)
        echo "Supported horizontal resolution: 1440 and 1680."
        exit
        ;;
esac



Experiencing Stumpwm
2008-07-24T00:43:00.002+08:00
What Emacs is to editors, Stumpwm is to window managers.

 --Bill Clementson (link)

Introduction

Recently I switched from GNOME to Stumpwm (wiki), which means
that I jumped out of a desktop environment to a simple window manager.
So far, I'm very satisfied with such a change and never looked back.
Stumpwm, as its website says, is a tiling, keyboard driven X11
Window Manager written entirely in Common Lisp. This definition
summarizes the reasons why Stumpwm is so suitable for me.

First, Stumpwm is a tiling, keyboard driven window
manager. This is very useful to boost productivity. When working, it
is desirable to minimize context switching. Tilting combined with
keyboard driven enables working with several applications
simultaneously as if your are dealing with one single program. Let me
give an example. As described in previous
post, with Emacs as development environment, it is easy to fire up
one browser within Emacs itself to preview blog post. However it is
painful to switch between Emacs and browser. Before using Stumpwm, I
am considering ways to integrate Emacs and Firefox together. With
Stumpwm, such dream is just trivial to fulfill: one can show
Emacs and browser side by side, with a few keystrokes to switch
between them. Everytime you make changes in Emacs and request the
results to be shown in Firefox, they are shown simultaneously. No more
need to leave your hand for mouse to click back and forth between
Emacs and Firefox. This gives you the feeling of integrated
environment. Note that I just use Emacs and Firefox as an example, and
such convenience is applicable for every application. Currently, the
problem is that I'm so accustomed to Stumpwm keystrokes that I somehow
confused the keystrokes of switching between windows in Stumpwm and
Emacs!

Second, Stumpwm is written in Common Lisp.  which
means that you can build Stumpwm with your favorite CL implementation
and you have a Lisp runtime when you work under Stumpwm. A powerful
programming language is just embedded within your window manager and
you can invoke it at any time. The other benefit is that you can
configure your window manager as you like, even when it is
running!

You may get a feeling on how Stumpwm works by watching this nice video.

Installation

Given the above lengthy introduction of Stumpwm, you may wonder how
to install this gem. Here we assume that you have some knowledge about
Common Lisp. Basically, installing Stumpwm is to compile it with your
preferred CL implementation, and tell your system to run Stumpwm as
your window manager.

Installing Stumpwm on Gentoo Linux is straightforward. Throughout
this section, we will assume that SBCL is the CL
implementation. Stumpwm website suggests turn off threading support in
SBCL (disable USE flag threads) for better performance. To
install, simply type emerge stumpwm. Note
that at the time of writing, there is also an ebuild called
stumpwm-cvs. Simply ignore it since it is actually a very old version,
not the bleeding edge version suggested by its name. Advanced users
might consider to get git version for latest cool features. 

There are mainly two ways to invoke Stumpwm. One is to run it by
calling SBCL, the other is to dump a core image containing Stumpwm
within SBCL and invoke that image. We will take the latter
approach. To proceed, first start SBCL by typing sbcl. Next
issue the following in REPL sequentially:

(asdf:oos 'asdf:load-op :stumpwm)
(sb-ext:save-lisp-and-die "stumpwm" :executable t
                          :toplevel #'(lambda () (stumpwm:stumpwm ":0")))


Next, put the generated executable stumpwm somewhere in
PATH (I put it under /usr/local/bin). Since I always start X Window by
typing startx, following command is used to use Stumpwm as
my window manager:

$ echo "exec stumpwm" >> ~/.xinitrc


Using Stumpwm

Configuration

All stumpwm configurations are stored in
file ~/.stumpwmrc, which is written in Common Lisp. Just
like .emacs, this file allows you to fully customize
Stumpwm. Following is my current configuration:

;;;; -*- Mode: Lisp -*-

(in-package :stumpwm)

;; Load swank.
(load "/usr/share/emacs/site-lisp/slime/swank-loader.lisp")
(swank-loader:init)
(define-stumpwm-command "swank" ()
  (setf stumpwm:*top-level-error-action* :break)
  (swank:create-server :port 4005
                       :style swank:*communication-style*
                       :dont-close t)
  (echo-string (current-screen) "Starting swank."))
(define-key *root-map* (kbd "C-s") "swank")       

;; Customize bars and modeline.
(setf *message-window-gravity* :center)
(setf *input-window-gravity* :center)
;; Turn on mode line.
(toggle-mode-line (current-screen) (current-head))
(setf *screen-mode-line-format* 
      (list "%w | "
            '(:eval (run-shell-command "date" t))))

(set-prefix-key (kbd "C-i"))
(define-key *root-map* (kbd "c") 
  "exec urxvt +sb -fn \"xft:Bitstream Vera Sans Mono:pixelsize=20\"")

(define-stumpwm-command "firefox" ()
  "Run or switch to firefox."
  (run-or-raise "firefox" '(:class "Firefox")))
(define-key *root-map* (kbd "f") "firefox")


Some explanations of my configuration:


  Prefix key: I use C-i instead of
  default C-t. The reason is that C-t is used in
  Firefox to open a new tab and also in Emacs for transpose. Then why
  choose C-i? I'd like to admit that it is quite difficult
  to select a prefix key for Emacs user. Before settle down
  on C-i, I fired up Emacs to see whether there is any key
  binding C-x where x from a to z is not used by
  Emacs. Unfortunately (fortunately?), Emacs binds every
  combination. So I can only choose one prefix key which is easy to
  type and I uses infrequently in Emacs. Then C-i is
  selected. Note that you can send C-i to application like
  Emacs by typing C-i i. For the following sections, please
  replace C-i with your favorite prefix key.
  SLIME: the section staring with
  comments Load swank provides ways to load
  swank. Type C-i C-s to start swank. To connect to swank,
  simply run Emacs, and type slime-connect within Emacs,
  and type RET and RET to accept default host
  (127.0.0.1) and default port (4005). Then you can play with Stumpwm
  as you wish: change parameters, add your own functions etc. Note
  that I do not start swank automatically for security reasons. 
  Key bindings: I use C-i c to start
  console: urxvt instead of xterm. Note that I have
  also set the font for urxvt. In addition I have setup using C-i
  f to start Firefox in case Firefox is not started, or bring
  Firefox window to front if it is already running.


Key Bindings

Following is a list of key bindings I used frequently. For
simplicity, I have omitted the prefix key.


  ?: Stumpwm help
  ;: Run Stumpwm commands
  :: Send commands to Common Lisp interpreter.
  Space: Go to next window
  c: Run X terminal
  e: Run Emacs or raise it if it is already running
  f: Run Firefox or raise it if it is already running
  k: Kill current window
  g c: Create a new group
  g k: Kill current group
  g m: Move current window to a specified group
  g Space: Next group
  o: Focus shifts to next frame
  Q: Remove all splits
  s: Vertical split
  S: Horizontal split

   



Using PulseAudio for OSS Applications
2008-05-21T20:13:00.000+08:00
In my previous
post, I discussed how to use PulseAudio to enable sound for
coLinux. This works fine for programs like MPlayer, but has issues
with some programs Solfege.

The solution is simple: if the problematic applications have
support for OSS, then emerge them with OSS USE flag on. In
addition, PulseAudio should also be emerged with OSS USE flag
(to enable the building of padsp, a PulseAudio OSS wrapper, which will
be discussed below). Under Gentoo Linux, one way is to simply
add OSS USE flag to /etc/make.conf. Alternative way
is to enable the USE flag application by application. For example, to
enable OSS USE flag for Solfege, type the following:

# echo "media-sound/solfege oss" >> /etc/portage/package.use


When running these applications, prefix the command line
with padsp like the following:

$ padsp solfege


Note that you may need additional configurations in the application
 to use OSS for audio output.

   



HV30: Camcorder and Accessories
2008-05-05T21:22:00.002+08:00
I just bought Canon HV30 (Amazon, Review)
recently for my baby. Before buying HV30, I was agonizing over the
selection between HV30 and Canon HF10 (Amazon, Review)
(although I know that HF100 is better than HF10 from money point of
view since the internal 16GB flash drive of HF10 does not justify the
price difference, I do like the black paint job of HF10 and HV30).

Comparison of HV30 vs. HF10

So why I chose HV30 over HF10? The following is my comparison.


Features HV30 HF10 Comments

  Price
  MSRP price is $999 in US and
  ¥10580.00 (about $1514) in China
  MSRP price is $1099 in US and ¥14080.00 (about $2014) in
  China
  The price difference in China is quite significant, which is
  true even for street price.


  Performance
  1/2.7 in. sensor
  1/3.2 in. sensor
  According to the review,
HV30 has slightly better performance over HF10 thanks to larger
sensor, however the gain seems to be marginal for most cases.


  Functionalities
  Viewfinder, Zebra, Peaking
  
  


  Accessories
  Standard accessory shoe
  Canon proprietary accessory shoe
  


  Backup
  Tapes are cheap for backup
  Additional backup is needed since SDHC cards are expensive and
  will be reused.
  


  Community
  A mature community (looking at hv20.com
  for example)
  Situation will improve
  


  Form Factor
  88x82x138 mm (3.5x3.2x5.4 in.), 535 g (1.2 lb.)
  73x64x129 mm (2.9x2.5x5.1 in.), 380 g
  (13.4 oz)
  HF10 is small and light, which is ideal for traveling. Normally
  the camcorder which you takes out often is the most useful one.


  Media
  Tape
  SDHC card + internal Flash
  Flash card is really convenient: just like small DCs. No hassles
  for rewinding and no worry for overriding.


  Zoom
  10x optical zoom
  12x optical zoom
  


  Noise
  Tape noise can be heard in silent environment
  No such noise since flash card is
  used.
  



Accessories

What I have:


  Battery and charger: compatible ones
  Camera bag: Lowepro Edit 120+
  CPL filter: Kenko 43mm
  HDMI cable
  IEEE 1394 cable: Belkin 4-pin to 4-pin
  Lens cap
  miniSD: Transcend 2G. Not sure whether 4G could be used (the
  manual only states that up to 2G has been tested).
  Tape: Panasonic AY-DVM63PQ
  UV filter: Kenko MC-UV 43mm


For future purchase:


  35mm adapter
  Close-up lens: +1/+2/+4
  Mic: Canon DM-50, or combination of Rode Videomic and stereo
  mic. Maybe XLR adapters with XLR Mics?
  Neutral density filters: ND2/4/8
  Shoulder strap
  Steadicam Merlin
  Telephoto lens: Canon TL-H43 (any black version?)
  Tripod and fluid head: Manfrotto 190XPROB (tripod) + 701RC2
  (fluid head) + MBAG70 (bag)
  Video light: Canon VL-3
  Wide angle lens: Canon WD-H43 (any black version?)
  Wrist strap


Final Words

After over 1 month's usage of HV30, I'm pretty satisfied with
it. There is a lot of things for me to learn: how to shoot, how to
edit, just to name a few. But all the efforts are worthy when you view
the video clips produced from it.

   



Automatic Backup
2008-05-02T16:30:00.000+08:00
We need to backup our stuff in case that some disasters might
happen. One way is to backup to some external medias like tape, USB
drives, or CD/DVDs. However working with them is somehow tedious and
manual intervention is needed. People tend to be lazy therefore we are
reluctant to backup if the process is inconvenient. If we have one
additional machine with link in between (throughout this article, we
assume the remote machine is named as remotebox), we can setup
an automatic backup mechanism.

Overview

We use crontab to schedule rsync jobs to backup our files to
another machine. Since rsync is secured with SSH, to suppress the
password prompt, we have to setup SSH keychain.

Setup SSH Keychain

Generate Keys

This section mainly follows this nice
guide.

We use the DSA approach.

$ ssh-keygen -t dsa
$ scp ~/.ssh/id_dsa.pub user@remotebox


Then log in to remotebox and append the public key to
~/.ssh/authorized_keys file like so:

cat id_dsa.pub >> ~/.ssh/authorized_keys
rm id_dsa.pub


Install Keychain

Keychain is a nice package which prompt you for your passphrase
once you log into the system, and automatically provides passwords for
later SSH session. Note this incurs security risks (there are always
tradeoffs between security and convenience) therefore attention is
needed.

This part mainly follows this guide.

First install keychain by typing emerge keychain.

Then add following lines to your ~/.bash_profile.

/keychain id_dsa
. ~/.keychain/$HOSTNAME-sh


Setup Crontab Job

We can then setup our automatic backup system now. Since there are
almost infinite number of combinations of crontab and rsync patterns,
I just shown one example below (many interesting examples of rsync
usage can be found by typing man rsync).

0 9,16 * * 1-5 rsync -az -e ssh --delete ~/work/ remotebox:~/backup/


The above example synchronizes local directory ~/work to
remote directory ~/backup on host remotebox, on 9
a.m. and 4 p.m. of every working day. Note that I have used
option --delete, which will try to delete extraneous files
from destination directory. This can ensure that the destination
directory is an exact copy of source directory but it would be
dangerous if the destination directory contains some files already. So
test this option before you actually use it.

   



Switching Browsers in Emacs
2008-05-02T16:24:00.001+08:00
Recently, I have met one dilemma in Emacs. Since I'm using Emacs for my blog writing, Firefox is used for
previewing. Also as an Common Lisp user, I use emacs-w3m to read
HyperSpec. Here comes the problem: I can only set one browser
for browse-url-browser-function. How can I suit my
requirements simultaneously?

Fortunately, Emacs is highly extensible. I just write one simple
interactive function toggle-browser, which can toggle between
Firefox and emacs-w3m when pressing M-x
toggle-browser. Following is the code:

(defun toggle-browser ()
  "Toggle browser between Firefox and emacs-w3m."
  (interactive)
  (setq browse-url-browser-function 
        (if (eql browse-url-browser-function 'browse-url-firefox)
            'w3m-browse-url
          'browse-url-firefox))
  (message "%s" browse-url-browser-function))


Update: please look at the 1st comment from pedro for a better solution.

   



Using Imaxima with Texlive in Gentoo Linux
2008-04-10T00:37:00.002+08:00
Imaxima is an Emacs extension to interface with Maxima (see here for a nice
tutorial). To use it, one need a LaTeX distribution. However with
default Texlive installation in Gentoo, imaxima cannot work. Following
is a sample session:

(%i1) integrate(x,x);
LaTex error in: \ifracn{x^{2}}{2}


The problem here is that there is another package breqn
needed. Breqn is inlcuded in Texlive
package texlive-mathextra, which is not installed when
you emerge texlive. The package is also masked by default,
therefore you need to unmask it before emerging. Following is a list
of commands to install the package (substitute x86
with amd64 for x86-64 systems).

# echo "dev-texlive/texlive-mathextra ~x86" >> /etc/portage/package.keywords
# emerge texlive-mathextra


After installation, fire up your Emacs and enjoy (I)maxima!

Update: if you are not comfortable with the
default equation size (as in my case), you can add the following line
into your .emacs (adjust the number to suit your own taste):

(setq imaxima-scale-factor 2.0)

   



Copy and Paste between Windows and Linux
2008-04-08T21:36:00.001+08:00
Many people use Windows as primary working machine and login to
remote Linux/Unix (we will use Linux as shorthand afterwards)
boxes. There are times when we need to copy and
paste information between Windows and Linux. Below we are going to
discuss convenient ways to do so.

Tools

PuTTY and Xming are two popular programs to run in Windows host to
facilitate communication to Linux. PuTTY provides console connection
while Xming X Window connection. For Xming, Ensure you start Xming
with parameter -clipboard (which is enabled by default). Our
discussion below is focused on how to copy/paste within
PuTTY/Xming. Copy/paste for other Windows applications is quite
general (Ctrl+C and Ctrl+V) and should be
familiar to the reader, therefore omitted.

Ways

PuTTY

Copy to clipboard: click-and-drag to
highlight. Just select, no need to press any key.

Paste from clipboard: right mouse
button.

Xming

For Xming applications, copy/paste follows the X window
standard as discussed below.

Copy to clipboard: click-and-drag to
highlight. Same as PuTTY.

Paste from clipboard: middle mouse
button.

   



Installation of Gentoo Linux 2008.0 Beta 1 with LiveCD
2008-04-07T21:17:00.002+08:00
I just started yet another process of installing Gentoo Linux, and
following is my record of the installation step by step. Previously I
preferred to install from stage 1/3 following
this
guide. I decided to try LiveCD this time.

1. Boot the machine.

Download the 2008.0 LiveCD from here. Burn the CD
and boot the machine with it.

Caveat: in my case, the boot process is stalled. I
solved it (according to this
article) by appending kernel options during GRUB screen
(press e to edit and append all-generic-ide irqpoll
pci=nommconf).

2. Installation.

After boot, I click icon "Gentoo Linux Installation (GTK+)" in the
desktop.

Partitioning: I selected Recommended
Layout.

Local Mounts: accept the defaults without any
changes.

Timezone: Asia/Shanghai.

Networking: Dhcp is used. Appropriate host and
domain name are provided.

Users: Just add the regular user for daily
usage.

Extra Packages: I selected screen and slocate. In
my first installation I also selected xorg-x11, gdm, firefox
etc. However the installation simply failed when emerging
x11-apps/xinit (and I cannot rescue the system since even /etc/fstab
is not correctly setup), therefore my second try is quite
conservative.

Startup services: I selected sshd.

Other Settings: Although I have not installed X11
or Gnome yet, I selected as following:


  Display Manager: gdm
  Clock: local
  XSession: Gnome


3. Post Installation.

This section is similar as what I have described for coLinux
(see here),
but is modified specifically for a standalone Gentoo Linux
installation.

3.0 Tiny Cleanups.

After rebooting, perform the following steps. Note that if your
system was stalled as described before, you can use same trick in GRUB
screen and you should add the kernel options to /boot/grub/grub.conf
after login.

To avoid some warning, you might need to perform following
steps.

# mkdir -p /usr/local/portage


To change host name (according to Gentoo
Handbook), edit file /etc/conf.d/hostname, if you call
your coLinux machine as tux, then set the following line.

HOSTNAME="tux"


In addition, remove string .\O from
file /etc/issue to get rid of the
annoying unknown_domain in the welcome message "This is
host.unknown_domain ...".

3.1 Network

We have already specified using DHCP. It works but the
configuration files is using old syntax. Let's clean it up to avoid
future complaints. Edit file /etc/conf.d/net as
follows.

config_eth0=( "dhcp" )


3.1.1 Proxy

If you connect to Internet directly, you can skip this section.

To use proxy server, add the
following lines into /etc/env.d/99local:

http_proxy="http://proxy.server.com:8080"
ftp_proxy="http://proxy.server.com:8080"


Replace the port and domain name according to your situation.  Then
run:

env-update && source /etc/profile


3.2 Add additional locales

Modify file /etc/locale.gen, select appropriate
locales. For me, the locales are:

en_US.UTF-8 UTF-8
zh_CN.UTF-8 UTF-8


Then run command locale-gen.

3.3 Modify /etc/make.conf

This is specific for your computer type and other
factors. Following is my configuration (I'm using a Core 2 Duo
CPU). Please refer to Gentoo documentation for your own
reference. GENTOO_MIRRORS reflects the fastest mirrors for my
connection, as discussed below.

CFLAGS="-march=nocona -O2 -fomit-frame-pointer -pipe -fno-ident"
CXXFLAGS="${CFLAGS}"
CHOST="x86_64-pc-linux-gnu"
LDFLAGS="-Wl,-O1"
MAKEOPTS="-j3"
ACCEPT_KEYWORDS="amd64"
FEATURES="parallel-fetch"
LINGUAS="en zh zh_CN"
INPUT_DEVICES="keyboard mouse"
VIDEO_CARDS="i810"
PORTDIR_OVERLAY="/usr/local/portage"
GENTOO_MIRRORS="http://ftp.iij.ad.jp/pub/linux/gentoo/ http://ftp.kaist.ac.kr/pub/gentoo/"

USE_DEV="bash-completion doc emacs latex ruby sbcl source spell"
USE_HW="cpudetection mmx sse sse2"
USE_LIB="curl gd glibc-omitfp ncurses readline zlib"
USE_MM="aac alsa encode ffmpeg flac midi mp3 ogg pulseaudio quicktime theora xvid"
USE_PIC="gif jpeg png raw svg tiff xpm"
USE_SECURITY="crypt pam ssl tcpd"
USE_SW="bzip2"
USE_SYS="cjk nls nptl nptlonly opengl truetype unicode xinerama xml"
USE_X11="cairo gnome gtk X xft xorg"
USE="-* ${USE_DEV} ${USE_HW} ${USE_LIB} ${USE_MM} \
     ${USE_PIC} ${USE_SECURITY} ${USE_SW} ${USE_SYS} ${USE_X11}"


3.4 Synchronize Portage

Use either emerge --sync or emerge-webrsync
-v to update the portage.

3.5 Mirror selection

First run emerge mirrorselect. To select the fastest
mirror. I use the following command mirrorselect -D -H
-s2 to select the mirror. It means
that:



  -D: actual file will be downloaded from the mirror to test speed.
  -H: test http mirrors only.
  -s2: only select 2 mirrors.


After running the command, original /etc/make.conf will be
backed up as /etc/make.conf.backup and your selected mirrors will be
reflected in make.conf. Type mirrorselect -h for
details.

3.6 Update system

Run the following suites of commands (we need the -p argument to
check what the commands are supposed to do since they're somehow
"dangerous"):

emerge -uDNpv world
emerge -uDNv world
emerge --depclean -pv 
emerge --depclean -v
revdep-rebuild -pv
revdep-rebuild -v


Caveat: if you encounter circular dependency when
emerge -uDNv world, try to remove gtk USE flag
from /etc/make.conf. After successful emerging, add
gtk flag again and run emerge -uDNv world again. 

After update, run dispatch-conf to update the
configuration files.

3.7 Install X11 and GNOME.

Run command emerge xorg-x11 to install X11. Then
use xorgcfg -textmode to configure X server.

I'm using a LCD with resolution of 1680x1050, which is not listed
in standard configurations. To use it, first run gtf 1680 1050
60, copy the output to the Monitor section
of /etc/X11/xorg.conf, and modify section Screen to
the resolution 1680x1050.

For GNOME, I prefer to install gnome-light to avoid the gigantic
full GNOME installation. To do so, simply run emerge
gnome-light.

If one use startx to run X11, one can simply
type echo "exec gnome-session" > ~/.xinitrc to configure
so.

To make gdm the default login screen, run the following commands:

# emerge gdm
# rc-update add xdm default


And then edit /etc/conf.d/xdm, changing DISPLAYMANAGER
to gdm.

3.8 Install Other programs.

This section is left empty intentionally :-)



Conclusion

It is quite convenient to install with LiveCD. There are some
hassles during installation, but I assume the reason is that I'm using
a beta version. Since beta version is not stable, my suggestion is to
install minimal components therefore you can quickly finish the
installation and boot from hard drive.

Although GRP install is not so optimal, the pre-compiled packages
should be quite close to optimal ones for 64-bit machines. Also as a
Gentoo user, I'm always in the process of customization the OS to suit
my needs, therefore a not-so-ideal-initial-setup is not an issue. In
summary, livecd approach can save a lot of time, with almost similar
end result as installing from stage 1/3.

   



Ripping CD to FLAC in Gentoo Linux
2008-04-07T21:15:00.000+08:00
This post is just a simple note, inspired from here.

First, install cdparanoia and flac:

# emerge cdparanoia flac


Next, issue the following commands (I deleted the original WAV
files after conversion). All tracks are extracted and converted to
FLAC format.

# cdparanoia -B
# find . -name "*.wav" -print0 | xargs -0 flac
# rm *.wav


Then enjoy the music with your favorite player. Since I'm using
mplayer, I call Ruby to generate the playlist and invoke mplayer with
it (in the 1st line below, the last track number is 16. Change it
according to your own situation).

$ ruby -e '(0..16).map{|i| printf("track%02d.cdda.flac\n", i)}' > pl
$ mplayer -playlist pl

   



Enable Sound for coLinux
2008-04-02T20:36:00.000+08:00
In my previous post about installing
coLinux with Gentoo, audio part is not touched. Here comes the
solution on how to use PulseAudio to bridge coLinux to sound
world. This is adapted from here
with contents modified for Gentoo Linux. 

Windows side

We need to install PulseAudio server on Windows. Frist
download binaries for Windows from here. Extract to one
directory (c:\bin\pulseaudio in my case).

In the directory of the binaries, create a text config file named
default.pa with the content:

load-module module-native-protocol-tcp auth-ip-acl=127.0.0.1;192.168.0.0/24
load-module module-esound-protocol-tcp auth-ip-acl=127.0.0.1;192.168.0.0/24
load-module module-detect


We start PulseAudio automatically when coLinux starts, by adding
the following line to the coLinux configuration file:

exec0=c:\bin\pulseaudio\pulseaudio.exe


Linux side

Install and configure PulseAudio client

Install PulseAudio by emerge pulseaudio. Then
edit /etc/pulse/client.conf to set the address of the Windows
host that runs the PulseAudio server:

default-server = 192.168.0.1


where 192.168.0.1 is the IP address of the Windows
host.

Make sure to add USE flag alsa
to /etc/make.conf. If you just add this USE flag, update your
system with emerge -uDNv world. ALSA would be automatically
emerged.

Install alsa-plugins

In addition, we should install alsa-plugins:

# echo "media-plugins/alsa-plugins pulseaudio" >> /etc/portage/package.use
# emerge alsa-plugins


Edit ~/.asoundrc with the following contents.

pcm.!default {
    type pulse
}

ctl.!default {
    type pulse
}

pcm.pulse {
    type pulse
}

ctl.pulse {
    type pulse
}


Test

As a final step, you can test with an ALSA player:

aplay /usr/share/sounds/alsa/Front_Center.wav


You should hear a voice saying "Front Center". Enjoy!

   



Install Gentoo with coLinux
2008-03-16T14:53:00.006+08:00
coLinux is gaining momentum as a way to run Linux programs on
Windows, and I decided to try it. As a user
of Gentoo, my first choice is to use Gentoo with coLinux. The
following description documents what I have done to run Gentoo
cooperatively in Windows laptop. I mainly follows up the Gentoo Wiki Howto article
and other coLinux resources.

1. Installation

First download the latest coLinux snapshot (you can
also get it from SourceForge). The
file I used is coLinux-0.7.2.exe from SourceForge. Run the
installer. Some notes during installation:


  In screen Choose Components, leave the default settings
  on (i.e. selecting all components), but uncheck Root FileSystem
  image download (we will download it separately).
   In addition, change the Destination folder to
  c:\bin\coLinux (note that all the description below assumes
  coLinux is installed into this directory. If you prefer other
  directory, please modify accordingly in steps below). 
  Installer prompts to download and install WinPCap separately,
  but my network configuration does not need WinPCap therefore I don't
  install it.


Then download Gentoo-colinux-i686-2007-03-03.7z from SourceForge,
extract the archive into c:\bin\coLinux, with a
gentoo subdirectory created during extraction. Copy file
gentoo.conf in gentoo to the parent directory and
modify it as follows:

kernel=vmlinux
cobd0=C:\bin\coLinux\gentoo\gentoo-root
cobd1=C:\bin\coLinux\gentoo\linux-swap
root=/dev/cobd0
hdc=\Device\Cdrom0
initrd=initrd.gz
mem=1024
eth0=tuntap
cofs0=c:\linux


Some explanations of the above file, details will be covered in
later sections.


  For mem, I set it to 1G. Choose according to your own
  situation.
  For network, I only select TAP driver.
  I shared Windows folder c:\linux to coLinux.
  CD/DVD-Drive is mounted. 


To test drive, open a command prompt, change to the directory
c:\bin\coLinux, and run the following command:

colinux-daemon @gentoo.conf


coLinux console should run. If there is something wrong, please
check previous steps. After some time, you are prompted for log in.
You can now login as root with password. Change your password
immediately.

Congratulations! You just successfully run Gentoo in coLinux! You
can now fool around with the system. Type halt when you are
tired. There is still a long road ahead to fully harness the
power. Following is two optional steps, but highly recommended.

Run coLinux during system startup

First, we need to install coLinux as a service. Open a command
prompt, change to the directory c:\bin\coLinux, and run the
following command:

colinux-daemon.exe @gentoo.conf --install-service "CoGentoo Linux" 


Then open Control Panel, first select Administrative
Tools, then Services, click service CoGentoo Linux, and
change its Startup type to Automatic.

Enlarge the image file

The original image size is 2G. This might not be sufficient. To
enlarge it, you need to download the tools from here. Extract the
archive and run file toporesize.bat. Use the program to
resize file c:\bin\coLinux\gentoo\gentoo-root to whatever
size you want. Make sure you uncheck "resize file
only no resize2fs".

2. Configuration

2.1 Timezone

Since I'm located in Beijing, I set the corresponding info as
follows. The related configuration files is
/etc/conf.d/clock. I changed two lines as follows.

CLOCK="local"
TIMEZONE="Asia/Shanghai"


Also run command ln -fs /usr/share/zoneinfo/Asia/Shanghai
/etc/localtime.

It is interesting that Beijing cannot manage to snap one place
inside /usr/share/zoneinfo/Asia.

2.2 Network

Based on coLinux
wiki article.

2.2.1 One network interface

This configuration only enable TAP, which is used to enable both
the high-speed private connection between Windows host and Linux
guest, and also to bridge the connection to Internet. One drawback of
this setup is that when you Internet connection changes (e.g. from
ADSL to WiFi, you have to adjust bridging and some parameters
manually).

File gentoo.conf discussed above already contains the
setup for coLinux. Next steps are as follows.

On hosting OS (Windows), configure TAP-Win32 Adapter V8
(coLinux) with following properties (make sure to leave
Default Gateway as blank):

Ethernet adapter:

        Connection-specific DNS Suffix  . :
        Description . . . . . . . . . . . : TAP-Win32 Adapter V8 (coLinux)
        Dhcp Enabled. . . . . . . . . . . : No
        IP Address. . . . . . . . . . . . : 192.168.0.1
        Subnet Mask . . . . . . . . . . . : 255.255.255.0
        Default Gateway . . . . . . . . . :


We need to bridge the Internet connection with TAP interface. For
example, assume you're using an ADSL PPPoE connection. In Windows
host, select Control Panel -> Network Connections, right
click the PPPoE interface, and select Properties. In tab
Advanced, for Internet Connection Sharing, check
Allow other network users to connect through this computer's
Internet connection, and for the Home networking
connection, select the TAP interface (e.g. Local Area Connection
2). When Windows prompts that TAP interface IP address will be set to
192.168.0.1, choose OK.

Under coLinux, edit file /etc/conf.d/net as follows.

config_eth0=( "192.168.0.20/24" )
routes_eth0=( "default via 192.168.0.1" )
dns_servers_eth0=(" xxx.xxx.xxx.xxx " )


Please change the DNS server information (shown as xxx above) to
those obtained in windows (e.g. via ipconfig).

We have to emphasize here that in this setup, Linux guest has IP
address 192.168.0.20 (this is arbitrarily selected, could be other
numbers like 192.168.0.x when 2 <= x <= 254), and Windows host has IP
address 192.168.0.1 interfacing Linux (of course Windows host has
external IP addresses as well).

2.2.2 For two network interfaces

Note this part is NOT compatible with other sections. The purpose
of this section is only to document alternatives to above solution. If
one interface solution is fine for you, you can skip this section.

This configuration enables both SLiRP and TAP connections. SLiRP is
used to connect outside world while TAP enables the high-speed private
connection between Windows host and Linux guest, which is desirable
for X protocol. The benefits of this configuration includes improved
security and that SLiRP can automatically use current Windows Internet
connection for outgoing connections (i.e. you do not need to enter DNS
server information) without reconfigurations. This works fine for
Wireless LANs, but I did meet problems for ADSL and finally decided to
use TAP only.

Anyway, let's look at how to setup two network interfaces.

First, we need to modify the aforementioned file
gentoo.conf located in coLinux directory. Change the line
eth0=tuntap with two lines as below.

eth0=slirp
eth1=tuntap


Then edit file /etc/conf.d/net on Linux guest. This
enables both SLiRP and TAP.

config_eth0=( "dhcp" )
config_eth1=( "192.168.37.20/24" )


We then need to setup DHCP client as following:

emerge dhcpcd
rc-update add dhcpcd default
/etc/init.d/dhcpcd start


Since only eth0 device is generated, we need to generate
eth1 device by 

cd /etc/init.d
ln -s net.lo net.eth1


On hosting OS (Windows), configure TAP-Win32 Adapter V8
(coLinux) with following properties (make sure to leave
Default Gateway as blank):

Ethernet adapter:

        Connection-specific DNS Suffix  . :
        Description . . . . . . . . . . . : TAP-Win32 Adapter V8 (coLinux)
        Dhcp Enabled. . . . . . . . . . . : No
        IP Address. . . . . . . . . . . . : 192.168.37.10
        Subnet Mask . . . . . . . . . . . : 255.255.255.0
        Default Gateway . . . . . . . . . :


2.2.3 Proxy

If you connect to Internet directly, you can skip this section.

To use proxy server, add the
following lines into /etc/env.d/99local:

http_proxy="http://proxy.server.com:8080"
ftp_proxy="http://proxy.server.com:8080"


Replace the port and domain name according to your situation.  Then
run:

env-update && source /etc/profile


2.3 Add additional locales

Modify file /etc/locale.gen, select appropriate
locales. For me, the locales are:

en_US.UTF-8 UTF-8
zh_CN.UTF-8 UTF-8


Then run command locale-gen.

2.4 Mirror selection

First run emerge mirrorselect. To select the fastest
mirror. I use the following command mirrorselect -D -H
-s2 to select the mirror. It means
that:


  -D: actual file will be downloaded from the mirror to test speed.
  -H: test http mirrors only.
  -s2: only select 2 mirrors.


After running the command, original /etc/make.conf will be
backed up as /etc/make.conf.backup and your selected mirrors will be
reflected in make.conf. Type mirrorselect -h for
details.

2.5 Modify /etc/make.conf

This is specific for your computer type and other
factors. Following is my configuration (I'm using a Core 2 Duo CPU on
32 bit Windows). Please refer to Gentoo documentation for your own
reference. GENTOO_MIRRORS reflects the fastest mirrors for my
connection, so you may skip that line if your have already performed
mirrorselect above.

CFLAGS="-march=prescott -O2 -fomit-frame-pointer -pipe -fno-ident"
CXXFLAGS="${CFLAGS}"
CHOST="i686-pc-linux-gnu"
LDFLAGS="-Wl,-O1"
MAKEOPTS="-j2"
ACCEPT_KEYWORDS="x86"
FEATURES="parallel-fetch"
LINGUAS="en zh"
USE="-* auctex bash-completion bzip2 cjk crypt curl doc emacs gd gif glibc-omitfp gtk \
     javascript jpeg mmx mozilla ncurses nls nptl nptlonly opengl pam png readline \
     ruby sbcl source spell sse sse2 ssl svg tcpd tiff truetype \
     unicode X xft xml xorg xpm zlib"
PORTDIR_OVERLAY="/usr/local/portage"
GENTOO_MIRRORS="http://ftp.iij.ad.jp/pub/linux/gentoo/ http://ftp.kaist.ac.kr/pub/gentoo/"


2.6 Update system

Since our original system is somehow outdated, we need to update
it. First run emerge --sync or emerge-webrsync
-v to synchronize portage, then run the following suites of
commands (we need the -p argument to check what the commands are
supposed to do since they're somehow "dangerous"):

emerge -uDNpv world
emerge -uDNv world
emerge --depclean -pv 
emerge --depclean -v
revdep-rebuild -pv
revdep-rebuild -v


Caveat: if you encounter circular dependency when
emerge -uDNv world, try to remove gtk USE flag
from /etc/make.conf. After successful emerging, add
gtk flag again and run emerge -uDNv world again. 

After update, run dispatch-conf to update the
configuration files (you need to avoid the update of
/bin/setfont. If you accidentally updated the file, you can
remedy the situation as discussed below):

Caveat: if during booting, you meet the error
message putfont: PIO_FONT: Function not supported, Solve the
problem as follows (as discussed here):

First, run command mv /bin/setfont /bin/setfont.old

Create a new file /bin/setfont with the following content:

#!/bin/bash

if ! uname -r | grep -q -e "-co-"
then
        /bin/setfont.old $@
fi


and make it executable by running chmod +x /bin/setfont

3. Run X

X.org is already installed in coLinux machine when we update the
system as discussed in section 2.6 (we have include X USE
flag). Therefore the remaining steps are just to install X server in
Windows host.

We choose Xming considering it is lightweight and widely used. To
install Xming, Download Xming
installer, font
installer and Xming-tools-and-clients. You
may use Xming-mesa
installer to replace Xming installer for stable OpenGL operation (only
do so if you meet problems with Xming installer).

My settings during installation of Xming are as follows:


  Xming
    
      Destination directory: changed to c:\bin\Xming.
    
  
  Xming-fonts
    
      Destination directory: changed to the same directory as
      Xming (c:\bin\Xming)
      Fonts to be installed: select according to your own taste.
    
  
  Xming-tools-and-clients extract the zip file of
  Xming-tools-and-clients to the directory where Xming is installed
  (c:\bin\Xming in this case).


Settings on Windows host

For file x0.hosts in Xming directory, make sure the IP
address of coLinux machines (192.168.0.20) is added.

If you want to run the tools and clients bundled with Xming in
Windows host, you need to type set DISPLAY=127.0.0.1:0 in
your Windows host.

Following steps are used to enable Windows fonts to be used by X
application. This step is optional.

Open a command prompt window, change to directory
c:\bin\Xming, and type the following commands:

mkfontscale C:/WINDOWS/Fonts
mkfontscale -b -s -l C:/WINDOWS/Fonts


Make sure that directory C:\Windows\Fonts appeared in the
file font-dirs in Xming directory.

On coLinux machine

Add the following line into /etc/profile.

export DISPLAY=192.168.0.1:0


After start Xming (preferably to start it automatically when
Windows start), you can simply invoke any command inside Linux console
like:

xterm &


4. Miscellaneous

Change host name

According to Gentoo
Handbook.

Edit file /etc/conf.d/hostname, if you call your coLinux
machine as tux, then set the following line.

HOSTNAME="tux"


Edit file /etc/conf.d/net, if you call your domain as
homenetwork, then set the following line.

dns_domain_lo="homenetwork"


Remove string .\O from file /etc/issue to get rid
of the annoying unknown_domain in the welcome message "This
is host.unknown_domain ...".

Create regular user accounts

With the following command, I created a new user named foo as a
powerful user (with the ability to su to root and also invoke some
portage related commands). Pleas substitute foo above with
your desired user name.

useradd foo -m -G users,wheel,portage
passwd foo


Access Windows folder

When we edit gentoo.conf above, we have already enabled
coLinux to share folder c:\linux. To make our regular user to access
this directory in Linux, edit file /etc/fstab and add the
following line:

cofs00  /home/foo/windows  cofs  uid=foo,gid=foo,dmask=0755,fmask=0644 0 0


The above line will enable the Windows folder to be shared in Linux
as a folder /home/foo/windows. You can modify above line for your own
user.

Additional programs

Following is a list of commands to install my own favorite
programs. Your situation would be of course different.

# Some system utilities
emerge gentoolkit slocate

# Development tools
echo "dev-lang/erlang hipe" >> /etc/portage/package.use
emerge aspell ruby sbcl erlang dev-util/git maxima

# Gfx (GLEidescope is one of my favorite XScreenSaver)
emerge inkscape blender xscreensaver

# Netowork
emerge mozilla-firefox

# X tools
emerge xlsfonts xfontsel


Before emerge texlive, I have to add a bunch of keywords
for dependencies in /etc/portage/package.keywords as
follows.

app-text/texlive ~x86
app-text/texlive-core ~x86
dev-texlive/texlive-metapost ~x86
dev-texlive/texlive-basic ~x86
dev-texlive/texlive-documentation-base ~x86
dev-texlive/texlive-psutils ~x86
dev-texlive/texlive-langcjk ~x86
dev-tex/cjk-latex ~x86
dev-texlive/texlive-documentation-english ~x86
dev-texlive/texlive-genericrecommended ~x86
dev-texlive/texlive-latexrecommended ~x86
dev-texlive/texlive-latex ~x86
dev-texlive/texlive-texinfo ~x86
dev-texlive/texlive-latex3 ~x86
dev-texlive/texlive-htmlxml ~x86
app-text/jadetex ~x86
dev-texlive/texlive-fontsrecommended ~x86
app-text/dvipdfm ~x86
dev-texlive/texlive-documentation-chinese ~x86
dev-texlive/texlive-langmanju ~x86


Use CVS version of Emacs

Run the following commands:

echo "app-editors/emacs-cvs ~x86" >> /etc/portage/package.keywords
emerge emacs-cvs


If you want to use this bleeding-edge version only, you
can uninstall the previous version of Emacs (if you have installed)
and type:

echo "virtual/emacs ~x86" >> /etc/portage/package.keywords


Otherwise (i.e. you have multiple versions of Emacs), when you
emerge packages requiring Emacs, a stable version
(e.g. v22.1) will be installed. For the multiple-version-installation,
you can use eselect emacs list to see a list of available
version. To use Emacs 23 just installed, you can type eselect
emacs set emacs-23. For details, please consult here.

Emacs extensions

I run the following commands to install a few Emacs
extensions. Note that in Gentoo, many extensions (e.g. those related
to development) are automatically installed for you when you emerge
the related tools and enable emacs USE flag.

echo "app-emacs/gentoo-syntax ~x86" >> /etc/portage/package.keywords
echo "app-emacs/imaxima ~x86" >> /etc/portage/package.keywords
emerge htmlize imaxima app-emacs/gentoo-syntax keywiz


Enable SSH server

You may want to enable SSH access to your coLinux machine. There
are some benefits. Examples are shown below (assuming PuTTY is
used):


  More flexibilities to configure the client (e.g. pretty fonts)
  and nice scrollbars.
  The title of PuTTY indicates what coLinux is doing (such
  information might be useful during a long emerge process).


To start SSH server, type the following commands.

rc-update add sshd default
/etc/init.d/sshd start


When you use PuTTY client, you should fill 192.168.0.20
as the host address to utilize the high speed internal connection.

GTK configrations

Change the default GTK+ font. Edit file ~/.gtkrc-2.0, add
the following line, and change the font name and size to your
preferred one.

gtk-font-name = "Arial 15"


Mount CD

Simply type the following

# mount -t iso9660 -o ro /dev/hdc /mnt/cdrom




Final Words

coLinux is an interesting way to run Linux applications in Windows
host. One drawback of coLinux is that the daemon process is pinned to
one processor only. This avoids some bizarre issues, however it does
not utilize the potentials of modern multi-core systems. This is
disappointing especially for a Gentoo user who requires lots of
computation power. Hope the situation could be improved in the
future.

Update: please refer this post on how to enable sound.

   



Reading List
2008-01-17T12:33:00.088+08:00
My previous post discussed my thoughts on how
to read programming books, here comes my reading list of
programming/computer science books. The books I have completed reading after this post are listed for each year, afterwards come the books I am reading and those I plan to read (all books I have completed before this post is not listed
here). Note that this is a living document.

 

 

Reading


  Concrete Mathematics 
  Structure and Interpretation of
  Computer Programs
  Lisp in Small Pieces
  Programming
  Pearls
  What Is Mathematics
  The Feynman Lectures
  on Physics
  Principles of Economics
  Mathematics and Plausible Reasoning (Volume I)
  Operating Systems Design and Implementation (3rd Edition)
  Professional Assembly Language
  Fundamentals of Wireless Communication
  Introduction to Algorithms (2nd Edition)
  Essentials of Programming Languages (3rd Edition)
  One Two Three . . . Infinity
  The History of the Decline and Fall of the Roman Empire, Vol. 1
  Aha! Insight
  Artificial Intelligence: A Modern Approach (2nd Edition)
  The Mind's I: Fantasies and Reflections on Self & Soul
  A Brief History of Time
  Practical Common Lisp
  围棋死活大全
  海子诗全集


To Read


  Operating System Concepts (7th Edition)
  Compilers: Principles, Techniques, and Tools (2nd Edition)
  Computer Graphics: Principles and Practice in C (2nd Edition)
  Art of Computer Programming, Volume 1: Fundamental Algorithms (3rd Edition)
  Code Complete: A Practical Handbook of Software Construction
  Astronomy Today (6th Edition)
  Walden
  Mathematical Thought from Ancient to Modern Times, Vol. 1
  A History of Western Philosophy
  The Seasoned Schemer
  Types and Programming Languages
  Concepts, Techniques, and Models of Computer Programming
  Information Theory, Inference & Learning Algorithms
  Psychology
  Genes IX
 



Update (2008-8-18): started to add books other
than CS/programming, e.g. mathematics, physics, literature,
communications.

Update (2009-2-25): replaced table format with lists (lists with year heading consists books completed in that year), partly because lists are easier to update, but mainly because that I tend to sneak more books into reading queue when no free "slots" available, which makes the table format itself somehow "cheating".

Update (2009-3-13): started to use Amazon widget; correct the 1st paragraph to reflect the status of reading.

   



Read Programming Books
2008-01-16T12:39:00.000+08:00
Although learning programming can be achieved via online tutorials,
reading books is still the preferred way for systematic learning (see
some discussions here,
here,
and here
from author's point of view). Following list is gathered along my
adventure on reading programming books, especially on programming
language books (for books not requiring
active participations, it is recommended that you follow advices
from Charles Petzold). Some suggestions are easier to be said than
to be done, therefore the list is also a reminder to myself.

1. Select the right book

Reading a programming book requires enormous time and
energy. Before embarking on the hundreds-hour-adventure, it is
justified to spend sometime to select the correct path. Sometimes
starting reading an excellent book itself is an achievement. So which
book to read? The simplest solution is to search in Google or Amazon
and select those with enthsiastic reviews.  Google/Amazon are always
handy since you can get more details/reviews on books. There are
alternatives of course, as we will discuss below.

If you just want to read great programming books without
any specific topics, you are lucky since this important question has
already been discussed again and again. For example, you may refer to
this Reddit
thread, or you can consult the opinions of some outstanding
programmers/bloggers (e.g. Steve Yegge's recommendations here and
here,
Joel
Spolsky's advices, and Jeff
Atwood's list).

If you do have some topics in mind (e.g. one particular programming
language), there are various ways in addition to Google/Amazon
approach.


  Lookup in Wikipedia. Typically for one programming language,
  there will be one related article. At the end of the article, there
  is one References section, where you can find some
  interesting links to books. 
 Consult the FAQ of the language newsgroup. Normally, language X
 has one corresponding newsgroup like comp.lang.X. Generally the FAQ
 (sometimes quite outdated) of the newsgroup contains some book
 recommendations.
 For some languages with official website (e.g. Haskell, Perl, and Python), you can easily find book
 recommendations in the site.


Let's name a few commonly recommended books for some programming
languages: SICP (a.k.a. Wizard Book) for
Scheme, Practical Common Lisp for Common Lisp,
Programming Ruby (a.k.a. The PickAxe Book) for Ruby, and
Javascript, the Definitive Guide (a.k.a. The Rhino Book) for
the Next Big Language.

2. Setup programming environment and prepare related resources

You can't learn programming without your hands dirty. To study one
programming language, you have to install necessary package(s).
"Modern" programming books (e.g. Practical Common Lisp,
Programming Ruby, Javascript, the Definitive Guide,
and Programming Erlang) typically have one chapter getting
you started. If your book does not contain such part, you can always
consult online tutorials to setup the environment.

Typically the package contains some kind of editor or IDE for
developing. You need to familiarize yourself with them either by
reading your book or tutorial. For Emacs users, you can install
related mode and simply fire up Emacs.

If the book is not so cutting-edge, you can find abundant resources
in the web: source code, discussion on exercises, and even video
lectures. You can bookmark or download these resources for your future
reference. SICP is an excellent example here: great video
lectures, and nice wiki to
discuss solutions to exercises.

3. Practice, practice, and practice

You can't learn a language without sustained practice. We will
focus on the examples and exercises contained in the book here. 

When you read one example, simply reading is not
enough. It is better if you type the example and run
it. Note: type, not copy & paste. The reason is similar
as when you write programs. After compilation, you think your code has
achieved the task. But typically you need several iterations to make
it right. Similar thing happens for reading. You think you understand
the code on paper, but actually you do not. You should type it in and
run it to really understand it. When you type, you can think about the
meaning of the code, syntax and idioms associated simultaneously. A
more paranoid yet effective way is that you first read the example,
and then enter it without even referring to the book. You then run the
code, analyze the results, and compare your code with the example
supplied in the book.

Now for exercises. Do them all if you can. Excellent books come
with excellent exercises, if any. You could gain more insight when you
employ the techniques you just learned. (If you have any doubt on the
value of exercises, you can look at Steve
Yegge's example on how doing exercises can change a person's view
about a language.) If you cannot solve one problem with
serious try, you can leave it for a while and attack it
later. If you really cannot handle it, you can refer to the solutions,
which you can typically find online. Believe me, you can easily find
the answers to these hard problems of excellent books in the web.

4. Develop hobby project

When you have already read several chapters of the book and feel
comfortable to write a few small programs with the language you just
learned, you may consider to setup some hobby projects. You can use
the project to automate some day-to-day tasks, either in home or at
work. Or you may try to implement/interface some (emerging) cool
things (e.g. Google Chart
API or Ray
tracing) with the language you are learning. Make sure the skills
to complete the project match the skills you just learned. You may
meet some problems you cannot handle with the skills you just
acquired. Don't panic. Just peep the later chapters of the book you
are reading, or Google around. Try to finish the project if you can
(and then you can blog about it if nobody has not done so or your way
is quite interesting). If you cannot finish it, you can leave it for a
later attempt. You may try several such projects when you progress in
the reading. The further you progress, the more ambitious task you can
try. Such hobby project itself can be viewed as extended exercises,
but designed by you, instead of the author.

(Note: this section is largely influenced by a nice article/post
related to reading On Lisp. Unfortunately I cannot find the
URL now.)

5. Never give up

The last but not the least advice is that you should read the book
from the beginning to the end. Spending a lot of time on one book but
giving it up when approaching say one third of it is just wasting
time. Since you only touched the surface of the book, which is easier
to learn and also easier to forget. Authors arrange the books
in such a way that you could linearly read along, which typically
means that advanced topics come later. You may get stuck with one
chapter and cannot understand the topics discussed there. At this
point, don't give up. Remember that you can only achieve a higher
level of understanding of one language/technique if you can understand
the difficult part of it. Just read it, understand it, try it, or
Google around to find different (maybe better) treatment of the
topic. Attack this gnarly
problem and you will gain lots from it. You have now learned some
unique things of the language, you can now look at the language in a
different yet more insightful view, and of course, you can now proceed
further to read the great book and learn more.

Happy reading!

   



MID for Programmers
2008-01-14T12:48:00.001+08:00
Lots of MIDs
were showed off in CES 2008. However, none of them satisfy my
requirements. What I need is a device for web surfing, RSS feed
consuming, programming, and of course blogging. The one closest to my
criterial is from
LG, however I shunned away from Windows. Following is a list of
features I expect an MID to have, from a programmer's perspective.

Mandatory factors:


  Around 5 inch touchscreen. Size matters. The screen
  size is the most difficult factor when making tradeoffs. We want it
  smaller for portability (otherwise why not a gorgeous 17 inch
  MacBook Pro?), but not so small to hurt our eyes. 5 inch seems to
  strike a nice balance. At this size, 800x600 resolution seems to be
  preferred.
  Full keyboard (similar as that of laptop in terms of
  keys equipped). Interface is critical. I am an Emacs fan, and the
  main usage of the MID is to try interesting programming ideas within
  Emacs and to compose blogs as well. Therefore I need the keyboard to
  have Ctrl and Alt equipped as a minimum. I also hope numerical keys
  have their independent positions (e.g. not shared with QWERTYUIOP as
  my Dopod 838, which is really painful). Keyboard from LG prototype
  satisfies all requirements here, therefore I do hope other vendors
  could follow suit.
  Fast CPU. Gentoo
  is my favorite Linux distribution, therefore I may try to
  install it (somebody has already done so
  on Q1 Ultra). In addition to satisfy a power-demanding
  distribution, a powerful CPU is also a right
  for programmer even on such a tiny gadget.
  Linux based OS. Linux will shine on MID, which is
  actually an old news. However, there are also devices
  running Windows at least for Menlow based MIDs, e.g. the one from
  LG. The reason for me to mandate Linux here is that I can easily
  install other flavors of Linux distributions if the device itself
  comes with one. With built-in Linux, we can make sure that source
  code for hardware is readily accessible: every vendor should learn
  something from the event of Eee PC GPL
  issues.
  WiFi access. Connection is important. Without WiFi
  access, I guess we can remove Internet from the name of
  MID.
  Miscellaneous. We should match the fast CPU with at
  least 512 MB RAM. 8 GB internal storage is a minimum to fool
  around. SD card slot is a must to connect the MID with other
  devices, and Bluetooth with A2DP support is necessary to avoid hairy
  wires.


Optional features:


  Phone capability. This feature is actually desperately
  needed. However I doubt whether I could find one MID if I insist on
  this feature. There are various reasons for vendors to drop this
  feature: maturity of the needed software, regulation aspects, and
  marketing issues (imagine the impact on N95 if N810 is a phone)
  etc.
  GPS. This is always a nice feature to enjoy.
  Nice camera. Just imagine the wow moment when you use
  your 5-inch-LCD-digital-camera.


Finally, is it possible for MID + Android?

   



Gentoo's Crisis
2008-01-12T20:08:00.000+08:00
Sad to
learn Gentoo's
crisis from Slashdot. I unsubscribed from Gentoo newsletter several months
ago (even the newsletter itself has not been updated for some
time), and I do not have any clue on this crisis. But according
to my knowledge, there is no Gentoo 2007.1 release, which is a
dangerous signal. Although the periodical release in Gentoo itself is
not as important as for other distributions (actually there is no
impact for users who have already installed Gentoo), bleeding-edge
release is a must for newcomers.

As
a fan
of Gentoo, I do hope that Robbins's offer will be accepted and all
the issues will be sorted out quickly.

   



Install Proggy Fonts for Emacs in Gentoo
2007-12-28T12:30:00.001+08:00
Coding horror discussed various nice programming fonts (see here
and here). Intrigued
by these posts, I decided to try these fonts on my Gentoo
box. Following is a record of what I have done.

Install Fonts in Gentoo

I decided to use Proggy Clean font (website is here). This section is mainly
related to this particular font, but I will discuss at the end of this
section how to handle other fonts in the same site.

Instead of manually installing the font, I determined to writer one
ebuild. Main rationale is to rely on font eclass to automate font
installation work. The ebuild could be easily reused and even
incorporate into Gentoo portage (hopefully). The other reason is that
I want to practice a little bit on portage overlay and to write my
first (trivial) ebuild.

Create portage overlay

If you have already setup a overlay, you may skip this
part. However it should be noted that all the description in this post
assumes that your overlay is located in
/usr/local/portage.

Run the following commands.

# mkdir -p /usr/local/portage
# echo 'PORTDIR_OVERLAY="/usr/local/portage"' >> /etc/make.conf    


Create the ebuild

First, create the directories.

mkdir -p /usr/local/portage/media-fonts/ttf-proggy-clean


Next, create a file named ttf-proggy-clean-1.0.ebuild under the
directory just created. Fill the contents as below.

inherit font

DESCRIPTION="Proggy Clean TTF font"
HOMEPAGE="http://www.proggyfonts.com/"
SRC_URI="http://www.proggyfonts.com/download/download_bridge.php?get=ProggyClean.ttf.zip"

LICENSE="Proggy"
SLOT="0"
KEYWORDS="alpha amd64 arm ia64 ppc s390 sh sparc x86"
IUSE=""

DEPEND=""

S="${WORKDIR}"
FONT_S=${S}
FONT_SUFFIX="ttf"
DOCS="Licence.txt Readme.txt"

RESTRICT="mirror" 


Some explanations of the ebuild:


  Version: I believe the font is quite stable, therefore version
  1.0 should be OK :-).
  License: the license in the package is quite permissive, but
  since I cannot link it to any existing license, I category it simply
  as Proggy.


Run the following command to sign the ebuild.

# ebuild /usr/local/portage/media-fonts/ttf-proggy-clean/ttf-proggy-clean-1.0.ebuild digest


BTW, I refers a lot to this
when writing this first ebuild (of course, the skeleton is from
man font.eclass).

Install font

Now comes our familiar emerge.

# emerge media-fonts/ttf-proggy-clean


Now we need to add the font path. Open file
/etc/X11/xorg.conf, find the section Section
"Files", and add the following line within the
section:

  FontPath "/usr/share/fonts/ttf-proggy-clean"


Then press Ctrl-Alt-Backspace to restart X server. After
login again, you may issue the following command (if you have not
installed xlsfonts, please do so by typing emerge
xlsfonts).

# xlsfonts | grep proggy


You should see several lines looking like
-altsys-proggycleantt.... If not, there is something wrong to
fix (please check previous typings).

Caveat: before I add the FontPath line in section
Files, that section is actually empty! Therefore X simply cannot
restart. The solution is to add some default paths into that section,
e.g. /usr/share/fonts/100dpi, /usr/share/fonts/misc.

How to use other fonts

Following is a guideline for other fonts listed in the same website.


  Use appropriate directory and file name to represent the
  font. For example, for Proggy Square X font, one might setup
  directory /usr/local/portage/media-fonts/proggy-square,
  and create a ebuild called
  proggy-square-1.0.ebuild. The directory should be added
  into /etc/X11/xorg.conf as described above.
  Modify the ebuild:
    
      Specify appropriate DESCRIPTION.
      Use correct SRC_URI.
      Specify DOCS correctly. I found that X font package
      does not contain Readme.txt.
      Specify FONT_SUFFIX. For X fonts, use gz;
      use ttf (like above) for TTF fonts.
    
  


Configure Emacs

Compared with the installation described above, this part is rather
easy. Assume that you want to use the font
-altsys-proggycleantt-medium-r-normal--0-0-0-0-m-0-iso8859-1
(you may select one from the output of xlsfonts as
described above)d, simply add the following line in your .emacs:

(set-default-font "-altsys-proggycleantt-medium-r-normal--0-0-0-0-m-0-iso8859-1")

   



Weirdness of Erlang
2007-12-24T17:32:00.000+08:00
But Erlang itself? It's too weird, and in my brief
experiments, the implementation shows its age; we have in fact learned
some things about software since way back then.


--Tim Bray, Prognostication

I'm just quite new to Erlang and read through Chapter 10 of Programming
Erlang so far. The language really expand my mind greatly,
with its pattern-matching capability (even for a Haskell user), and of
course concurrency oriented programming. Inter-process communication,
even across different machines, is a quite simple task in
Erlang. However, as stated by Tim Bray, Erlang is too weird. In the
following, I will list some weird things I met when studying
Erlang. The focus is mainly on syntax, which seems to be non-essential
but should have been easily improved (if backward compatibility is not
an issue).

Usage of punctuation

A newcomer to Erlang will almost forget to type period after an
expression one or more times. Why do we need such punctuation even in
an interactive shell? I admit that using period, semicolon and comma
in Emacs makes source editing more automatic, but I prefer not using
them at all. Erlang should learn something from the elegance of
Haskell in this aspect.

Awkward functional programming with fun

There is no fun when using fun. When writing an anonymous
function, one have to use fun and append
end. Even when calling a named function within a higher
order function, one has to use fun like this
lists:map(fun math:sqrt/1, [1.0, 2.0, 3.0]). This is
awkward and Haskell way is way better again.

Handling Records

Although records are tuples in disguise, they are treated in a
C-style way: i.e. if one record definition is to be used by multiple
source files, it has to be put into an include file. Weird.

Macro

Fans of C/C++ may applaud when they learn that Erlang provided
similar macro facilities. However it is doubtful whether this
controversial feature is needed for such a functional programming
language.

Makefile

Again advocates of C/C++ might be pleased to know that Erlang also
relies on makefile for package management/build. Shed by the light of
Haskell/Cabal, Common Lisp/ASDF or X/Y (put your favorite language and
package management system here), one may expect that Erlang has a more
modern way?

Conclusion

Apparently, above weird aspects are not show-stopper for the
popularity of Erlang. However, appreciating the Ruby principle of
Least Surprise, I cannot help but rant my impressions
gathered when climbing the mind-blown-away learning curve.

   



Minimize Diagnostics in Common Lisp
2007-12-17T13:03:00.002+08:00
Diagnostics generated by compilers are quite useful for programmers
to spot bugs or inefficiencies in the first place. It is quite often
to observe that command
options -Wall -pedantic (or even -Werror)
are used to invoke gcc. When investigating the situation in Common
Lisp, a newcomer would be surprised to see that diagnostics and ways
to handle them are actually standardized (just like other
implementation-like-stuff-in-other-languages, say
disassemble). In addition to the already standardized
diagnostics like warning
and style
warning (the latter is actually a subtype of the former), some CL
implementations (notably CMUCL and SBCL) provides efficiency
notes, which alerts the user that the compiler has chosen a rather
inefficient implementation for some operations. For SBCL users, it is
highly recommended to read one
part of SBCL manual to learn how to interpret SBCL
diagnostics.

In typical development cycle, after writing/modifying some code and
compilation, if compiler complains with such diagnostics, you know
that somewhere needs your attention. However, you can only sense these
new diagnostics if the number of existing diagnostics is
quite small or zero. Otherwise you many not even notice the new
diagnostics since you already have a bunch of them. In this sense,
existing diagnostics are like broken windows, and they
discourage you to identify new diagnostics. Your program will
deteriorate if you do not fix these broken windows. In summary, to
make the diagnostics useful, you have to minimize, or better yet,
eliminate existing diagnostics.

There are two ways to stop the compiler emitting diagnostics:
removal and muffling. We will discuss them separately. Note that we
will use SBCL as our CL implementation in the following
discussion.

1. Removing Diagnostics

This is of course our first choice, since diagnostics are
indications of some risks in the code. In the following, we give some
simple examples to illustrate how to delete such annoying (but useful
somehow) complaints.

1.1 Warnings

Typically warnings are serious issues to be resolved
immediately. One exception is that sometimes if one file contains
multiple style warnings, a warning will be issued for the file itself
as well. Therefore to remove such warning requires eliminating related
style warnings, as discussed below.

1.2 Style Warnings

1.2.1 Variable X defined but never used.

The first consideration here should be to remove the variable
causing diagnostics if possible, which has the additional benefit of
simplifying code. However there are cases when we cannot change the
function interface. For example, when we write reader macros with
set-dispatch-macro-character, the 3rd argument is itself
a function with 3 arguments like (stream char1 char2),
and typically we do not use either char1 or
char2. In this situation, we can use declarations like
(declare (ignore char1 char2)). Sometimes, if the
argument is used in some scenarios while not in other cases (typically
in macro definitions), we can use declaration
ignorable. An example here is the anaphoric macro
acond2 defined in On Lisp, a declaration
(declare (ignorable it)) is needed for the ubiquitous
it.

1.2.2 Redefining FOO in BAR.

Typically such style warnings are issued when one file is reload,
hence can be simply ignored. However there are some more complicated
cases. One example is again related to the reader macro. If one want to
use the reader macro in the same file, the definition of the reader
macro should be encompassed with (eval-when (:compile-toplevel
:load-toplevel :execute), as illustrated in HyperSpec. The
problem is that if you write a function instead, every time you load
the file, this style warning pops up. The solution? Either use
anonymous functions as illustrated in HyperSpec, or spin out related
part into a separate file without the eval-when stuff.

1.3 Efficiency Notes

It should be noted that efficiency notes are emitted only when we
turn on optimization declarations, e.g. (declare (optimize
(speed 3) (safety 0))). It is well known that premature
optimization is the root of all evil. Therefore before battling
against those notes, one should check whether the function in question
is the bottleneck or not. If not, one should remove those optimization
declarations, which can make the core simpler and more maintanable;
otherwise (when the optimization is really necessary), one can proceed
further with techniques discussed below.

As indicated by the informative CMUCL
manual on efficiency notes, the solution to dismiss these notes is
to provide sufficient type declarations. It should be noted that
current SBCL is smart enough to perform type inference, therefore it
is unnecessary to clutter the code with type declarations for every
expressions.

1.3.1 Doing X to pointer coercion

Such notes are typically emitted for function return values which
are of boxed types. For example, on 64-bit machines, double-float is
still boxed (it requires exactly 64-bits!), therefore compiling the
following functions will get a note doing float to pointer coercion
(cost 13) to "<return value>".

Code-list-1
(defun df+ (x y)
  (declare (optimize (speed 3) (safety 0))
           (double-float x y))
  (+ x y))


How to remove these notes? One way is to use unboxed types. For
example, in above example, one can use single-float
instead of double-float for 64-bit CPUs. However such solution is not
always available: one reason is that there are only a few unboxed
types, and the other one is that sometimes the boxed types are what we
really need: e.g. double-float is required from accuracy point of
view. Another solution is to use local functions. Since compiler know
how to utilize the return value, the efficiency notes will not be
emitted. One example is shown below:

Code-list-2
(defun df-outer ()
  (flet ((df+ (x y)
           (declare (optimize (speed 3) (safety 0))
                    (double-float x y))
           (+ x y)))
    (coerce (df+ 1.0d0 2.0d0) 'float)))


Note that the above example is simply contrived to illustrate that
using local functions can remove efficiency notes.

2. Muffling Diagnostics

There are many occasions that above solutions cannot be applied. In
this case, we can muffle the diagnostics. The intention of muffling is
not that we are going to adopt an ostrich policy for those
diagnostics. The underlying rationale is actually as following:


  We acknowledge that there is no way to eliminate the diagnostics.
  The diagnostics do not impose any risks to the program per se.
  The diagnostics have to be suppressed in order not to obscure
  other diagnostics.


One example is the above code-list-1. If such a standalone function
optimized for double-float is really what we want, we cannot remove
the efficiency note on 64-bit machines (when will 128-bit CPUs become
mainstream?). Since there is no problem with the code, we can safely
muffle the annoying note to avoid it to distract our attention
further.

Common Lisp does provide a standard way to muffle standard
diagnostics (i.e. warnings and style warnings, but not notes since
they are not standardized). This is function muffle-warning,
however its usage seems not straightforward. SBCL wraps it up and
provides a pair of declarations 
sb-ext:muffle-conditions and
sb-ext:unmuffle-conditions to muffle diagnostics. We
will use them in the following discussion. If portability is
desirable, one should use #+sbcl; however in the
following examples, we do not use it for simplicity.

2.1 Local Control

It is preferred that we muffle the diagnostics in a local
definition if possible. We can specify which types of diagnostics to
muffle and use the pair of extensions for advanced purposes, as
illustrated in SBCL manual. Following is an example to illustrate how
it is applied to our example (code-list-1) above.

Code-list-3
(defun df+ (x y)
  (declare (optimize (speed 3) (safety 0))
           (double-float x y)
           (sb-ext:muffle-conditions sb-ext:compiler-note))
  (+ x y))


Note the usage of sb-ext:muffle-conditions in line 4
above.

2.2 Global Control

One can muffle diagnostics globally in the way like (declaim
(sb-ext:muffle-conditions sb-ext:compiler-note)). However it is
not recommend to do so since we will lose the whole point of using
diagnostics. Nevertheless using pairs of global declarations is useful
sometimes, since it can muffle the diagnostics issued for top level
structures, and allow the compiler to complain again if it meets
issues in other places.

Let's use code in On Lisp again as another example. When
emulating Scheme-like continuations in Common Lisp, Paul Graham
defined parameter *cont* as (setq *cont*
#'identity) in top level. SBCL emits a warning undefined
variable: *cont* when compiling the code. It should be noted that
we cannot use defvar for this parameter, as discussed in
the text. To muffle the warning, we can add a pair of declarations as
in code-list-4 below.

Code-list-4
(declaim (sb-ext:muffle-conditions warning))
(setq *cont* #'identity)
(declaim (sb-ext:unmuffle-conditions warning))


Update: as pointed out by Lars Rune Nøstdal
in the comment below, a cleaner way is to use locally. The
above example can be simplified as following:

Code-list-5
(locally (declare (sb-ext:muffle-conditions warning))
  (setq *cont* #'identity))


Conclusion

To make the best of compiler diagnostics, it is good to remove the
existing ones. Happy hacking beautiful code!

   



The Last Clip of SICP Video
2007-12-11T12:51:00.000+08:00
This morning, I finished watching SICP
video lectures with my tiny Dopod 838 QVGA screen when commuting
on company shuttle bus. As I have pointed out in previous
post, learning such a mind-expanding book has never been such a
pleasure with the aid of videos.

When watching the video, one may have already noticed at the beginning that the atmosphere
was somehow different, with Abelson wearing a foolscap and almost
everybody wearing sunglasses. In this very last clip, after Sussman
explained the simple-but-very-effective Minsky garbage collector and
discussed the halting problem, the usual Q&A section started. The
last tricky question was Is this the last
question?, and Sussman replied Apparently yes after a long
pause. And the lecture just closed here, with every student (actually
HP employees) getting a close-up. 

I just studied the lectures along with them, learned a lot of
things, and was delighted and blown away many times. The video was
taken in July 1986, when I had no idea of what computer is. It's interesting that I
managed to learn things together with them. Although I have not really
dived into the books and the exercises, I'm sure I will do so quite
soon, at least to meet the authors again during reading.

   



Nicer Fonts for Gnuplot
2007-12-07T12:52:00.001+08:00
Gnuplot is a great
visualization tool to use. However the default font is not so pleasant
for me. Following is the record on my adventure for alternative
fonts. The description is mainly Gentoo oriented.

Which font to use?

I'd like to try TTF font for nicer looking, and Bitstream Vera
bundled with Gnome just fits my taste. In case that you have not
installed Gnome (e.g. you're pretty much satisfied with the plain
terminal, or you use X only without any desktop environment, or you
are a fan of other desktop environments like KDE or E17), you may
install the font with one command in Gentoo: emerge
ttf-bitstream-vera.

Build Gnuplot

Gnuplot should be built with libgd support. To do so, you should
enable the USE flag "gd". You may add gd to your make.conf, or use the
following command to add it for gnuplot alone:

echo "sci-visualization/gnuplot gd" >> /etc/portage/package.use

After that, build gnuplot by typing emerge gnuplot. The
version I'm using for this post is 4.2.2.

Configure environment variable

Add the following line to one of the shell profiles
(e.g. ~/.bash_profile):

export GDFONTPATH=/usr/share/fonts/ttf-bitstream-vera

Try it out
Now launch gnuplot. To test whether our settings take effect, type
the following at gnuplot prompt:

gnuplot> set term png enhanced font "Vera,12"
Terminal type set to 'png'
Options are 'nocrop enhanced font Vera 12 '
gnuplot> set output "test.png"
gnuplot> plot sin(x)
gnuplot> quit


Now start your favorite photo viewer to examine test.png (I use
Emacs in this case). Does the figure looks prettier than before? (Note: it seems that the font name is case sensitive and should be consistent with the actual file name of the font: "Vera" works in above example while "vera" not.)

One limitation is that TTF fonts are not available to X11 terminal
therefore you cannot directly enjoy the effect within gnuplot. Emacs
users using gnuplot-mode might not feel too much switching pain in
this case. Another solution is: if you have ImageMagick installed, you can use png terminal, and pipe the output to ImageMagick by typing set output '|display png:-' instead of plain test.png as shown above.

   



Format Source Code for Web
2007-11-28T12:52:00.000+08:00
Introduction

As both a programmer and blogger, one needs to publish source code
on blogs frequently. So how to do it? As a newcomer to blogger
community, I looked into the problem for a while. There are two issues
related to this if you only want to display your code plainly, without
syntax highlighting:


  HTML treats consecutive spaces, tabs, line breaks as a single
  space, which removes indentations of the code.
  Special characters like &, <, >, and " need
  escaping, especially for HTML code itself.


My initial thought is to use sed, since the above issues can be
simply solved with line by line processing, which is what sed is
designed for. Later on I realized that why not use Javascript? Then I
can publish code everywhere, which means that I can mobilize web
authoring, in addition to mobilizing
web surfing.

Solution

Following is the tool to convert source code to HTML fragments to
be inserted in blogs or normal web pages. Simply paste your source
code into the 1st textarea, and copy the HTML code out from the 2nd
textarea. 

It should be noted that I use CSS to format the source code in
<pre> element, therefore one attribute "code" appears
in the 1st line of the HTML output. One may tweak it according to
his/her own preference, e.g. change to inline style sheets.


  
    Source code:

            
    
    HTML output:

            
    
  




Implementation

The source code for above form and related script is shown below
formatted with itself.

<form name="converter" action="">
  <table>
    <tr><td>Source code:<br/>
      <textarea name="source" onchange="formatCode();" rows="10" cols="80">
      </textarea>
    </td></tr>
    <tr><td>HTML output:<br/>
      <textarea name="output" rows="10" cols="80">
      </textarea>
    </td></tr>
  </table>
</form>

<script type="text/javascript">
  //<![CDATA[
  function formatCode () {
    var form = document.converter;
    var s = form.source.value.replace(/&/g, '&amp;');
    s = s.replace(/</g, '&lt;').replace(/>/g, '&gt;').replace(/"/g, '&quot;');
    form.output.value = "<pre class=\"code\">\n" + s + "\n</pre>\n";
    form.output.select();
  }
  //]]>
</script>      


We use HTML element <pre> to solve the 1st issue listed
above. In the script, function formatCode escapes special
characters; note that we should handle character & first.

Update (2008-1-17): for Emacs users, you may find
htmlize
useful for this task (with many cool features). Gentoo users can
simple emerge htmlize. I just stumbled upon this cool
extension when reading the comments for Steve Yegge's
nice post.

   



Mobilize Web Surfing
2007-11-15T13:06:00.000+08:00
At first glance, web surfing is naturally in a mobile way. You can
surf anywhere, anytime nowadays. However, the important data
associated with web surfing are not mobilized at the begining. The
most noteworthy example is your favorite websites. Traditionally, you
have to store them locally in your computer, in a format only known by
your chosen web browser. You have to copy your bookmarks around and
import them whenever you use a different computer, or even try a new
browser. That is possibly why web browsers are bundled with features
like import/export bookmarks and even the tool to migrate bookmarks
from competing browsers. Even armed with those weapons, you still have
the problem of synchronizing your bookmarks between different
computers; not to mention that the import/export process is really
boring.

So here comes the so-called social bookmark, with the most
successful example: del.icio.us. As
its name suggests, it is really a delicious tool to
use. After registration, you can install browser extentions to make
the service provided by del.icio.us to integrate with your browser
seamlessly. Now you can access your bookmark anywhere: even on a
computer without the extension installed, you can simply access your
bookmarks by visiting the del.icio.us website. With the latest Firefox
extension, one bookmark sidebar is installed, which gives you instant
access to your bookmarks. In addition to these handy bookmark
functionalities, you also get social effects. When you add
one nice page to your collection, you now can choose popular
tags which is automatically provided by the service by examining who
else chosen to tag the same page. You now can select the
right tag easily. 

A more efficient way, compared with checking updates of your
favorite web pages individually, is to syndicate them and process them
in a RSS reader. Most web surfers have already done so. At first, we
use a offline RSS reader, i.e. subscription file is stored
locally and RSS updates are also fetched to local computer. A question
then arises, similar to the bookmark problem: how can we read RSS
updates in different places: e.g. work and home? We can synchronize
the subscription (although tedious), but we then have to manually mark
those already read in one place (note that typically new items pop up
very quickly). Online RSS reader comes into rescue now. Simple
register one, e.g. Google
Reader, import your subscription file and start reading
everywhere. For my own experience, the online reader can compete with
the offline reader in almost every area, and the developing pace of
such online readers is also extremely fast!

Yet another example is online notebook like Google Notebook. Remember
those yellow stick notes? One certainly take many notes during web
surfing. Now you can write and read them everywhere.

There seems to be endless examples to fully mobilize your web
surfing experience: you can put your calendar online, write your
report online, investigate your financial status online, or prepare
your presentation online. Such experience is really excellent, and the
only thing we should be cautious is to guarantee the network
connection to avoid any agony.

Features	HV30	HF10	Comments
Price	MSRP price is $999 in US and ¥10580.00 (about $1514) in China	MSRP price is $1099 in US and ¥14080.00 (about $2014) in China	The price difference in China is quite significant, which is true even for street price.
Performance	1/2.7 in. sensor	1/3.2 in. sensor	According to the review, HV30 has slightly better performance over HF10 thanks to larger sensor, however the gain seems to be marginal for most cases.
Functionalities	Viewfinder, Zebra, Peaking
Accessories	Standard accessory shoe	Canon proprietary accessory shoe
Backup	Tapes are cheap for backup	Additional backup is needed since SDHC cards are expensive and will be reused.
Community	A mature community (looking at hv20.com for example)	Situation will improve
Form Factor	88x82x138 mm (3.5x3.2x5.4 in.), 535 g (1.2 lb.)	73x64x129 mm (2.9x2.5x5.1 in.), 380 g (13.4 oz)	HF10 is small and light, which is ideal for traveling. Normally the camcorder which you takes out often is the most useful one.
Media	Tape	SDHC card + internal Flash	Flash card is really convenient: just like small DCs. No hassles for rewinding and no worry for overriding.
Zoom	10x optical zoom	12x optical zoom
Noise	Tape noise can be heard in silent environment	No such noise since flash card is used.