shad's blog

How to know all of your unfollowers on Twitter

Fri, 04 Mar 2011 18:55:00 -0800

Here is my easy way to know who unfollows me on Twitter. Or more precisely, how to monitor the changes on your followers list.

First of all, create a simple script named "twitter_changes.sh":

#!/bin/sh

SCREEN_NAME="sam_alba"

EMAIL="myemail@domain.com"

( cd $(dirname $(readlink -f $0))

mv latest.log previous.log

curl -s -X GET "http://api.twitter.com/1/statuses/followers.xml?screen_name=${SCREEN_NAME}" | grep -E '^ <(screen_name|name)>' > latest.log

data=$(diff previous.log latest.log)

if [ ! -z "$data" ] ; then

echo "$data" | mail -s "[twitter] changes" -a "From: twitter@shad.cc" $EMAIL

fi

)

Then, execute the script daily with a crontab:

# cd /etc/cron.daily ; ln -s /path/of/your/script twitter_changes

That's it, every changes on your twitter account will be emailed to you every morning.

Permalink

Released CirruxCache 0.4.2

Sun, 23 Jan 2011 08:15:00 -0800

I have just released a new minor version of CirruxCache. This new version includes several bugfixes and third party library upgrades.

Here is the change list of the 0.4.2:

Admin panel: upgraded jquery, jquery-ui et jquery-form to the latest versions.
Store service: fixed file upload (which was not working anymore on appengine with billing mode enabled).
Some code cleanup

This update is not critical, I invite you to upgrade only if you are experiencing problems with the admin panel or the store service.

Thanks to Doug Tung who shared me an appengine account with billing mode enabled, that was really helpful for debugging.

Permalink

CirruxCache 0.4.1 is out!

Tue, 26 Oct 2010 20:10:00 -0700

An easy way to use Google Appengine as a personal CDN! CirruxCache 0.4.1 is out, so let me show you a set of new cool features!

CirruxCache provides a software solution to dynamically cache HTTP objects on Google Appengine (using the Datastore and the Memcache services).

Since the last release, I received a lot of good comments and smart suggestions. Fortunately, Some of you increased the size of my todo-list and a bunch of nice ideas are still in the box.

Let’s take a look at this release:

Admin panel now includes a config helper (find here screenshots and documentation).
Object flushing has been fixed in the admin panel, give it a try!
The Cache service now includes a “Content Prefetching” feature. When it is enabled, HTML pages are interpreted. All associated content are loaded in the cache before the user request them. So it speeds up the first page loading. It is experimental for the moment, but it works (it is enabled on this blog!).
A new Image service enables you to delegate image transformation to CirruxCache. For example, you won’t need to upload images previews or thumbnail on your website anymore, let CirruxCache generate them and cache them on the fly! (All images on this page are generated this way).
Config has been simplified: you don’t need to edit the main “app.py” file. It is located in a file named “config.py”. On top of that, all internal services (admin, cron, store) are now included by default, you can forget them! (Install documentation page has been updated).
A lot of bugfixes which improved the caching mechanism.

I hope you will enjoy this new release, check it out on the download page.

I would like to share with you the Users page, I try to list some of you using CirruxCache. This pagedoes not have much success, please take a look!

Finally, I warmly encourage you to subscribe to the CirruxCache group, it is really cool to get help and to provide advices to other users.

Spread the word!

Permalink

Google Appengine on Ubuntu Maverick - without Python 2.5

Sat, 16 Oct 2010 20:10:00 -0700

First of all, install the required ubuntu packages:

$ sudo aptitude install libssl-dev libsqlite3-dev

Then, fetch python2.5 source tarball:

$ wget http://python.org/ftp/python/2.5/Python-2.5.tar.bz2
$ tar xjf Python-2.5.tar.bz2

Before compiling, we need to tweak the file Python-2.5/Modules/Setup.dist (line 207) as following:

_ssl _ssl.c \
-DUSE_SSL -I/usr/include/openssl \
-L/usr/lib -lssl -lcrypto

Compile and install:

$ mkdir /opt/python2.5
$ ./configure --prefix=/opt/python2.5
$ make $ sudo make install

Finally, after having deployed the appengine framework package, we need to modify shebangs:

$ cd /path/to/appengine
$ find . -name '*.py' -exec sed -i "s/#!\/usr\/bin\/env python/#!\/opt\/python2.5\/python/" {} \;
$ chmod -R 755 .

Permalink

CirruxCache: Store API documentation

Sat, 07 Aug 2010 19:29:00 -0700

I have just written some documentation about the Store WebService of CirruxCache. This little page will help you to upload big files on CirruxCache in the same way than the current admin panel does.

http://code.google.com/p/cirruxcache/wiki/StoreAPI

Just to remember, the Store WebService has been developed to bypass the 1MB limitation of appengine. It is not possible to dynamically cache objects with a size greater than 1MB, in this case, you need to store explicitly such an object using this WebService. It uses the Blobstore service, so file size cannot exceed 2GB.

Permalink

CirruxCache: Advanced configuration sample

Wed, 21 Jul 2010 22:46:00 -0700

That's it! This whole blog is cached and directly delivered by CirruxCache (only static files were cached before). My origin server is a tiny eeebox connected through my personal ISP. So this configuration is a good challenge to offload my tiny web server as much as possible. I think this is a good opportunity to show an example of a configuration which is a little bit more evolved. The point is, I cannot set the same cache TTL for the whole website, and actually, I want to cache several websites...

# URL mapping
urls = {}

base = (
'/_admin/(.*)', 'Admin',
'/_store/(.*)', 'Store',
'/_cron/(.*)', 'Cron'
)

urls['default'] = base + (
'(/debug/.*)', 'Debug',
'/(.*)', 'Root'
)

urls['www.shad.cc'] = base + (
'(/themes/.*)', 'Blog_Static',
'(/plugins/.*)', 'Blog_Static',
'(/admin/.*)', 'Blog_Forward',
'(/.*)', 'Blog_Page'
)

urls['www.zaphod.eu'] = base + (
'(/pub/.*)', 'Zaphod_Redirect',
'(/.*)', 'Zaphod'
)

# still supporting the old config

urls['cdn.shad.cc'] = base + (
'/blog(/.*)', 'Blog_Static',
'/(.*)', 'Root'
)

urls['cdn.zaphod.eu'] = base + (
'(/admin/.*)', 'Zaphod',
'/(.*)', 'Root'
)

# POP definition
# You can define and configure your Point Of Presence

class Blog_Static(cache.Service):
origin = 'http://orig.shad.cc'
forceTTL = 2592000 # 1 month
ignoreQueryString = True
forwardPost = False
allowFlushFrom = ['x.x.x.x']

class Blog_Page(cache.Service):
origin = 'http://orig.shad.cc'
forceTTL = 3600 # 1 hour
ignoreQueryString = True
forwardPost = True
allowFlushFrom = ['x.x.x.x']

class Blog_Forward(forward.Service):
origin = 'http://orig.shad.cc'

class Zaphod(cache.Service):
origin = 'http://orig.zaphod.eu'
forceTTL = 2592000 # 1 month
ignoreQueryString = True
forwardPost = False
allowFlushFrom = ['x.x.x.x']

class Zaphod_Redirect(redirect.Service):
origin = 'http://zaphod.eu'

# !POP
I think this configuration is enough readable to avoid any explanation. However, do not hesitate to leave any comments.

Finally, I created a google groups to centralize all help requests. So if you need help, go to http://groups.google.com/group/cirr... or send an email to cirruxcache 'at' googlegroups 'dot' com.

Permalink

New release: CirruxCache 0.3.1

Sun, 18 Jul 2010 15:21:00 -0700

I am really glad to announce a new major release of CirruxCache.

This new release includes the following changes:

A storage webservice: store big files (<= 2GB) on the Blobstore in order to deliver them through CirruxCache. This feature is useful to bypass the 1MB limit on appengine.
An admin panel that enables users to flush objects, manage big files and see some statistics about the resources used.
Bugfixes

It is really important to note there are few limitations on the panel admin:

There is no error reporting on the flush panel (it only displays the number of objects trying to be flushed).
Storage manager displays a "500 Internal Error" when uploading. It only happens when you don't have a billing account (the Blobstore is only available on billing accounts, refer to appengine).

These two limitations will be improved in the next release, and there will be more informations in the statistics panel.

The Storage WebService will be documented really soon, but you can access the admin panel through "http://your.cirruxcache.app/_admin/"

I make the most of this opportunity to announce some changes on the project website:

Roadmap page
Users list: this page has just been started, so do not hesitate to notify me to add you.

I hope you will enjoy this new release.

Permalink

Minor release: CirruxCache 0.2.2

Wed, 19 May 2010 17:45:00 -0700

CirruxCache 0.2.2 has just been released. It contains some bugfixes (thanks to Devattas to have reported errors on Datastore latency). Webpy has been updated to the last version.

I have also updated the documentation, especially I brought more details on Point of Presence configuration and usage of cron tasks for garbage collection.

Finally, some of the users reported me that there is a real problem with the cached object size limit (currently 1MB). I am working on the solution, I will take advantage of the new Blobstore service on AppEngine to store objects. Maybe I will keep the Datastore only for meta-data. This solution will raise the cache object limit to 50MB.

Stay tuned :)

Permalink

CirruxCache 0.2.1 is released

Thu, 11 Mar 2010 17:20:00 -0800

I have just released a new version (0.2.1) of CirruxCache. To remember:

CirruxCache provides a software solution to dynamically cache HTTP objects on Google Appengine (using the Datastore and the Memcache services).

This new version includes an interesting set of features:

allow object flushing from restricted IP
configure a PoP (Point of Presence) according to a virtual host
several behaviors (cache, redirect, forward)

In more details, the last feature is the ability to configure a point of presence to differ from a classical caching mechanism. For example, I may want to configure "/admin/*" on my website to be redirected on the origin without caching.

Of course, this release includes several bugfixes, especially a fix on the "Expires" HTTP header which improves the caching performances.

Do not hesitate to test this new version and to comment any bugs or any suggestions.

Permalink

Adding virtual host support to webpy

Fri, 19 Feb 2010 18:11:00 -0800

Webpy is a tiny web framework. I use it a lot for my web-services applications. In general, I let my web server (lighttpd) to handle virtual hosting. But as you may know, I am working on a CDN solution on top of Google App Engine, named CirruxCache. In that case, while I have absolutely no control on the server configuration, I need to handle virtual hosting from the code. Webpy maps urls by iterating through a tuple. So my solution is quite simple: wrapping the tuple to override the __iter__ function according to an environment variable (HTTP_HOST). Let's take this basic webpy example, without vhosting:

import web

urls = ('/(.*)', 'hello')

class hello(object):
def GET(self, name):
if not name:
name = 'World'
return 'Hello, %s' % name

if __name__ == "__main__":
app = web.application(urls, globals())
app.run()
Let's add the VhostMapper class:

import web

urls = {
'default' : ('/(.*)', 'hello'),
'my-vhost.domain.tld' : ('/(.*)', 'helloVhost')
}

class hello(object):
def GET(self, name):
if not name:
name = 'World'
return 'Hello, %s !' % name

class helloVhost(object):
def GET(self, name):
return 'Hello %s' % web.ctx.environ['HTTP_HOST']

class VhostMapper(object):
def __iter__(self):
url = urls['default']
if 'HTTP_HOST' in web.ctx.environ:
vhost = web.ctx.environ['HTTP_HOST']
if vhost in urls:
url = urls[vhost]
return iter(url)

if __name__ == "__main__":
app = web.application(VhostMapper(), globals())
app.run()
Finally, you can use curl or wget to test your vhosts:

$> curl -H "Host: my-vhost.domain.tld" http://localhost:8080/

It is not so early to announce that the next version of CirruxCache will handle virtual hosting :) I am sure this simple hack can be easily reproduced to use virtual hosting in some other Rest frameworks.

Permalink

Exploiter le SIP de sa Freebox

Thu, 04 Feb 2010 12:53:00 -0800

J'ai commencé à jouer récemment avec le compte SIP de ma Freebox.

Le compte SIP se paramètre via le panel de gestion de Free, dans l'onglet Téléphonie. Deux options sont possibles:

Rediriger les appels vers la Freebox
Rediriger les appels vers le serveur SIP

Dans le deuxieme cas, il vous faudra un téléphone IP si voulez conserver un téléphone physique connecté à la Freebox. Pour ma part, ayant un téléphone analogique standard, je redirige les appels vers la Freebox. J'utilise donc le serveur SIP uniquement pour passer des appels, et non en recevoir.

Il existe de nombreux clients SIP. J'utilise Linphone sur Linux que je trouve léger et stable.

Un point interessant, le serveur SIP est accessible de l'exterieur, autrement dit, vous pouvez appeler en illimités sur des fixes en France et dans certains pays depuis n'importe où en utilisant votre compte SIP Freephonie.

De plus, si vous êtes l'heureux possesseur d'un iPhone (ou tout autre téléphone sur Symbian, Windows Mobile ou Android), il vous suffit d'installer un client SIP (j'utilise iSipSimple sur iPhone) et de solliciter votre forfait data illimité (ou pas) pour passer vos appels vers les fixes tout en utilisant votre généreux forfait Free.

Attention aux appels vers les mobiles qui ne seraient pas avantageux.

Permalink

How to auto-update Chromium on Mac OS X, Linux and other

Wed, 06 Jan 2010 12:23:00 -0800

I want to try to use Chromium as my first web browser. But for the moment, Chromium suffers from a lack of package maintaining though it targets developers. On my Gentoo Linux, I let portage to handle my Chromium version. But I also use Mac OS X at work, so I have written a little shell script to auto update my Chromium.app. Put the content of this scripts wherever you want and make it executable ($> chmod 755 chromium_update.sh).

#!/bin/sh
# Chromium update script
# - by shad <shad@zaphod.eu>
# mac | linux | ...
OS=mac
LATEST=$(wget -q -O - http://build.chromium.org/buildbot/snapshots/chromium-rel-${OS}/LATEST)
INSTALL_DIR=/Applications
TMP="/tmp/update-chrome-$RANDOM"
(
mkdir $TMP ; cd $TMP
echo Download...
wget -q -O chrome.zip http://build.chromium.org/buildbot/snapshots/chromium-rel-${OS}/${LATEST}/chrome-${OS}.zip
if [ $? -ne 0 ] ; then
echo Cannot update.
exit 1
fi
echo Unzip...
unzip -qq chrome.zip
echo Copying...
rm -rf "${INSTALL_DIR}/Chromium.app"
mv chrome-$OS/Chromium.app "$INSTALL_DIR"
)
rm -rf $TMP
Then, set a crontab rule to choose the update interval ($> crontab -e). Mine runs every day at 12:00.

# Run Chromium update script every day at 12:00
00 12 * * * /Users/shad/mbin/chromium_update.sh

I hope it will be useful for someone as much as for me.

Permalink

CirruxCache: speeds up your HTTP app using Google Appengine as a CDN

Fri, 30 Oct 2009 22:05:00 -0700

It is a great moment, for the first time since I have started to work at Zoomorama, I have just released as open-source an important part of our server platform.

I previously explained how to use Google AppEngine as a Content Delivery Network (CDN). CirruxCache project concretizes this idea. I released the first version based as the one we use in production.

Here is the features it currently supports:

honor Cache-Control
cache TTL override
several POP (Point Of Presence) configuration mapped on custom base-url
ignore query string
POST forwarding
expired entries garbage collection
extensibility

CirruxCache is not documented at the moment even if you would be able to use it after reading the comments in the app.py file. I'll document this app in the next few days, but if you need more documentation, don't hesitate to contact me.

The project website.

Permalink

Speed up HTTP delivering using Google AppEngine

Thu, 30 Jul 2009 15:16:00 -0700

Google AppEngine provides an high-level cloud service which means that your application will be distributed automatically on top of the Google platform. All of your code will depends on the AppEngine SDK, so it could be risky to develop complex application on it. I develop a webservice application for content delivering and content publishing at Zoomorama. We currently use Akamai CDN as a simple cache layer to improve data delivering accross the world. It is interesting for me to use AppEngine in the same way: without changing anything on my existing code base. I have found some posts on blogs dealing with this AppEngine usage, but they are not focused on dynamic HTTP caching like a real CDN. Principle is very simple, all HTTP requests on my AppEngine application will be copied to the AppEngine Datastore. Moreover data which are delivered through AppEngine are cached by AppEngine servers. The code below is a tiny proof of concept:

# HTTP caching on Google App Engine
# - by shad <shad@zaphod.eu>
#

import web # webpy 0.3x
from google.appengine.ext import db
from google.appengine.api import urlfetch

origin = 'http://my.website.com'

urls = (
'(/.*)', 'Root'
)

class Cache(db.Model):
data = db.BlobProperty(default=None)
headers = db.ListProperty(str)

class Root(object):
def GET(self, request):
cache = self.readCache(request)
if cache is None:
cache = self.writeCache(request)
for h in cache.headers:
print h
return cache.data

def readCache(self, key):
cache = cache = Cache.get_by_key_name(key)
if cache:
return cache

def writeCache(self, request):
url = origin + request
response = urlfetch.Fetch(url=url)
if response.status_code != 200:
raise web.NotFound()
cache = Cache(key_name=request)
cache.data = db.Blob(response.content)
cache.headers = []
for k, v in response.headers.iteritems():
cache.headers.append('%s: %s' % (k, v))
cache.put()
return cache

if __name__ == '__main__'
app = web.application(urls, globals())
app.cgirun()
I use webpy to depends on the AppEngine SDK as less as possible.

I have almost finished the production version of this application. I am doing some performance tests. This application is closed source for now. But I am going to release the code source in few weeks. This version will include:

Fetch from Memcache (about 10 times faster).
Headers forwarding.
Read "Cache-Control" and "Expires" to define a TTL (rfc 2616).
Multi origins (according to url mount points).
Other small features (force TTL, ignore query string, etc...).

It is important to note that AppEngine does not keep running instances of your application (your CGI is distributed and it is executed on demand). So this application have to start very quickly (no configuration file, no dynamic generation, etc...).

Permalink

Generates your C++ bindings easily with SWIG

Wed, 01 Jul 2009 19:20:00 -0700

Today, I have to develop a client library. This client library aims to allow users to use my storage webservice app more easily. Mainly to avoid them to read my HTTP Rest API documentation. I want to target the most used languages, and of course, I don't want to develop the same library for each languages. The main reason is that my client library is a draft for the moment, and I want to make changes on the client API over the time. This library is very high level, so I take only the HTTP core class for the example:

#ifndef __HTTP_H__
#define __HTTP_H__

#include <iostream>
#include <fstream>
#include <string>
#include <map>
#include <vector>
#include <exception>

#include <curl/curl.h>

namespace HTTP
{
class Error : public std::exception
{
private:
std::string _message;

public:
Error(const std::string &msg) : _message("HTTPError: ")
{
_message.append(msg);
}
virtual const char* what() const throw()
{
return _message.c_str();
}
virtual ~Error() throw() {}
};

class Response
{
private:
friend class Request;
std::map<std::string, std::string> _headers;
std::string  _version;
std::string  _code;
std::string  _message;
std::string  _body;

void  _parseHeaderLine(const std::string &headerData);
Response() {}

public:
const std::map<std::string, std::string> &getHeaders() const { return _headers; }
const std::string  &getVersion() const { return _version; }
const std::string  &getCode() const { return _code; }
const std::string  &getMessage() const { return _message; }
const std::string  &getBody() const { return _body; }
friend std::ostream&  operator<<(std::ostream &out, const Response &resp)
{
std::map<std::string, std::string>::const_iterator it, ite = resp._headers.end();
for (it = resp._headers.begin(); it != ite; ++it)
out << it->first << ": " << it->second << std::endl;
return out;
}
};

class Request
{
private:
std::string      _host;
unsigned int    _port;
CURL        *_curl;
struct curl_slist  *_headerList;
Response      _response;
std::ofstream    _outFile;
std::ifstream    _inFile;
std::string      _credentials;
std::vector<char>  _buffer;

void  _request(const std::string &uri);

public:
Request(const std::string &host, const unsigned int port = 80);
virtual ~Request();

virtual void  setHeader(const std::string &name, const std::string &value);
virtual void  setCredentials(const std::string &login, const std::string &passwd);
virtual void  get(const std::string &uri, const std::string &outFile = "");
virtual void  post(const std::string &uri, const std::string &data);
virtual void  postFile(const std::string &uri, const std::string &inFile);
virtual void  setVerbose();
virtual void  setHost(const std::string &host, const unsigned int port = 80);
static size_t  writeHeaderCallback(void *ptr, size_t size, size_t nmemb, void *stream);
static size_t  writeCallback(void *ptr, size_t size, size_t nmemb, void *stream);
static size_t  readCallback(void *ptr, size_t size, size_t nmemb, void *stream);
virtual const Response  &getResponse() const { return _response; }
};
}

#endif /* !__HTTP_H__ */
SWIG uses interface files to parse and to generate bindings. IMHO, a good thing is to have an only one interface file to add language bindings much easier.

Here is some documentation on SWIG interface syntax. First, here is my interface file (http.i):

%module http

%include stl.i
%include exception.i

%exception
{
try
{
$function
}
catch (std::exception &e)
{
SWIG_exception(SWIG_RuntimeError, e.what());
}
catch (...)
{
SWIG_exception(SWIG_RuntimeError, "Unknown exception");
}
}

%include "http.h"
%{
#include "http.h"
%}
All C++ code between %{ ... }% tags will be copied during generation. SWIG provides a lot of interface files. Some of them are language-dependant. So I have written a generic exception handler to use the same interface file for all targetted languages.

Finally, I just have to write a shell script to build each languages with the same http.i:

#!/bin/sh

CFLAGS_python_Darwin=-I/usr/include/python2.5
LDFLAGS_python_Darwin="-dynamiclib -lpython2.5 -o _zws.so"

CFLAGS_java_Darwin=-I/System/Library/Frameworks/JavaVM.framework/Headers
LDFLAGS_java_Darwin="-dynamiclib -o libzws.dylib"

CFLAGS_ruby_Darwin=-I/System/Library/Frameworks/Ruby.framework/Headers
LDFLAGS_ruby_Darwin="-dynamiclib -lruby -o zws.bundle"

CFLAGS_php_Darwin="$(php-config --includes)"
LDFLAGS_php_Darwin="-bundle -o zws.so"

if [ $# -lt 1 ] ; then
echo Usage $0 language
exit 1
fi

LANG=$1
ROOT=$(dirname $0)
OSNAME=$(uname -s)
ARCH=$(uname -p)

# All the source files
SRC="http.cpp"

(
DIR=${LANG}_${OSNAME}_${ARCH}
rm -rf ${DIR}/*
mkdir -p ${ROOT}/${DIR}
cd ${ROOT}/${DIR}

# This options will be always added to the compilation/linking
CFLAGS+="-undefined suppress -flat_namespace"
LDFLAGS+="-lcurl"

CFLAGS+=" $(eval echo \$CFLAGS_${LANG}_${OSNAME})"
LDFLAGS+=" $(eval echo \$LDFLAGS_${LANG}_${OSNAME})"

swig -I./src -${LANG} -c++ -o wrapper.cpp -outdir . ./http.i
g++ ${CFLAGS} ${SRC} wrapper.cpp ${LDFLAGS}
)
Notes:

./script.sh python to generate python bindings.
Some of the code below has been modified for the example, the most important thing to understand is the way of use.
I work on Mac OS X, so you just have to add (CFLAGS|LDFLAGS)_<lang>_<os> for other configs.
It would be cleaner with a Makefile ;)

Cheers !

Permalink

How to handle SSL certificates smartly in Firefox

Fri, 03 Apr 2009 15:16:00 -0700

SSL certificates validation is often annoying. In most of cases, the certificate is not acceptable by Firefox because it is self-signed by the server. Some of personal blogs does not have their certificate signed by a known authority.

I think Perspectives is a good solution for this problem, which does not concern only Firefox.

According to perspectives website:
Perspectives is a new approach to help clients securely identify Internet servers in order to avoid "man-in-the-middle" attacks. Perspectives is simple and cheap compared to existing approaches because it automatically builds a robust database of network identities using lightweight network probing by "network notaries" located in multiple vantage points across the Internet.

Firefox extension: http://www.cs.cmu.edu/~perspectives/firefox.html

Permalink

FreeBSD on the ASUS Eee Box using an USB flash drive

Sun, 18 Jan 2009 20:28:00 -0800

I wanted my personal server to be closer to me. So I invested few days ago in the Asus Eee Box, thinking it is the best configuration for a light server at home.

I moved this blog and all my services from my dedicated server to my new one.

To setup FreeBSD on this kind of machine, you have two ways to do it:

Install FreeBSD using a pxeboot image and a dhcp server.
Install FreeBSD using a bootable USB flash drive.

Creating a bootable USB flash drive by using the original ISO image is a bit annoying if you haven't got a FreeBSD station yet. I have been using the fbsd2img.sh script,

I share my USB flash drive image: 7.1-RELEASE-i386-bootonly.img.
To setup the USB flash drive, use "dd" (/dev/XXX is the device of your USB flash drive):

dd if=7.1-RELEASE-i386-bootonly.img of=/dev/XXX bs=1m

Here is the result of the server hosting this blog :)

Permalink

Subversion repository corruption: something to try when svnadmin recover does not work.

Fri, 18 Jul 2008 11:04:00 -0700

I had the wrong habit to keep my old projects in a subversion repository. It's not the good way of keeping code safe. svnadmin recover works when there is no corruption in your repository. If you have an old corrupted repository that you can't recover, you can first try a berkeley db recovery:

cd /var/svn/myrepos
dbX.X_recover

X.X is the berkelay db version of the repository. If svnadmin recover and db_recover don't work, try the solution below, it worked for me. You have to know the database version:

REPOS_PATH="/var/svn/myrepos"
DB_VERSION="4.3"
cd ${REPOS_PATH}/db
rm log.* __db.*
mv changes changes.old
mv copies copies.old
mv nodes nodes.old
mv representations representations.old
mv revisions revisions.old
mv strings strings.old
mv transactions transactions.old
mv uuids uuids.old
mv lock-tokens lock-tokens.old
mv locks locks.old
db${DB_VERSION}_dump changes.old | db${DB_VERSION}_load changes
db${DB_VERSION}_dump copies.old | db${DB_VERSION}_load copies
db${DB_VERSION}_dump nodes.old | db${DB_VERSION}_load nodes
db${DB_VERSION}_dump representations.old | db${DB_VERSION}_load representations
db${DB_VERSION}_dump revisions.old | db${DB_VERSION}_load revisions
db${DB_VERSION}_dump strings.old | db${DB_VERSION}_load strings
db${DB_VERSION}_dump transactions.old | db${DB_VERSION}_load transactions
db${DB_VERSION}_dump uuids.old | db${DB_VERSION}_load uuids
db${DB_VERSION}_dump lock-tokens.old | db${DB_VERSION}_load lock-tokens
db${DB_VERSION}_dump locks.old | db${DB_VERSION}_load locks
svnadmin recover ..
If binaries dbX.X_dump and dbX.X_load does not exist, you have to compile a old version of subversion with the berkeley database corresponding.

Thanks to dmp. Link: http://svn.haxx.se/users/archive-2005-07/0576.shtml

Permalink

Gmail-Notify improvements and a modified behavior

Thu, 17 Jul 2008 14:21:00 -0700

Gmail-notify is a small program written in Python. I use it because it is light and it just do what I want. I am a regular user of the firefox extension gmail-notifyer and I was uncomfortable with the Gmail-notify behavior, so I wrote a little patch to make it more like I want. I also added the possibility to execute a command when a new message arrive. I use it to play a sound (aplay ~/sounds/mail.wav). I don't know your habits but maybe you could enjoy my modifications. Here is a small description:

A click on the event box close it.
A click on tray icon open the mailbox.
Mailbox is now open as HTTPS.
A command can be executed when a new mail is received (like playing a sound).

To use my modifications, download my patch, and apply it to gmail-notify-1.6.1.1:

wget http://garr.dl.sourceforge.net/sourceforge/gmail-notify/gmail-notify-1.6.1.1.tar.gz
wget http://zaphod.eu/pub/gmail-notify-1.6.1.1.patch tar zxvf gmail-notify-1.6.1.1.tar.gz
(cd gmail-notify ; patch -p1 < ../gmail-notify-1.6.1.1.patch)
rm gmail-notify-1.6.1.1.tar.gz

Then run gmail-notify.

gmail-notify/notifier.py

Permalink

Ushare on Gentoo Linux

Wed, 09 Jul 2008 00:01:00 -0700

Ushare is a UPnP media server on linux. I use it to browse media files from my Xbox. There is no official gentoo ebuild. Sunrise Gentoo overlay provides an ebuild for Ushare, it requires some modifications to be used. I noticed some problems:

Ushare daemon is configured to run as root. Without being paranoid, I don't want to entrust a root daemon which manipulate my precious media directory.
Boot script in init.d directory is written for debian, so it does not work at all on Gentoo.
Default configuration file contains some wrong variable names. So this configuration directives are ignored.

I rewrote the init.d script to work on Gentoo and to launch ushare daemon as nobody user. To sum up, Ushare can be deployed rapidly as follow: 1. Checkout Sunrise overlay with layman

emerge -v layman
layman -a sunrise
echo source /usr/portage/local/layman/make.conf >> /etc/make.conf
echo media-video/ushare >> /etc/portage/package.keywords
emerge -v ushare
2. Replace the Ushare boot script /etc/init.d/ushare by my script:

#!/sbin/runscript
#- by shad
DAEMON="/usr/bin/ushare"
NAME="ushare"
CONFIGFILE="/etc/ushare.conf"
OPTS="-f $CONFIGFILE"
PIDFILE="/var/run/ushare.pid"
USER="nobody"
[ -x $DAEMON ] || exit 0
start() {
ebegin "Starting $NAME"
start-stop-daemon --start --quiet --background --pidfile $PIDFILE \
--make-pidfile --exec /bin/su -- --command "$DAEMON $OPTS" $USER
eend $?
}
stop() {
ebegin "Stopping $NAME"
start-stop-daemon --stop --quiet --pidfile $PIDFILE
eend $?
}
3. The last thing to check is the name of configuration variables in /etc/ushare.conf. Make sure each variable starts with the "USHARE_" prefix.

Then,

/etc/init.d/ushare start

and watch your favorite movie on your Xbox 360 :)

Permalink