I have recently made my Adium useless by moving all my IM accounts to my beloved Nokia N9. While moving my buddy lists of all major instant messaging services, I did a quick check on each of the contact to see past interaction. It is sort of surprising to see I don’t actually chat with them as frequent as I thought, so why do I “need” my Adium opened all the time?

There’s a new app (in beta) released for Nokia N9 called wazapp, which is an unofficial port of WhatsApp for MeeGo Harmattan/Maemo (a port for Fremantle, which runs on N900 is coming). The main reason I installed this is because I got tired of friends asking me whether I have one.

The funny thing about instant messaging services is that we don’t appear to have enough of them. There was once ICQ was the only choice until the UIN gets ridiculously long. Then people switch to MSN/WLM or whatever crap Microsoft decide to call it now. Then it comes Skype, Google Talk and Facebook Chat, chances are, anyone who is relying Internet for communication purpose, would have more than one instant messaging service account, regardless whether they are being actively used.

Then comes the era of smartphone applications, which spurs the development of more applications which is supposed to make us more “sociable” and “approachable”. Aren’t we approachable enough already? What is wrong with just phone calls and text messages? Am I feeling insecure that people may forget about me so I keep on registering myself to new services?

It’s really tiring having to maintain so many instant messaging accounts and I doubt if I would jump into another one any time soon. In fact, I am in the process of pruning my contact list to contain just real close friends to ironically, and reduce my reliance to the phone.

Everyone knows folksonomy is (or was) cool and useful, however, when it is applied in real life, then problem arises. The idea of blogging this came while I am struggling to get my literature review report done (been doing it for months, I am being so ridiculous, I know). As a matter of fact, as I am dying to get it done, there are a couple of things that I found to be blog-worthy. So, I will be publishing a couple of brief overview to some of the topics involved in the coming days in a really casual (read: lazy, and full of personal speculations) way to this very humble little blog of mine.

Let’s start with a brief overview of folksonomy, which is typically a system (usually web 2.0 applications like flickr, youtube and del.icio.us) that allow users to annotate published content with keywords, also known as tags. The tagging may be done by all users or just the publisher of the particular content depending on the policy applied by the system. The usage of these tags is mainly for organization, but it may also help in discovering some unseen content. Well, I can really write a whole article just about folksonomy, but let’s just move on to the topic of the day.

Firstly, one may ask, WTF is a ramp-up problem? Imagine when a system grows as large as youtube for instance, where hours of video clips get uploaded in a minute (or even seconds, I don’t quite remember, but first heard here), expecting the publisher to individually tag their videos with relevant keywords is not really practical, considering there are already a number of them don’t even bother to provide description. No big deal then, one may just say. However, this pose a problem to
users like us, if there’s is no enough information provided, how are we supposed to find out what the video is about without having to waste our time watching it that has a chance of becoming yet another yet-another-friggin-cat-video-that-waste-n-minutes-of-my-life when I am really looking for something else?

If we can’t, how can youtube decide whether to put that video to a search result? Of course, it may be possible for a site like youtube to actually generate a transcript using their voice recognition technology. However, if the web application is developed by just a small startup, how are they going to find out the content of a multimedia content like video clips for instance?

What about cold-start problems then? A cold-start problem happens when a new user registers himself (or herself) to a system and the system doesn’t know anything about him due to the lack of information. It is a problem because the system wouldn’t be able to recommend him to other content published by other users that is relevant to him unless he start publishing and annotate his content or rather start ‘liking’ (rating) content published by others.

Wait a minute, didn’t I just read cold-start problem also applies to the scenario described in ramp-up problem mentioned above, you may ask. It’s true that there are people who generalize these two as cold-start problem, but I prefer to have them separated. Although it is true that they are both really similar in nature, but they are still two different cases, it makes explanation easier without the risk of confusing the audience.

So how does people respond to these problems? Some system would try to work out some implicit annotations from what they have in hand, however the result wouldn’t be just as accurate but still better than nothing most of the time. One of the real solutions to the problem is by encouraging users to tag by setting some challenging goals, aka. making it fun. In order to make the process fun, some uses the badge (or achievement) system like Stackoverflow, and some make tagging as a game. The other solution would be the implementation of auto-tagging feature, but that often involves a lot of computing magic and may not be really feasible.

On the other hand, for cold-start problem, a questionnaire can sometimes help in building a profile for the user to kick start his participation. However, the questionnaire can be left unanswered for uninterested user, or answered randomly. Another way of building a useful profile would be require a new user to rate or annotate a set of content until enough information is captured. Unless the process involved is fun, otherwise I don’t really see it as something that is doable. Demographic information would be used as a workaround to this problem by using it to complement the user’s profile. So, a new user would be assumed to share the same taste as the whole population in the beginning until they develop their own taste from the system’s POV.

I guess that’s it for this very brief note on cold-start and ramp-up problems. I would try to publish more in these couple of days while I am wrapping up my literature review report. My supervisor is gonna get so pissed off with me.

Some references (in no particular order):

• Chumki Basu, Haym Hirsh, and William Cohen. Recommendation as clas- sification: Using social and content-based information in recommendation.
  13
  In Proceedings Of The National Conference On Artificial Intelligence, vol- ume pp, pages 714–720. JOHN WILEY & SONS LTD, JOHN WILEY \& SONS LTD, 1998. ISBN 0262510987. URL http://scholar.google.com/ scholar?hl=en&btnG=Search&q=intitle:Recommendation+as+Classification: +Using+Social+and+Content-Based+Information+in+Recommendation#0.
• R Farzan and P Brusilovsky. Social navigation support in a course recommendation system. In V Wade, H Ashman, and B Smyth, editors, Adaptive Hypermedia and Adaptive WebBased Systems 4th International Conference AH 2006, volume 4018 of Lecture Notes in Computer Science, pages 91–100. Springer Berlin Heidelberg, 2006. ISBN 9783540346968. doi: 10.1007/11768012. URL http://www.springerlink. com/content/v4221425gm747700/.
• Rong Hu and Pearl Pu. Using Personality Information in Collaborative Filtering for New Users. Proceedings of the 2nd ACM RecSys10 Workshop on Recommender Sys- tems and the Social Web, 2010. URL http://www.dcs.warwick.ac.uk/~ssanand/ RSWeb_files/Proceedings_RSWEB-10.pdf#page=23.
• Stefan Siersdorfer and Sergej Sizov. Social recommender systems for web 2.0 folk- sonomies. ACM Press, New York, New York, USA, 2009. ISBN 9781605584867.
  17
  doi: 10.1145/1557914.1557959. URL http://portal.acm.org/citation.cfm? doid=1557914.1557959.
• Martin Szomszor, Ciro Cattuto, Harith Alani, Kieron OHara, Andrea Baldassarri, Vittorio Loreto, and Vito D P Servedio. Folksonomies, the Semantic Web, and Movie Recommendation. eprintsecssotonacuk, pages 71–84, 2007. URL http://eprints. ecs.soton.ac.uk/14007/.
• Andrew I Schein, Alexandrin Popescul, Lyle H Ungar, and David M Pennock. Methods and metrics for cold-start recommendations. Proceedings of the 25th an- nual international ACM SIGIR conference on Research and development in infor- mation retrieval SIGIR 02, (Sigir):253, 2002. doi: 10.1145/564376.564421. URL http://portal.acm.org/citation.cfm?doid=564376.564421.
• Douglas Eck, Paul Lamere, Thierry Bertin-Mahieux, and Stephen Green. Auto- matic Generation of Social Tags for Music Recommendation. Learning, 20:1–8, 2007. URL http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.67. 7891&rep=rep1&type=pdf.
• H Berger, M Denk, M Dittenbach, A Pesenhofer, and D Merkl. Photo-Based User Profiling for Tourism Recommender Systems. In Giuseppe Psaila, editor, ECommerce and Web Technologies, volume 4655/2007, pages 46–55. Springer, 2007. doi: 10.1007/978-3-540-74563-1\ 5. URL http://www.springerlink.com/index/ jv44060013345257.pdf.
• Hao Ma, Haixuan Yang, Michael R Lyu, and Irwin King. Sorec: social recom- mendation using probabilistic matrix factorization. In Proceeding of the 17th ACM conference on Information and knowledge management, pages 931–940. ACM, 2008. ISBN 9781595939913. doi: 10.1145/1458082.1458205. URL http://portal.acm. org/citation.cfm?id=1458205.
• Upendra Shardanand and Pattie Maes. Social information filtering: algorithms for automating word of mouth. In I R Katz, R Mack, L Marks, M B Rosson, and J Nielsen, editors, Proceedings of the ACM Conference on Human Fac- tors in Computing Systems, volume 1 of Proceedings of ACM CHI’95 Conference on Human Factors in Computing Systems, pages 210–217. ACM Press/Addison- Wesley Publishing Co., 1995. ISBN 0201847051. doi: 10.1145/223904.223931. URL http://portal.acm.org/citation.cfm?id=223931&coll=portal& d…N=53154003&ret=1.

It is very difficult to like the way vim handle plugins by default, so I was really thrilled to find out about pathogen when a geek I followed tweeted about it. It took me some time to actually re-organize my current configuration to this new format. Then I thought why not reorganize my .vimrc as well, as my current version looks a bit cryptic after a while.

So I have updated my .vimrc to something like this (link shows the diff). It is now a lot more elaborative, so whenever I need to change the settings I know what I am doing without having to refer to the documentation.

Hg-git is a great tool to enable Mercurial to work with git repositories. I know I should be picking up git but I still find myself liking Mercurial for now. So with pathogen around, I decided to install plugins in the right way by setting up proper sub-repositories. However, the problem comes when I was trying to push my changes. Apparently Mercurial will attempt to check with all repositories to decide whether to push changes. However, this is a problem when I do not have changes in the sub-repositories, and I do not have access to those repositories. So, according to one of the proposed solution to this problem posted at stackoverflow, I needed to fall back using an older version of Mercurial where there was no sub-repository support, and I ended up pushing my changes using version 1.2.1.

Another discovery while updating my configuration is that apparently there’s a transparency option for MacVim. So if one is using MacVim, to achieve the gorgeous transparency effect, just put this in .vimrc

" Set transparency for MacVim
if has("gui_macvim")
    set transparency=15
endif

I sort of like the current state of my MacVim right now. Will see how it works when I get this set of settings on a linux machine.

Should have done this earlier, I was just being lazy to go through all the steps to publish it properly. So here it is, the full source is published to bitbucket. Feel free to fork the project if you are interested. I have not attach a licence to it but it will most probably be BSD licence. I have also uploaded the latest 0.0.2 release to bitbucket and would update the download link posted previously soon.

Besides attaching a licence, there are a couple of things that needs to be done. For example, I would need to cough out a README.md to actually describes what the app does properly. I am still thinking whether I would want to provide proper support by enabling the issue tracker provided by bitbucket. For now, the app needs PySide to be installed and that would involve some manual work, at least a terminal app is needed to add new repositories.

I am also sort of interested in participating in this competition. There’s another application that I would want to develop but don’t think I would be able to make it in time as I plan to learn QML properly while doing it. Would have to see how everything goes.

This is basically a small incremental update to my script published here. For some reason, the previous version of the script didn’t really work, so this release should fix the problem. Besides fixing the problem where the daemon did not actually launched at start up, I have added a settings applet for this script as well.

Don’t feel like going the more complicated route, so I decided to just create a very simple applet using xml declaration. So I started with this documentation, and then followed the example given here.

The settings applet for the script looks like this

Switching the toggle on will stop the daemon from changing the ‘Allow background connections’ switch as I would sometimes need the data network to stay on for whatever reason. So I suppose this is a useful feature to have. Don’t ask me why the settings start in the middle of the screen, I have no idea, and I don’t care as long as it is working.

Again, the final product can be downloaded here, and there will be no support or whatsoever if you decided to install this. So install at your own risk, but you can still leave me comments here if you found problems. I have tested this on PR1.2, not sure if this would work for PR1.1 though.

Another half a day spent on figuring out how to package my daemon properly, fortunately with help from friends over at #harmattan IRC channel as well as cckwes, I finally get the deb package generated properly. So just a quick reminder on what my daemon does, it is just a quick hack that toggles the ‘allow background connections’ on and off depending which kind of data network a user is connected to. Apparently I am not the only one who are looking for this, as a feature request was filed long long time ago.

Preparing an Upstart Configuration File

After spending much time reading through the Upstart Cookbook and a brief explanation on how to develop a daemon for harmattan, all I could vomit was (reads: still need time to actually understand the whole document):


author "Choon-Siang Lai "
description "My workaround for enhancement "
 
expect daemon
 
exec /usr/bin/aegis-exec -s -u user /opt/restricted-mode-toggler/bin/control.py start


Just a quick note on the execution at startup, it would typically take minutes before the daemon is executed after the GUI is started. Not sure why this is, but don’t worry if the daemon is not executed right after startup, it takes time.

Packaging using Qt Creator under OS X

The reason why OS X is mentioned is because apparently the package building process is bugged. To fix the problem, edit dh_installdeb located at $QtSdkRoot/Madde/madbin according to the comments to the above mentioned bug report.

To make sure Qt Creator is building the package properly, I was told to add a line at the top of the .pro file as I am building a package that consists only python scripts.

TEMPLATE = lib

TL;DR The Final Product

The final product of my daemon can be downloaded here. Currently it is set to update the setting every 60-65 seconds. A GUI may probably be done if I feel like doing it.

Please note that I provide no support or whatsoever to the script and hold no responsibility if it kills your phone. I didn’t have time to add in dependency check in it, in order to run this properly, you need python-gconf, python-pyside and python-qtmobility installed. For instructions on how to enable the developer repository, read this wiki for more information.

Call me a cheapskate, as I still have not subscribe to a mobile data plan after purchasing my second smartphone, namely Nokia N9. There’s this ‘allow background connections’ option but it doesn’t care whether the connected network is a WLAN network or mobile data network. After finding out that Nokia has no interest in creating another separate option so that each type of network has their respective ‘allow background connections’ switch, I decided to make one for my own.

Turns out it is not too difficult to implement it via Python and PySide, thanks to djszapi, frals, itsabigtruck for their guidance over #harmattan IRC channel . Writing the daemon part is easy (but still took me half a day to get it work properly), however I have not figure out how to put everything together in Qt Creator. As I have not done any daemon in the past (I am a web developer, remember?), and Python not being my favorite language, it did took me quite a while to get the daemon part done. I followed the example published by Sander Marechal and have my simple daemon done.

Then I proceeded with the actual script, apparently toggling on and off the settings is not as complicated as it seems. Using python-gconf, the actual code to do the work is just:


gconf.client_get_default().set_bool('/system/osso/connectivity/network_type/restricted_mode', True);


So I proceeded with the actual QSystemAlignedTimer code (I used QTimer initially to get the idea) as this is the recommended way of creating a daemon as stated here. It took me quite some time to get everything working, and to my surprise it involves using of threads. If this is not confusing enough, apparently there’s a right way of working with threads.

However, after pulling a considerable amount of hair, and some help from Stackoverflow, I managed to vomit out something that works, as follows


class Updater(QObject):
def update_mode(self):
gconf.client_get_default().set_bool('/system/osso/connectivity/network_type/restricted_mode', self.is_wlan() == False);
 
def is_wlan(self):
return QSystemNetworkInfo().networkStatus(QSystemNetworkInfo.WlanMode) == QSystemNetworkInfo.Connected;
 
def run(self):
timer = QSystemAlignedTimer(self);
timer.timeout.connect(self.update_mode);
timer.start(300, 330);
 
class MyDaemon(Daemon):
def run(self):
app = QApplication(sys.argv);
 
thread = QThread();
thread.start();
 
update = Updater();
update.moveToThread(thread);
update.run();
 
app.exec_();


Now, the only question left is, how do I put everything together as a Qt project in Qt Creator so I can deploy this on my N9 properly. I am currently reading this so that I can make my daemon to run at startup.

Been trying my best to stick to the well-known UNIX Philosophy – “Do one thing and do it well”, so I have been breaking down my projects into numerous pieces of small tasks and rely on existing tools whenever possible. One of the existing tool that I use a lot is the GNU sort tool. Generally sort utility is really doing fine and dandy without having to configure anything, at least not until I realize the problem that leads to this post.

Considering I am having input in this form

http://flickr.com/photos/tags/スラム http://flickr.com/photos/13980928@N03/6001200971 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/13980928@N03/6001200971 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/13980928@N03/6001200971 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/21207178@N07/5441742937 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/21207178@N07/5441742937 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/21207178@N07/5441742937 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/25845846@N06/3033371575 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/25845846@N06/3033371575 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/25845846@N06/3033371575 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/30366924@N08/5772100510 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/30366924@N08/5772100510 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/30366924@N08/5772100510 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/31343451@N00/5957189406 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/31343451@N00/5957189406 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/36662563@N00/4815218552 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/36662563@N00/4815218552 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/36662563@N00/4815218552 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/38583880@N00/5686968462 1
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/38583880@N00/5686968462 1
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/46857830@N03/5651576112 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/99996011@N00/5396566822 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/99996011@N00/5396566822 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/99996011@N00/5396566822 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/38583880@N00/5686968462 1
http://flickr.com/photos/tags/スラム http://flickr.com/photos/43335486@N00/5794673203 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/43335486@N00/5794673203 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/43335486@N00/5794673203 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/46857830@N03/5651576112 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/31343451@N00/5957189406 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/46857830@N03/5651576112 0

I am expecting result in this order

http://flickr.com/photos/tags/スラム http://flickr.com/photos/13980928@N03/6001200971 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/21207178@N07/5441742937 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/25845846@N06/3033371575 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/30366924@N08/5772100510 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/31343451@N00/5957189406 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/36662563@N00/4815218552 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/38583880@N00/5686968462 1
http://flickr.com/photos/tags/スラム http://flickr.com/photos/43335486@N00/5794673203 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/46857830@N03/5651576112 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/99996011@N00/5396566822 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/13980928@N03/6001200971 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/21207178@N07/5441742937 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/25845846@N06/3033371575 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/30366924@N08/5772100510 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/31343451@N00/5957189406 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/36662563@N00/4815218552 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/38583880@N00/5686968462 1
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/43335486@N00/5794673203 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/46857830@N03/5651576112 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/99996011@N00/5396566822 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/13980928@N03/6001200971 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/21207178@N07/5441742937 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/25845846@N06/3033371575 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/30366924@N08/5772100510 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/31343451@N00/5957189406 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/36662563@N00/4815218552 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/38583880@N00/5686968462 1
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/43335486@N00/5794673203 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/46857830@N03/5651576112 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/99996011@N00/5396566822 0

but I am getting this instead

http://flickr.com/photos/tags/スラム http://flickr.com/photos/13980928@N03/6001200971 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/13980928@N03/6001200971 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/13980928@N03/6001200971 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/21207178@N07/5441742937 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/21207178@N07/5441742937 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/21207178@N07/5441742937 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/25845846@N06/3033371575 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/25845846@N06/3033371575 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/25845846@N06/3033371575 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/30366924@N08/5772100510 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/30366924@N08/5772100510 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/30366924@N08/5772100510 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/31343451@N00/5957189406 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/31343451@N00/5957189406 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/31343451@N00/5957189406 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/36662563@N00/4815218552 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/36662563@N00/4815218552 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/36662563@N00/4815218552 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/38583880@N00/5686968462 1
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/38583880@N00/5686968462 1
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/38583880@N00/5686968462 1
http://flickr.com/photos/tags/スラム http://flickr.com/photos/43335486@N00/5794673203 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/43335486@N00/5794673203 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/43335486@N00/5794673203 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/46857830@N03/5651576112 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/46857830@N03/5651576112 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/46857830@N03/5651576112 0
http://flickr.com/photos/tags/スラム http://flickr.com/photos/99996011@N00/5396566822 0
http://flickr.com/photos/tags/ホーム http://flickr.com/photos/99996011@N00/5396566822 0
http://flickr.com/photos/tags/ブラジル http://flickr.com/photos/99996011@N00/5396566822 0

After a quick search, it turns out that GNU sort actually rely on LC_COLLATE to do the sorting. I don’t really care about the character sequence order as long as I get consistent output, therefore the easiest way to fix this problem is obviously changing either LC_COLLATE, LC_ALL or LANG environment variable to “C”. However, to ensure the utility works the same elsewhere without interfering other scripts/programs relying on those environment variables, it would be easier to do this instead

$ <whatever command that produce output to stdOut> | LC_COLLATE="C" sort

Just a quick update to the previous post, the virtuoso storage engine works with redland provided the required packages are properly installed (yes, yes, yes, I know I haven’t release my PHP OO wrapper for Redland). Now that the package is installed, we need to do some configuration so that Redland can use it.

Installing Virtuoso

Just a quick update to the previous post, the virtuoso storage engine works with redland provided the required packages are properly installed (yes, yes, yes, I know I haven’t release my PHP OO wrapper for Redland). Now that the package is installed, we need to do some configuration so that Redland can use it.

Running Virtuoso

virtuoso-opensource and unixodbc-dev would have to be installed before virtuoso database can be started. After installing the required packages, make a copy of virtuoso.ini from /etc/virtuoso-opensource-6.1/virtuoso.ini to the directory where you want virtuoso to store the database.

$ mkdir foo
$ cp /etc/virtuoso-opensource-6.1/virtuoso.ini foo

Edit the ini file as desired, just make sure the configured ports aren’t being used by other processes (by default, virtuoso runs at port 1111 with web interface at port 8890, so if you are running another instance of it, change the port settings). If you are not interested in running another instance of virtuoso database, you can actually skip this part.

After everything is configured, just run this command at the directory.

$ cd foo
$ virtuoso-t -fd

For more information, please refer to this quick start guide.

Setting Up ODBC

By referring to this tutorial, add these lines to /etc/odbc.ini (assuming the database name is SENSEIDB)

[SENSEIDB]
Driver = /usr/lib/odbc/virtodbc.so
Description = Sensei DB
Address = localhost:11111

Making Redland to Work With Virtuoso

Just make sure the ODBC configuration is done correctly, then follow the guide for further instructions.

I wanted to try using virtuoso as the storage engine for Redland but unfortunately there is no librdf-storage-virtuoso package for Ubuntu. After getting some help from @dajobe, I attempted to build the packages myself. Although it takes quite some time to build packages, but not too difficult it seems.

The version of Redland that I am attempting to build is 1.0.13, and unfortunately this means I would have to build librasqal3 too as the version provided by Ubuntu Natty is not up to date. Installing the packages may cause some breakage, so… YMMV.

Pre-requisites

By following the instructions here, add this source repository to /etc/apt/sources.list

deb-src http://download.librdf.org/binaries/debian/unstable ./

And then run an apt-get update

$ sudo apt-get update

Building Rasqal

To build Rasqal, first grab the source package

$ sudo apt-get source librasqal

Grab the source packages

$ sudo apt-get source librasqal3

.. install the dependencies

$ sudo apt-get build-dep librasqal3 librasqal3-dev

Then build them

$ sudo apt-get -b source librasqal3

Then install librasqal3, librasqal3-dev, rasqal-utils

Build Redland

First we will need to grab the packages

$ sudo apt-get -b source librdf0

You may get an error saying some packages can’t be downloaded, in particular the source package. Just download this into the same directory as the other downloaded tarballs and rename it as redland_1.0.13.orig.tar.gz (Note: as of writing, source packages for 1.0.14 is not out yet, you may want to adjust when package for 1.0.14 is out).

There are some other dependencies besides the listed ones that needs to be installed, to install all of them, issue these commands

$ sudo apt-get install virtuoso-opensource unixodbc-dev
$ sudo apt-get build-dep librdf0

Now we can proceed to build Redland

$ sudo apt-get -b source librdf0

To build librdf-storage-virtuoso, Navigate to $REDLAND_SRC/redland-1.0.13/debian and edit a file named ‘control’ (without the quotes). Append this to the end of the file.

Package: librdf-storage-virtuoso
Section: libs
Architecture: any
Depends: ${shlibs:Depends}, ${misc:Depends}
Replaces: librdf0 (<< 1.0.9-3)
Description: RDF library, Virtuoso backend
 Redland is a library that provides a high-level interface for RDF
 (Resource Description Framework) implemented in an object-based API.
 It is modular and supports different RDF/XML parsers, storage
 mechanisms and other elements. Redland is designed for applications
 developers to provide RDF support in their applications as well as
 for RDF developers to experiment with the technology.
 .
 This package contains the Virtuoso storage backend.

Edit a file named 'files' (without the quotes) and append this line

librdf-storage-virtuoso_1.0.13-2_i386.deb libs optional

Add a new file named librdf-storage-virtuoso.install with the following content

debian/tmp/usr/lib/redland/librdf_storage_virtuoso.so

Add another file named librdf-storage-virtuoso.substvars

shlibs:Depends=libc6 (>= 2.3.6-6~), libc6 (>= 2.4), libdb4.8, librdf0 (>= 1.0.13), unixodbc (>= 2.2.11)
misc:Depends=

Not sure what happens, but that was the edited version after building the packages, my original version is shown below (will update once I find out why)

shlibs:Depends=libc6 (>= 2.3.6-6~), libc6 (>= 2.4), libdb4.8, librdf0 (>= 1.0.13), unixodbc-dev (>= 2.2.14p2-2ubuntu1), virtuoso-opensource (>= 6.1.2+dfsg1-1ubuntu4)
misc:Depends=

Lastly, create a new file named librdf-storage-virtuoso.debhelper.log from librdf-storage-mysql.debhelper.log Apparently this file is generated automatically, thanks @dajobe for the tip again.

Lastly, navigate back to $REDLAND_SRC and build the packages again

# apt-get -b source librdf0

Building Redland-bindings

Same thing again, first grab the source packages

$ sudo apt-get source redland-bindings

Since I am just interested in the php5-librdf package, I issue this to fetch the dependencies

$ sudo apt-get build-dep php5-librdf

Proceed with building

$ sudo apt-get -b source php5-librdf

Everything looks fine as of now, but I am seeing some random conflicts. Will try to work with the virtuoso storage engine and post a follow up later today.