IdeaMonk

Scraping in Python for fun and profit

2011-01-05T16:18:00.000+05:30

Missed out on this one lately.

Q1 of BORQ - Mad Libs

2010-10-24T21:08:00.000+05:30

My solution for Mad Libs, I need to practice some more regular expression. Rubular is a great help for people like me. RegExp is in a more free-form in Ruby compared to python, which is quite good for quick experiments on irb.

I wish I could even get rid of the "if"s and make it sexier. Can't think more on this one right now.

Q2. of BORQ - LCD display

2010-10-24T21:06:00.000+05:30

This is an old one, pretty famously seen on uva, etc. I found Choice to be better than OptParse, as said in Choice docs - it like writing poems for command-line parsing :D

Initially I was doing a string[j].chr.to_i which seemed like a required bullshit, I wish it could've been string[j].to_i but string[i] gives back a decimal. Now one could talk about the C style '5'-'0' conversion to number, but then, that machine dependent (think ascii, ebcdic, etc). The slightly more ruby-ish way - string[j..j].to_i :)

Here's my number -

abhishekmishra@mbp [~/code/BORQ]> ruby q2_lcd_numbers.rb -s 3 9535009187
 ---   ---   ---   ---   ---   ---   ---         ---   ---
|   | |         | |     |   | |   | |   |     | |   |     |
|   | |         | |     |   | |   | |   |     | |   |     |
|   | |         | |     |   | |   | |   |     | |   |     |
 ---   ---   ---   ---               ---         ---
    |     |     |     | |   | |   |     |     | |   |     |
    |     |     |     | |   | |   |     |     | |   |     |
    |     |     |     | |   | |   |     |     | |   |     |
       ---   ---   ---   ---   ---               ---

Surprisingly the PyMos core code is 180 lines while this reaches upto 108. Python => more results per line? The absence of 'end' in python is one big reason for this. Besides I dont find my ruby code so ruby-ish at this stage.

Speaking at PyCon this weekend :)

2010-09-22T00:45:00.000+05:30

Here are the slides for talk at the upcoming PyCon India 2010. See you this weekend.

Scraping with Python for Fun and Profit - PyCon India 2010

View more presentations from Abhishek Mishra.

See you there!

Poorly implementing caching in python - eye opener

2010-07-17T06:10:00.021+05:30

So after looking at some slides on caching function returns in javascript, I was keen on trying out so in Python. And LOL I came up with this logic -

fun(val):
  if val in cache.keys():
    return cache[val]
  else:
    do the right thing...

But it seems, though "if val in cache.keys()" sounds very human friendly, it definitely would suck for a very big cache, and so it does in the following test.

I guess I'm not using timeit in the classical way where I would pass some statements in string and ask it to do them for N number of times, but it seems passing some variables from existing code to statement string is a pain, tried global etc, didn't work. Hence a simple time diff test.

# -*- coding: utf-8 -*-
import random
from timeit import Timer

class PlainColorParser:
    def parse(self,value):
        rgb = int(value[1:], 16)
        r = rgb >> 16 & 0xff
        g = rgb >>  8 & 0xff
        b = rgb & 0xff
        return (r,g,b)

    def __call__(self,value):
        return self.parse(value)

class NoBitColorParser:
    def __call__(self,value):
        return (int(value[1:3],16), int(value[3:5],16), int(value[5:7],16))

class PoorCachedColorParser(PlainColorParser):
    def __init__(self):
        self.cache = {}

    def __call__(self,value):
        if value in self.cache.keys():
            return self.cache[value]
        self.cache[value] = self.parse(value)
        return self.cache[value]

class CachedExceptionColorParser(PlainColorParser):
    def __init__(self):
        self.cache = {}

    def __call__(self,value):
        try:
            return self.cache[value]
        except KeyError:
            self.cache[value] = self.parse(value)
            return self.cache[value]

class CachedColorParser(PlainColorParser):
    def __init__(self):
        self.cache = {}

    def __call__(self,value):
        if value in self.cache:
            return self.cache[value]
        else:
            self.cache[value] = self.parse(value)
            return self.cache[value]

if __name__ == "__main__":
    t = Timer()
    pccParse = PoorCachedColorParser()
    cecParse = CachedExceptionColorParser()
    ccParse = CachedColorParser()
    pcParse = PlainColorParser()
    nbParse = NoBitColorParser()

    # setup some random data to test
    colors = []
    for i in xrange(100000):
        colors.append("#" + hex(random.randint(0xfe0000, 0xff0aff))[2:])


    def timeDiff(obj):
        start = t.timer()
        for c in colors:
            obj(c)
        stop = t.timer()
        return ((1000000*stop - 1000000*start)/1000000)

    # ---- test poorly cached
    print "Poorly Cached - %.2fs" % timeDiff(pccParse)

    # ---- test exception cached
    print "Exception Cached - %.2fs" % timeDiff(cecParse)

    # ---- test cached
    print "Cached - %.2fs" % timeDiff(ccParse)

    # ---- test uncached
    print "Non Cached - %.2fs" % timeDiff(pcParse)

    # ---- test no bitwise, uncached
    print "Not Bitwise, Non Cached - %.2fs" % timeDiff(nbParse)

So we've got 4 5 classes to represent different ways of parsing an html hex code for color e.g. "#f00f00" into a tuple of (r,g,b) integers. PlainColorParser and NoBitColorParser could easily be functions with no need of classes over them as they do not cache, but to bring them a little equal to other two cached ones, I've bound them in classes.

NoBitColorParser does string manipulations and parses 3 times before returning a tuple. PlainColorParser does better than that, it uses bit shifts and AND masks to filter out content after 1 round of parsing integer from string. PoorCachedColorParser does caching in an obvious way "if its there in cache keys... else ...", and ~~CachedColorParser~~ CachedExceptionColorParser complies to the philosophy of "Fail early, fail often", which is quite interesting :D
but recent findings reveal that CachedColorParser is the right, fast, pythonic way.

What the test does is - generate 100000 random color codes, pick them from a range of 2816 colors (0xff0aff - 0xff0000 + 1). Obviously many colors are bound to get repeated says pigeon hole.

Here's what goes on in an average run on my machine -

abhishekmishra@mbp [~/code]> python pycaching.py
Poorly Cached - 340.23s
Exception Cached - 0.30s
Cached - 0.18s
Non Cached - 0.15s
Not Bitwise, Non Cached - 0.16s

"if value in self.cache.keys():" in PoorCachedColorParser gives you a thumbs down with a sucky performance, obviously not the right thing to do!!! (I was mistaken)

CachedExceptionColorParser gives a sweet 0.30s, "Fail early, Fail often" works :)
But wait, CachedColorParser goes further even a bit more with just 0.18s.

NoBitColorParser is suckier than the Non Cached PlainColorParser, which points out that string ops, parsing integers is one costly affair.

So much for food to my sleeplessness. Oh I remember doing something similar in PyMos too :D.

More updates -

Looking at Dhananjay's code on BangPypers, I think I was too excited to throw in the idion of fail fast in this place, it can albeit be done in a cleaner way. So instead of -

try:
    return self.cache[value]
except KeyError:
    self.cache[value] = self.parse(value)
    return self.cache[value]

You could just write in much cleaner way -

if value in self.cache:
    return self.cache[value]
else:
    self.cache[value] = self.parse(value)
    return self.cache[value]

So the issue was with cache.keys(), which now seems like an obvious slow and shitty way.
New stats reveal that even Try: Except:... fail fast is even not the right way.

Notice the almost double time difference between "try.. except.." way and "if value in self.cache".

Lesson learnt :)

अपने घर के चोर

2010-07-14T15:13:00.000+05:30

You find it to be a sorry ass pussy system 
that fails to respond to situations in time, 
you read up, watch stuff and blame on them for "not doing anything", 
but hardly would it strike to your conditioned imagination 
that they might just be so pre-occupied in doing something else 
that you could have never imagined. 

And yet you passive cunt, you take it all for granted
and sleep for the next day to come by,
expecting everything to be normal.

Who knows for everything,
that you quoted the system to be pussy about,
over your cup of tea and a morning newspaper,
there were these "अपने घर के चोर",
thiefs, filling their pockets.


On a side note,
Now that things have become so well connected,
the systems are so well informed, fed by numerous channels,
and fingers just like the ones that typed out this piece of brainf*ck,
clicking out zillions of preferences,
we're not far from the point when
it would be possible to crunch these into
a whole new understanding, meaning, life.

Just that in this spiderweb,
analyzing nodes, reaching from one to another,
predicting their minds and way of thinking has become easier.

DWM-ified

2010-06-21T11:58:00.003+05:30

Arch + Dwm + Conky + Dzen2 + links + vim + irssi (plus ram eaten by chrome, firefox, thunderbird, and eden) otherwise awesome at ~100MB, ideal light & fast setup for my netbook :)

Release early, release often, release immediately?

2010-05-21T12:19:00.002+05:30

Okay nothing much for this post, but I was just thinking about the RERO philosophy apparently popularized by ESR in CATB (I guess I won't need to read the 256 pages).

Anyways so I was wondering if releasing immediately is a good thing to do? Picture yourself working whole night over a cool project that you intend to release for the public. Its 6 AM, you're done with your last commit, nothing seems to be broken, yet is it the right time to release? The problem has nothing to do with any sort of technicalities, but is a psychological one and completely based on my experiences with such quick releases.

The question is, are you left with enough energy after working for 2-3 days straight to take the criticism, suggestions and comments seriously in a positive way? At least I as a lone developer of something that I put forward to the public haven't been able to harness the feedback that people gave me. I ponder over the reasons, and I'm not sure of why is that. But mostly it went something like this -

"Could you possibly include feature XYZ into it?"
and I think - oh cummon, I just got over with this work and now I need a break.
"This stuff seems broken, I cant move X to Y."
and I think - who said I made it for you :/
"It would be nice if you include ABC and XYZ also"
and I just write a good reply to the person, add it to my non-existant list of todos in mind or Tasks in gmail and it stays there forever.

This has happened many a times, even valuable suggestions have just ended up in a todo list rather than getting implemented and the projects hardly getting touched again.

How do you go about it? or is it just me who keeps hopping over ideas and leaves the old ones half done. Or is it the procrastination that engulfs on dawn of winter and summer vacations.

An interesting observation over procrastination. During this end sem exams, we were given a 5 day prep leave, and LOL I completely spent the whole time writing some piece of code instead of doing maths. Same story for any intermediate breaks between exams, reading something, watching some anime, writing some code but zero preparation for any exam. And yes a hell lot of ideas cooking in your mind for coming vacations.

Surprisingly it happens, not once but many times, that when exams end - I go completely blank over things I had planned. And with "Oh come on! exams just got over!" mentality I start spending days being useless to myself and everyone else.

And even though some new work arrives, I still end up behaving as if I'm super busy with things. Hoping to get rid of this soon and maybe hack this "Not doing what I am supposed to do at this time." way of life for greater goods.

</procrastination>

Okay the blogger WYSIWYG editor is really not good on chrome. It doesn't render <, > properly and end up putting in loads of DIVs instead of beautiful Ps as in firefox. Why can't they have a standard across two browsers for same webapp.

Summer 2010 #1

2010-05-16T13:10:00.006+05:30

So this is 16th already. Today, being a Sunday - we still had some college classes to attend, namely the placement classes. Glad that I bunked and got proxy in majority of them. I mean seriously, it sucks to have your college delay you from your planned schedule.

So this is the action plan for today -

Setup local repositories, branches for gsoc on launchpad and a latest copy of web2py.
Go through the all mails you missed out during exams.
Do something with web2py quickly to go through everything it offers (maybe port bugbase to web2py)
Make notes on my understandings, raise doubts, go though UI testing chapters sent by Michael.

I will end it there to KISS as the evening might be a little pre-occupied with a friend coming over.

This leaves me with a week before community bonding ends, to finish off with getting as familiar as possible.

Keeping myself on the right track would be the job of Korganizer and RTM.

I had been using KDE's RTM plasmoid to keep track of things till April end when I got my netbook.

On gnome I'm sticking with GTG (Getting things Gnome) but nothing beats a todo reminder like RTM plasmoid on KDE which omnipresently reminds you of what needs to be done.

RTM plasmoid - adding another step to KDE's social desktop

GTG - syncs and sits in try making you lazier

Some afterthoughts on yesterday's talks

2010-05-16T10:30:00.000+05:30

One interesting observation that comes to my mind after

Yay! Freebies arrive :D

2010-05-15T16:41:00.003+05:30

Just got back home from WebApps 2010 to find these two awesome books from Linux Format magazine kept on my chair. Thanks to Graham Morrison for the initiative. Love the color print in the one from Yahoo Press.

Met a good designer from Shrishti college Chennai, had great time discussing the borderline between a developer's and a designer's perspective. Interesting day by chance, otherwise apart from a few talks from rediff.com VP, and Mr. Satyadeep Vusuthy of Yahoo India R&D and @hsivaram of Adobe, the others were a little boring. Had to be like that, for the geekiness quotient was low in the masses.

Linux++

Orkut spotted in Doga

2010-05-05T23:05:00.002+05:30

Orkut is pretty much a part and parcel of our daily lives - even in Indian comics now - Doga- the best Indian comic action hero with some serious balls. (courtesy @ixpu)

ASUS eee PC 1005P review

2010-05-02T17:04:00.006+05:30

1. Small form Keeps the desk clean Easy to carry	2. Glossy screen :( reflects bright light :) gives good picture quality	3. 11 hours battery backup. Works for 6hrs max on linux, starts from 10hrs on Windows 7
4. as wide as a book	5. as thin as a book	6. as thick as 2 books :D
7. Great touchpad, never becomes slippery/sticky	8. Chicklet keyboard, effortless to type, takes time getting used to	9. Runs Ubuntu 10.04 decently
10. 30 lines of code at a time	11. Completing a college project in QtCreator was an okay experience, project build was slow.	12. Wrote a PHP webapp on it, wasn't a pain!
13. Plays HD as well, good to catchup with anime while travelling	14. Decent fullscreen performance	15. No No No don't do this with it

Been using it for a month now, bought it from Staples, Koramangala for Rs. 18900. Somehow the prices of netbooks are little hiked up here compared to ones mentioned on Amazon. In some more depth -

In its small form it encloses decent power -
- Atom N450 (1.66 Ghz) processor
- a 250 GB harddisk
- a 0.3 mp webcam
- 1 GB RAM (which is enough if you run fvwm2 or awesome wm)
  Basically Ubuntu+Chrome+Flashplayer+20 tabs is destined to screw this machine, don't even run two browsers!
It has a 10.1 inch glossy display and supports upto 1024x600 resolution. Its pretty decent for web browsing with Chrome or a customized Firefox with menus, statusbar, etc removed using addons.
Claims 11 hours of battery life, well that's if you keep running what it came with - Windows 7. Unfortunately I removed it 2 days after the purchase and have been looking for a large capacity USB to restore in future. On linux, battery life is a pain, though on Arch linux sans a functioning window manager/desktop, I could run it for 3 hours watching movies in mplayer and the battery drained till 48%.
Copying things from an external harddisk sucks battery very quickly. Don't keep ntfs partitions mounted otherwise flush and jdb2 calls every 3 seconds will suck more.
On average, use wifi, watch youtube, keep surfing for 4 hours without any worries.
Again pretty light on my college bag, just like a book.
Decently thin too, and it is something that you can really hold in your lap.
Beware of dust though, looking at heat outlet openings, and being able to see some circuitary from there, it gives you scary thoughts of "what if... ?"
Little thicker towards battery side though. It does generate less heat than my previous laptop, though I haven't tried compiling KDE on this.
I like the touchpad, look at those protrusions over it, it never gets sticky at all, now matter how bad your fingers are. Though touchpad sometimes interferes with your typing.
Small keyboard sucks initially, specially the Fn key and Home and End button placement. While coding I do a lot of Home, Ctrl+Home, now I have to press 2 and 3 keys respectively. Once you get used to this keyboard, you can type faster.
I tried both Ubuntu and Arch on it. It started from Ubuntu 10.04 alpha on which 3d accerelation magically worked. Upon dist-upgrade to beta and subsequently to LTS, it gave up on 3d acceleration and runs in VESA mode, which I'm okay with, just that I can't enjoy Quake 3 for some time.
The Poulsbo graphics chipset is pretty new and would take time to get support here. Some guys in Mandriva have found solution, while the ones in Arch community are still in discussions. The problem is that Xorg-server 1.7.x as on arch, doesn't even run with VESA on this hardware, which is something that should work.
I wish to put Arch as my main OS here soon, will try by downgrading the Xorg-server to 1.5.x-gentoo from AUR.
Coding on a netbook - yes its possible. All IDEs and editors gave me decent 30 line view after some tweak :)
Coming to IDEs I wouldn't recommend running Eclipse on it to anyone, though Qt Creater did a good performance. Build takes some 3-5 seconds more than what it took on old laptop. The embedded Qt designer in QtCreater is impossible to use though, better stick to launching Qt Designer externally.
Web development is what goes good with netbooks, you really don't need to compile anything here, and having an instance of apache2 and mysql in background doesn't hurt at all. Though you won't be able to do a lot of Firebug here as it would eat most of the browser real estate. Same goes with multi-resolution testing. Good part is that it attaches to an external very well and you can still get resolution like 1920x1080 to work on.
I've used GIMP - again a pain to use on a netbook and Photoshop 7 over wine - which is relatively less painful.
Coming to moves, yes, this is something the device is very good at. Is able to play full HD quality movies at full screen :)
You can watch around 2-3 movies in a row given it works for 6hrs on your distro. Besides what people commonly say about display eating most of the battery, is true. You can see instant gains in battery life when you dim the display. Perfect for catching up with movies before you sleep with lights off in your room.
Ah! don't think of doing this. Virtualization, bah! obviously not. Though I did it for the same Qt project to write some more lines to make it crossplatform compatible, horrible build time in a QtCreater inside vm - almost 2-4 minutes to build the project./
Also a netbook isn't the right hardware to compile anything substantially big.

~~The only problem I face right now is the Poulsbo compatibility issues with Arch and some more distros, am little annoyed to see that X is freezing even in vesa mode.~~

Update: As of now the lastest release of Arch linux works like a charm, I'm running dwm on it and its the best thing that could happen to a netbook.

Otherwise a netbook is the best way to roam around with a full-fledged linux box in your back pack. I sync my mails offline, catch them up whenever you got time. Play school-time Super Nintendo roms when bored. Catchup an Anime while coming back home from college. Experimenting with some code when it comes to mind. Giving presentations wherever you may be, writing down your thoughts as they come to your mind anytime anywhere, and what not...

New upcoming models from asus are even more awesome - 1201N - metal body - 2GB ram - 1366x768 killer resolution! - 256 MB Nvidia ION all for around $499

Inspiration and the inspired

2010-05-01T19:21:00.009+05:30

Dad just brought over this clipping from way back in 2001 to me today. OMAN this is so cool, they covered our GW-BASIC programs in news paper :)

I had written a wheel of fortune program with colors, wheel animation (the wheel never rotated, the colors did), funny sounds from the PC speakers, all in 320x200 screen 13 :). I remember Mohit's brother fortunately did win the complete game.

While my brother @ixpu , who is more into chasing electrons and writes more C# these days, had made a clone of Kaun Banega Crorepati, again in GW BASIC, I guess he was using screen 12 (640x400 I guess)

The only difference between today and those days - it was hell lot more exciting back then with all the time to create things like these.

While today it happens to be a mixture of passion and time-management between pointless acads and programming.

I wish I could see some of the source codes from those days, which eventually got destroyed in format wars between us.

Wooh 9 years, well that was the un-procedural language with line numbers and many GOTOs here and there.

GSOC 2010 - Would be working on Sahana Eden this summer :)

2010-05-01T11:59:00.009+05:30

I applied for two orgs for GSoC 2010 this time - Sahana Software Foundation, PSF and have been selected for my "HTML/JS based reusable frontend for S3XRC" proposal for Sahana Eden (previously known as SahanaPy). I wish I had applied about an year ago too.

Most of all I'm so because I'm gonna be working on things I love -

Sahana Eden is an Emergency Development Environment - a framework to rapidly build powerful applications for Emergency Management. It is something which you can take and quickly customize to suit the nature of disaster, location, language, etc. It can be morphed into many faces, as per the demands from the use-case. A really life saving app which has a proven record of assistance in disaster struck areas. Sahana Eden has been used in Haiti, World Food Project, etc while previous versions of Sahana have helped a lot in Sri Lanka, Bihar.

A brief history of project Sahana here.

Working with the Sahana Eden team at FOSS.in '09 was a delight, from there on I could contribute little more till my winter break ended, but haven't been able to hack more since then due to the _busy_ _busy_ college life in India :/ (which ofcourse I hate a lot). I'm gonna quench all my thirst this summer.

Introducing BugBase 1.0

2010-04-27T03:56:00.003+05:30

Here's something about what has kept me busy since last week - a college miniproject. The quickiest dirtiest php app I ever wrote. Still incomplete with loads of things to fix, add, etc. Nevertheless - the intro :)

Introducing BugBase 1.0

View more presentations from Abhishek Mishra.

VRML, Direc!Tax, book, shops, kapoorthala...

2010-04-25T23:58:00.005+05:30

Looks like they just picked books from the end of an alphabetical list - VRML, Visual J++. Just reminds me of the time in that Universal bookshop in Kapoorthala, Lucknow when me and @ixpu used to drool at a wall of Computer programming books.

If you asked for a book on DirectX the shop owner usually used to get confuse it for Direct Tax. Wooh boy that was like 8-9 years ago down the line. There was a couple which was correcting each other on right pronunciation of C# telling "Its not See-Hash its See Sharp" and Dipanjan sir or someone said 'This is M$'s answer to Java', etc.

For each ...

2010-04-24T19:00:00.000+05:30

Weird ways I learnt a few things down the line -

'for each' construct from the source code of love worm
PHP because writing perl cgi scripts was a pain on the free tripod account, and PHP for all wrong reasons ;)
VB - 1st to make pacman, then again for all wrong reasons.

more to come as I recall.

My GSOC proposal for SahanaPy - HTML/JS based reusable frontend for S3XRC

2010-04-09T21:14:00.003+05:30

1. Personal Details

Name- Abhishek Mishra
Communication details -
Email - ideamonk@gmail.com
Freenode IRC Nickname - ideamonk
Google talk - ideamonk
Skype - ideamonk
Twitter - ideamonk
Blog - http://blog.ideamonk.in
I am from a city called Patna in Bihar, India (+5:30 GMT). Currently I am pursuing a B.Tech (a 4 year course) in Amrita School of Engineering, Bangalore. I spend most of my day working on my side projects and cooking new ideas in mind when free from college. In free time and holidays I also work as a freelance web developer with my friends - http://madetokill.com
Resume - http://ideamonk.in/abhishek_mishra_resume.txt

Exposure to similar technologies and/or FOSS in general.

I started relying heavily on open source projects to learn from and use them ever since I switched to Linux about 1.5 years ago. I started learning Python about 1 year ago and it has in a way become my primary language to implement anything that comes to my mind. I have worked on SahanaPy in the past (the current steel theme, custom error pages, a bit of ExtJS work) and am quite familiar as to how it works. Interestingly SahanaPy is the first Open Source project I've contributed to.

Some thoughts over Free/Open Source Software -
Subtle - as they may look, ignore - as many people may do, but at the end of the day, there's no denying that "Open Source is the best infrastructure you can rely on" to - solve complex problems of the world, help people around the world (the way SahanaPy does!), or even to start your business.
I believe FOSS has saved humanity a lot of time by showing them that "Re-inventing the wheel" is not always the right approach to get things right.
Some of my side projects - http://github.com/ideamonk

“Why would you like to help the Sahana project?”

I was initially introduced to SahanaPy on IRC by a few friends. Later in Dec 2009, I got a chance to participate in SahanaPy workout at FOSS.in. I believe that I could put in my experience in developing Web Applications very well into S3XRC frontend and help developers deal with compound results in an easier, uniform, organized way.
Besides, to the common question that people ask "How would a software help in disaster?", Sahana stands as the best example you could give them, given that it has actually been used at many incidents. The recent support shown to Haiti crisis by Sahana foundation is something that would inspire us to do good at all times. I believe I could too make significant contribution to such a great endeavour.

2. Personal Availability

Yes I have reviewed the important dates and timeline of GSOC 2010
Apart from my institute's end-semester exams that fall from April 30th to May 13th, I have no other conflicts with the schedule.
Will you need to finish your project prior to the end of the GSOC?
Yes, I would take utmost care to finish it before GSOC ends so that developers could start utilizing it fruitfully by then. Though S3XRC frontend would always have my support post-gsoc too in case any issues/bugs come up.
I would be available for entire duration of GSOC 2010 starting from May 14th when my exams end.

3. Project Abstract

Currently Sahanapy uses two backends for REST - S3XRC (XML+JSON interface of the restful API) and CRUD (Web2Py crud for html interfaces). Web2Py crud is good at a few things e.g. client side validation, ease of use, but it also has limitations, i.e. It can access only single table at a time.

On current version of SahanaPy this is evident when you take a look at http://127.0.0.1:8000/sahana/hms/hospital/1.xml [1] and compare it to http://127.0.0.1:8000/sahana/hms/hospital/1 [2]

While [1] is generated by S3XRC, it contains nested resources, not only do you get hospital details, but also current status of beds.
On the other hand [2] generated by web2py is not able to do anything worthwhile to this information.

Unfortunately S3XRC has no HTML front-end yet. The backend is able to handle XML and JSON but web2py front-end (SQLTABLE and SQLFORM) are not able to handle the complex resources.

A good way out would be to replace them with new handlers - XRCTABLE and XRCFORM

Which would understand and represent or interpret S3XRC's XML/JSON output appropriately. After that jQuery/ExtJS based web interface would be made which would be more responsive and would be able to take advantage of JSON outputs of S3XRC.

4. Project Plan

This section is to provide the detail of your project proposal. Take as much space as is necessary.

Project Deliverable -
- The ability to present the complex resources returned by S3XRC to the user.
- A more responsive jQuery/ExtJS based UI that would change less, and actively load data through AJAX
- We would be able to save good about of bandwidth, would be able to work in low bandwidth conditions when data gets passed in JSON format.
- Since aimed to be written in a generic and reusable way, this would be reused in other parts of code too.
- Besides this work would also reduce the number of requests, which implies quick data entry at times of disaster management work.
Project Justification -
- The number of useful inter-related data we wish to retrieve and represent would definitely increase in time. This calls for a way to represent then to the user in a RAD fashion.
- One can never be sure about availability of bandwidth at sites of disaster. At times one might have to work in very low bandwidth conditions. The reduced size and frequency of requests that the S3XRC frontend would bring, will help rescue workers easily operate SahanaPy in such conditions.

Implementation Plan -

A sample compound resource -


<s3xrc success="True" results="1" domain="<span style=" class="J-JK9eJ-PJVNOc">rocksolid</span>"  <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">url</span>="http://127.0.0.1:8000/<span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">sahana</span>" <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">latmin</span>="-90.0"  <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">latmax</span>="90.0"  <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">lonmin</span>="-180.0" <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">lonmax</span>="180.0">
  <resource name="<span style=" class="J-JK9eJ-PJVNOc">hms</span>_hospital"  <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">uuid</span>="eb903352-0a6c-4d3c-bb8c-f18f6b7a3179"  created_on="2010-04-02  15:14:07" modified_by="<span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">Abhishek</span> <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">Mishra</span>" created_by="<span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">Abhishek</span> <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">Mishra</span>"  modified_on="2010-04-02 15:14:07"  <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">url</span>="http://127.0.0.1:8000/<span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">sahana</span>/<span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">hms</span>/hospital/1">
    <data field="website"></data>
   [[ snip snip ]]
   <data field="info_source"></data>
   <data field="phone_exchange"></data>
   <resource name="<span style=" class="J-JK9eJ-PJVNOc">hms</span>_<span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">hrequest</span>" <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">uuid</span>="c9aa43df-11fc-4b63-977f-d56abaefd1cf"   modified_by="<span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">Abhishek</span> <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">Mishra</span>"  created_by="<span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">Abhishek</span> <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">Mishra</span>"  created_on="2010-04-02 15:14:58" modified_on="2010-04-02 15:14:58"  <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">url</span>="http://127.0.0.1:8000/<span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">sahana</span>/<span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">hms</span>/hospital/1/<span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">hrequest</span>/1">
      <data field="city"></data>
       [[ snip snip ]]
     </resource>
 </resource>
</s3xrc>

^^ Notice the nested

Approach 1 -
Addition of new view generators for complex resources as shown above, namely XRCTABLE and XRCFORM to SahanaPy. Using these to generate ExtJS code that would do the interface. We already use ExtJS a lot, so this new front-end would be more generic and fast.
So, this would be an ExtJS based front-end with S3XRC as backend.

Approach 2 -
Exploiting S3REST pre and post hooks. When one calls a REST controller, the following happens --
1. REST parses the original server request into an XRequest object (jr)
2. XRequest contains hooks to export / import directly in XML or JSON Eg. jr.export_xml()
3. After parsing a pre-processor is called - which is dynamic, i.e. a controller can set the pre-processor as response.s3.prep
4. This means we can pre-process the data before REST action is executed, and even change the REST action itself. Eg. s3rest.set_handler ("foo", my_foo_handler)
5. If the Pre-processor returns False, then the REST action is not taken at all. This means we can manipulate the flow as we wish to.
6. At last step, a post-processor is called, which receives both the output of the method handler and the XRequest object. This can be set using response.s3.postp
7. At this stage we can generate the HTML and ExtJS code required by the views.
8. The view then builds the generated HTML and ExtJS code into the HTML template.
An optimal approach is to have the ExtJS objects pre-loaded, and retrieve data to them as needed. XRequest supports server-side validation and responds in Ext-compliant JSON. This could be very ideal for a full-fledged Ext-Based interface.

Another point to be taken care of while developing this would be to make it re-usable, generic and easy to customize. For eg. An update-data popup for entries listed in a table could be used where-ever update kind of operation needs to be done, and so on.

Future Options -
- Adding usability support for browsers that don't have javascript support enabled. This could be either done by making the interface fall back to web2py crud or having an alternate non-js interface rendered by s3xrc
Relevant Experience -
- I have contributed earlier to SahanaPy - https://launchpad.net/sahana/+topcontributors
- Some of the SahanaPy related work I did -
  - creating the current steel theme, - gives me a good understanding of js, css and views used in sahanapy.
  - improving the main and sub menu (created css based crossbrowser compatible menus),
  - javascript based popup for coordinate conversion and, - I've used ExtJS here for coordinate conversion popup.
  - adding custom error pages - involved tweaking routes.py of web2py framework
  - ( http://is.gd/bj82z http://is.gd/bj86d http://is.gd/bj87m )
- I have worked with MVC architecture in the past ( CodeIgniter, ASP.NET MVC) and even while working for SahanaPy. Recently started learning Web2Py - http://hello-poll.appspot.com/polls
- I am quite comfortable with Python, jQuery, writing good HTML & CSS and have basic idea of ExtJS.

Other references - http://trac.sahanapy.org/wiki/S3XRC http://is.gd/bj8Tn

5. Project Goals and Timeline

Work already undertaken -
- Reading more about S3XRC, S3REST and drafts of S3XRC_2
- Familiarize with web2py
First trimester (26 April - 23 May) - Community bonding
- Low activity till 13th May due to college exams.
- As S3XRC work is also related to ucore, I would spend good time before I start coding in learning it more indepth.
- Creating a list of all things that could be made generic with S3XRC frontend. Right now I'm aware of tables and forms, yet inside forms we could provide a generic way to make them interactive as seen in a work by Michael - http://58.137.55.83/adpcdemo/drrpp/project/create
- Researching some more possible generic widgets that could be implemented in S3XRC frontend and reused later.
- Collecting feedbacks from developors as to what kind of tasks are restricted due to current web2py crud.
Second trimester (24 May - 12 July) - Coding starts
- Basic XRCTABLE 24th May to June 13th
- ExtJS interface to XRCTABLE to provide on the spot update/add feature - June 14th to June 27th
- XRCFORM as a replacement to existing SQLFORM - June 28th to July 12th
Third trimester (13 July - 9 August) -
- ExtJS interface for XRCFORM elements to reduce number of requests, and to provide inline edits, assistive popups, etc - July 13th to July 20th
- Weekly implementation of widgets and generic elements decided in First Trimester - July 21st to Aug 7th
- Rigorous testing and bug fixes - Aug 7th to Aug 9th

PyKata - Bulkuploader Patch

2010-04-04T15:58:00.000+05:30

GSoC 2010 - PyKata - Proposal

2010-04-01T05:30:00.020+05:30

Taking PyKata to another Level

developing a useful supplement to Python in education

Contact Information

Name - Abhishek Mishra
Email Address - ideamonk@gmail.com
Phone Number - +91-9535009187
IRC nickname - ideamonk (irc.freenode.net)
Twitter - http://twitter.com/ideamonk
Skype username - ideamonk
Webpage/Blog - http://blog.ideamonk.in
College/University - Amrita School of Engineering, Bangalore
Major - B.Tech in Computer Science

Project Details

Project Description - Help develop PyKata - the website for Python practice (http://pykata.appspot.com/vision) We want this to be the prime site serving students, teachers, and professionals wanting to learn Python via self-study.
Mentor - Andy Harrington and David MacQuigg
Key Technologies - Python, AppEngine, Web2Py/Django

What is PyKata?

PyKata is a community-authored website with contributions from teachers and students in math and science, as well as technical professionals. It provides you a way to quickly write programs on the web, get immediate feedback on errors, and work through a set of exercises designed by the best math and science teachers all over the world. For teachers, who may not be programmers themselves, PyKata? offers a way to integrate simple programminmg into their lesson plans, without distracting from the subject being taught.

http://en.wikipedia.org/wiki/Kata
Website - http://pykata.appspot.com/
Vision - http://pykata.appspot.com/vision/
Code - http://code.google.com/p/pykata/ , currently based on webapp+django
PyKata uses similar doctest environment as seen in Crunchy.

What features are provided currently?

1. It lets you write, execute and automatically evaluate your code online.
2. Saves your work-in-progress.
3. Shows your overall progress as to problems solved.
4. Lets community contribute problems.
5. Static help

Why should PyKata be developed further under GSoC '10?

Python has been a liberating force for both programmers and for scientists and engineers who don't have time for traditional programming. It has a also added some fun and playfullness to programming.
I believe Python is the best when it comes to teaching programming at school and college levels. http://is.gd/aYO08 and personally while discussing python with some friends, I've noticed a fair amount of excitement they've shown towards Python.
PyKata has great potential of making these ideas popular and assisting in quick learning for say, college students who wish to do quick simulations, or say, new engineers on the team who do not wish to spend excessive amounts of time on learning to write programs.

What features am I planning to implement in PyKata?

I am very excited to see a Python+AppEngine task on PSF's Summer of Code Page. Here are some of the ideas that I've come up with after discussions on the PyWhip mailing list.

Porting to Web2Py
After some experiments and discussions over choice of framework, it seems that web2py is a suitable framework for PyKata development. Django happens to have steep learning curve for new comers who would contribute to PyKata in future. To keep the friction in entry of new developers low, Web2Py seems like an appropriate framework for quick contribution and RAD development.
Though it is said that web2py follows some anti-patterns, but then "Practicality beats Purity" at times. Also deploying a web2py app on AppEngine is almost effortless.
Porting Python 3
Right now, Google App Engine uses Python 2.5.2. But there are two interesting things related here -
- opposed to the propaganda on #python about not using Python 3 yet (http://python-commandments.org/python3.html), the Python community is more interested in porting existing packages to Python 3
- This discussion on web2py and Python 3 compatibility is a relief as the core developer believes that it won't take too much to port web2py to Python3.
Instead of waiting for App Engine to shift to Python 3, I would create a Python 3 compatible branch of PyKata in parallel. Once App Engine shifts to Python 3, PyKata too would get updated in no time. This would also help us give web2py developers useful feedbacks on porting to Python 3.
UI/UX improvements

PyKata could benefit from an improved design (more web2.0-ish). This would not only make it attractive but also help us present the progress data, user profiles, live statistics, tutorials, news, etc in a better way as it would encourage more social interaction upon growing. A rough mock-up - http://twitpic.com/1aux8n/full

I aim to create an addictive and interactive user experience. Using game style interaction to drive participation, possibly include points, badges, rewards a great example for that would be http://stackoverflow.com

Creating a flow - once a student gets a problem right, PyKata would show her which way to proceed from there.

If a student goes wrong repetitevely, PyKata could also show him/her possible tutorials related to it. http://twitpic.com/1auytp/full

Some examples - http://codingbat.com http://spoj.pl http://codegolf.com/
User Profiles
PyKata can provide each user his/her own personal profile to track progress. I also plan to add two levels of views to these profiles - a personal view, a public view. Someone whome the student designates as a teacher will have full access to the student's data. Currently the only way to track progress is to either look at ticks on 'All problems page' or to do the same on category pages. A mockup for this feature - http://twitpic.com/1av0x0/full

Another important detail that a user profile should expose, when a teacher views a student's profile would be the areas where one is weak at. This could be found out by monitoring number of times a student fails in a problem or a class of problems. While the public view would not show any such details.
Live execution results
Currently PyKata takes you to another page to show results of current code execution. Then the user hits back button to modify the code, or copy-pastes the traceback for reference and hits back to debug the code. This user experience could be simplified by placing a live result box on right side of the editor. As shown here - http://twitpic.com/1auytp/full
Content Management

PyKata relies heavily on user submitted content. As for content submissions, all new submissions would get reviewed by PyKata editors and then moved to appropriate categories. If that doesn't happen, then a problem remains in 'new' category for the time it accumulates favourable votes.
As per current plan, only a PyKata contributor would be able to modify the problem/content they submitted. Another group of users - PyKata editors would have the rights to modify problems and content for the purpose of moderation.
When a user proposes improvements to a problem, the author of the problem would have the right to decide whether to accept it or not.
As per modifications to problems, the system would preserve previous version and show the history of the problem. This would be useful for Task 9 - Course planning feature.
Batch upload problems/ solution-uploads
A contributor of problems/tutorials might have many contents already written in say, a python script itself. Copy pasting from them and uploading one by one might not be a great user experience. I would simplify this task by allowing flash/javascript based bulk uploads (to select multiple files). A server-side validation and processing would be provided to such uploads which would process them according to rules specified on PyKata Project Wiki - http://code.google.com/p/pykata/wiki/UploadFormat
Good news is that I've already written an experimental bulk uploader for PyKata ( http://code.google.com/p/pykata/source/detail?r=36 ), extending that to above mentioned feature should be easy :)
Export/Import of Data
This involves creating a way for people to run a local copy of PyKata on their infrastructure. As such it would be nice for them to be able to export and import problems/content from main PyKata website to their local copies.
Here the batch upload code can be easily reused as PyKata already specifies an upload format. This task would involve generating uploadable file in PyKata specified format. The user then can take it to any other instance of PyKata and merge the updates.
* Course planning feature
It is expected that PyKata once launched and contributed to, would become a large repository of Python learning problems. It could also include support for classes in Maths, Science and Engineering.
We could do something interesting here by allowing teachers to create lessons by handpicking existing problems and tutorials and making a definite learning track or a roadmap out of them. This would also reduce the efforts that go on a teachers behalf in creating tutorials.
A problem editor would mark an edit with "major change" for the purpose of maintaining a history of a problem. This is useful in maintaining consistency in already created learning traks prior to modification.
* PyLint / Pyflakes integration
Though not sure if it would be possible to run them over AppEngine, I would experiment with it and try to integrate it as well to PyKata. If this feature gets added, PyKata would be able to pinpoint indentation and other minor errors immediately. Crunchy's main developer Andre Roberge suggests that this could be done in a similar way as seen in Crunchy.
* Community Ratings

This involves creating a way for the community to rate contents - including users, problems, tutorials. This would help us gather good statistical data for further analysis. (this is a non priority task)
* Making PyKata more Social
Though not at all a priority task, and not even a tough thing to do. At a later finishing stage, it would be nice to have integration to social networks like Facebook, twitter, etc to gain more popularity. Though not at all a concern right now.
A good way to encourage participation, as suggested by Andrew, would be to expose different users. Different user types - learners, one-by-one contributors, experts, content contributors, admins, developers.
I believe a 'karma ranking' (as seen on Launchpad, Hacker News, Reddit, etc) or community rating approach to classify users would be great. This would encourage already contributing members to contribute more.
Problems can also be rated in this manner - "XX out of YYY people found this tutorial useful"
Documentation
Though not a separate task at all, documentation will be written in parallel with code development. This will be sufficient that a student not familiar with web development will be able to take over as maintainer of PyKata. One of our mentors, Dr. MacQuigg, is experienced in documentation of commercial software, and he will assist with this effort, so that our "lead developer" will not be slowed by documentation problems in future.

Future ideas - a canvas based whiteboard tool for teachers to create better tutorials,

Having mentioned everything that came to my mind for PyKata's improvement, I would like to point out that not every feature might be possible to be implemented within the time frame. Hence I've kept feature 9, 10, 11, 12 marked with a (*) as tentative or post-gsoc plans for PyKata if they don't fit in.
** Feature 9 would possible to implement once all other feaures get completed as it would require all other features to be functional.

Personal Availability

I have reviewed the major dates of GSoC
Apart from my Institutes end-semester exams that fall from April 30th to May 13th, I have no other conflicts with the schedule.
I would be available for the entire duration of GSoC 2010

Major Milestones (weekly plans)

Community Bonding Period ( April 26 - May 23 )

April 30th - May 13th - Low action due to exams

May 13th - May 23rd - Start studying existing codebase, old discussions for getting as familiar as possible with the project. Getting familiar with different aspects of web2py and AppEngine with possible experiments.
First Semester ( May 24th - July 11th )

May 24th - May 30th - Task1 - Porting existing codebase to web2py.
May 31st - June 6 - Task4 - User Profiles
June 7 - June 13th - Task5 - Live execution results (AJAX based)
June 14th - June 27th - Task6 - Content Management
June 28st - July 4th - Task 7 - Batch Uploads
July 5th - July 11th - Task3 - UI/UX Improvements (partial) and testing+review for mid term submission
Second Semester ( July 16th - August 16th )

July 16th - July25th - Task 8 - Export Import of data
July 26th - Aug 8th - Task9 - Course Planning feature
August 9 - August 16th - Task3 (completion), Code Review, documentation wherever necessary, testing.

* Each week is a 40 hour work week.
* Since my GSoC work would bring in a lot of changes to PyKata, I will continue as a committer and will support and evangelize PyKata.

Source Code Repository and Communication Plan

I would be working on SVN repository of PyKata on Google Code - http://code.google.com/p/pykata/source/browse/ and I already have a committor access there. Most probably I would push my changes to test branch which could be merged to trunk after a simple review/test.

I plant to communicate quickly as for any doubts and discussions coming in between the week. In the end of every week, I plan to submit a report to my mentor. As per my schedule, I'm mostly implementing 1 feature per week, with an exception of two weeks to Content Management feature and Course planning feature. So a weekly report would be well suited.

Why me?

I have been programming since my school days QuickBASIC, mostly writing games http://blog.ideamonk.in/search/label/QBASIC
I do programming puzzles on SPOJ sometimes. Initially in C/C++ now in Python - http://www.spoj.pl/status/ideamonk/signedlist/
I believe in giving back to the community and wrote SPOJ Rank Tracker addon for Firefox and SpojBackup tool in Python.
During my first year at college I wrote a web based programming contest platform along with a friend - CodeFire (PHP/DevC++)
Some Python related work I've done so far -
- PyPaste http://blog.ideamonk.in/2009/08/pypaste-pastebins-at-your-terminal.html
- I felt the need of a twitter panel applet for gnome and learnt some PyGTK to create it - http://blog.ideamonk.in/2009/11/introducing-gtweetbar-alpha.html
- Interested into making webapps, I also tried Google AppEngine and created a small app called Web2Hunter -
  http://blog.ideamonk.in/2009/09/web2hunter-find-that-awesome-domain.html
  http://www.youtube.com/watch?v=XyURw35v2u8 (a short intro to it at PyCon India 09)
- I wanted to create photo-mosaics one day, I wrote pymos for it along with a friend who helped me modularize and package it - http://blog.ideamonk.in/2009/11/pymos-creating-mosaics-of-your-photos.html I used it to create a poster for FOSS.in 2009 http://foss.in/promote/posters , that really made me happy as I wrote something useful.
- I felt the need to receive sms alerts for new incoming mails as I do not own a gprs phone. I came up with an OAuth based solution - http://lenny.in which runs on AppEngine and uses twitter to send alerts.
- Another AppEngine project I did and am yet to add finishing touches is Poorman's datastore - http://github.com/ideamonk/poorman It lets one use multiple free appengine instances as a unified, distributed datastore.
- A fun project exploring WebSockets with mod_pywebsocket, pyAIML, CSS3, redis - http://ideamonk.in/bakabot/

Some relevant OpenSource Experience -
- Participated in Sahanapy workout at FOSS.in http://trac.sahanapy.org/wiki/WorkOutFossIn2009 (ref: ideamonk) and contributed more to it https://launchpad.net/sahana/+topcontributors , which mostly involved font-end work, the current theme, nice error pages, etc.
- I am writing a PyQt based gui for apt-offline - an offline APT based package manager for debian based distros - as a hobby project http://github.com/ideamonk/apt-offline
  http://packages.debian.org/unstable/apt-offline
My github page - http://github.com/ideamonk
My Blog - http://blog.ideamonk.in
I started http://madetokill.com/ with friends to fetch us some work in holidays. I learnt a good deal of PHP, HTML, CSS, jQuery and a bit of Photoshop from this experience.
Recently I also got committer access to PyKata, and have started exploring it more.

As you can clearly observe I've been doing odd small things here and there, Google Summer of Code and PyKata really brings me an opportunity where I can create, give support to, be responsible for something that a lot of people would use and be proud about it.

Python is bound to play a big role in education, rapid prototyping and what not. At PyCon India 09, I was really happy at the announcement of FOSSEE (Free Open source Software for Science and Engineering Education) group about introducing Python to engineering colleges in India. And now I'm delighted at the fact that I too might get a chance to contribute to Python Education in a big way!

Thanks!

Abhishek

We <3 MadeToKill

2010-03-20T21:26:00.000+05:30

2+ years and still counting . . .

Quine attempt :)

2010-03-20T19:17:00.002+05:30

Just tried writing a quine in python, I'm ashamed that it took me too much time to find a solution, so the hide the shame I covered my quine in an ascii art -


#!/usr/bin/env python
# -*- coding: utf-8 -*-
if __name__ == '__main__':
 """
  'adppppba,                 pp
 dp"      `"pb                ""
dp`        `pb
pp          pp  pp       pp  pp  pb,dPPYba,    ,adPPYba,
pp          pp  pp       pp  pp  ppP`   `"pa  apP_____pp
Yp,    "pp,,pP  pp       pp  pp  pp       pp  pPP'""'"""
 print open(__file__).read();"""pp       pp  "pb,   ,aa
  `"YppppY"Ypa   `"YbbdP'Yp  pp  pp       pp   '"Ybbdp'
  """

Another approach with sys.exit() -

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import sys
if __name__=='__main__':
    print open(__file__).read()
    sys.exit(0)

    for x in xrange(1,10):
        for y in xrange(x):
            print chr(y),
        print

    # ^^ this one is just a cover up, actual quine ends at exit

and yet another approach with muting the stdout :P -

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import sys

class Mute:
  def write(self,msg):
    ''' I am a quine alright '''
  def flush(self):
    ''' so am i :P '''

if __name__=='__main__':
  print open(__file__).read()
  sys.stdout, sys.stderr = Mute(),Mute()
  for x in xrange(1,10):
    for y in xrange(x):
      print chr(y),
    print
    
  foozooooo < iz_this_an_error

  if now_this_too:
    if you_too:
      if santa:
        if you:
          if me:
            if no_one_at_all:
             '  /(       )\
                \ \__   / /\
                /- _ `-/  / \
               (/\/ \ \    \
               / /   | `    \
               O O   )      |\
               `-^--``<    /\
              (_.)  _ )   / \
               `.___/`   |\
                 `-----` \
    <----.     __ / __   \
    <----|====O)))==) \) \
    <----`    `--` `.__,` \
                 |         |\
                  \       /\
              ____( (_   / \_____\
            ,`  ,----`   |        \
            `--{__________)       \/'

Fun with PyGame's camera module

2010-03-16T13:00:00.006+05:30

OMG T2 starts tomorrow, I'm probably gonna fail in Random Processes this time.
But before I start preparing, I had to try out the tempting Camera Module of PyGame.
PyGame has pretty cool stuff under pygame.transform and pygame.mask, which makes the task of thresholding very easy.
I came up with an interactive Tux based on the "Capturing a Live Stream" code in the Camera Module Introduction.

My program tries to detect red colored objects, finds the centroid of such points and draws a ghost (find in your /usr/share/icons/oxygen/32x32/apps/) and makes Tux (/usr/share/icons/oxygen/128x128/apps/tux.png) run away from it.

Here's how the code looks like -


# -*- coding: utf-8 -*-

# An interactive Tux, interact with it using any red colored object
#
# based on PyGame camera module intro by Nirav Patel
#   http://www.pygame.org/docs/tut/camera/CameraIntro.html
#                         -- Abhishek Mishra (ideamonk # gmail.com)

import os
import pygame
import pygame.camera
from pygame.locals import *

class Capture(object):
    ''' A Capture class to get location of a desired blob '''

  def __init__(self, ccolor=(248, 111, 115), threshold=(60, 10, 10)):
      self.size = (640,480)
      # create a display surface. standard pygame stuff
      self.display = pygame.display.set_mode(self.size, 0)
      # initialize camera module
      pygame.camera.init()
      # this is the same as what we saw before
      self.clist = pygame.camera.list_cameras()
      if not self.clist:
          raise ValueError("Sorry, no cameras detected.")
      self.cam = pygame.camera.Camera(self.clist[0], self.size)
      self.cam.start()

      # create a surface to capture to.  for performance purposes
      # bit depth is the same as that of the display surface.
      self.snapshot = pygame.surface.Surface(self.size, 0, self.display)
      # target color to detect -- default is red
      self.ccolor = ccolor
      # by default we give more priority to shades of red
      self.threshold = threshold

  def get_blob_location(self):
      self.snapshot = self.cam.get_image(self.snapshot)
      # threshold against the color we got before
      mask = pygame.mask.from_threshold(self.snapshot, self.ccolor, self.threshold)
      # keep only the largest blob of that color
      connected = mask.connected_component()
      # these numbers are purely experimental and specific to your room and object
      # print mask.count() # use this to estimate
      # make sure the blob is big enough that it isn't just noise
      if mask.count() > 7:
          # find the center of the blob
          return mask.centroid()
      return (None,None)

class Ghost():
    ''' Ghost class, to have a rect for collision detection '''
  def __init__(self):
      self.image = pygame.image.load (os.path.join ("./","gv.png"))
      self.rect = self.image.get_rect()

  def set_rect(self,position):
      self.left,self.top=position
      self.rect = pygame.Rect(self.left,self.top,32,32)
    
class Tux(Capture):
    ''' Tux class extends Capture and does stuff using get_blob_location '''

  def __init__(self):
      Capture.__init__(self)
      self.location = [ x/2 for x in self.size ]
      self.set_rect (self.location)
      self.image = pygame.image.load (os.path.join ("./","tux.png"))
      self.ghost = Ghost()
      self.backbuffer = pygame.Surface (self.size)
      self.force = 5

  def set_rect(self,position):
      left,top=position
      self.rect = pygame.Rect(left,top,128,128)
    
  def interact_tux(self):
      if (pygame.sprite.collide_rect(self,self.ghost)):
          # ghost collides with tux
          if self.ghost.left<self.location[0]+64:
              self.location[0] += self.force
          if self.ghost.left>self.location[0]+64:
              self.location[0] -= self.force
          if self.ghost.top<self.location[1]+64:
              self.location[1] += self.force
          if self.ghost.top>self.location[1]+64:
              self.location[1] -= self.force
            
        
  def main(self):
      going = True
      old_coord = (0,0)
    
      while going:
          events = pygame.event.get()
          for e in events:
              if e.type == QUIT or (e.type == KEYDOWN and e.key == K_ESCAPE):
                  # close the camera safely
                  self.cam.stop()
                  going = False

          new_coord = self.get_blob_location()
          if new_coord != (None,None):
              # delta = sum( [(x-y)**2 for (x,y) in zip(new_coord,old_coord)])    # for less fuzziness
              # if delta>200:
              old_coord = new_coord
              self.ghost.set_rect(old_coord)
            
          self.set_rect (self.location)
          self.interact_tux()
          self.backbuffer.blit(self.snapshot,(0,0))
          self.backbuffer.blit(self.image,self.location)
          self.backbuffer.blit(self.ghost.image, old_coord)
          self.backbuffer = pygame.transform.flip(self.backbuffer,True,False)
          self.display.blit(self.backbuffer,(0,0))
          pygame.display.flip()
        
if __name__=='__main__':
  t = Tux()   # all default params
  t.main()

And that's me doing weird things with tux :D -

Kalman Filter would be more interesting.

Better than stealing...

2010-02-21T01:17:00.006+05:30

Remember way back when your were that kid whose only tools of trade in teh bag o' tricks were pencil and crayons and stuff like that, ah the small bricks of water colors, the cheap blue color box or color bricks where you'd brush the watery brush and ... blah ends here.
So, yes, better than even copy pasting, not caring about copyrights, etc why not trace the stuff you like as in the old school days :). After all there's no harm/copyright violation in getting inspired from an artform. So for lenny.in (crap this is 3rd day in making and its not yet out)... I needed some logos, some cool graphics for gmail, twitter and Lenny itself. Here's how I got it done -

1. Load original image/photo into ArtRage or MyPaint (on linux)
2. Start drawing... draw ... draw... draw some more
3. Resize into iconic/useful proportion, and smile.

Lol I might just have mislead you... but hey this obviously sounds like a safer practice than outright copy pasting media content from here and there.

Anyways so this is what I could get out of it (zoom to see brush strokes) -

and finally ->