<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" media="screen" href="/~d/styles/rss2full.xsl"?><?xml-stylesheet type="text/css" media="screen" href="http://feeds.feedburner.com/~d/styles/itemcontent.css"?><rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:wfw="http://wellformedweb.org/CommentAPI/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:sy="http://purl.org/rss/1.0/modules/syndication/" xmlns:slash="http://purl.org/rss/1.0/modules/slash/" xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#" xmlns:creativeCommons="http://backend.userland.com/creativeCommonsRssModule" version="2.0">

<channel>
	<title>xion.log</title>
	
	<link>http://xion.org.pl</link>
	<description>Programming on (many) different levels of abstraction</description>
	<lastBuildDate>Thu, 10 May 2012 18:44:29 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.3.2</generator>
		<atom10:link xmlns:atom10="http://www.w3.org/2005/Atom" rel="self" type="application/rss+xml" href="http://feeds.feedburner.com/xion" /><feedburner:info xmlns:feedburner="http://rssnamespace.org/feedburner/ext/1.0" uri="xion" /><atom10:link xmlns:atom10="http://www.w3.org/2005/Atom" rel="hub" href="http://pubsubhubbub.appspot.com/" /><geo:lat>53.25</geo:lat><geo:long>21</geo:long><creativeCommons:license>http://creativecommons.org/licenses/by/2.0/</creativeCommons:license><xhtml:meta xmlns:xhtml="http://www.w3.org/1999/xhtml" name="robots" content="noindex" /><image><link>http://xion.org.pl</link><url>http://xion.org.pl/wp-content/themes/xion.log/images/rss-logo.jpg</url><title>xion.log</title></image><item>
		<title>DreamPie with virtualenv</title>
		<link>http://xion.org.pl/2012/05/10/dreampie-with-virtualenv/</link>
		<comments>http://xion.org.pl/2012/05/10/dreampie-with-virtualenv/#comments</comments>
		<pubDate>Thu, 10 May 2012 18:44:29 +0000</pubDate>
		<dc:creator>Xion</dc:creator>
				<category><![CDATA[Applications]]></category>
		<category><![CDATA[Programming]]></category>
		<category><![CDATA[bash]]></category>
		<category><![CDATA[DreamPie]]></category>
		<category><![CDATA[Linux]]></category>
		<category><![CDATA[Python]]></category>
		<category><![CDATA[virtualenv]]></category>

		<guid isPermaLink="false">http://xion.org.pl/?p=3556</guid>
		<description><![CDATA[If you haven't heard about it, DreamPie is an awesome GUI application layered on top of standard Python shell. I use it for elaborate prototyping where its multi-line input box is a significant advance over raw, terminal UX of IPython. However, up until recently I didn't know how to make DreamPie cooperate with virtualenv. Because [...]]]></description>
			<content:encoded><![CDATA[<p>If you haven't heard about it, <a href="http://dreampie.sourceforge.net/">DreamPie</a> is an awesome GUI application layered on top of standard Python shell. I use it for elaborate prototyping where its multi-line input box is a significant advance over raw, terminal UX of <a href="http://ipython.org/">IPython</a>.</p>
<p><img style="float:right; margin: 7px" src="http://xion.org.pl/wp-content/uploads/2012/05/dreampie-cloud.png" alt="" />However, up until recently I didn't know how to make DreamPie cooperate with <a href="http://www.virtualenv.org/">virtualenv</a>. Because it's a GUI program, I scoured its menu and all the preference windows, searching for any trace of option that would allow me to set the Python executable. Having failed, I was convinced that authors didn't think about including it - which was rather surprising, though.</p>
<p>But hey, DreamPie is open source! So I went to look around its <a href="http://bazaar.launchpad.net/~dreampie-devel/dreampie/trunk/files">code</a> to see whether I can easily enhance it with an ability to specify Python binary. It wasn't too long before I <a href="http://bazaar.launchpad.net/~dreampie-devel/dreampie/trunk/view/head:/dreampielib/gui/__init__.py">stumbled</a> into this vital fragment:</p>
<div class="syntax_hilite">
<div id="python-4">
<div class="python"><span style="color: #ff7700;font-weight:bold;">def</span> main<span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>:<br />
&nbsp; &nbsp; usage = <span style="color: #483d8b;">"%prog [options] [python-executable]"</span><br />
&nbsp; &nbsp; version = <span style="color: #483d8b;">'DreamPie %s'</span> % __version__<br />
&nbsp; &nbsp; <span style="color: #dc143c;">parser</span> = OptionParser<span style="color: black;">&#40;</span>usage=usage, version=version<span style="color: black;">&#41;</span><br />
&nbsp; &nbsp; <span style="color: #808080; font-style: italic;"># ...</span><br />
&nbsp; &nbsp; opts, args = <span style="color: #dc143c;">parser</span>.<span style="color: black;">parse_args</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span></div>
</div>
</div>
<p>
The conclusions we could draw from this anecdote are thereby as follows:</p>
<ul>
<li>It is indeed true that source code is often the best documentation...</li>
<li>...especially for open source programs where <em>actual</em> docs often suck.</li>
</ul>
<p>With this newfound knowledge about <kbd>dreampie</kbd> arguments, it wasn't very hard to make it use current virtualenv:</p>
<div class="syntax_hilite">
<div id="code-5">
<div class="code">$ dreampie $<span style="color:#006600; font-weight:bold;">&#40;</span>which python<span style="color:#006600; font-weight:bold;">&#41;</span></div>
</div>
</div>
<p>
And after doing <a href="http://superuser.com/questions/177218/how-to-start-gui-linux-programs-from-the-command-line-but-separate-from-the-com">some</a> more <a href="http://stackoverflow.com/questions/876239/bash-redirect-and-append-both-stdout-and-stderr">research</a>, I ended up adding the following line to my <em>~/.bash_aliases</em>:</p>
<div class="syntax_hilite">
<div id="code-6">
<div class="code">alias dp=<span style="color:#CC0000;">'(dreampie $(which python) &amp;&gt;/dev/null &amp;)'</span></div>
</div>
</div>
<p>
Now I can simply type <kbd>dp</kbd> to get a DreamPie instance operating within current virtualenv but independent from terminal session. Very useful!</p>
<img src="http://feeds.feedburner.com/~r/xion/~4/AXHxvol6KUU" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://xion.org.pl/2012/05/10/dreampie-with-virtualenv/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Hacking Python Imports</title>
		<link>http://xion.org.pl/2012/05/06/hacking-python-imports/</link>
		<comments>http://xion.org.pl/2012/05/06/hacking-python-imports/#comments</comments>
		<pubDate>Sun, 06 May 2012 17:05:09 +0000</pubDate>
		<dc:creator>Xion</dc:creator>
				<category><![CDATA[Programming]]></category>
		<category><![CDATA[import]]></category>
		<category><![CDATA[modules]]></category>
		<category><![CDATA[packages]]></category>
		<category><![CDATA[Python]]></category>
		<category><![CDATA[pytz]]></category>

		<guid isPermaLink="false">http://xion.org.pl/?p=3535</guid>
		<description><![CDATA[...for fun and profit! I'm still kind of amazed of how malleable the Python language is. It's no small feat to allow for messing with classes before they are created but it turns out to be pretty commonplace now. My latest frontier of pythonic hackery is import hooks and today I'd like to write something [...]]]></description>
			<content:encoded><![CDATA[<p><small>...for fun and profit!</small></p>
<p>I'm still kind of amazed of how malleable the Python language is. It's no small feat to allow for messing with classes <a href="http://www.voidspace.org.uk/python/articles/metaclasses.shtml">before they are created</a> but it turns out to be pretty commonplace now. My latest frontier of pythonic hackery is <strong>import hooks</strong> and today I'd like to write something about them. I believe this will come handy for at least a few pythonistas because the topic seems to be rather scarcely covered on the 'net.</p>
<h3>Importing: a simplistic view</h3>
<p>As you can easily deduce, the name 'import hook' indicates something related to Python's mechanism of <strong>imports</strong>. More specifically, import hooks are about injecting our custom logic directly into Python's importing routines. Before delving into details, though, let's revise how the imports are being handled by default.</p>
<p>As far as we are concerned, the process seems to be pretty simple. When the Python interpreter encounters an <code>import</code> statement, it looks up the list of directories stored inside <code>sys.path</code>. This list is populated at startup and usually contains entries inserted by external libraries or the operating system, as well as some standard directories (e.g. <em>dist-packages</em>). These directories are searched in order and in greedy fashion: if one of them contains the desired package/module, it's picked immediately and the whole process stops right there.</p>
<p>Should we run out of places to look, an <code>ImportError</code> is raised. Because this is an exception we can catch, it's possible to try multiple imports before giving up:</p>
<div class="syntax_hilite">
<div id="python-15">
<div class="python"><span style="color: #ff7700;font-weight:bold;">try</span>:<br />
&nbsp; &nbsp; <span style="color: #808080; font-style: italic;"># Python 2.7 and 3.x</span><br />
&nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">import</span> json<br />
<span style="color: #ff7700;font-weight:bold;">except</span> <span style="color: #008000;">ImportError</span>:<br />
&nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">try</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; <span style="color: #808080; font-style: italic;"># Python 2.6 and below</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">import</span> simplejson as json<br />
&nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">except</span> <span style="color: #008000;">ImportError</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">try</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;<span style="color: #808080; font-style: italic;"># some older versions of Django have this</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;<span style="color: #ff7700;font-weight:bold;">from</span> django.<span style="color: black;">utils</span> <span style="color: #ff7700;font-weight:bold;">import</span> simplejson as json<br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;<span style="color: #ff7700;font-weight:bold;">except</span> <span style="color: #008000;">ImportError</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;<span style="color: #ff7700;font-weight:bold;">raise</span> <span style="color: #008000;">Exception</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">"MyAwesomeLibrary requires a JSON package!"</span><span style="color: black;">&#41;</span></div>
</div>
</div>
<p>
While this is extremely ugly boilerplate, it serves to greatly increase portability of our application or package. Fortunately, there is only handful of worthwhile libraries that we may need to handle this way; <code>json</code> is the most prominent example.</p>
<h3>More details: about <code>__path__</code></h3>
<p>What I presented above as Python's import flow is sufficient as description for most purposes but far from being complete. It omits few crucial places where we can tweak things to our needs.</p>
<p>First is the <strong><code>__path__</code> attribute</strong> which can be defined in package's <em>__init__.py</em> file. You can think of it as a <em>local extension</em> to <code>sys.path</code> list that only works for submodules of this particular package. In other words, it contains directories that should be searched when a package's submodule is being imported. By default it only has the <em>__init__.py</em>'s directory but it can be extended to contain different paths as well.</p>
<p>A typical use case here is splitting single "logical" package between several "physical" packages, distributed separately - typically as different PyPI packets. For example, let's say we have <code>foo</code> package with <code>foo.server</code> and <code>foo.client</code> as subpackages. They are registered in PyPI as separate distributions (<em>foo-server</em> and <em>foo-client</em>, for instance) and user can have any or both of them installed at the same time. For this setup to work correctly, we need to modify <code>foo.__path__</code> so that it may point to <code>foo.server</code>'s directory and <code>foo.client</code>'s directory, depending on whether they are present or not. While this task sounds exceedingly complex, it is actually very easy thanks to the standard <code>pkgutil</code> module. All we need to do is to put the following two lines into <em>foo/__init__.py</em> file:</p>
<div class="syntax_hilite">
<div id="python-16">
<div class="python"><span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">pkgutil</span><br />
__path__ = <span style="color: #dc143c;">pkgutil</span>.<span style="color: black;">extend_path</span><span style="color: black;">&#40;</span>__path__, __name__<span style="color: black;">&#41;</span></div>
</div>
</div>
<p>
There is much more to <code>__path__</code> manipulation than this simple trick, of course. If you are interested, I recommend reading an issue of <a href="http://www.doughellmann.com/PyMOTW/">Python Module of the Week</a> devoted <a href="http://www.doughellmann.com/PyMOTW/pkgutil/">solely to <code>pkgutil</code></a>.</p>
<h3>Actual hooks: <code>sys.meta_path</code> and <code>sys.path_hooks</code></h3>
<p>Moving on, let's focus on parts of import process that let you do the <em>truly</em> amazing things. Here I'm talking stuff like pulling modules <a href="http://docs.python.org/library/zipimport.html">directly from Zip files</a> or remote repositories, or just <em>creating them dynamically</em> based on, say, WSDL description of Web services, symbols exported by DLLs, REST APIs, command line tools and their arguments... pretty much anything you can think of (and your imagination is likely better than mine). I'm also referring to "aggressive" interoperability between independent modules: when one package can adjust or expand its functionality when it detects that another one has been imported. Finally, I'm also talking about security-enhanced Python sandboxes that intercept import requests and can deny access to certain modules or alter their functionality <em>on the fly</em>.<br />
<span id="more-3535"></span><br />
All of these (and possibly much more) can be achieved through the usage of <em>import hooks</em>. There are two distinct types of them, usually referred to as <strong>meta hooks</strong> (defined in <code>sys.meta_path</code>) and <strong>path hooks</strong> (defined in <code>sys.path_hooks</code>). Although they are invoked at slightly different stages of the import flow, they are both built upon the same two concepts: that of a module <em>finder</em> and a module <em>loader</em>.</p>
<p><strong>Module finder</strong> is simply an object which implements one specific method - <code>find_module</code>:</p>
<div class="syntax_hilite">
<div id="python-17">
<div class="python">finder.<span style="color: black;">find_module</span><span style="color: black;">&#40;</span>fullname, path=<span style="color: #008000;">None</span><span style="color: black;">&#41;</span></div>
</div>
</div>
<p>
It receives a fully qualified name of the module to be imported, along with <code>path</code> where it's supposed to be found. and it is expected that the method does one of three things:</p>
<ul>
<li>Raises an exception, aborting the import process completely.</li>
<li>Returns <code>None</code>, meaning that given module cannot be found <em>by this particular finder</em>. It can still be found during the next stages of import flow, either by some other custom finder or just the standard Python import mechanism.</li>
<li>Returns a <em>loader</em> object that is capable of actually loading the module.</li>
</ul>
<p>The last case is of course the most interesting one, as it gracefully leads us to the concept of <strong>module loader</strong>. This one is an object that implements the <code>load_module</code> method:</p>
<div class="syntax_hilite">
<div id="python-18">
<div class="python">loader.<span style="color: black;">load_module</span><span style="color: black;">&#40;</span>fullname<span style="color: black;">&#41;</span></div>
</div>
</div>
<p>
Again, the <code>fullname</code> parameter is a fully qualified name of the module that we want to import. Return value should be a module <em>object</em> - a final result of the whole importing process. Note that this could be something that was already imported; for such "duplicate" imports the loader should simply return the existing module:</p>
<div class="syntax_hilite">
<div id="python-19">
<div class="python"><span style="color: #ff7700;font-weight:bold;">def</span> load_module<span style="color: black;">&#40;</span><span style="color: #008000;">self</span>, fullname<span style="color: black;">&#41;</span>:<br />
&nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">if</span> fullname <span style="color: #ff7700;font-weight:bold;">in</span> <span style="color: #dc143c;">sys</span>.<span style="color: black;">modules</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #dc143c;">sys</span>.<span style="color: black;">modules</span><span style="color: black;">&#91;</span>fullname<span style="color: black;">&#93;</span><br />
&nbsp; &nbsp; <span style="color: #808080; font-style: italic;"># otherwise, do the loading magic </span></div>
</div>
</div>
<p>
If anything goes wrong at this stage, loader should raise an exception (typically is just an <code>ImportError</code>).</p>
<h3>Writing your own importer</h3>
<p>Here's where most of the theory ends, as conveniently described in <a href="http://www.python.org/dev/peps/pep-0302/">PEP 302</a>. In practice both finder and loader can be the same object and the <code>find_module</code> method can simply <code>return self</code>. As an example, consider this simple hook which is intended to block some specific modules from being imported at all:</p>
<div class="syntax_hilite">
<div id="python-20">
<div class="python"><span style="color: #ff7700;font-weight:bold;">class</span> ImportBlocker<span style="color: black;">&#40;</span><span style="color: #008000;">object</span><span style="color: black;">&#41;</span>:<br />
&nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">def</span> <span style="color: #0000cd;">__init__</span><span style="color: black;">&#40;</span><span style="color: #008000;">self</span>, *args<span style="color: black;">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; <span style="color: #008000;">self</span>.<span style="color: black;">module_names</span> = args</p>
<p>&nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">def</span> find_module<span style="color: black;">&#40;</span><span style="color: #008000;">self</span>, fullname, path=<span style="color: #008000;">None</span><span style="color: black;">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">if</span> fullname <span style="color: #ff7700;font-weight:bold;">in</span> <span style="color: #008000;">self</span>.<span style="color: black;">module_names</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #008000;">self</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #008000;">None</span></p>
<p>&nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">def</span> load_module<span style="color: black;">&#40;</span><span style="color: #008000;">self</span>, name<span style="color: black;">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">raise</span> <span style="color: #008000;">ImportError</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">"%s is blocked and cannot be imported"</span> % name<span style="color: black;">&#41;</span></p>
<p><span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">sys</span><br />
<span style="color: #dc143c;">sys</span>.<span style="color: black;">meta_path</span> = <span style="color: black;">&#91;</span>ImportBlocker<span style="color: black;">&#40;</span><span style="color: #483d8b;">'httplib'</span><span style="color: black;">&#41;</span><span style="color: black;">&#93;</span></div>
</div>
</div>
<p>
Once installed in <code>sys.meta_path</code>, it will intercept every attempt to import a new module and check whether its name exists on our list. This applies to indirect imports as well: if we attempt to use the <a href="http://www.python-requests.org">Python Requests</a> library:</p>
<div class="syntax_hilite">
<div id="python-21">
<div class="python"><span style="color: #ff7700;font-weight:bold;">import</span> request</div>
</div>
</div>
<p>
then it will also fail, as <code>requests</code> internally uses <a href="http://urllib3.readthedocs.org"><code>urllib3</code></a>, which in turn uses the restricted <code>httplib</code> package.</p>
<p>A hook that is a total blocker doesn't seem very useful, so let's try something slightly different. Rather than refusing to import a particular module, we'll proceed normally and issue a warning instead. Such a hook can help detect when deprecated Python modules are introduced to the project:</p>
<div class="syntax_hilite">
<div id="python-22">
<div class="python"><span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">logging</span><br />
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">imp</span><br />
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">sys</span></p>
<p><span style="color: #ff7700;font-weight:bold;">class</span> WarnOnImport<span style="color: black;">&#40;</span><span style="color: #008000;">object</span><span style="color: black;">&#41;</span>:<br />
&nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">def</span> <span style="color: #0000cd;">__init__</span><span style="color: black;">&#40;</span><span style="color: #008000;">self</span>, *args<span style="color: black;">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; <span style="color: #008000;">self</span>.<span style="color: black;">module_names</span> = args</p>
<p>&nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">def</span> find_module<span style="color: black;">&#40;</span><span style="color: #008000;">self</span>, fullname, path=<span style="color: #008000;">None</span><span style="color: black;">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">if</span> fullname <span style="color: #ff7700;font-weight:bold;">in</span> <span style="color: #008000;">self</span>.<span style="color: black;">module_names</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span style="color: #008000;">self</span>.<span style="color: black;">path</span> = path<br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #008000;">self</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #008000;">None</span></p>
<p>&nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">def</span> load_module<span style="color: black;">&#40;</span><span style="color: #008000;">self</span>, name<span style="color: black;">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">if</span> name <span style="color: #ff7700;font-weight:bold;">in</span> <span style="color: #dc143c;">sys</span>.<span style="color: black;">modules</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #dc143c;">sys</span>.<span style="color: black;">modules</span><span style="color: black;">&#91;</span>name<span style="color: black;">&#93;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; module_info = <span style="color: #dc143c;">imp</span>.<span style="color: black;">find_module</span><span style="color: black;">&#40;</span>name, <span style="color: #008000;">self</span>.<span style="color: black;">path</span><span style="color: black;">&#41;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; module = <span style="color: #dc143c;">imp</span>.<span style="color: black;">load_module</span><span style="color: black;">&#40;</span>name, *module_info<span style="color: black;">&#41;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; <span style="color: #dc143c;">sys</span>.<span style="color: black;">modules</span><span style="color: black;">&#91;</span>name<span style="color: black;">&#93;</span> = module</p>
<p>&nbsp; &nbsp; &nbsp; &nbsp; <span style="color: #dc143c;">logging</span>.<span style="color: #008000;">warning</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">"Imported deprecated module %s"</span>, name<span style="color: black;">&#41;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">return</span> module</p>
<p><span style="color: #dc143c;">sys</span>.<span style="color: black;">meta_path</span> = <span style="color: black;">&#91;</span>WarnOnImport<span style="color: black;">&#40;</span><span style="color: #483d8b;">'getopt'</span>, <span style="color: #483d8b;">'optparse'</span>, <span style="color: #808080; font-style: italic;"># etc.</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;<span style="color: black;">&#41;</span><span style="color: black;">&#93;</span></div>
</div>
</div>
<p>
In order to access the normal importing mechanism, we can use the <a href="http://docs.python.org/library/imp.html"><code>imp</code> package</a>. Its functions <code>find_module</code> and <code>load_module</code> are roughly the equivalents of our import hook's methods with the same names. But <code>imp</code> offers much more, as it also contains functions capable of creating modules from various inputs (e.g. <code>load_source</code>, <code>load_compiled</code>) or even creating them completely from scratch (<code>new_module</code>). </p>
<h3>What gives?</h3>
<p>While this all is surely very interesting, we may doubt whether import hooks actually have any notable applications at all. There is surely potential for some really impressive things, including importing Python modules straight from <a href="http://urlimport.codeshift.net/">remote URLs</a> (security concerns notwithstanding). In my case, though, I had an actual need that import hooks seemed to satisfy best.</p>
<p>There is this great <a href="http://pytz.sourceforge.net/"><code>pytz</code> package</a> for supporting date and time calculations involving timezones. In general, it is a <a href="https://twitter.com/#!/DEVOPS_BORAT/status/141342379612450816">really shaky</a> ground to thread upon, where issues related to daylight saving time are among the easier ones to deal with. For the most part, <code>pytz</code> helps navigating through the obstacles in elegant manner but there is one thing where it falls short.</p>
<p>The <a href="http://www.twinsun.com/tz/tz-link.htm">underlying timezone database</a> has apparently no notion of usable, "generic" timezones - ones based solely on offset from GMT rather than precise location, and without any DST. That's bad because generic timezones are useful in web development as temporary choice for new users, based on automatic detection through <a href="http://www.w3schools.com/jsref/jsref_gettimezoneoffset.asp">timezone offset obtained with JavaScript</a>.<br />
But the keyword here is 'usable'. As a matter of fact, <code>pytz</code> has <em>Etc/GMT+X</em> timezones. However, due to some obscure, decades-old <a href="http://en.wikipedia.org/wiki/Tz_database#Area">compatibility imperative</a> they are the <em>exact opposite</em> of what we would expect them to be: their offsets are effectively negated. It means that <em>Etc/GMT+2</em>, for example, doesn't refer to normal time in eastern Europe (or DST in center/western) but to a timezone on the other side of Prime Meridian which is almost unused except as DST for few South American countries, and Greenland. It goes without saying that this is completely and utterly insane.</p>
<p>In cases like this there are usually two solutions. You can put a thin layer with appropriate fix in front of (already perfect) library interface and make sure that no one uses the library directly - and this is of course impossible. Or you can fork it and make necessary changes - but then you'll have to maintain the fork, manually pulling changes from upstream whenever the original timezone database is updated (which is every few months). Neither is satisfying; couldn't we just use the library as it is, but somehow patch it on the fly, just before it's used?...</p>
<p>Why, of course - hello import hooks! Using a relatively simple module finder and loader, we can easily achieve the desired effect and transparently expand the <code>pytz</code> library to include more useful generic timezones. The full code can be seen in <a href="https://gist.github.com/2621334">this gist</a> and it isn't even long or complex.</p>
<h3>Further reading</h3>
<p>That would be it for today's write-up. If you want to learn more about the intricacies of Python's import hooks - such as the meaning of <code>sys.path_hooks</code>, for example - the canonical source will of course be <a href="http://www.python.org/dev/peps/pep-0302/">the appropriate PEP</a>. Beyond that, there isn't really any wealth of information to point at: some blog posts <a href="http://orestis.gr/blog/2008/12/20/python-import-hooks/">here</a> and <a href="http://blog.dowski.com/2008/07/31/customizing-the-python-import-system/">there</a>, with <a href="http://www.doughellmann.com/PyMOTW/sys/imports.html">this one</a> being probably the most useful.</p>
<img src="http://feeds.feedburner.com/~r/xion/~4/Vxsx5OBfxSM" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://xion.org.pl/2012/05/06/hacking-python-imports/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Bootstrap, a UI framework for the modern Web</title>
		<link>http://xion.org.pl/2012/04/22/bootstrap-a-ui-framework-for-the-modern-web/</link>
		<comments>http://xion.org.pl/2012/04/22/bootstrap-a-ui-framework-for-the-modern-web/#comments</comments>
		<pubDate>Sun, 22 Apr 2012 18:11:00 +0000</pubDate>
		<dc:creator>Xion</dc:creator>
				<category><![CDATA[Internet]]></category>
		<category><![CDATA[Programming]]></category>
		<category><![CDATA[Thoughts]]></category>
		<category><![CDATA[GUI]]></category>
		<category><![CDATA[standard]]></category>
		<category><![CDATA[Twitter Bootstrap]]></category>
		<category><![CDATA[user interface]]></category>
		<category><![CDATA[web apps]]></category>

		<guid isPermaLink="false">http://xion.org.pl/?p=3505</guid>
		<description><![CDATA[It was almost exactly one year ago and I remember it quite vividly. I was attending an event organized by Google which was about the Chrome Web Store, as well as HTML5 and web applications in general. After the speaker finished pitching about awesomeness of this stuff (and how Chrome was the only browser that [...]]]></description>
			<content:encoded><![CDATA[<p>It was almost exactly one year ago and I remember it quite vividly. I was attending an event organized by Google which was about the Chrome Web Store, as well as HTML5 and web applications in general. After the speaker finished pitching about awesomeness of this stuff (and how Chrome was the only browser that supported them all, of course), it was time for a round of questions and some discussion. I seized this opportunity and brought up an issue of <strong>user interface inconsistencies</strong> that plague the contemporary web apps. Because the Web as a platform doesn't really enforce any constraints on UI paradigms, we can experience all sorts of "creative" approaches to user interface. In their pursuit of novelty and eye candies, web designers often sacrifice usability by not adhering to well known interface metaphors, and shying away from uniform UI elements.</p>
<p>At that time I didn't really get a good answer that would address this issue. And it's an important one, given the rate at which web applications are springing to life and replacing their equivalent desktop programs. Does it mean we'll be stuck with this UI bonanza for the time being?...</p>
<p>Fortunately, there are some early first signs that it might not necessarily be so.</p>
<h3>Enter Bootstrap</h3>
<div style="float:right; margin-left: 4px; margin-bottom: 5px; text-align:center"><img src="http://xion.org.pl/wp-content/uploads/2012/04/twitter-logo.png" alt=""/><br/><img src="http://xion.org.pl/wp-content/uploads/2012/04/bootstrap-intro.png" alt=""/></div>
<p>Few months later, in August 2011, Twitter released the first version of <a href="http://twitter.github.com/bootstrap">Bootstrap</a> framework. Originally intended for internal use, this set of common HTML, CSS and JS patterns was made open source and released into the wild. The praise it subsequently gained is definitely well deserved, for it is a great set of tools for kickstarting development of any web-related project. Its features include:</p>
<ul>
<li>a flexible grid system for establishing a skeleton of the web page or app</li>
<li>a set of great-looking styles for HTML form elements</li>
<li>many complex UI components, like collapsible alerts, dropdowns, navigation bars, modal windows, and so on</li>
<li>customizable set of CSS styles for typical markup elements, such as headers or tables</li>
</ul>
<p>Along with universal acclaim came also the popularity: it is currently <a href="http://github.com/twitter/bootstrap">the most watched project on GitHub</a>.</p>
<h3>The value of consistency</h3>
<p>However, some want to argue that being so popular has also an unanticipated, <a href="http://drawar.com/d/great-another-bootstrap-site/">negative side</a>. It makes the developers lazy, convinced they can get away without a "proper" design for their site or app. Even more: it allegedly shows <em>disrespect</em> for their users, as if the creator simply didn't care how does their product look like.</p>
<p>Does it compute? I don't think so. Do you complain if you find that any particular <em>desktop</em> application uses the very same looks for UI components, like buttons or list boxes?... Of course not. We learned to value the consistency and predictability that this entails, because it frees us from the burden of mentally adjusting to every single GUI app that we happen to use. Similarly, developers appreciate how this makes their work easier: they don't have to code dropdown menus or modal dialogs, which in turns allows them to spend more time on actual, useful functionality.</p>
<p>Sure, it didn't happen overnight when desktop OS' were emerging as software platforms. Also, there are still plenty of apps whose creators - willfully or unintentionally - chose not to adhere to the standards. Sometimes it's even for the good, as it allows for new, useful UI patterns to emerge and be adopted by the mainstream. The resulting process is still that of convergence, producing interfaces which are more consistent and easier to use.</p>
<h3>Bootstrap shapes the Web</h3>
<p>The analogous process may just be happening to the Internet, considered as a "platform" for web applications. By steadily raising in popularity, Bootstrap has a chance of becoming <strong>the</strong> UI framework for Web in general - an obvious first choice for any new project.</p>
<p>Of course, even if this happens, it would be terribly unlikely that it starts reigning supreme and making every website look almost exactly the same - i.e. transforming the Web into equivalent of desktop. What's much more likely is following the footsteps of mobile platforms. In there, every app strives to be original and appealing but only those that correctly balance usability with artsy design provide really compelling user experience.</p>
<p>It will not be without differences, though. Mobile platforms are generally ruled with <a href="http://en.wikipedia.org/wiki/IOS">iron</a> (or <a href="http://en.wikipedia.org/wiki/Android_%28operating_system%29">clay</a>) fist and have relevant <a href="http://developer.apple.com/library/ios/#documentation/UserExperience/Conceptual/MobileHIG/Introduction/Introduction.html">UI guidelines</a> spelled out <a href="http://developer.android.com/guide/practices/ui_guidelines/index.html">explicitly</a>. For Web it's very much not the case, so any potential "standardization" is necessarily a bottom-up process whose benefits have to be indisputable and clearly visible. Despite some opposition, Bootstrap seems to have enough momentum to really (<em>ahem</em>) bootstrap this process. It already wins <a href="http://blog.avgs.ca/2012/04/why-bootstrap-is-awesome-and-why-you-should-care-about-it/">hearts and minds</a> of many web developers, so it may be just a matter of time.</p>
<p>If it happens, I believe the Web will be in better place.</p>
<img src="http://feeds.feedburner.com/~r/xion/~4/q5V3gww5A-Q" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://xion.org.pl/2012/04/22/bootstrap-a-ui-framework-for-the-modern-web/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>The Infernal Comma</title>
		<link>http://xion.org.pl/2012/04/16/the-infernal-comma/</link>
		<comments>http://xion.org.pl/2012/04/16/the-infernal-comma/#comments</comments>
		<pubDate>Mon, 16 Apr 2012 17:58:02 +0000</pubDate>
		<dc:creator>Xion</dc:creator>
				<category><![CDATA[Computer Science & IT]]></category>
		<category><![CDATA[arguments]]></category>
		<category><![CDATA[comma]]></category>
		<category><![CDATA[Python]]></category>
		<category><![CDATA[syntax errors]]></category>

		<guid isPermaLink="false">http://xion.org.pl/?p=3511</guid>
		<description><![CDATA[It came up today as a real surprise to me. Up until then, I thought that long gone were the times when I stared at plain old syntax errors in confused bewilderment. Well, at least if we're talking languages I have some experience with, like Python. So when it happened to me today, I was [...]]]></description>
			<content:encoded><![CDATA[<p>It came up today as a real surprise to me. Up until then, I thought that long gone were the times when I stared at plain old <em>syntax errors</em> in confused bewilderment. Well, at least if we're talking languages I have some experience with, like Python. So when it happened to me today, I was really caught off-guard.</p>
<p>The crux of the issue can be demonstrated in the following, artificial example:</p>
<div class="syntax_hilite">
<div id="python-28">
<div class="python"><span style="color: #ff7700;font-weight:bold;">from</span> lxml.<span style="color: black;">builder</span> <span style="color: #ff7700;font-weight:bold;">import</span> E</p>
<p><span style="color: #ff7700;font-weight:bold;">def</span> user_to_xml<span style="color: black;">&#40;</span><span style="color: #dc143c;">user</span><span style="color: black;">&#41;</span>:<br />
&nbsp; &nbsp; address = <span style="color: black;">&#91;</span>E.<span style="color: black;">address</span><span style="color: black;">&#40;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; street=<span style="color: #dc143c;">user</span>.<span style="color: black;">address</span>.<span style="color: black;">street</span>,<br />
&nbsp; &nbsp; &nbsp; &nbsp; zipcode=<span style="color: #dc143c;">user</span>.<span style="color: black;">address</span>.<span style="color: black;">zipcode</span>,<br />
&nbsp; &nbsp; &nbsp; &nbsp; city=<span style="color: #dc143c;">user</span>.<span style="color: black;">address</span>.<span style="color: black;">city</span>,<br />
&nbsp; &nbsp; <span style="color: black;">&#41;</span><span style="color: black;">&#93;</span> <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: #dc143c;">user</span>.<span style="color: black;">address</span> <span style="color: #ff7700;font-weight:bold;">else</span> <span style="color: black;">&#91;</span><span style="color: black;">&#93;</span><br />
&nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">return</span> E.<span style="color: #dc143c;">user</span><span style="color: black;">&#40;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; <span style="color: #008000;">dict</span><span style="color: black;">&#40;</span>first_name=<span style="color: #dc143c;">user</span>.<span style="color: black;">first_name</span>,<br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;last_name=<span style="color: #dc143c;">user</span>.<span style="color: black;">last_name</span><span style="color: black;">&#41;</span>,<br />
&nbsp; &nbsp; &nbsp; &nbsp; *address,<br />
&nbsp; &nbsp; <span style="color: black;">&#41;</span></div>
</div>
</div>
<p>
The goal is to build some simple XML tree using the most convenient interface, i.e. the <a href="http://lxml.de/tutorial.html#the-e-factory"><code>lxml.builder.E</code> manipulator</a> from the <a href="http://lxml.de/"><code>lxml</code> library</a>. The real code is somewhat longer and more complicated but this snippet encapsulates the issue pretty neatly.</p>
<p>And strange as it may seem, this little piece produces a <code>SyntaxError</code> at the final closing parenthesis:</p>
<div class="syntax_hilite">
<div id="code-29">
<div class="code">SyntaxError: invalid syntax <span style="color:#006600; font-weight:bold;">&#40;</span>at line <span style="color:#800000;">13</span> col <span style="color:#800000;">5</span><span style="color:#006600; font-weight:bold;">&#41;</span></div>
</div>
</div>
<p>
In such case, the first obvious thing anyone would do is of course to look for unmatched opening brace. With the aid of modern editors (or even <a href="http://vim.wikia.com/wiki/Moving_to_matching_braces">not so modern ones ;></a>) this is a trivial task. Before too long we would therefore find out that... all the braces are <em>fine</em>. Double-checking, just to be sure, will have the same result. Everything appears to be in order.</p>
<p>But, of course, we still have the syntax error. What the hell?!</p>
<p><img style="float:right; margin:0" src="http://xion.org.pl/wp-content/uploads/2012/04/comma.jpg" alt="" />As it turns out, the offending line is just above the seemingly erroneous parentheses. It's this one:</p>
<div class="syntax_hilite">
<div id="python-30">
<div class="python">*address,</div>
</div>
</div>
<p>
Or, to be more specific, it is the very last character of this line that the interpreter has problems with:</p>
<div class="syntax_hilite">
<div id="python-31">
<div class="python">*address, <span style="color: #808080; font-style: italic;"># &lt;-- comma! </span></div>
</div>
</div>
<p>
See, Python <em>really doesn't like</em> this trailing comma. Which, admittedly, is more than surprising, given how lenient it is in pretty much any other setting. You may recall that it's perfectly OK to include the additional comma after the final element of a list, tuple, or dictionary, and it is quite useful to do so in practice. Not only that - it is also possible for argument lists in function call. Indeed, this very fragment has one instance of such trailing comma that appears after a keyword argument (<code>city=user.address.city,</code>).</p>
<p>But apparently this doesn't really work for all kinds of arguments. If we unpack some positional ones (using <code>*</code> operator), we <em>cannot</em> put a comma afterwards. The relevant part of <a href="http://docs.python.org/reference/grammar.html">Python grammar specification</a> is stating this, of course:</p>
<div class="syntax_hilite">
<div id="code-32">
<div class="code">arglist: <span style="color:#006600; font-weight:bold;">&#40;</span>argument <span style="color:#CC0000;">','</span><span style="color:#006600; font-weight:bold;">&#41;</span>* <span style="color:#006600; font-weight:bold;">&#40;</span>argument <span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#CC0000;">','</span><span style="color:#006600; font-weight:bold;">&#93;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|<span style="color:#CC0000;">'*'</span> test <span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#CC0000;">','</span> argument<span style="color:#006600; font-weight:bold;">&#41;</span>* <span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#CC0000;">','</span> <span style="color:#CC0000;">'**'</span> test<span style="color:#006600; font-weight:bold;">&#93;</span> <br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|<span style="color:#CC0000;">'**'</span> test<span style="color:#006600; font-weight:bold;">&#41;</span></div>
</div>
</div>
<p>
but I wouldn't call it very <em>explicit</em>. And it seems that you actually <em>can</em> have a comma after <code>*foo</code> but only if another argument follows. If my intuition of formal grammars is correct, the reason for this rule to prohibit <code>foo(*args,)</code> (or <code>foo(**kwargs,)</code> for that matter) is strictly related to the fact than Python's grammar is <a href="http://en.wikipedia.org/wiki/LL%281%29">LL(1)</a>. And this, by the way, is here to stay. Quoting <a href="http://www.python.org/dev/peps/pep-3099/">PEP 3099</a>:</p>
<blockquote><p>Simple is better than complex. This idea extends to the parser. Restricting Python's grammar to an LL(1) parser is a blessing, not a curse. It puts us in handcuffs that prevent us from going overboard and ending up with funky grammar rules like some other dynamic languages that will go unnamed, <strong>such as Perl</strong>.</p></blockquote>
<p>I, for one, deem this attitude completely reasonable - even if it results in 20 minutes of utter confusion once in a blue moon.</p>
<p><small>Footnote: The title is of course a not-so-obvious reference to <a href="http://news.ycombinator.com/item?id=3844302">The Infernal Semicolon</a>.</small></p>
<img src="http://feeds.feedburner.com/~r/xion/~4/d_z9v67GOa4" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://xion.org.pl/2012/04/16/the-infernal-comma/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>Self-Reinforcement and Exponential Functions</title>
		<link>http://xion.org.pl/2012/04/11/self-reinforcement-and-exponential-functions/</link>
		<comments>http://xion.org.pl/2012/04/11/self-reinforcement-and-exponential-functions/#comments</comments>
		<pubDate>Wed, 11 Apr 2012 16:23:08 +0000</pubDate>
		<dc:creator>Xion</dc:creator>
				<category><![CDATA[Computer Science & IT]]></category>
		<category><![CDATA[Math]]></category>
		<category><![CDATA[Thoughts]]></category>
		<category><![CDATA[cognitive biases]]></category>
		<category><![CDATA[exponential functions]]></category>
		<category><![CDATA[intuitions]]></category>
		<category><![CDATA[IT trends]]></category>
		<category><![CDATA[prediction]]></category>
		<category><![CDATA[trends]]></category>

		<guid isPermaLink="false">http://xion.org.pl/?p=3464</guid>
		<description><![CDATA[Special relativity is really kind of mean. Not only it prohibits anything from going faster than the speed of light (therefore destroying our Star Trek-inspired dreams of interstellar travel) but also threatens with extreme adverse effects should anyone dare to even come close to the impenetrable barrier of c. Assuming we can deal with the [...]]]></description>
			<content:encoded><![CDATA[<p><a href="http://en.wikipedia.org/wiki/Special_relativity">Special relativity</a> is really kind of mean. Not only it prohibits anything from going faster than the speed of light (therefore destroying our <em>Star Trek</em>-inspired dreams of interstellar travel) but also threatens with extreme adverse effects should anyone dare to even come close to the impenetrable barrier of <em>c</em>. Assuming we can deal with the steady increase of mass as the speed goes up, there is always this issue of <em>time dilation</em>. While you are taking your short (i.e. few years-long) trip to nearby star, time passed on Earth could very well be measured in centuries. Having a millennium to catch up might prove cumbersome, and rather frustrating. Just think of all the iPhone models you would have missed!</p>
<p>As a solace, though, you could get quite a pile of cash waiting for you to pick up. Let's say you've put 10,000 dollars (or euro, or your favorite currency) into investment with a yearly interest rate of 10 percent. Every year, this deposit will therefore increase by one tenth, and this will happen continuously over the next 1000 years. Could you quickly tell how big the final amount will be, compared to the initial one? How many times will it increase?...</p>
<p>You shouldn't be very hard on yourself if you answered instinctively with e.g. 100 times or something similar. I mean, such figures are totally, utterly wrong by many orders of magnitude because the actual value is <em>bigger than 10<sup>40</sup></em>. But it's absolutely common to have problems with grasping <strong>exponential functions</strong> intuitively. In many ways this is quite pitiful, for they accurately describe many phenomenons that occur in nature, civilization, technology and culture. Yet they often escape understanding, leading to unfulfilled predictions, incorrect extrapolations, and plain old cognitive biases.</p>
<p>What is so bizarre about these functions that they tend to confuse a significant fraction, if not the majority of people?...</p>
<p><span id="more-3464"></span></p>
<h3>A derivative problem</h3>
<p>First, let's clarify that <em>the</em> exponential function is a very specific one. It can be defined as the only function which is <em>its own derivative</em>. We can write it down as:</p>
<p align="center"><img src="http://xion.org.pl/wp-content/ql-cache/quicklatex.com-1476785ee608114ee4e3a7c6ce1ab554_l3.png" class="ql-img-inline-formula" alt="&#92;&#100;&#105;&#115;&#112;&#108;&#97;&#121;&#115;&#116;&#121;&#108;&#101;&#32;&#92;&#102;&#114;&#97;&#99;&#123;&#100;&#125;&#123;&#100;&#120;&#125;&#101;&#94;&#120;&#32;&#61;&#32;&#101;&#94;&#120;" title="Rendered by QuickLaTeX.com" style="vertical-align: -11px;"/></p>
<p>where <em>e</em> (equal to approximately 2.71) is typically known as Euler's number and is easily the most important constant in all of mathematics. Yes, much more significant that <img src="http://xion.org.pl/wp-content/ql-cache/quicklatex.com-6ebba43c950581540afe674d8f721d10_l3.png" class="ql-img-inline-formula" alt="&#92;&#112;&#105;" title="Rendered by QuickLaTeX.com" style="vertical-align: 0px;"/>, <img src="http://xion.org.pl/wp-content/ql-cache/quicklatex.com-dcd29d17e8488b1e2f1256023e46f13d_l3.png" class="ql-img-inline-formula" alt="&#92;&#116;&#97;&#117;" title="Rendered by QuickLaTeX.com" style="vertical-align: 0px;"/>, <img src="http://xion.org.pl/wp-content/ql-cache/quicklatex.com-d8de3ceeceb388c48e121a25ff3f337b_l3.png" class="ql-img-inline-formula" alt="&#92;&#112;&#104;&#105;" title="Rendered by QuickLaTeX.com" style="vertical-align: -3px;"/> or whatever the current fashion in geometry dictates.</p>
<p>In practice, we often deal with exponentials of different base. For example, computer scientists are especially fond of base 2 for variety of reasons. While the above equality isn't satisfied for those functions in exact manner, the general principle still stands. Differentiating an exponential function always leaves us with another exponential function.</p>
<p>And this, I believe, might be one of the roots of the problem. A significant portion of human brain - the <em>sensory cortex</em> - is basically a very complicated differentiator that operates on incoming neural signals. We experience it as <a href="http://en.wikipedia.org/wiki/Weber-Fechner_Law">Weber-Fechner law</a>, which basically states that our senses detect changes in stimuli only in proportion to the already perceived one. In other words: the brain is, for the most part, conveniently ignoring the absolute values of sensory data as it's mostly interested in their proportional changes. It would seem that applying differentiation across the board makes the data more manageable for cognitive processing.</p>
<p>How this relates to our poor intuitions towards exponential functions? Well, we could speculate that some equivalent of before-mentioned law works on deeper level: the one related to our "perception" of numbers, measures, quantities and trends - or just mathematics for short. If that's indeed true, then we can easily see how exponentials fail to comply. Because they are immune to differentiation, we are unable to subconsciously transform them into more "natural" form which would be easier for our mind to act upon. Instead, they appear as conceptual black box that we might never <em>really</em> grasp on a fundamental level.</p>
<h3>Reinforced by itself</h3>
<p>Regardless of whether the above supposition is sound, it is certainly true that exponential trends mess badly with one of our basic intuitions: that future will be more or less similar to the past. Despite countless fluctuations of human condition over the centuries, this heuristic held ground because significant and overarching changes almost always took longer than a single lifetime. It is only through history (written or spoken) that we can learn of the curious and alien ways of living in the distant past.</p>
<p>But cultural and technological development is a <strong>self-reinforcing</strong> process. This means that any amount of progress not only has an immediate effect in the present but also results in further improvement of speed and effectiveness of the <em>process itself</em>. In physics, we encounter many phenomena that also exhibit this property (e.g. <a href="http://en.wikipedia.org/wiki/Drag_%28physics%29">air resistance</a>), and they are described using certain class of differential equations. Their solutions always take a form of - you guessed it - exponential functions.</p>
<p>Maybe this is why we fail so hard at making any kind of mid- to long-term predictions. Linear extrapolation of exponential trend is bound to diverge from reality at <em>very</em> fast pace. On the other hand, this can be a reason why some things remain almost invisible until they "explode" into ubiquity in very short time frame. If they feed upon themselves - being self-reinforcing processes - this is no coincidence, and we could see that happen in case of social networks and mobile applications. I suppose this is also what founders of technological startups should make sure their idea is like.</p>
<h3>Adjusting perception</h3>
<p>It would appear that exponential functions are quite a pain to comprehend on intuitive, subconscious level. But, as I believe to have shown above, developing more effective intuitions towards them offers plenty of benefits. I'm just not entirely sure what's the best approach to achieve this end. For one, though, I think that gaining awareness of how exponential trends are interleaving our daily lives seems like a good start.</p>
<img src="http://feeds.feedburner.com/~r/xion/~4/yKqscc4QY5U" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://xion.org.pl/2012/04/11/self-reinforcement-and-exponential-functions/feed/</wfw:commentRss>
		<slash:comments>3</slash:comments>
		</item>
		<item>
		<title>Counting Lines in Multiple Files</title>
		<link>http://xion.org.pl/2012/04/06/counting-lines-in-multiple-files/</link>
		<comments>http://xion.org.pl/2012/04/06/counting-lines-in-multiple-files/#comments</comments>
		<pubDate>Fri, 06 Apr 2012 11:40:03 +0000</pubDate>
		<dc:creator>Xion</dc:creator>
				<category><![CDATA[Computer Science & IT]]></category>
		<category><![CDATA[command line]]></category>
		<category><![CDATA[files]]></category>
		<category><![CDATA[Linux]]></category>
		<category><![CDATA[piping]]></category>
		<category><![CDATA[regular expressions]]></category>

		<guid isPermaLink="false">http://xion.org.pl/?p=3482</guid>
		<description><![CDATA[Looks like using Linux is really bound to slowly - but steadily - improve your commandline-fu. As evidence, today I wanted to share a little piece of shell acolyte's magic that I managed to craft without very big trouble. It's about counting lines in files - code lines in code files, to be specific. For [...]]]></description>
			<content:encoded><![CDATA[<p>Looks like using Linux is really bound to slowly - but steadily - improve your <em>commandline-fu</em>. As evidence, today I wanted to share a little piece of shell acolyte's magic that I managed to craft without very big trouble. It's about counting lines in files - code lines in code files, to be specific.</p>
<p>For a single file, getting the number of text rows is very simple:</p>
<div class="syntax_hilite">
<div id="code-43">
<div class="code">$ wc -l some.<span style="">file</span><br />
&nbsp; <span style="color:#800000;">142</span> some.<span style="">file</span></div>
</div>
</div>
<p>
Although the name <code>wc</code> comes from "word count", the <code>-l</code> switch changes its mode of operation into counting rows. The flexibility of this little program doesn't end here; for example, it can also accept piped input (as <em>stdin</em>):</p>
<div class="syntax_hilite">
<div id="code-44">
<div class="code">$ cat some.<span style="">file</span> | wc -l<br />
<span style="color:#800000;">142</span></div>
</div>
</div>
<p>
as well as multiple files:</p>
<div class="syntax_hilite">
<div id="code-45">
<div class="code">$ wc -l some.<span style="">file</span> other.<span style="">file</span><br />
&nbsp; <span style="color:#800000;">142</span> some.<span style="">file</span><br />
&nbsp; &nbsp;<span style="color:#800000;">54</span> other.<span style="">file</span><br />
&nbsp; <span style="color:#800000;">196</span> all</div>
</div>
</div>
<p>
or even wildcards, such as <code>wc -l *.file</code>. With these we could rather easily count the number of lines of code in our project:</p>
<div class="syntax_hilite">
<div id="code-46">
<div class="code">$ wc -l **<span style="color:#008000;">/*.py<br />
&nbsp; &nbsp; 3 foo/__init__.py<br />
&nbsp; 189 foo/main.py<br />
&nbsp; &nbsp;89 foo/utils.py<br />
&nbsp; &nbsp;24 setup.py<br />
&nbsp; 133 tests.py<br />
&nbsp; 438 all </span></div>
</div>
</div>
<p>
Unfortunately, the exact interpretation of <code>**/*</code> wildcard seems to vary between shells. In <em>zsh</em> it works as shown above, but in <em>bash</em> I had it omit files from current directory. While it <em>might</em> make some sense here (as it would give a total without setup script and tests), I'm sure it won't be the case all projects.</p>
<p>And so we need something smarter.<br />
<span id="more-3482"></span><br />
A certain way to list all files matching given property (e.g. name) in current and (recursively) child directories is to use the <code>find</code> command:</p>
<div class="syntax_hilite">
<div id="code-47">
<div class="code">$ find -name <span style="color:#CC0000;">"*.py"</span><br />
./foo/__init__.<span style="">py</span><br />
./foo/main.<span style="">py</span><br />
./foo/utils.<span style="">py</span><br />
./setup.<span style="">py</span><br />
./tests.<span style="">py</span></div>
</div>
</div>
<p>
Can we feed such a list into command taking multiple files, like <code>wc</code>? As it turns out, it is perfectly possible, and the utility that allows to do this is called <code>xargs</code>. Numerous are its features, of course, but the simplest usage is totally option-less; we only need to supply the target command and pipe our list to <code>xargs</code>' standard input:</p>
<div class="syntax_hilite">
<div id="code-48">
<div class="code">$ find -name <span style="color:#CC0000;">"*.py"</span> | xargs cat<br />
# ...<span style="">many</span>, many lines of code from all the *.<span style="">py</span> files...</div>
</div>
</div>
<p>
This is how we get all the lines printed, so counting them is trivial now:</p>
<div class="syntax_hilite">
<div id="code-49">
<div class="code">$ find -name <span style="color:#CC0000;">"*.py"</span> | xargs cat | wc -l<br />
<span style="color:#800000;">438</span></div>
</div>
</div>
<p>
The real power of this technique lies in the fact that we can inject additional modifiers, or filters, at any stage. We can, for instance, eliminate some files we are not interested in by using <code>grep -v</code>:</p>
<div class="syntax_hilite">
<div id="code-50">
<div class="code">$ find -name <span style="color:#CC0000;">"*.py"</span> | grep -v <span style="color:#CC0000;">"test"</span> | xargs wc -l<br />
&nbsp; &nbsp; <span style="color:#800000;">3</span> foo/__init__.<span style="">py</span><br />
&nbsp; <span style="color:#800000;">189</span> foo/main.<span style="">py</span><br />
&nbsp; &nbsp;<span style="color:#800000;">89</span> foo/utils.<span style="">py</span><br />
&nbsp; &nbsp;<span style="color:#800000;">24</span> setup.<span style="">py</span><br />
&nbsp; <span style="color:#800000;">305</span> all</div>
</div>
</div>
<p>
Likewise, we can get rid of comment lines if we push the output of <code>cat</code> through another regex-based filter:</p>
<div class="syntax_hilite">
<div id="code-51">
<div class="code">$ find -name <span style="color:#CC0000;">"*.py"</span> | xargs cat | grep -vx <span style="color:#CC0000;">"^<span style="color:#000099; font-weight:bold;">\s</span>*<span style="color:#000099; font-weight:bold;">\#</span>.*"</span> | wc -l<br />
<span style="color:#800000;">312</span></div>
</div>
</div>
<p>
Obviously, both <code>grep</code>s can be present at once:</p>
<div class="syntax_hilite">
<div id="code-52">
<div class="code">$ find -name <span style="color:#CC0000;">"*.py"</span> | grep -v <span style="color:#CC0000;">"test"</span> | xargs cat | grep -vx <span style="color:#CC0000;">"^<span style="color:#000099; font-weight:bold;">\s</span>*<span style="color:#000099; font-weight:bold;">\#</span>.*"</span> | wc -l<br />
<span style="color:#800000;">223</span></div>
</div>
</div>
<p>
Complexity of this command likely exceeds many typical use cases for <code>find</code> or <code>grep</code>, although seasoned shell hackers may think otherwise. In any case, I think the power of this technique is very evident, and not only for counting lines.</p>
<img src="http://feeds.feedburner.com/~r/xion/~4/4hJBRq48iBo" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://xion.org.pl/2012/04/06/counting-lines-in-multiple-files/feed/</wfw:commentRss>
		<slash:comments>4</slash:comments>
		</item>
		<item>
		<title>Prison Escape: Game from IGK Compo</title>
		<link>http://xion.org.pl/2012/04/03/prison-escape-game-from-igk-compo/</link>
		<comments>http://xion.org.pl/2012/04/03/prison-escape-game-from-igk-compo/#comments</comments>
		<pubDate>Tue, 03 Apr 2012 16:50:52 +0000</pubDate>
		<dc:creator>Xion</dc:creator>
				<category><![CDATA[Events]]></category>
		<category><![CDATA[Games]]></category>
		<category><![CDATA[Programming]]></category>
		<category><![CDATA[Compo]]></category>
		<category><![CDATA[game development]]></category>
		<category><![CDATA[IGK]]></category>

		<guid isPermaLink="false">http://xion.org.pl/?p=3472</guid>
		<description><![CDATA[Yesterday I came back from IGK conference (Engineering of Computer Games) in Siedlce. Among few interesting lectures and presentations it also featured a traditional, 7-hour long game development contest: Compo. Those unfamiliar with the concept should know that it's a competition where every team (of max. 4 people) has to produce a playable game, according [...]]]></description>
			<content:encoded><![CDATA[<p>Yesterday I came back from <a href="http://genbit.info/igk/">IGK conference</a> (Engineering of Computer Games) in Siedlce. Among few interesting lectures and presentations it also featured a traditional, 7-hour long game development contest: Compo. Those unfamiliar with the concept should know that it's a competition where every team (of max. 4 people) has to produce a playable game, according to particular topic, e.g. theme or genre. When the work is done, results are being presented to the public while a comittee of organizers votes for winners.</p>
<p>As usual, I took part in the competition along with Adam Sawicki "<a href="http://asawicki.info">Reg</a>" and Krzystof Kluczek "<a href="http://devkk.net/">Krzysiek K.</a>". Topic for this year revolved around the idea of "escape" or "survival" kind of game, so we designed an old school, pixel-artsy scroller where you play as a prisoner trying to escape detention by running and avoiding numerous obstacles. We coded intensely and passionately, and in the end it paid off really well because we actually managed to snatch the first place! Woohoo!</p>
<p align="center"><a href="http://xion.org.pl/wp-content/uploads/2012/04/pe-splashscreen.png"><img src="http://xion.org.pl/wp-content/uploads/2012/04/pe-splashscreen-300x234.png" alt="" title="Prison Escape - splash screen" width="300" height="234" /></a>&nbsp;<a href="http://xion.org.pl/wp-content/uploads/2012/04/pe-gameplay.png"><img src="http://xion.org.pl/wp-content/uploads/2012/04/pe-gameplay-300x234.png" alt="" title="Prison Escape - gameplay" width="300" height="234" /></a></p>
<p>A whopping amount of 15 teams took part in this year's Compo, so it might take some time before all the submissions are available online. Meanwhile, I'm mirroring our game here. Please note that it uses DirectX 9 (incl. some shaders), so for best results you should have a non-ancient GPU and Windows OS. (It might work under Wine; we haven't tested it).<br />
Note: There is a file embedded within this post, please visit this post to download the file.</p>
<img src="http://feeds.feedburner.com/~r/xion/~4/FAOpC1JDYxE" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://xion.org.pl/2012/04/03/prison-escape-game-from-igk-compo/feed/</wfw:commentRss>
		<slash:comments>4</slash:comments>
		</item>
		<item>
		<title>Working within Temporary Directory</title>
		<link>http://xion.org.pl/2012/03/24/working-within-temporary-directory/</link>
		<comments>http://xion.org.pl/2012/03/24/working-within-temporary-directory/#comments</comments>
		<pubDate>Sat, 24 Mar 2012 14:35:18 +0000</pubDate>
		<dc:creator>Xion</dc:creator>
				<category><![CDATA[Programming]]></category>
		<category><![CDATA[contextlib]]></category>
		<category><![CDATA[Git]]></category>
		<category><![CDATA[Python]]></category>
		<category><![CDATA[with]]></category>

		<guid isPermaLink="false">http://xion.org.pl/?p=3452</guid>
		<description><![CDATA[Few days ago I needed to write a script which was supposed to run inside a temporary directory. The exact matter was about deployment from an ad hoc Git repository, and it's something that I may describe in more detail later on. Today, however, I wanted to focus on its small part: a one that [...]]]></description>
			<content:encoded><![CDATA[<p>Few days ago I needed to write a script which was supposed to run inside a temporary directory. The exact matter was about deployment from an <em>ad hoc</em> Git repository, and it's something that I may describe in more detail later on. Today, however, I wanted to focus on its small part: a one that (I think) has neatly captured the notion of executing something within a non-persistent, working directory. Because it's a very general technique, I suppose quite a few readers may find it pretty useful.</p>
<p>Obtaining a temporary file or even directory shouldn't be a terribly complicated thing - and indeed, it's very easy in case of Python. We have a <a href="http://docs.python.org/library/tempfile.html">standard <code>tempfile</code> module</a> here and it serves our needs pretty well in this regard. For one, it has the <code>mkdtemp</code> function which creates a temporary directory and returns path to it:</p>
<div class="syntax_hilite">
<div id="python-56">
<div class="python">temp_dir = <span style="color: #dc143c;">tempfile</span>.<span style="color: black;">mkdtemp</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span></div>
</div>
</div>
<p>
That's what it does. What it <em>doesn't</em> do is e.g ensuring a proper cleanup once the directory is not needed anymore. This is especially important on Windows where the equivalent of <code>/tmp</code> is not wiped out at boot time.<br />
We also wanted our fresh temp directory to be set as the program's working one (<code>PWD</code>), and  obviously this is also something we need to manually take care of. To combine those two needs, I think the best solution is to employ a <strong>context manager</strong>.</p>
<p>Context manager is basically a fancy name for an object that the <code>with</code> statement can be applied upon. You may recall that some time ago <a href="http://xion.org.pl/2012/01/21/on-clever-usage-of-python-with-clauses/">I wrote</a> about interesting use cases for the <code>with</code> construct. This one could also qualify as such, but the principles are very typical. It's about introducing a <em>scope</em> where some resource (here: a temporary directory) remains accessible as long as we're inside it. Once we leave the <code>with</code> block, it is cleaned up - just like file handles, network sockets, concurrent locks and plenty of other similar objects.</p>
<p>But while semantics are pretty clear, there are of course several ways to do this syntactically. I took this opportunity to try out the supposedly simplest one which I learned recently on <a href="http://pygrunn-paylogic.blogspot.com/2012/03/pygrunn-monthly-3-lesser-known-modules.html">local Python community meet-up</a>: the <code>contextlib</code> library. It includes the <code>contextmanager</code> decorator: a simple and clever way to write <code>with</code>-enabled objects as simple functions. It is based on particular usage of <code>yield</code> statement which makes it very interesting even by itself.</p>
<p>So without further ado, let's look at the final solution I wanted to present:</p>
<div class="syntax_hilite">
<div id="python-57">
<div class="python"><span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">os</span><br />
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">shutil</span><br />
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">tempfile</span><br />
<span style="color: #ff7700;font-weight:bold;">from</span> contextlib <span style="color: #ff7700;font-weight:bold;">import</span> contextmanager</p>
<p>@contextmanager<br />
<span style="color: #ff7700;font-weight:bold;">def</span> temp_directory<span style="color: black;">&#40;</span>*args, **kwargs<span style="color: black;">&#41;</span>:<br />
&nbsp; &nbsp; <span style="color: #483d8b;">""</span><span style="color: #483d8b;">"Allows the program to operate inside temporary directory.<br />
&nbsp; &nbsp; Sets the app's working dir automatically and restores it<br />
&nbsp; &nbsp; to original one upon existing the `with` clause.<br />
&nbsp; &nbsp; "</span><span style="color: #483d8b;">""</span><br />
&nbsp; &nbsp; orig_workdir = <span style="color: #dc143c;">os</span>.<span style="color: black;">getcwd</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span><br />
&nbsp; &nbsp; temp_workdir = <span style="color: #dc143c;">tempfile</span>.<span style="color: black;">mkdtemp</span><span style="color: black;">&#40;</span>*args, **kwargs<span style="color: black;">&#41;</span><br />
&nbsp; &nbsp; <span style="color: #dc143c;">os</span>.<span style="color: black;">chdir</span><span style="color: black;">&#40;</span>temp_workdir<span style="color: black;">&#41;</span></p>
<p>&nbsp; &nbsp; <span style="color: #ff7700;font-weight:bold;">yield</span> temp_workdir</p>
<p>&nbsp; &nbsp; <span style="color: #dc143c;">os</span>.<span style="color: black;">chdir</span><span style="color: black;">&#40;</span>orig_workdir<span style="color: black;">&#41;</span><br />
&nbsp; &nbsp; <span style="color: #dc143c;">shutil</span>.<span style="color: black;">rmtree</span><span style="color: black;">&#40;</span>temp_workdir<span style="color: black;">&#41;</span></div>
</div>
</div>
<p>
As we can see, <code>yield</code> divides this function into two parts: <em>setup</em> and <em>cleanup</em>. Setup will be executed when we enter the <code>with</code> block, while cleanup will run when we're about to exit it. By the way, this scheme of multiple entry and exit points in one function is typically referred to as <strong>coroutine</strong>, and it allows for several <a href="http://www.dabeaz.com/coroutines/Coroutines.pdf ">very intriguing techniques</a> of smart computation.</p>
<p>Usage of <code>temp_directory</code> function is pretty obvious, I'd say. Here's a simplified excerpt of the Git-based deployment script that I used it in:</p>
<div class="syntax_hilite">
<div id="python-58">
<div class="python"><span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">subprocess</span><br />
shell = <span style="color: #ff7700;font-weight:bold;">lambda</span> <span style="color: #dc143c;">cmd</span>: <span style="color: #dc143c;">subprocess</span>.<span style="color: black;">call</span><span style="color: black;">&#40;</span><span style="color: #dc143c;">cmd</span>, shell=<span style="color: #008000;">True</span><span style="color: black;">&#41;</span></p>
<p>orig_repo = <span style="color: #dc143c;">os</span>.<span style="color: black;">getcwd</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span><br />
with temp_directory<span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>:<br />
&nbsp; &nbsp; shell<span style="color: black;">&#40;</span><span style="color: #483d8b;">'git clone --shared %s .'</span> % orig_repo<span style="color: black;">&#41;</span><br />
&nbsp; &nbsp; shell<span style="color: black;">&#40;</span><span style="color: #483d8b;">'./build'</span><span style="color: black;">&#41;</span><br />
&nbsp; &nbsp; shell<span style="color: black;">&#40;</span><span style="color: #483d8b;">'git add -f '</span> + build_products<span style="color: black;">&#41;</span><br />
&nbsp; &nbsp; shell<span style="color: black;">&#40;</span><span style="color: #483d8b;">'git commit -m &quot;%s&quot;'</span> % message<span style="color: black;">&#41;</span><br />
&nbsp; &nbsp; shell<span style="color: black;">&#40;</span><span style="color: #483d8b;">'git push %s master'</span> % deploy_remote<span style="color: black;">&#41;</span></div>
</div>
</div>
<p>
Note how the meaning of <code>'.'</code> (current directory) shifts depending on whether we're inside or outside the <code>with</code> block. Users of <a href="http://www.fabfile.org">Fabric</a> (Python- and SSH-based remote administration tool) will find this very similar to its <a href="http://docs.fabfile.org/en/1.4.0/api/core/context_managers.html#fabric.context_managers.cd"><code>cd</code> context manager</a>. The main difference is of course that directory we're <code>cd</code>-ing to is not a predetermined one, and that it will disappear once we're done with it.</p>
<img src="http://feeds.feedburner.com/~r/xion/~4/sDEB6CGSZvY" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://xion.org.pl/2012/03/24/working-within-temporary-directory/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>How Does this Work (in JavaScript)</title>
		<link>http://xion.org.pl/2012/03/18/how-does-this-work-in-javascript/</link>
		<comments>http://xion.org.pl/2012/03/18/how-does-this-work-in-javascript/#comments</comments>
		<pubDate>Sun, 18 Mar 2012 20:19:06 +0000</pubDate>
		<dc:creator>Xion</dc:creator>
				<category><![CDATA[Programming]]></category>
		<category><![CDATA[classes]]></category>
		<category><![CDATA[functions]]></category>
		<category><![CDATA[JavaScript]]></category>
		<category><![CDATA[methods]]></category>
		<category><![CDATA[this]]></category>

		<guid isPermaLink="false">http://xion.org.pl/?p=3442</guid>
		<description><![CDATA[Many caveats clutter the JavaScript language. Some of them are quite hilarious and relatively harmless, but few can get really nasty and lead to insidious bugs. Today, I'm gonna talk about something from the second group: the semantics of this keyword in JavaScript. Why's this? It is worth noting why JS has the this keyword [...]]]></description>
			<content:encoded><![CDATA[<p>Many caveats clutter the JavaScript language. Some of them are <a href="http://www.destroyallsoftware.com/talks/wat">quite hilarious</a> and relatively harmless, but few can get really nasty and lead to insidious bugs. Today, I'm gonna talk about something from the second group: the semantics of <code>this</code> keyword in JavaScript.</p>
<h3>Why's <code>this</code>?</h3>
<p>It is worth noting why JS has the <code>this</code> keyword at all. Normally, we would expect it only in those languages which also have the corresponding <code>class</code> keyword. That's what C++, Java and C# have taught us: that <code>this</code> represents the <strong>current object</strong> of a class when used inside one of its methods. It only makes sense, then, to use <code>this</code> keyword in a class scope, denoted by the <code>class</code> keyword - both of which JavaScript doesn't seem to have. So, why's <code>this</code> even there?</p>
<p>The most likely reason is that JavaScript actually <em>has</em> something that resembles traditional classes - but it does so very poorly. And like pretty much everything in JS, it is written as a <code>function</code>:</p>
<div class="syntax_hilite">
<div id="javascript-68">
<div class="javascript"><span style="color: #003366; font-weight: bold;">function</span> Greeting<span style="color: #66cc66;">&#40;</span>text<span style="color: #66cc66;">&#41;</span> <span style="color: #66cc66;">&#123;</span><br />
&nbsp; &nbsp; <span style="color: #000066; font-weight: bold;">this</span>.<span style="color: #006600;">text</span> = text<br />
<span style="color: #66cc66;">&#125;</span><br />
Greeting.<span style="color: #006600;">prototype</span>.<span style="color: #006600;">greet</span> = <span style="color: #003366; font-weight: bold;">function</span><span style="color: #66cc66;">&#40;</span>who<span style="color: #66cc66;">&#41;</span> <span style="color: #66cc66;">&#123;</span><br />
&nbsp; &nbsp; <span style="color: #000066;">alert</span><span style="color: #66cc66;">&#40;</span><span style="color: #3366CC;">"Hello, "</span> who + <span style="color: #3366CC;">"! "</span> + <span style="color: #000066; font-weight: bold;">this</span>.<span style="color: #006600;">text</span><span style="color: #66cc66;">&#41;</span>;<br />
<span style="color: #66cc66;">&#125;</span></p>
<p><span style="color: #003366; font-weight: bold;">var</span> greeting = <span style="color: #003366; font-weight: bold;">new</span> Greeting<span style="color: #66cc66;">&#40;</span><span style="color: #3366CC;">"Nice to meet you!"</span><span style="color: #66cc66;">&#41;</span>;<br />
greeting.<span style="color: #006600;">greet</span><span style="color: #66cc66;">&#40;</span><span style="color: #3366CC;">"Alice"</span><span style="color: #66cc66;">&#41;</span>;</div>
</div>
</div>
<p>
Here, the <code>Greeting</code> is technically a function and is defined as one, but semantically it works more like <strong>constructor</strong> for the <code>Greeting</code> "class". As for <code>this</code> keyword, it refers to the object being created by such a constructor when invoked by <code>new</code> statement - another familiar construct, by the way. Additionally, <code>this</code> also appears inside <code>greet</code> method and does its expected job, allowing access to the <code>text</code> member of an object that the method was called upon.</p>
<p>So it would seem that everything with <code>this</code> keyword is actually fine and rather unsurprising. Have we maybe overlooked something here, looking only at half of the picture?...</p>
<p>Well yes, very much so. And not even a half but more like a <em>quarter</em>, with the remaining three parts being significantly less pretty - to say it mildly.<br />
<span id="more-3442"></span></p>
<h3>Four meanings of <code>this</code></h3>
<p>It might not have been immediately visible, but in the previous example the <code>this</code> keyword has been used in <em>two</em> different meanings. They just happen to blur into single semantic, thanks to our associations with other object-oriented languages. Incidentally, those associations were probably the main reason behind adding the whole <code>this</code>/<code>new</code> mechanism to JavaScript. Unfortunately, it doesn't look <em>quite</em> as appealing when viewed in the context of language as a whole. Because at this wider perspective, <code>this</code> sucks big time.</p>
<p>For starters, here are the possible referents of <code>this</code>, i.e. things it can point to depending on where and <em>how</em> the keyword is used:</p>
<ul>
<li>objects constructed using the <code>new</code> keyword, as seen in the <code>Greeting</code> function above</li>
<li>objects we're calling methods on, as seen in the <code>greet</code> function above</li>
<li>the <strong>global object</strong> where all the global variables reside; for browsers, this would be the <code>window</code> object</li>
<li><strong>any object</strong> at all</li>
</ul>
<p>What you might have noticed is the massive drop of sanity level between the second and third option. There is absolutely no mistake here: <code>this.x = 1;</code> can mean the same as <code>window.x = 1;</code> and create a global variable, happily cluttering our global namespace. In fact, this is the <em>default</em> behavior of <code>this</code> when used inside a function. It's the first two, non-crazy variants that require a special call syntax, while the following "just works":</p>
<div class="syntax_hilite">
<div id="javascript-69">
<div class="javascript"><span style="color: #003366; font-weight: bold;">function</span> foo<span style="color: #66cc66;">&#40;</span>arg<span style="color: #66cc66;">&#41;</span> <span style="color: #66cc66;">&#123;</span><br />
&nbsp; &nbsp; <span style="color: #000066; font-weight: bold;">this</span>.<span style="color: #006600;">x</span> = arg;<br />
<span style="color: #66cc66;">&#125;</span></p>
<p>foo<span style="color: #66cc66;">&#40;</span><span style="color: #CC0000;">42</span><span style="color: #66cc66;">&#41;</span>;<br />
<span style="color: #000066;">alert</span><span style="color: #66cc66;">&#40;</span>window.<span style="color: #006600;">x</span> === <span style="color: #CC0000;">42</span><span style="color: #66cc66;">&#41;</span>; <span style="color: #009900; font-style: italic;">// true </span></div>
</div>
</div>
<p>
Chances are, it doesn't work the way we would expect it to work.</p>
<h3>It's all in the call</h3>
<p>Time for a small brain teaser. Could you compare the <code>Greeting</code> and <code>foo</code> functions, and find some differences between them? Preferably, it should be a difference which is unrelated to naming and would persist even if we <del>compiled</del> minified both functions... Can you find at least one?</p>
<p>You could try for a while, but it would ultimately prove futile: <em>there are none</em>. Nothing is special about the so-called constructor of our <code>Greeting</code> "class": it is an ordinary JavaScript function. While it seems to exhibit some class-like behavior, it is only due to our usage of <code>new</code>. If we called it directly, we would end up with the exact same behavior as with <code>foo</code> example above: <code>this</code> would bind to <code>window</code> and <code>text</code> would end as a global variable.</p>
<p>Not only this is surprising, confusing and error-prone, but it also <em>generalizes</em> to every case where <code>this</code> is used inside a function! The gruesome conclusion is that:</p>
<blockquote><p>The meaning of <code>this</code> depends on how the function was <strong>called</strong>, regardless of how it was defined.</p></blockquote>
<p>To realize how unpleasant consequences this may have, consider a simple object that has a "method":</p>
<div class="syntax_hilite">
<div id="javascript-70">
<div class="javascript"><span style="color: #003366; font-weight: bold;">var</span> counter = <span style="color: #66cc66;">&#123;</span><br />
&nbsp; &nbsp; inc: <span style="color: #003366; font-weight: bold;">function</span><span style="color: #66cc66;">&#40;</span><span style="color: #66cc66;">&#41;</span> <span style="color: #66cc66;">&#123;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; <span style="color: #000066; font-weight: bold;">this</span>.<span style="color: #006600;">value</span> = <span style="color: #66cc66;">&#40;</span><span style="color: #000066; font-weight: bold;">this</span>.<span style="color: #006600;">value</span> || <span style="color: #CC0000;">0</span><span style="color: #66cc66;">&#41;</span> + <span style="color: #CC0000;">1</span>;<br />
&nbsp; &nbsp; <span style="color: #66cc66;">&#125;</span>,<br />
<span style="color: #66cc66;">&#125;</span>;</div>
</div>
</div>
<p>
Let's say that we want <code>counter.inc</code> to be called in response to some external event. In asynchronous, callback- and event-based environments - where almost all of the JavaScript code operates - this would be an extremely common occurrence. Setting aside any specific details, we can safely assume that our method will at some point end up in a variable. It will be deep inside the guts of a framework or library of choice, but in the end, it will be essentially equivalent to this line:</p>
<div class="syntax_hilite">
<div id="javascript-71">
<div class="javascript"><span style="color: #003366; font-weight: bold;">var</span> callback = counter.<span style="color: #006600;">inc</span>;</div>
</div>
</div>
<p>
When the event in question actually happens, <code>callback</code> is of course invoked:</p>
<div class="syntax_hilite">
<div id="javascript-72">
<div class="javascript">callback<span style="color: #66cc66;">&#40;</span><span style="color: #66cc66;">&#41;</span>;</div>
</div>
</div>
<p>
And it is invoked like a normal function (rather than a method), so <code>this</code> acquires meaning specific to that invocation. Which means the global (<code>window</code>) object, <em>not</em> the <code>counter</code> object as we would expect... Yuck!</p>
<p>Fortunately, this can be worked around in a relatively straightforward way: by capturing <code>counter</code> object in a closure, therefore ensuring that it ends up as <code>this</code> inside its <code>inc</code> method:</p>
<div class="syntax_hilite">
<div id="javascript-73">
<div class="javascript"><span style="color: #003366; font-weight: bold;">var</span> callback = <span style="color: #003366; font-weight: bold;">function</span><span style="color: #66cc66;">&#40;</span><span style="color: #66cc66;">&#41;</span> <span style="color: #66cc66;">&#123;</span><br />
&nbsp; &nbsp; counter.<span style="color: #006600;">inc</span><span style="color: #66cc66;">&#40;</span><span style="color: #66cc66;">&#41;</span>;<br />
<span style="color: #66cc66;">&#125;</span>;</div>
</div>
</div>
<p>
Just think, though: what's the chance that someone notices the "superfluous" function later on and decides to "refactor" it out?... If you ever use this trick in practice, make sure you add some adequate comment.</p>
<h3><code>this</code> and that</h3>
<p>What about the fourth point, stating that <code>this</code> can actually refer to anything at all? Again, it is only a matter of how the function gets called - and it so happens that there are some <em>very</em> special ways of calling a function. Those are the <code>apply</code> and <code>call</code> methods, where one can completely specify not only the argument list, but also the referent of <code>this</code> inside the function:</p>
<div class="syntax_hilite">
<div id="javascript-74">
<div class="javascript"><span style="color: #003366; font-weight: bold;">var</span> greeting = <span style="color: #66cc66;">&#123;</span><span style="color: #66cc66;">&#125;</span>;<br />
Greeting.<span style="color: #006600;">apply</span><span style="color: #66cc66;">&#40;</span>greeting, <span style="color: #66cc66;">&#91;</span><span style="color: #3366CC;">"Have a nice day!"</span><span style="color: #66cc66;">&#93;</span><span style="color: #66cc66;">&#41;</span>;<br />
greeting.<span style="color: #006600;">greet</span><span style="color: #66cc66;">&#40;</span><span style="color: #3366CC;">"Bob"</span><span style="color: #66cc66;">&#41;</span>;</div>
</div>
</div>
<p>
Don't be mistaken: this is a <em>very</em> cool feature, especially when used appropriately (and judiciously) in some well-known libraries:</p>
<div class="syntax_hilite">
<div id="javascript-75">
<div class="javascript"><span style="color: #003366; font-weight: bold;">var</span> names = <span style="color: #66cc66;">&#91;</span><span style="color: #3366CC;">"Charlie"</span>, <span style="color: #3366CC;">"David"</span>, <span style="color: #3366CC;">"Eva"</span>, <span style="color: #3366CC;">"Fanny"</span><span style="color: #66cc66;">&#93;</span>;<br />
$.<span style="color: #006600;">each</span><span style="color: #66cc66;">&#40;</span>names, <span style="color: #003366; font-weight: bold;">function</span><span style="color: #66cc66;">&#40;</span><span style="color: #66cc66;">&#41;</span> <span style="color: #66cc66;">&#123;</span><br />
&nbsp; &nbsp; greeting.<span style="color: #006600;">greet</span><span style="color: #66cc66;">&#40;</span><span style="color: #000066; font-weight: bold;">this</span><span style="color: #66cc66;">&#41;</span>;<br />
<span style="color: #66cc66;">&#125;</span><span style="color: #66cc66;">&#41;</span>;</div>
</div>
</div>
<p>
Problems arise when we already have <code>this</code> in our outer function, because it will be inaccessible ("overwritten") in the inner one. As a remedy, it is suggested to capture the outer <code>this</code> in a local variable that will come as part of inner function's closure:</p>
<div class="syntax_hilite">
<div id="javascript-76">
<div class="javascript">Greeting.<span style="color: #006600;">prototype</span>.<span style="color: #006600;">greetAll</span> = <span style="color: #003366; font-weight: bold;">function</span><span style="color: #66cc66;">&#40;</span>who<span style="color: #66cc66;">&#41;</span> <span style="color: #66cc66;">&#123;</span><br />
&nbsp; &nbsp; <span style="color: #003366; font-weight: bold;">var</span> self = <span style="color: #000066; font-weight: bold;">this</span>;<br />
&nbsp; &nbsp; $.<span style="color: #006600;">each</span><span style="color: #66cc66;">&#40;</span>who, <span style="color: #003366; font-weight: bold;">function</span><span style="color: #66cc66;">&#40;</span><span style="color: #66cc66;">&#41;</span> <span style="color: #66cc66;">&#123;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; <span style="color: #000066;">alert</span><span style="color: #66cc66;">&#40;</span><span style="color: #3366CC;">"Hello "</span> + <span style="color: #000066; font-weight: bold;">this</span> + <span style="color: #3366CC;">"! "</span> + self.<span style="color: #006600;">text</span><span style="color: #66cc66;">&#41;</span>;<br />
&nbsp; &nbsp; <span style="color: #66cc66;">&#125;</span><span style="color: #66cc66;">&#41;</span>;<br />
<span style="color: #66cc66;">&#125;</span>;</div>
</div>
</div>
<p>
Conventionally the variable is often called <code>that</code>, although <code>self</code> is also popular for somewhat <a href="http://docs.python.org/tutorial/classes.html#class-objects">obvious reasons</a> :)</p>
<h3><code>this</code> doesn't look pretty</h3>
<p>As we can see, <code>this</code> keyword in JavaScript is not quite as simple as in many other languages. Care must be taken when calling any function that uses it internally; otherwise we might end up with totally unintended results.</p>
<img src="http://feeds.feedburner.com/~r/xion/~4/duY9nsN0Fno" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://xion.org.pl/2012/03/18/how-does-this-work-in-javascript/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>Ideas Worth Spreading: Issue Tracking</title>
		<link>http://xion.org.pl/2012/03/11/ideas-worth-spreading-issue-tracking/</link>
		<comments>http://xion.org.pl/2012/03/11/ideas-worth-spreading-issue-tracking/#comments</comments>
		<pubDate>Sun, 11 Mar 2012 16:15:50 +0000</pubDate>
		<dc:creator>Xion</dc:creator>
				<category><![CDATA[Computer Science & IT]]></category>
		<category><![CDATA[Life]]></category>
		<category><![CDATA[Thoughts]]></category>
		<category><![CDATA[ideas]]></category>
		<category><![CDATA[issue trackers]]></category>
		<category><![CDATA[software engineering]]></category>
		<category><![CDATA[startups]]></category>

		<guid isPermaLink="false">http://xion.org.pl/?p=3431</guid>
		<description><![CDATA[Real-world metaphors are quite abundant when discussing topics related to IT and programming. They seem to be particularly useful when introducing newcomers, although it's equally easy to point out mature and established techniques that originate from non-digital world (OOP anyone?...). Regardless, the flow of ideas seems to be extremely one-directional, and I think that's very [...]]]></description>
			<content:encoded><![CDATA[<p>Real-world metaphors are quite abundant when discussing topics related to IT and programming. They seem to be particularly useful when introducing newcomers, although it's equally easy to point out mature and established techniques that originate from non-digital world (OOP anyone?...). Regardless, the flow of ideas seems to be extremely one-directional, and I think that's very unfortunate. There's wealth of concepts specific to IT or software industry that the general public would benefit from knowing about.</p>
<p>One of those, in my not-so-humble opinion, is the idea of <strong>issue tracking</strong>. I suppose vast majority of readers is intimately familiar with it but let's put it into words anyway, for the sake of clarity and explicitness. The concept revolves around a system which allows its users to create <em>tickets</em> describing various issues pertaining to some particular project, or part of it, or process, or any similar endeavor. Those tickets necessarily consist of title and content, very much like emails. Usually though, they also have few additional fields that are more <em>meta</em>, and describe the ticket itself. Typical examples include:</p>
<ul>
<li>A <strong>type</strong> or category for ticket. In software project, the distinction between <em>bug</em> and <em>feature request</em> is of utmost importance, although several more kinds of tickets (e.g. documentation-related tasks) are pretty well known too.</li>
<li><strong>Status</strong> of a ticket, indicating what's currently happening with the issue in question. Did it arise only recently, or some work on it has been already done? Maybe it was successfully resolved, or maybe there are more information needed to push the case further... Either way, that's what ticket status should tell us.</li>
<li>Person currently responsible for issue - the one it has been <strong>assigned to</strong>. For new issues, this field usually points at project manager, who is subsequently dividing work among members of their team.</li>
</ul>
<p>Lastly, every ticket allows for discussion in a forum-like manner, and for adding comments to any metadata changes we make.</p>
<p>That's it, in a nutshell. It doesn't seem very complicated and frankly, it may not sound very innovative either. Why do I think such a concept is worthy of attention in broader context, then?...<br />
<span id="more-3431"></span><br />
Because, pardon the pun, <em>it just works</em>.</p>
<p>Issue trackers solve a specific problem (managing cases and incidents related to software project) but the general premise behind them is much more general. They answer a practical question: <em>what should I do if I spot a problem</em>? Not every single little problem, of course, but a defect in a complex system: no matter how small a problem and no matter <em>what system</em> we're talking about, by the way.</p>
<p>Imagine you're driving a straight road going through some secluded area and you notice a speed limit sign. It tells you to slow down significantly for no apparent reason: there is nothing ahead (or around the road) that seems to require turtling. The sign feels out of place and is just confusing, but there is nothing you can do about it. You just leave it behind and maybe slow down slightly, just in case. Before too long, you forget about the whole thing.</p>
<p>Result? The sign will probably stay there <em>for years</em>, precisely because no one really knows what to do once they encounter it.</p>
<p>What could be done, though?... Well, how about pulling over, grabbing your phone, taking a picture of misplaced sign and using a mobile app to <strong>create a ticket</strong> about the whole issue? "<em>road 42 useless sign</em>" is certainly enough to adequately describe it, along with the photo and GPS coordinates. The whole process takes like two minutes, requires hardly any effort and only one person needs to do it for the issue to not go unnoticed.</p>
<p>On the other side, someone from the transport department (or similar institution) gets notified and glances at your ticket. It's hardly an emergency so it's given low priority and generous deadline but it's nevertheless assigned to appropriate person who should see to it being resolved. And it better is, for her job/salary/bonus/etc. depends on successfully dealing with cases such as these! And besides, if you're <em>really</em> interested, you can follow what's happening to your ticket, up until it is eventually resolved and closed.</p>
<p>A pleasant dream, isn't it? ;) To implement it in reality, there would obviously be numerous obstacles - with technical difficulties of constructing and managing a huge system being the <em>smallest</em> of them. For a massive crowd of people using it, we would certainly need to employ some clever moderation techniques - at least at the level of <a href="http://stackexchange.com/">StackExchange</a> communities. But what I foresee as the biggest challenge is to intertwine this novelty with existing bureaucracy. Starting small and showing tangible results in micro scale seems to be a mandatory first step.</p>
<p>So, are there any brave entrepreneurs who dare to tackle the challenge I hereby lay forth? :) Because, you know, back when the word 'disruptive' still had a meaning, this idea would probably deserve being called like that. But if you prefer the modern wording, then '<a href="http://paulgraham.com/ambitious.html">frighteningly ambitious</a>' is equally good. In any case, I wouldn't complain to see it happen...</p>
<img src="http://feeds.feedburner.com/~r/xion/~4/MJZLMVhqGeU" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://xion.org.pl/2012/03/11/ideas-worth-spreading-issue-tracking/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
	</channel>
</rss>

