Faruk Akgul

Things I Dislike In IntelliJ IDEA

2011-12-24T00:00:00-08:00

Don't get me wrong. I'm a big fan of IntelliJ IDEA and have been a happy user. I do believe it's the best IDE for Java but I've a very small dislike list;

Let's say I've this class; MyClass. Unfortunately, when I want to create an instance, IntelliJ doesn't autocomplete the generics. It only shows the MyClass part.

Here's an example;

   public abstract class SomeClass<K, T> implements Some<K, T> {
        public abstract void doSomething(K key, Node<K, T> root, Node<K, T> node);
    }

Let's say I've changed the Node part to Node accidentally. Of course, IntelliJ says there's an error, no problem with that. However, its recommendation list is nowhere relevant to project:

Change 'root' type to 'java.util.concurrent.ConcurrentLinkedQueue.Node'
Change 'root' type to 'java.util.concurrent.LinkedBlockingDeque.Node'
Change 'root' type to 'java.util.concurrent.LinkedBlockingQueue.Node'

which are completely irrelevant. In this scenario, problem occurs with the Node class which is in the project, not a Node class from another library. I'd expect to fix the actual Node class in the project as recommendation. If I'm not wrong, Eclipse automatically makes these changes in interfaces and other classes.

Eclim: Eclipse Meets Vim And Emacs

2011-09-09T00:00:00-07:00

Here's a great tool named Eclim which brings Eclipse to vim and there's an emacs port as well. I've tested for both vim and emacs and I think vim version is better than the emacs version. Emacs version needs work. I'm jealous.

Emacs Setup

Follow these instructions to install eclim.
Download emacs plugin from here.
Move downloaded emacs plugin to your .emacs.d/ Write the following to your .emacs;

(add-to-list 'load-path (expand-file-name "~/.emacs.d/eclim/vendor"))
(require 'eclim)
(setq eclim-auto-save t)
(global-eclim-mode)

Whenever you need to do Java development, first open a console and run the eclimd and then open emacs.

Cool Tools: ZeroMQ, Gevent, Websockets, node.js, web.py and Java

2011-08-12T00:00:00-07:00

In this post, I'll share my experiences with the cool technologies of the Internet by creating a simple real time application.

ZeroMQ: Message/Queue socket library (I'm a contributor) with async I/O used by Mongrel2. Unlike RabbitMQ, it doesn't need a broker.
Gevent: Co-routine based Python network library based on libevent which provides high level sync API.
WebSockets: One of the cool features of HTML5 which provides communication with browsers via a TCP.
node.js: Evented I/O based on V8.
web.py: A Python web framework which we'll use in this post via gevent's high performance WSGI server based on libevent-HTTP.
Java: An island of Indonesia.

So, we have all the cool tools together and we can begin our little project. The scenario is;

We have a Producer that does some calculations.
Producer sends the results of the calculations to Server.
Server sends those information to WebSocket server.
We serve those information via a simple web.py application powered by gevent.

For the sake of simplicity, we don't do any fancy calculations instead we'll just send some random values to connected users. Our ZeroMQ Producer and Server will be in Java whereas the rest of the will be in Python (and of course a little bit Javascript).

Producer

Here's our simple ZeroMQ'd producer.

   import org.zeromq.ZMQ;
    import java.util.ArrayList;
    import java.util.Random;

    public class Producer {
        private static ArrayList<String> list = new ArrayList<String>();
        static {
            list.add("Milla Jovovich");
            list.add("David Duchovyn");
            list.add("Naomi Watts");
            list.add("Keanu Reeves");
        }

        static String choice() {
            int random = new Random().nextInt(list.size());
            return list.get(random);
        }

        public static void main(String[] args) throws InterruptedException {

            ZMQ.Context context = ZMQ.context(1);
            ZMQ.Socket pub = context.socket(ZMQ.REQ);
            pub.connect("tcp://localhost:6060");
            while (true) {
                String msg = choice();
                pub.send(msg.getBytes(), 0);
                String reply = new String(pub.recv(0));
                System.out.println("reply: " + reply);
                Thread.sleep(1000);
            }
        }
    }

What we do here is, we send random data from the list and send it server.

Server

The ZeroMQ'd server takes the data from Producer and sends it WebSocket. We also send back the data to Producer (no need to. I can't remember why I did it in the first place but let's hope I had a good reason).

   import org.zeromq.ZMQ;

    public class Server {

        public static void main(String[] args) throws InterruptedException {
            ZMQ.Context context = ZMQ.context(1);
            ZMQ.Socket sub = context.socket(ZMQ.REP);
            ZMQ.Socket pub = context.socket(ZMQ.PUB);

            sub.bind("tcp://*:6060");
            pub.bind("tcp://*:3030");
            while (true) {
                String msg = new String(sub.recv(0));
                System.out.println("received: " + msg);
                sub.send(msg.getBytes(), 0);
                pub.send(msg.getBytes(), 0);
                Thread.sleep(1000);
            }

        }
    }

Gevent WebSocket Server

Here, gevent waves its magic wand. We connect to ZeroMQ server and once we get some data from the ZeroMQ server, we send it to our web application via wsgi.websocket. This is a gevent WSGI server.

   import zmq
    import gevent
    from geventwebsocket.handler import WebSocketHandler
    from gevent.pywsgi import WSGIServer

    def app(environ, start_response):
        context = zmq.Context()
        ws = environ['wsgi.websocket']
        socket = context.socket(zmq.SUB)
        socket.setsockopt(zmq.SUBSCRIBE, '')
        socket.connect('tcp://localhost:3030')
        while True:
            msg = socket.recv()
            ws.send(msg)
            gevent.sleep(0.1)

    if __name__ == '__main__':
        print 'here we go...'
        server = WSGIServer(('', 4040), app,
        handler_class=WebSocketHandler)
        server.serve_forever()

And now, we can move on to our web application part, a simple web.py application.

web.py via gevent WSGI

Here's our simple gevent powered web.py application. Look out to monkey.patch_all() which turns thread-local storage of web.ctx into a greenlet-local storage.

   import web
    from gevent import monkey
    from gevent.pywsgi import WSGIServer

    monkey.patch_all()

    class Index(object):
        def GET(self):
            return """
                <div id="place"></div>
                <script>
                    var ws = new WebSocket('ws://localhost:4040');
                    var place = document.getElementById('place');
                    ws.onopen = function(e) {
                        document.write('connected');
                    };
                    ws.onmessage = function(e) {
                        place.innerHTML = e.data;
                    }
                </script>
            """

    urls = (r'^/', 'Index')
    app = web.application(urls, globals()).wsgifunc()

    if __name__ == '__main__':
        WSGIServer(('', 8080), app).serve_forever()

Here's a small problem; WebSocket isn't supported by all browsers. For example; Firefox disables it by default whereas Google Chrome supports it. In this case using socket.io, Faye (based on Bayeux protocol) for cross-browser support might be a better choice (or we could write a WebSocket server in node.js).

I must say that I'm quite getting started to warm up with these cool tools. I frequently make the same mistake which is I connect to wrong port and look at the blank screen while wondering "Why am I seeing a blank screen?" so my gentle advice would be double check the port you're connecting :)

SQLAlchemy: The ORM

2011-07-23T00:00:00-07:00

Long story short; SQLAlchemy is an ORM for Python. SQLAlchemy is far more complex than Django's ORM. You can actually call stored functions in SQLAlchemy. For a Django vs. SQLAlchemy article, have a look at Armin Ronacher's post.

There are two different ways to define a new table. One is; defining a table within the MetaData. Second one is; creating table, class and mapper with Declarative Base. I'm more like a declarative_base guy and I must say that I've never used MetaData.

Warming Up The Engine

This might be a way to define the SQLAlchemy engine. In SQLAlchemy, everything is done via a Session().

from sqlalchemy import (Column, SmallInteger, Unicode, UnicodeText,
    CHAR, VARCHAR, DateTime, ForeignKey, create_engine)
from sqlalchemy.orm import (scoped_session, sessionmaker, relation, relationship)
from sqlalchemy.ext.declarative_base import declarative_base

First, we imported data types such as SmallInteger, Unicode, VARCHAR etc. In the second line, we imported sessionmaker and relations which we use in our tables since SQLAlchemy won't wave its magic wand to define relations between tables. Third line; the declarative_base.

engine = create_engine(ENGINE, echo=True, pool_recycle=3600)
Base = declarative_base()
Session = scoped_session(sessionmaker(bind=engine, autoflush=True,
    autocommit=True, expire_on_commit=True))
session = Session()

Above code defines the engine, and declarative_base class and the session. Once the setup is complete, we need to create the table classes.

class User(Base):
    __tablename__ = 'user'
    __table_args__ = {'mysql_charset': 'utf8', 'mysql_engine': 'InnoDB'}

    uid = Column(CHAR(32), primary_key=True)
    name = Column(Unicode(255), nullable=False)
    username = Column(VARCHAR(32), nullable=False, unique=True)
    password = Column(CHAR(64), nullable=False, index=True)
    registered = Column(DateTime, default=datetime.utcnow())

    def __init__(self, name, username, password):
        self.uid = uuid4().hex
        self.name = name
        self.username = username
        self.password = password

    def __repr__(self):
        return u'<User (%s, %s)>' % (self.uid, self.username)

Here, we define a new class named User. Remember, SQLAlchemy expects you to explicitly define tablenames and all table options so we define the tablename as user.

Then we defined the table options. In table_args we explicitly stated to SQLAlchemy that this table uses utf8 as charset and it uses InnoDB as the engine.

Then we defined the table fields. nullable=False says this field won't accept NULL. unique=True says this field is unique indexed. default=datetime.utcnow() says field's default value.

Many-To-Many, Many-To-One

Let's say we've simple blog where users' post their articles.

class Article(Base):
    __tablename__ = 'article'
    __table_args__ = {'mysql_charset': 'utf8', 'mysql_engine': 'InnoDB'}

    aid = Column(CHAR(32), primary_key=True)
    author_id = Column(CHAR(32), ForeignKey('user.uid', ondelete='cascade'))
    title = Column(Unicode(255), nullable=False)
    content = Column(UnicodeText, nullable=False)
    posted = Column(DateTime, default=datetime.utcnow())

    author = relation(User, primaryjoin=User.uid == author_id, lazy='select')

    # Let's assume there's also a Comments table.
    # In this case we could make a relation between tables by ordering the results.
    comments = relation('Comments', order_by='Comments.posted.desc()', backref='article')

In the above example, we create a relation to User table. To create a many-to-many tables, we need to create the classes and the necessary association class.

CRUD Operations

CRUD is simple.

Create

For example, let's insert some data to Article class.

def insert_article(args):
    res = Article(**args)
    session.add(res)
    session.commit()
    return res

Retrieve

For example; let's retrieve an article with the specified article_id.

def retrieve_article(article_id):
    res = None
    try:
        res = session.query(Article).filter(Article.aid == article_id).first()
    except (NoResultFound):
        session.rollback()
    return res

Another example which we return some articles of a user.

def retrieve_user_articles(user_id):
    res = None
    try:
        res = session.query(Article).filter(Article.author_id == user_id).order_by(Article.posted.desc()).all()
    except (NoResultFound):
        session.rollback()
    return res

Update

def update_article(article_id):
    res = session.query(Article).get(article_id)
    res.title = 'New Title'
    session.add(res)
    session.commit()
    return res

Delete

For example, let's delete an article with specified article_id.

def delete_article(article_id):
    try:
        res = session.query(Article).filter(Article.aid == article_id).delete(synchronize_session='fetch')
        session.commit()
    except (NoResultFound):
        session.rollback()
    return res

SQL/Stored Functions

Calling SQL functions such as MySQL's COUNT() or your own stored functions is very easy with SQLAlchemy.

Let's say we want to count the number of rows in the table.

from sqlalchemy.sql import func

def count_rows():
  res = session.query(Article.aid, func.count(Article.aid).label('rows')).all()
  return res

label stands for SQL's AS (for example; SELECT value AS data FROM table. label is the AS data part). Calling your stored functions is precisely the same as calling SQL functions. Assume that we have a stored function named my_func. In this case, the proper SQLAlchemy code would be; func.my_func(arg).

I didn't cover joins and some other more SQLAlchemy features but SQLAlchemy is an endless sea.

A Gentle Intro To Backbone.js: Part 1

2011-06-28T00:00:00-07:00

Backbone.js is a javascript framework which follows a MVC pattern (I couldn't find any better way to explain what it's). It makes your web application incredibly easy to maintain.

Model

Models are the fundamental structure of a Javascript application. Creating a model in Backbone is plain simple;

var Person = Backbone.Model.extend({
});

In order to create our own model, we need to extend Backbone.Model and define properties of our own model. You don't have to define initialize (but you'll when working in a real example) but if you define initialize, it'll be called when the model is created.

Getters And Setters

When we create a new instance of our model, we need to set some values.

Setters

  var person = new Person({name: 'Milla Jovovich'});
  // or
  var person = new Person();
  person.set({name: 'Milla Jovovich'});

Getters

To get an attribute;

person.get('name');

You could check the documentation of Backbone to see the methods of models. For example; to create a JSON format of the model, we could simply do;

JSON.stringify(person);

Here's our Person model with some methods defined.

var Person = Backbone.Model.extend({
    defaults: function() {
      'name': 'Milla Jovovich',
      'age': 36,
    },

    changeName: function(name) {
      this.set({name: name});
    },

    clear: function() {
      this.destroy();
    },

  });

View

Views identify how your models look like. Backbone.js needs Underscore.js. Creating a view is simple;

  var PersonView = Backbone.View.extend({
    initialize: function() {
      _.bindAll(this, 'render');
    },

    render: function() {
      this.el.html(_.template($('#some_html_container').html()));
      // or
      // this.el.html(_.template(this.model.toJSON()));
      return this;
    },
  });

If the element we want to work on already defined in the DOM then;

var person_view = new PersonView({el: $('#container')});

Controller

Controllers define how our models interact. Backbone.Controller provides methods for routing. For example; application such as Twitter change the URL via #path (For example; http://twitter.com/#!/MillaJovovich). Here's a simple Controller example;

  var PersonController = Backbone.Controller.extend({
    routes: {
      'about': 'about',
      'search/:query': 'search',
    },

    about: function() {
      alert('A Message');
    }

    search: function(query) {
      this.doSearch(query);
    },

    doSearch: function(query) {
      $.getJSON('/ajax/search?q='+e(query), function(data) {
        $('#result_container').html(data);
      });
    }

  });

routes table is similar to Google App Engine's, Django's, or web.py's way of defining URLs. The above routes will match; http://domain/#about and http://domain/#search/milla.

As always, we need to create an instance;

var person_controller = new PersonController();
Backbone.history.start();

Backbone.history is a global router. It's mandatory to start it.

You may want to check out Thomas Davis' Backbone.js Tutorials website which you could find better explanations than this post.

Storing Passwords: Password Hashing

2011-06-25T00:00:00-07:00

Another Sony hacking (hacking Sony has become a tradition) revealed approximately 1 million passwords which are stored in plain text. Storing passwords in plain text is a very stupid idea.

The least we could do is that a password needs to be hashed and salted. We could either create our own mechanism or taking advantage of libraries which support these things out of the box. For example; Apache Shiro. For those who may not be familiar with Apache Shiro; It's a Java security library which makes authentication, authorization and session management easy. What makes Apache Shiro special is its hashing mechanism also supports iterations.

Password Hashing With Apache Shiro

Apache Shiro supports MD2, MD5, SHA1, SHA256, SHA384 and SHA512 hashing. Since it also supports iterations in its hashing mechanism, we could simply do;

Sha512Hash s = new Sha512Hash("milla", "jovovich", 1024);

And Apache Shiro will create a SHA512 value for password "milla" salted with "jovovich" and iterated 1024 times (random salt and iteration would be a good approach). By default Apache Shiro iterates 1 time.

Here's a very simple example where the scenario is to store the username - password combination in an auth.ini.

Factory<SecurityManager> factory = new IniSecurityManagerFactory("/tmp/auth.ini");
SecurityManager manager = factory.getInstance();
SecurityUtils.setSecurityManager(manager);
Subject user = SecurityUtils.getSubject();

try {
    Sha512Hash s = new Sha512Hash("milla", "jovovich", 1024);
    UsernamePasswordToken token = new UsernamePasswordToken("milla", s.toString());
    System.out.println(s.toString());
    user.login(token);
} catch (IncorrectCredentialsException e) {
    // password is wrong
} catch (UnknownAccountException e) {
    // username is wrong
}
// should say "true"
System.out.println("user is authenticated " + user.isAuthenticated());

Strange Behavior Of SQLite In Different OSes

2011-04-15T00:00:00-07:00

When I was doing some SQLite I've noticed, accidentally, its SUBSTR() function behaves differently in different OSes.

Consider the following example;

import sqlite3
con = sqlite3.connect('test.db')
c = con.cursor()
r = c.execute('SELECT SUBSTR(post, 0, 30) FROM entries WHERE user=1 LIMIT 1').fetchone()
len(r[0])

The above code returns 29 in OSX and Linux but not in Windows. Unlike OSX and Linux, it returns 30 in Windows. Some may say, it's not a bug since SUBSTR() expects the lowest start integer as 1 but this strange behavior creates confusion. If it didn't work in the first place when I was testing the code in OSX and Linux, I'd think I'm doing something wrong and fix the problem immediately but it has worked which led me to believe that my code was bug-free (in fact I created the bug).

For example, consider the following query in OSX and Linux;

SELECT SUBSTR(post, 0, 31)

The above query returns exactly the same value when you type the following query in Windows;

SELECT SUBSTR(post, 0, 30)

They're all buggy, the correct query should be;

SELECT SUBSTR(post, 1, 30)

This creates a confusion. They make you believe that you're writing the correct code whereas you're not. SUBSTR() functions of PostgreSQL and MySQL also start with 1 but when you code in programming languages, SUBSTR() starts with 0 so you might make such mistakes sometimes and this was my mistake but since it has worked in SQLite I thought I was doing right (but I wasn't).

Just curiosity, I've decided to look at the source code of SQLite to see if I can figure out what's causing this confusion. SUBSTR() function of SQLite is in sqlite.c and it starts on line 81727 (Yeah, I just grabbed the entire thing).

Line 81748; i64 p1, p2;

These two values define the [start, end] part of the SUBSTR function. They're in i64.

Line 81758; p1 = sqlite3_value_int(argv[1]);

This defines the [start] part of SUBSTR.

Line 81814; sqlite3_result_text(context, (char*)z, (int)(z2-z), SQLITE_TRANSIENT); (int) (z2-z) defines the total length of the substring.

After the line 81758 I've written if(p1 == 0) p1 = 1;. I know it's stupid but it does the trick. When you compile it, SUBSTR starts to behave in the same way.

The reason I've written specifically if(p1 == 0) is, according to comment on line 81735, p1 is calculated as abs(p1) so if(p1 < 1) might break it.

After I've shared this with someone else, he sent me http://www.sqlite.org/cgi/src/fdiff?v1=e536218d193b8d32&v2=1ddc1c93a0bb3509.

Apparently, I've just reinvented the wheel. My ignorance made me waste quite amount of time for a rookie mistake.

Hacking Python: Adding A New Keyword

2011-03-28T00:00:00-07:00

As you may know Python and Ruby are very similar languages of course there are huge differences but they're influenced from the same ideas but every language has unique keywords such as Python's pass.

For example, in Ruby, there are some keywords which aren't in Python such as unless and end. I thought it'd be fun to implement these keywords to Python (I'm not suggesting that such keywords should exist in Python). I've decided to implement the unless keyword of Ruby to Python.

How "unless" works

Here's a simple example how unless works in Ruby;

def test(n)
  return n unless n < 6
end

Ruby has a very flexible syntax so this one is also valid;

def test(n)
  unless n < 6
    return n
  end
end

Let's Start Hacking Python

Here we go. We're gonna implement Ruby's unless keyword to Python by following the instructions of PEP-0339^3.

Go to python.org and download the Python 2.7.1 source code.
Extract it to somewhere and go into that folder.
????
Profit.

Python's Parser Tree is an LL Parser^1 which puts the source code to a parse tree to define the grammar rules. The reason behind this action is so compiler can traverse the tree. The grammar file of Python is in Grammar/Grammar file. In this file we see all the grammar rules.

For example; for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite]

can be a rule for;

for k, v in mydict.items():
  print v
else:
  print 'a message'

We need to define the rule for our unless keyword. After if_stmt rule, write the following;

unless_stmt: 'unless' test ':' suite ['else' ':' suite]

We need to put unless_stmt that we've just defined to compound_stmt list so put it. For example;

compound_stmt: if_stmt | unless_stmt | ....

Abstract Syntax Trees(AST)

ASTs are a tree structure to represent the abstract syntax structure of the source code of a programming language^2.

ASTs create a clean interface between the parser and the compiler and make it ease to compile. Java, Python, C# (technically all modern languages) take advantage of ASTs but legacy languages such as C couldn't take the advantage of them since memory was a big issue.

I'm finishing ASTs short so I'm not going to say anything about how to track the position, or doing the parsing and analysis of semantics, maybe in another post.

The definition of AST nodes for Python is found in Parser/Python.asdl^3. Open that file and find the part of for, while, if, with and write the following there;

| Unless(expr test, stmt* body, stmt* orelse)

The AST tree that is generated is in Python/ast.c. Open that file and search for ast_for_stmt function where you'll see the functions that are called according to their statement types. (Remember, our unless_stmt is in compound_stmt), and find the switch/case where the if_stmt, while_stmt... are defined(basically the statements of compound_stmt) and add the following line

case unless_stmt:
    return ast_for_unless_stmt(c, ch);

After that, we need to define the ast_for_unless_stmt function which is the following function;

static stmt_ty
ast_for_unless_stmt(struct compiling *c, const node *n)
{
    /*
      unless_stmt: 'unless' test ':' suite ['else' ':' suite]
     */
    char *s;

    REQ(n, unless_stmt);

    s = STR(CHILD(n, 4));

    if (NCH(n) == 4) {
        expr_ty expression;
        asdl_seq *suite_seq;

        expression = ast_for_expr(c, CHILD(n, 1));
        if (!expression)
            return NULL;
        suite_seq = ast_for_suite(c, CHILD(n, 3));
        if (!suite_seq)
            return NULL;

        return Unless(expression, suite_seq, NULL, LINENO(n), n->n_col_offset,
                  c->c_arena);
    }

    if (s[2] == 's') {
        expr_ty expression;
        asdl_seq *seq1, *seq2;

        expression = ast_for_expr(c, CHILD(n, 1));
        if (!expression)
            return NULL;
        seq1 = ast_for_suite(c, CHILD(n, 3));
        if (!seq1)
            return NULL;
        seq2 = ast_for_suite(c, CHILD(n, 6));
        if (!seq2)
            return NULL;

        return Unless(expression, seq1, seq2, LINENO(n), n->n_col_offset,
                  c->c_arena);
    }

    PyErr_Format(PyExc_SystemError,
                 "unexpected token in 'unless' statement: %s", s);
    return NULL;
}

Once we've created the AST, the next step is to create the CFG (Control Flow Graph). Just read the 4th part, which I'm using to do this, to understand CFG.

To create the CFG, open the file Python/compile.c and add the following function somewhere (Just follow the structure of compound_stmt: to make the things easier).

static int
compiler_unless(struct compiler *c, stmt_ty s)
{
    basicblock *end, *next;
    int constant;
    assert(s->kind == Unless_kind);
    end = compiler_new_block(c);
    if (end == NULL)
        return 0;

    constant = expr_constant(s->v.Unless.test);
    /* constant = 0: "if 0"
     * constant = 1: "if 1", "if 2", ...
     * constant = -1: rest */
    if(constant == 0) {
      VISIT_SEQ(c, stmt, s->v.Unless.orelse);
    }
    else if(constant == 1) {
      VISIT_SEQ(c, stmt, s->v.Unless.body);
    } else {
      if(s->v.Unless.orelse) {
        next = compiler_new_block(c);
        if(next == NULL) return 0;
      } else {
        next = end;
      }
      VISIT(c, expr, s->v.Unless.test);
      ADDOP_JABS(c, POP_JUMP_IF_TRUE, next);
      VISIT_SEQ(c, stmt, s->v.Unless.body);
      ADDOP_JREL(c, JUMP_FORWARD, end);
      if(s->v.Unless.orelse) {
        compiler_use_next_block(c, next);
        VISIT_SEQ(c, stmt, s->v.Unless.orelse);
      }
    }
    compiler_use_next_block(c, end);
    return 1;
}

You'll notice this is very similar to compiler_if function but unless and if are very similar so I almost copy/pasted the compiler_if function :)

So, that's it. just run ./configure && make and you'll have a custom built Python.

Here's a sample usage of our unless keyword;

def test(n):
  unless n < 4:
    print n
  else:
    print 'a message'

References

LL Parser, http://en.wikipedia.org/wiki/LL_parser
AST, http://en.wikipedia.org/wiki/Abstract_syntax_tree
PEP-0339, http://www.python.org/dev/peps/pep-0339/
PEP-0339, http://www.python.org/dev/peps/pep-0339/#ast-to-cfg-to-bytecode

Google App Engine - The Missing Libraries: Python Socket Library

2011-02-10T00:00:00-08:00

As you may know Google App Engine does not allow developers to use the imp, ftplib, select, socket libraries. As stated in the documentation, they are empty.

However I needed to use the inet_aton and inet_ntoa functions from the socket library. In order to use these functions I had to implement these two functions and I have decided to code the rest of the functions and put them on GitHub. If I needed, someone else might need them as well.

Long story short, here's a mini project, gae-missing-libraries, which I have implemented the socket library's conversion functions.

Feel free to use, fork or whatever you want to do with it. If there's already a library exist in GAE somewhere then forgive my ignorance and ignore this mini library.

The Sleeping Beauty: Emacs

2010-10-11T00:00:00-07:00

I think everyone would agree on that there are only two text editors in the history of mankind: emacs and vim. I'm not going to say which one is better than the other.

I'm a programmer and I work with files everyday so just like every other developer I care about the editor I use and how it looks to increase my efficiency.

Configuring Emacs - Fullscreen Mode

I use emacs in fullscreen mode but unfortunately Cocoa Emacs 23 doesn't support fullscreen mode. However, I found out a Japanese guy (Daisuke Murase) has patched it and put it on github (ありがとう、ダイスケ！)

Here are the instructions to build it;

git clone git://github.com/typester/emacs.git
cd emacs
./configure --with-ns
make bootstrap
make install
mv nextstep/Emacs.app /Applications

The next you should do is to bind a key for fullscreen mode in your .emacs file.

(global-set-key (kbd "BIND_A_KEY") 'ns-toggle-fullscreen)

Customizing The Emacs

If you don't want to see the welcome message every time you open emacs;

(setq inhibit-startup-message t)

To highlight the line which contain the cursor in every buffer;

(global-hl-line-mode 1)

I use indent space as 2 rather than 4;

(setq standard-indent 2)

Emacs uses both spaces and tabs to indent lines but I prefer only spaces.

(setq-default indent-tabs-mode nil)

To show line number and column number;

(line-number-mode 1)
(column-number-mode 1)

I don't want Emacs to backup the files.

(setq make-backup-files nil)

To swap the windows;

(defun swap-windows ()
  (interactive)
  (cond ((/= (count-windows ) 2)
         (message "you need 2 windows"))
        (t
         (let* ((w1 (first (window-list)))
                (w2 (second (window-list)))
                (b1 (window-buffer w1))
                (b2 (window-buffer w2))
                (s1 (window-start w1))
                (s2 (window-start w2)))
           (set-window-buffer w1 b2)
           (set-window-buffer w2 b1)
           (set-window-start w1 s2)
           (set-window-start w2 s1))))
  (other-window 1))

(global-set-key (kbd "C-c s") 'swap-windows)

I look at Emacs window every single day for long hours so it must look pretty to my eyes. For that, I use molokai theme for emacs.

(load "~/.emacs.d/plugins/color-theme-molokai.el")
(color-theme-molokai)

And you need to choose a decent monospaced font. Inconsolata, Consolas, Monaco, Menlo are good choices whereas Courier, Courier New are not. Menlo has become Mac OS X's default monospace font since Snow Leopard.

(set-default-font "Menlo-12")

I always prefer some transparency in my editor and console. Like, when I type pseudocode, say in Java, it's good to have transparency. 85% transparency level is enough for me.

(eval-when-compile (require 'cl))
 (defun toggle-transparency ()
   (interactive)
   (if (/=
        (cadr (find 'alpha (frame-parameters nil) :key #'car))
        100)
       (set-frame-parameter nil 'alpha '(100 100))
     (set-frame-parameter nil 'alpha '(85 60))))
 (global-set-key (kbd "C-c t") 'toggle-transparency)

That's pretty much covers my .emacs file. I use YASnippet, SLIME, Org-Mode and some other plugins which I didn't cover in this post.