Esistgut

A Django / jQuery Mobile application for the Raspberry PI

Fri, 22 Aug 2014 11:41:00 +0000

This is a late post about a skeleton project I published some time ago: django-rpi-jqm-sample. It features a sample application composed of two parts: the first one is a standard Django application, the second one is a stand-alone daemon. The daemon should be started before the Django application, you can find the details in the above Youtube video.

The two components communicate with each other using the Django’s cache framework. Every time the jQuery Mobile form is submitted the data is pushed to the cache:

def form_valid(self, form):
    cache.set('servo', form.cleaned_data['servo'])
    cache.set('led1', form.cleaned_data['led1'])
    cache.set('led2', form.cleaned_data['led2'])
    return super(Demo1View, self).form_valid(form)

The same happens the other way around, every time the form is initialized it gets its initial data from the cache:

def get_initial(self):
    servo = cache.get('servo', 0)
    led1 = cache.get('led1')
    led2 = cache.get('led2')
    initial = {'servo': servo, 'led1': led1, 'led2': led2}
    return initial

The daemon will do continous polling on the cache data and take proper actions when changes are detected:

while True:
    cache_tmp['servo'] = cache.get('servo', "0")
    if cache_cur['servo'] != cache_tmp['servo']:
        servo_control.set_servo(25, int(1200/60*float(cache_tmp['servo']))+900)
        cache_cur['servo'] = cache_tmp['servo']

Of course the cache framework is not supposed to be used like this, a proper message queueing solution like RabbitMQ is usually the way to go for this kind of problems, it just seemed a little bit overkill in this particular context so I opted for the cache framework hack .

Blog moved to GitHub Pages

Thu, 21 Aug 2014 17:00:00 +0000

I just migrated my old blog from WordPress.com to GitHub Pages. It is built using Jekyll and the theme is based on Start Bootstrap blog templates, Bootstrap, Font Awesome and Google Fonts.

The old domain is still active but for now it will work only as a redirect to this one.

It is my first time using a static content generator but the Jekyll platform seems nice, I look forward on adding some more features like a tag cloud or an automatic résumé generator.

Corbucci CMS blog admin preview

Wed, 20 Feb 2013 00:20:00 +0000

This is how the admin for the CMS I’m developing looks right now.

Dynamic URLs with Django

Fri, 08 Feb 2013 11:51:00 +0000

While implementing a configurable permalink structure feature for the Corbucci CMS project I had the need to inject new URLs in the Django routing mechanism. As the URLs are normally designed to fit in the hardcoded settings.py I had to find another way to edit them at runtime and a middleware was the way to go: as you can read in the documentation the process_request allows to define an urlconf attribute, when processing the request Django will search for the .urlconf.urlpatterns attribute and this will be used as the “standard URLs” for the routing.

This is how I implemented it:

class GlueMiddleware(object):
    def process_request(self, request):
        urlpatterns = patterns('',
            url(r'^articles/$', ArticleListView.as_view()),
            (r'^', include('urls')),
        )
        u = type('DynUrlConf', (object,), dict(urlpatterns = urlpatterns))
        request.urlconf = u

Lines 3-6 contain the URLs in the standard format for Django, line 4 is the “dynamic” one, line 5 just include the previous ones (as this will override them). Lines 7 is used to create a class with the urlpatterns attribute at runtime, check the type function documentation for further details.</pre>

NAS with Ruby, NILFS2 and Samba

Mon, 07 Feb 2011 21:10:00 +0000

Here comes a simple script I wrote to setup a couple of file servers using Linux and NILFS2 with daily incremental snapshots, sharing of the snapshots via Samba and optional sync on an external NTFS hard disk on USB: nilfs2-ruby-nas.

As usual be extremely careful with this code as it did not receive proper testing and should be seen as a starting point for your own setups rather than a ready to use solution.

Take a look on the class I wrote to run NILFS2 commands, it isn’t exactly fail proof :-D:

class NILFS2
    def initialize(device)
        raise IOError, "can't find device file" unless File.exists?(device)
        @device = device
    end
    
    def get_checkpoints()
        t = `lscp #{@device}`
        r = Array.new
        t.split("\n")[1..-1].each do |l|
            a = l.split
            time = Time.parse("#{a[1]} #{a[2]}")
            r.push({:CNO => a[0], :MODE => a[3], :FLG => a[4],
            :NBLKINC => a[5], :ICNT => a[6], :time => time})
        end
        return r
    end

    def get_snapshots()
        r = Array.new
        get_checkpoints.each { |c| r.push(c) if c[:MODE] == "ss" }
        return r
    end

    def ss_to_mount(snapshot, path)
        return {:dev => @device, :path => path, :fs => "nilfs2", :opts => {"ro"=>nil, "cp" => snapshot[:CNO]} }
    end
    
    def make_checkpoint(snapshot=false)
        snapshot ? `mkcp -s #{@device}` : `mkcp #{@device}`
    end
    
    def make_snapshot()
        make_checkpoint(true)
    end
    
    def remove_checkpoint(checkpoint)
        `chcp cp #{@device} #{checkpoint[:CNO]}` if checkpoint[:MODE] == "ss"
        `rmcp #{@device} #{checkpoint[:CNO]}`
    end
    
    def get_total_space()
        `df -Pk #{@device} |grep ^/ | awk '{print $2;}'`.to_i * 1024
    end
    
    def get_used_space()
        `df -Pk #{@device} |grep ^/ | awk '{print $3;}'`.to_i * 1024
    end

    def get_free_space()
        `df -Pk #{@device} |grep ^/ | awk '{print $4;}'`.to_i * 1024
    end
    
    def get_free_space_percent()
        #total:100=used:x
        get_free_space()*100/get_total_space()
    end
end

Throwing some sources

Sat, 11 Oct 2008 11:06:00 +0000

I just made public under the terms of the GPLv3 a couple of old university projects hoping they will help someone. Here they are: ufs (micro file system, a fat-alike user space file system with some unix extensions) and Vespasiano (small movie theater manager, done when learning sockets in Java).

Please note that they both are so bugged to be broken. Be careful :-).

Cerberus Project

Wed, 01 Oct 2008 19:55:00 +0000

What is this cool drooling puppy doing on my blog? His name is Cerberus, he came from Greek mithology and he is nothing less than the guardian of the gate to Hades, also known as Hell.

No, I haven’t joined satanism, this is just a mascotte, the mascotte for a new project I’ve been thinking on lately: a Debian based Linux distribution specialized on video surveillace, its name will obviously be Cerberus.

Why am I building another Linux distribution? Well, the answer is very simple: in the last two years I’ve been selling video surveillance systems build on top of Debian or Ubuntu with Motion and Vigilante. Assembling such systems requires to repeat each time a set of very specific steps such as installing the distribution with the needed packages, configuring the modprobe options for driver of the video capture card (often based on the bt878 chip), setup Motion and Vigilante, linking it all together on the user’s desktop. Most of these actions (excluding hardware probing/configuration) could be automatized, or better, saved prebuild in a proper environment. This is where Cerberus Linux enters the game: it is meant to act like firmwares on embedded Linux devices, a main read-only partition containing the distribution with a small overlay read-write (freezable to read-only) partition containing the configuration files. Why am I following the “embedded-firmware”-like way in place of a more common standard installation? First reason is robustness: making the system read-only prevents damage to the file system when doing hard resets (which are very common on this kind of systems, often seen as vhs recorders or table dvd players by the inexperienced users who will use them). Second reason is easy of use: you can (re)install and update the base system by rewriting the partition without caring about backups or installation process, is it just a call to dd, automatizable. Third reason: users are idiots, give them a writeable home and they will find a way to mess up the entire system, give them a writeable root file system and they will bring back the system as an ash heap claiming you sold them a broken thing.

So the roadmap so far is to build the distribution with the listed features, put a new Vigilante (based on Cluttermm/Gstreamermm) on it and maybe develop a remote web configuration system (maybe with ExtJS and Rails or Zend Framework or Webtoolkit).

Of course if you have any advice or proposal for Cerberus please contact me, any good idea will be welcomed.

SQLite3 support for Motion

Tue, 30 Sep 2008 14:49:00 +0000

One of the first problem occurred when developing the specialized video surveillance Linux distribution I’ve been working on these days is the database engine to be used to archive the list of recording files saved by Motion. As every write operation will be placed on a read-write file system layer on top of the main read-only layer (containing the whole “monolithic” distribution, something like the firmware on an embedded device) I have the need to store the whole database data in a compact manner, something like only one file and not similar to the /var/lib/mysql/ directory with its many entries. The problem with this is that Motion only supported MySQL and PostgreSQL so I had to add the SQLite layer by myself. As a result I have published a patch on the Motion’s wiki, test it please :-).

What I don’t like in Ruby

Mon, 07 Apr 2008 11:02:00 +0000

I’m beginning to learn the well hyped Ruby language, in many ways it looks useful, elegant and pleasant, still it has something that, at least at a first glance, looks odd to me. I will use this post to take track of those odd things that I will meet when studying the language.

So, here we go:

1) Ends, take a look on this snippet:

if !r.nil? then
  rr = r.right
  if !rr.nil? then
    if ((r.color == RedBlackTree::RED)and(rr.color == RedBlackTree::RED)) then
      x = rot_left()
      copy(x) unless x.nil?
      @color = RedBlackTree::BLACK
      l = @left
      l.color = RedBlackTree::RED unless l.nil?
    end
  end
end

and here comes the Python equivalent:

]if (right != None):
    rr = right.__right
    if (rr != None):
        if ((right.__color == RedBlackTree.red)and(rr.__color == RedBlackTree.red)):
            x = self.__rotLeft()
            if (x != None):
                self.__copy(x)
            self.__color = RedBlackTree.black
            left = self.__left
            if (left != None):
                left.__color = RedBlackTree.red

The Python way looks just the best way in my opinion. What’s the reason for the ends when you’ll indent the code anyway (and if you don’t then you deserve nothing else than suffering and pain)? Even the { C/C++/Java approach } appears way better or at least less intrusive.

2) “Fuzzy” expressions terminators. Try to compile and execute the following C++ code:

#include <iostream>
using namespace std;

class Foo {
    public:
    void bar() {
        int x = 3;
        int y = 2;
        cout << x
        +y;
        cout << endl;
    }
};

int main() {
    Foo *f = new Foo;
    f->bar();
    delete(f);
    return 0;
}

What do you expect it to print? Right, it will correctly print 5. Now Ruby’s turn:

class Foo
  def bar
    x,y = 3,2
    puts x
    +y
  end
end

f = Foo.new()
f.bar

Now what output do you expect? The answer is 3. This is because Ruby doesn’t use semicolons as C, C++, Java, PHP and many other languages do to terminate statements. It is still the better way to go, semicolons are redundants useless things almost everytime, still you have to pay attention: note that the above Ruby code isn’t wrong so the interpreter will not warn you, it will just print 3 instead of 5 and the method bar will return the value2 (remember that in Ruby the last expression in a method or function is the actual return value of it so if you replace f.bar with puts f.bar it will print the “missing” 2).

3) No method overloading. The following code is self-explanatory:

class Foo
    def bar(a, b)
        puts b
    end

    def bar(a)
        puts a
    end
end

gh = Foo.new()
gh.bar("hello", "world")

if you try to execute it then you will get something like this:

asd.rb:12:in `bar’: wrong number of arguments (2 for 1) (ArgumentError) from asd.rb:12

4) GIL (global interpreter lock). To me it’s nothing else than an alternative reading for “fake threading”. If you are on a multicore/multicpu system try to execute this snippet while monitoring the cpu activity, it will start 8 threads and compute fibonacci(40) on each thread:

class Fibonacci
    def fib(i)
        if (i <= 2)
            return 1
        else
            return fib(i - 1) + fib(i - 2)
        end
    end
end

threads = []
8.times { |i| threads[i] = Thread.new {Fibonacci.new().fib(30) } }
threads.each {|t| t.join}

If you come from C/C++ or Java you may expect this code to use up to 8 cores/cpus on your system at the same time… wrong! It will execute the code using only one core/cpu at once because the GIL prevents the threads from taking control of the interpreter at the same time. Cool, isn’t it?

5) Method invocation hooking. The following is the best way I found to intercept whenever a method is called:

class Foo
  def bar1(gh1)
    puts "bar1 #{gh1}"
  end
end

class Foo
  def bar1_hook(*args)
    puts "bar1 hooked!"
    bar1_asd(args)
  end

  alias_method(:bar1_asd, :bar1)
  alias_method(:bar1, :bar1_hook)
end

a = Foo.new
a.bar1("foobar")

So bar1 is the method to be hooked: you have to monkey patch the Foo class defining the hooking method and hack it all together with alias_method. Doing this you are ripping the original class apart: if someone will edit thebar1 method after your hook he/she will actually be editing the bar1_hook method without even noticing it (well, he will notice the unexpected behaviour and maybe try to debug his/her own code… have fun :-

), I really hope to be wrong about this point…

XML importer from Danea Easyfatt to Cubecart

Mon, 30 Jul 2007 14:56:00 +0000

This is ugly, bugged, wrote in PHP (I’m not a PHP programmer) and will probably send death threats emails to all of your customers (really, I didn’t have the time to test it). If you still want it, click to download or just read it online:

<?php
/**
 * ----------------------------------------------------------------------------
 * 
 *  Copyright (C) 2007 Giacomo Graziosi (g.graziosi@gmail.com)
 * 
 * ----------------------------------------------------------------------------
 * 
 *  This program is free software; you can redistribute it and/or modify
 *  it under the terms of the GNU General Public License as published by
 *  the Free Software Foundation; either version 3 of the License, or
 *  (at your option) any later version.
 *
 *  This program is distributed in the hope that it will be useful,
 *  but WITHOUT ANY WARRANTY; without even the implied warranty of
 *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 *  GNU General Public License for more details.
 *
 *  You should have received a copy of the GNU General Public License
 *  along with this program.  If not, see .
 *
 * ----------------------------------------------------------------------------
**/
 
 
error_reporting(E_ALL);
 
//Ugly importer interface abstraction
class Outputter
{
    private $file_handle;
 
    public function __construct($file)
    {
        $this->file_handle = fopen($file, 'a');
    }
 
    public function __destruct()
    {
        fclose($this->file_handle);
    }
 
    public function write($buffer)
    {
        fwrite($this->file_handle, $buffer."\n##################################\n");
    }
}
 
//Importer implementation for the Cubecart
abstract class Importer
{
    protected $logger;
    protected $dbh;
    protected $xml_products;
 
    public function __construct($import_str, $host, $dbname, $user, $pass, $op)
    {
        $this->logger = $op;
        $this->dbh = new PDO('mysql:host='.$host.';dbname='.$dbname, $user, $pass);
        //$xml = simplexml_load_file($xml_file);
        $this->import_data($import_str);
        $this->update_and_delete();
        $this->insert_new();
    }
 
    public function __destruct()
    {
        //print_r($this->cats);
    }
 
    protected function query_has_row($sql_query, $str_id = "cat_id")
    {
        //echo "$sql_query\n";
        //$s = $this->dbh->query($sql_query);
        //if ($row = $s->fetch())
        foreach ($this->dbh->query($sql_query) as $row)
            return $row[$str_id];
        return false;
    }
 
    abstract protected function import_data($import_str);
    abstract protected function update_and_delete();
    abstract protected function insert_new();
}
 
 
class CubeImporter extends Importer
{
    private $cats = array();
 
    protected function import_data($import_str)
    {
        $xml = simplexml_load_string($import_str);
        $this->xml_products = $xml->Products->Product;
    }
 
    protected function update_and_delete()
    {
        foreach ($this->dbh->query('SELECT * from CubeCart_inventory') as $row)
        {
            foreach ($this->xml_products as $product)
            {
                if( $row['productId'] == $product->InternalID )
                {
                    //print($row['productId']." update\n");
                    $this->update_row($product);
                    continue 2;
                }
            }
            //print($row['productId']." delete\n");
            $this->delete_row($row['productId']);
        }
    }
 
    protected function insert_new()
    {
        foreach ($this->xml_products as $product)
        {
            foreach ($this->dbh->query('SELECT * from CubeCart_inventory') as $row)
            {
                if( $row['productId'] == $product->InternalID )
                {
                    continue 2;
                }
            }
 
            $this->add_row($product);
        }
    }
 
 
 
 
    private function add_cat($cat_name, $cat_fat_id = "0")
    {
        $this->dbh->exec("INSERT INTO CubeCart_category (cat_name, cat_father_id)"
        ."values ('$cat_name', $cat_fat_id)");
 
        //echo "ho inserito ".$this->dbh->lastInsertId()."\n";
        return $this->dbh->lastInsertId();
    }
 
    private function check_categories($xml_product) //needs some refactoring
    {
        if (array_key_exists("Subcategory", $xml_product))
        {
            $a = array_search($xml_product->Subcategory, $this->cats);
            if ($a == false)
            {
                $a = array_search($xml_product->Category, $this->cats);
                if ($a == false)
                {
                    $a = $this->query_has_row("SELECT * FROM CubeCart_category WHERE cat_name = '$xml_product->Category'");
                    if ($a == false)
                    {
                        //inserire cat
                        $a = $this->add_cat($xml_product->Category);
                    }
                    //$this->cats["$xml_product->Category"] = $a;
                }
                //$a deve contenere id di cat
                $fa = $a;
                $a = $this->query_has_row("SELECT * FROM CubeCart_category"
                ." WHERE cat_name = '$xml_product->Subcategory' AND cat_father_id = $fa");
                if ($a == false)
                {
                    $a = $this->add_cat($xml_product->Subcategory, $fa);
                }
                //$this->cats["$xml_product->Subcategory"] = $a;
 
            }
        } else { //If Subcategory doesn't exist then there must be at least a Category
            $a = array_search($xml_product->Category, $this->cats);
            if ($a == false)
            {
                $a = $this->query_has_row("SELECT * FROM CubeCart_category WHERE cat_name = '$xml_product->Category'");
                if ($a == false)
                {
                    $a = $this->add_cat($xml_product->Category);
                }
                //$this->cats["$xml_product->Category"] = $a;
            }
        }
        return $a;
    }
 
 
    private function add_row($xml_product)
    {
        $cat_id = $this->check_categories($xml_product);
        $tax_id = $this->query_has_row("SELECT id FROM CubeCart_taxes WHERE taxName = "."'IVA'", "id");        
        $s = $this->dbh->prepare("INSERT INTO CubeCart_inventory (productID, productCode,"
        ."price, name, cat_id, sale_price, stock_level, taxType)"
        ." VALUES (:productID, :productCode, :price, :name, :cat_id, "
        .":sale_price, :stock_level, :taxType)");
 
        $s->bindParam(':productID', $xml_product->InternalID);
        $s->bindParam(':productCode', $xml_product->Code);
        $s->bindParam(':price', $xml_product->GrossPrice3);
        $s->bindParam(':name', $xml_product->Description);
        $s->bindParam(':cat_id', $cat_id);
        $s->bindParam(':sale_price', $xml_product->GrossPrice3);
        $s->bindParam(':stock_level', $xml_product->AvailableQty);
        $s->bindParam(':taxType', $tax_id);
        $s->execute();
 
 
        $this->dbh->exec("INSERT INTO CubeCart_cats_idx (cat_id, productId)"
        ." VALUES ($cat_id, $xml_product->InternalID)");
        $this->dbh->exec("UPDATE CubeCart_category SET noProducts = noProducts + 1 "
        ."WHERE cat_id = $cat_id");
    }
 
    private function update_row($xml_product)
    {
        $cat_id = $this->check_categories($xml_product);
        $tax_id = $this->query_has_row("SELECT id FROM CubeCart_taxes WHERE taxName = "."'IVA'", "id");    
        $sql = "UPDATE CubeCart_inventory SET productCode = :productCode, price = :price, name = :name, cat_id = :cat_id, "
        ."sale_price = :sale_price, stock_level = :stock_level, taxType = :taxType WHERE productID = :productID";
        $s = $this->dbh->prepare($sql);
 
        $s->bindParam(':productID', $xml_product->InternalID);
        $s->bindParam(':productCode', $xml_product->Code);
        $s->bindParam(':price', $xml_product->GrossPrice3);
        $s->bindParam(':name', $xml_product->Description);
        $s->bindParam(':cat_id', $cat_id);
        $s->bindParam(':sale_price', $xml_product->GrossPrice3);
        $s->bindParam(':stock_level', $xml_product->AvailableQty);
        $s->bindParam(':taxType', $tax_id);
        $this->logger->write($sql);
        $s->execute();
    }
 
    private function delete_row($id)
    {
        //echo("$xml_product->Description\n");
        $cat_id = $this->query_has_row("SELECT cat_id FROM CubeCart_inventory WHERE productId = $id", "cat_id");
        $this->dbh->exec("DELETE FROM CubeCart_inventory WHERE productId = $id");
        $this->dbh->exec("UPDATE CubeCart_category SET noProducts = noProducts - 1 "
        ."WHERE productId = $cat_id");
    }
}
 
 
$op = new Outputter("/path/to/log/asd.txt");
 
if (array_key_exists("file", $_FILES)) {
    move_uploaded_file($_FILES['file']['tmp_name'], "file.xml");
    $xmlstr = file_get_contents("file.xml");
    $op->write($xmlstr);
    $ci = new CubeImporter($xmlstr, 'localhost', 'database_name', 'username', 'password', $op);
}
$op->write(var_export($_FILES, true));
$op->write(var_export($_POST, true));
echo("OK");
?>