Unstable

GLib: Transfer Full

2014-11-29T07:32:00.003-08:00

While reading the GLib Reference Manual, or any other manuals for glib based libraries such as GObject, Gtk+ or GStreamer, you might notice that some parameters or returns are annotated as Transfer Full. The glib reference manual explains in the annotation glossary as

Free data after the code is done.

What exactly does this mean? This means that all responsibility of the objects that annotated variables point to will be transferred to the other side beyond the API borders.

GObject Introspection Reference Manual or GObject-Introspection Annotations Wiki has a much better description of the annotation, including other modes.

Returns

When a function of the return is annotated, you are the one to control the life of the returned object. You decide when to free() or not. All the responsibility is fully transferred to you after the function returned.

Out Parameters

This is similar to the returns. Everything is transferred to you. Free when you do need them any more.


/* -*- compile-command: "gcc -Wall -Wextra -g $(pkg-config --cflags --libs gio-2.0) transfer-full-out.c" -*- */
#include <gio/gio.h>
#include <stdio.h>

int main()
{
        GFile *file;
        char *buf;
        gsize len;

        file = g_file_new_for_path("/etc/passwd");
        g_file_load_contents(file, NULL, &buf, &len, NULL, NULL);

        fwrite(buf, len, 1, stdout);
        g_free(buf);

        return 0;
}

In Parameters

You pass your data to a function with Transfer full annotation, you are no longer the owner of the object.


/* -*- compile-command: "gcc -Wall -Wextra -g $(pkg-config --cflags --libs gio-2.0) transfer-full-in.c" -*- */
#include <glib.h>
#include <stdio.h>
#include <stdlib.h>

int main()
{
        void *buf;
        gsize len;
        GBytes *bytes;

        len = 100;
        buf = malloc(len);
        sprintf(buf, "%s", "hello");
        bytes = g_bytes_new_take(buf, len);

        // It might be a good idea to remove the reference
        // to the buf.  This ensures that we are no longer
        // able to free it by accident.
        buf = NULL;
        len = 0;

        fwrite(g_bytes_get_data(bytes, NULL),
               g_bytes_get_size(bytes),1, stdout);

        // buf will be freed when byts are freed
        g_bytes_unref(bytes);

        return 0;
}

Elisp: String Compare (Less than)

2014-10-27T07:25:00.000-07:00

Less Than

There are many elisp functions for comparisons. The "less than" sign < is a mathematical sign to compare inequality. Elisp has the function < to compare Number or Marker.

(< 1 2)
t
(< 2 1)
nil

It is obvious enough that no explanation is needed.

String Less Than

Elisp also has a "less than" function for Strings, called string-lessp.

(string-lessp "abc" "def")
t
(string-lessp "abc" "ab")
nil

It is a predicate function, meaning that it returns t if the first string, "abc" in the first example, is less than the second strings "def", or nil otherwise.

Elisp have a convention, as noted in elisp info, to end a predicate function with letter "p". Thus, following this convention, string-lessp ends with "p".

Elisp does have string< as an alias for string-lessp. So that you can write the above example:

(string< "abc" "def")
t
(string< "abc" "ab")
nil

"<" vs "p" vs "-p"

It seems like a majority of "less than" functions follow this "lessp" convention, however, there are other functions in Emacs. Here is an incomplete list of "less than" functions I have got from my Emacs with M-x apropos <$\|lessp$\|less-p$.

What functions do you have in your Emacs?

Incomplete List of "less than" functions

<
<<
bibtex-lessp
byte-string<
coding-system-lessp
customize-version-lessp
elmo-time-less-p
elmo-time<
file-attributes-lessp
gnus-pseudos<
gnus-string<
ido-file-extension-lessp
ido-file-lessp
mailcap-viewer-lessp
org-entries-lessp
org-table-formula-less-p
org-time<
recentf-string-lessp
s-less-p
skk-string-lessp-in-coding-system
skk-string<
string-lessp
string<
time-less-p
version-list-<
version<

Validate

mruby first impression

2012-04-23T12:04:00.000-07:00

Matz released mruby, a yet another ruby implementation for small systems, at GitHub. Mruby is a new implementation of Ruby Language based on RiteVM, which seems to be a register based VM with tri-color marking and mark-and-sweep collector GC.

I'll just quick look at the following points on the VM, which is said to be the most costly part of any virtual machine^[1].

The Instruction Dispatcher
Accessing the Operands
Performing the Computation

The instruction dispatcher

The instruction dispatcher, mrb_run(), is a simple C switch dispatcher. There seems to be a compile time condition code for a label dispatcher, too, but the default one is the C switch.

The instruction dispather is compiled to 17200 bytes (0x4330 bytes) of function under GCC 4.3 for ARM (the number depends on compiler options, of course). I haven't checked how this works in any type of ARM CPU cache.

$ objdump -t vm.o |grep mrb_run
000001d8 g     F .text  00004330 mrb_run

Accessing the Operands

The VM instructions are encoded in 32 bit fixed size instruction set. There seems to be three types of instructions. You can find this out in src/opcode.h.

The opcode is 7 bit long, which means that RiteVM can support 127 opcodes. There already are 75 opcodes and five reserved opcodes defined in src/opcode.h. So, 6 bit (64 opcodes) is too short RiteVM.

If you look at the comment above, it seems to be that the opcode is at MSB, but it is actually at the LSB side.

So, it's more like A:B:C:OP instead of OP:A:B:C.

A:B:C:OP takes three operands, the first two are 9 bits each and the last one is 7 bits. So, to retrieve the operand A from this vm instruction, you need to shift 23(9+7+7) bits to the right. You can do that in one instruction on ARM CPU.

A:Bx:OP takes two operands, the first one is 9 bits and the second one is 16 bits.

Ax:OP takes one 25 bits operand.

Performing the Computation

I'm still not sure how much computation is required for each instructions. This is fun to try but takes some time.

Register files

I also checked mruby registers. They are an array of mrb_values, which is a union of C variables and a byte for type indicator, as you can see below.

Virtual Machines and Bytecode Architecture

2012-03-02T20:04:00.000-08:00

I was curious about types of popular VMs / bytecodes. So here is a result of quick survey I did:

Register based

Dalvik
Parrot
Lua
LLVM
Sqlite 3.5.5 and later

Stack based

Java VM
CPython
CIL
Sqlite 3.4.4 and earlier

Abstract Syntax Tree based

Perl
Ruby
Dart

Android loves small LCD, too

2008-11-10T03:51:00.000-08:00

One of my colleague just finished hooking up a small LCD to Armadillo-500 FX! Android seems to be happy at this lovely new home. I just can stop thinking about what I can do with this tiny, cell phone sized, combo dev toy.

This hack was to show that FX works just fine with other LCD, or any LCD per se, than the default 5.7 inch one. Hope everyone gets the idea.

This will be showed off at ET2008(link is to japanese page...). Hope to see you guys there.

Armadillo Panel Computer

2008-05-08T21:13:00.000-07:00

Hmm.. It seems like shaping up quite nicely.

Android - Runs on a real hardware

2007-11-22T06:52:00.000-08:00

Here is a clip to show you Android running on a real hardware, Armadillo-500.

All credit is due to Ben "Benno" Leslie for his incredible work. All I've done is put them together.

By the way, the kernel is compiled as OABI with two additional feature enabled.

EABI support
OpenBinder

The kernel itself doesn't have to be EABI to run EABI binaries.

Android - OpenBind: protocol doesn't match!

2007-11-22T04:54:00.000-08:00

It seems like some people are trying to get as much OpenBinder information on the Android Internals.

I've downloaded OpenBinder source code archive from Dianne Hackborn's site, fixed a few places to meet the current API, compiled, put it on my board and still can't make Home App to run.

So, I've downloaded strace binary from Benno's page and run system_server under it. Now, I've got:


open("/dev/binder", O_RDWR|O_LARGEFILE) = 7
fcntl64(7, F_SETFD, FD_CLOEXEC)         = 0
ioctl(7, 0xc0046209, 0xbecdac9c)        = 0
writev(4, [{"\6", 1}, {"ProcessState\0", 13}, {"Binder driver protocol does not "..., 59}], 3) = 73
close(7)


Binder driver protocol does not match user space protocol!

Ouch.... I can go disassemble ioctl() to see how the application is checking binder version. However, if Android components are using new feature, there is no way I can re-implement that feature. I guess I have to wait for code to be open. Sigh...

Android - Binder

2007-11-20T03:35:00.000-08:00

I haven't been able to run android on my arm platform. I have been too lazy to compile strace to see what's going on. Fortunately, Benno have been working on it and it seems like he's successfully got a strace log.

He's log reveal that the android depends on /dev/binder which is not a standard feature on my kernel. Here is the snippet from his log.

This bit here is pretty important. /dev/binder is the character device for OpenBinder, which is something that you see in the kernel source. This appears to be an IPC sub-system for the Linux kernel. It was at some stage donated by Palm, and there was an OpenBinder.org website for sometime, although that site is now dead. Finding more information on this will be essential.
05:30:51.733954 open("/dev/binder", O_RDWR|O_LARGEFILE) = 7
05:30:51.735488 fcntl64(7, F_SETFD, FD_CLOEXEC) = 0
05:30:51.736483 ioctl(7, 0xc0046209, 0xbef86cbc) = 0
05:30:51.737443 ioctl(7, 0x40046205, 0xbef86cb8) = 0
05:30:51.738385 mmap2(NULL, 8388608, PROT_READ, MAP_PRIVATE|MAP_NORESERVE, 7, 0) = 0x40008000

As he said, OpenBinder.org is already down. But, I've found a link to OpenBinder Documentation on #android.

I haven't read the doc yet but it seems like this is required to run android on my system.

bsdiff - binary diff

2007-11-19T20:47:00.000-08:00

So I've tested the bsdiff. In conclusion, it does patch up the binary file as advertised. But, doesn't have any check for original file like GNU Patch has. So, here is how it works:


$ cat hello.c
#include 

int main()
{
        printf("hello world!\n");
        return 0;
}
$ cat hello-bsdiff.c
#include 

int main()
{
        printf("hello bsfile!\n");
        return 0;
}

Prepare two source code as above. The difference is bellow.


$ diff -u hello.c hello-bsdiff.c
--- hello.c     2007-11-20 13:44:39.000000000 +0900
+++ hello-bsdiff.c      2007-11-20 14:11:09.000000000 +0900
@@ -2,6 +2,6 @@
 
 int main()
 {
-       printf("hello world!\n");
+       printf("hello bsdiff!\n");
        return 0;
 }

As you can see the code differ only in text section. Now compile both code to create binaries "hello.out" and "hello-bsdiff.out".


$ gcc -Wall hello.c -o hello
$ gcc -Wall hello-bsdiff.c -o hello-bsdiff

Generate binary diff with bsdiff.

The synopsis for bsdiff is:


bsdiff <oldfile> <newfile> <patchfile>


$ bsdiff hello hello-bsdiff hello-hello-bsdiff.bsdiff

Let's see how well bsdiff did, in size.


$ ls -l *.out *.bsdiff
-rwxr-xr-x 1 user user 8963 2007-11-20 14:12 hello-bsdiff.out
-rw-r--r-- 1 user user  199 2007-11-20 14:12 hello-hello-bsdiff.bsdiff
-rwxr-xr-x 1 user user 8956 2007-11-20 14:11 hello.out

Patch up the hello.out to create hello-new.out, which should be identical to hello-bsdiff.out.

The synopsis for bspatch is similer to bsdiff:
bspatch <oldfile> <newfile> <patchfile>


$ bspatch hello.out hello-new.out hello-hello-bsdiff.bsdiff
$ md5sum hello-bsdiff.out hello-new.out
3564457345c42a2e0cce1c0a2191bece  hello-bsdiff.out
3564457345c42a2e0cce1c0a2191bece  hello-new.out

See that hello-bsdiff.out and hello-new.out has same hash value, 356445....

Let's also apply the binary patch to the /dev/null to see how does the bspatch works.


$ bspatch /dev/null hello-new-bad.out hello-hello-bsdiff.bsdiff
$ md5sum hello-bsdiff.out hello-new-bad.out
3564457345c42a2e0cce1c0a2191bece  hello-bsdiff.out
5f52d2ca0b2c01a1d05865ef2af102f4  hello-new-bad.out

bspatch doesn't even complain at all and, of cause, the generated file, hello-new-bad.out, is completely different from hello-bsdiff.out.

New eBook - Amazon Kindle

2007-11-19T20:18:00.000-08:00

It seems like Amazon Kindle is out. As a linux kernel book author, Robert Love, reported that the device is based on Linux 2.6.10.

Amazon has already released Linux kernel and other components source code, which includes, except usual uclibc, busybox, and other library and utils, bsdiff: binary diff and patch utility, and taglib: Audio Meta-Data Library.

It would be interesting to test bsdiff and report how well the utility reduce the size of update images.

Android - Dex

2007-11-18T23:23:00.001-08:00

So, it's been a week since Android SDK is out. Unfortunately, I haven't been able to poke around as much as I want last a few days. but there are many people interested in the internal of Android instead of Java wrapper; which is good thing.

My main focus is everything below the Dalvik. because once the underneath layer is done, we can just run Dalvik on any platform. That's gonna be fun.

But some people are interested in DEX format. so here is a list of links.

Android - Videos

2007-11-13T00:52:00.000-08:00

It's been an interesting day for me. many people are checking, talking, asking, playing with Android. I'll post a few entry to summarize what I've been reading so far.

The first post is a list of video clips released by google. These helped me to grasp what Android is all about.

Androidology - Part 1 of 3 - Architecture Overview

Androidology - Part 2 of 3 - Application Lifecycle

Androidology - Part 3 of 3 - APIs

Convert video/audio for my new cell phone

2006-01-18T21:18:00.000-08:00

I just got a new cell phone, which can play MPEG4 AAC audio and video.

To convert arbitrary video, such as Google Video, for my new N902i:

ffmpeg -y -i downloaded.mp4 -bitexact -fixaspect -s 176x144 -r 14.985 -b 400 -acodec aac -ac 2 -ar 16000 -ab 32 -f 3gp MOL001.3GP

to convert audio:

ffmpeg -i my.ogg -acodec aac -ac 2 -ab 128 test.m4a

2006-01-18T20:49:00.001-08:00

how about gnome-blog

does it work?

No more RSS Reader

2005-10-08T00:55:00.000-07:00

Google's new toy is Google Reader. It has vi-like key-binding too. Is there a count-down timer for Google's Desktop Domination? :-P

kernel hacker's guide to git

2005-09-29T07:36:00.000-07:00

It's still kinda old. here is some replacements.

Update local kernel tree to latest 2.6.x upstream ("fast-forward merge")

$ git pull

Undo all local modifications:

$ git reset --hard

List all branches

$ git branch

Examine which branch is current

ditto, or

$ git show-branch --list

/usr/bin/X11 is not in PATH

2005-09-26T22:45:00.000-07:00

Since apt-get upgrade this morning, /usr/bin/X11 isn't in my zsh's PATH env var.

I don't remember which package was upgraded this morning... sigh.

Testing Blogger

2005-09-26T22:43:00.000-07:00

I'm just trying to see how well blogger does. This entry is posted with gnome-blog 0.8.