Daws Design

Mochiweb Documentation

dawsdesign — Mon, 01 Jun 2009 05:52:22 -0500

I've noticed it can be hard to find the mochiweb docs online, so I compiled them and I'm now hosting them at mochiweb documentation. Feel free to ask in the comments if you'd like me to host anything else at http://dawsdesign.com/drupal/man-pages.

Erlang Driver Walkthrough with Berkeley DB in C

dawsdesign — Tue, 26 May 2009 14:28:42 -0500

While thinking about ErlFS, I realized I'd need at least two subsystems. One for finding a node with the data I am looking for and one to actually store that data. The first will be covered by Chordial. For the storage portion, I was originally going to use a homegrown method where I would just store the data in files named after their key. I decided I should explore the various possiblities and so I did a little research and decided to take a crack at writing a port driver for Erlang. Thanks go to Kevin Smith for his article on writing linked-in drivers for Erlang.

There are several way to write an Erlang driver. One way is to spawn a process and communicate over a pipe to your program and another is to bring your program in the same memory space and communicate directly with in-memory vectors. The first is safe; if your program crashes, your Erlang application can recover using standard OTP principles, such as supervisors. The in-vm-memory model is much more dangerous; if your program crashes, the whole Erlang VM crashes with it. This method, however, is much faster, as it does not have to copy memory and then have the OS switch context for each call. We will be covering this method.

Writing your first Erlang driver can be very difficult as there are very limited examples and the documentation can be very obscure. In this article, I will walk you through writing one for a common purpose-- writing to a database.

All sources can be found at the project's GitHub page.

Build Directory and Makefiles

First let's start with a simple OTP build directory structure. I'll create a new folder called erl_bdb_store and in that make the following folders: ebin, include, priv, and src. This is what it should look like:

erl_bdb_store/
- ebin (to store the erlang bytecode/binaries)
- include (to store erlang include files)
- priv (to store non erlang projects/code)
- src (to store erlang source code)

Next, we'll need a couple simple makefiles. Please note that these currently only compile under linux (tested on Ubuntu). The first one just goes to the /priv and /src directories and runs the Makefiles in each.

./Makefile:

The next one compiles the C source (the meat-and-potatoes of the driver) into a shared library. This is different on each operating system and will need to be changed for each. I will do this sometime in the future.

./priv/Makefile:

The erlang Makefile is simple
./src/Makefile

Erlang Source

Now we will make our Erlang wrapper for the driver. It is a standard OTP gen_server behavior callback.
./src/bdb_store.erl

The two important parts are the init function and the handle_call function. The init function loads and starts the driver.

The handle_call function communicates with the driver to put, get and delete records from the database file. This just forwards and translates the commands to the actual driver, where all of the real work happens:

Notice the primary difference between each call is the first byte of each Message binary. This byte is inspected inside the driver and there it is determined which function to perform:

C Source

The C code does most of the work. It handles creating all of the error messages and performing all of the commands the Erlang source communicates to it.

Let's take a look at the C source piece-by-piece.

First we'll create a header file with settings and function prototypes.
./priv/bdb_drv.h:

At the top we include necessary Erlang headers followed by a couple standard C headers and then the Berkeley DB header. Next we define the path where we want our database to store data, and then we define constants for the byte values of the commands sent to the driver from the Erlang VM. The _bdb_drv_t struct is very important, as we cannot use global variables and so we must keep our state in a struct. The reason is that the Erlang VM can and will start a new instance of the driver many times for performance and we want to keep our state localized. The last entries in the header file are our function prototypes.

Now to our C implementation file (./priv/bdb_drv.c).
Include our header file:

Specify the callbacks we will be implementing:

This array defines the callback functions that will be invoked at various points by the Erlang VM. In this example, we are implementing start, stop and outputv. This means that when the Erlang VM starts an instance of our driver, it will call our function start, when it sends a message, it will call outputv and when it stops the instance, it will call stop. outputv is only used if output is not defined in the array. output uses a buffer whereas outputv does not, meaning there is no overhead in copying the data, so it is faster. See driver_entry documentation for more detail.

Next is boilerplate which tells the VM which struct to use as the state holder:

Here, we define the start function and open up the database for reading and writing.

If Berkeley DB returns an error, it is propagated to the Erlang VM as a typical {error, Reason} tuple.

The stop function closes the database and releases the driver:

outputv will serve as our entry point when a message is sent to the driver:

It interprets the first byte of the message to determine which function to call.

As it's name implies, the put function inserts a record into the database:

The function grabs the first 20 bytes after the first command byte and uses it as the key. This leaves room for 160 bits-- enough for a SHA1 hash. All of the remaining bytes are used as the value to be stored. This function returns the atom ok or the typical {error, Reason} tuple.

The get function is similar to put, except we have to free the memory allocated by Berkeley after the record has been retrieved:

First we call an Erlang function named driver_alloc_binary (see erl_driver#driver_alloc_binary) which keeps a reference count, starting with 1. We then return the value with driver_output_term which increases the reference count to 2 and then call driver_free_binary which brings it to 1 again. When the Erlang VM has finished with it, it will also call driver_free_binary which brings the reference count to 0 and it will be freed from memory.

The del function deletes a record and is the most simple of the database functions:

Finally we have our catch-all function which will be called if an unrecognized command byte is sent. It returns the tuple {error, unkown_command}:

Usage

Let's compile it and jump into an Erlang shell to test it out:

Now, let's start the gen_server we wrote and perform a couple tests:

That's it for the walkthrough. If you have any questions, please post a comment or use the contact form. You can also email me at the email address listed in the source. I hope it helps you on a future project! Please let me know if it does!

All sources can be found at the project's GitHub page.

Chordial - Chord DHT in Erlang

dawsdesign — Tue, 10 Feb 2009 22:45:14 -0500

I've been stalled with ErlFS for a while now, but my interest has been flared up due to someone cloning my SVN repo into GitHub, where it ended up on the front page for interesting projects.

So, I had actually been scheming a bit and was thinking about using a distributed hash table instead of a mesh network, due to the ability to scale massively. This is one of the goals of ErlFS, so it seemed natural.

I'm dead set on it now and I've started a project called chordial, which is my open source _interpretation_ of MIT's chord algorithm. Once it is finished, it should greatly simplify ErlFS and I will likely start from scratch.

You can find chordial at http://github.com/dawsdesign/chordial/.

I can currently add multiple nodes to the ring, but I am having some issues with the finger function on the nodes. It will just take some time to get sorted out, but I am feeling great about this project and it is going quite well.

Use MySQL with Erlang

dawsdesign — Tue, 23 Dec 2008 11:11:35 -0500

If you want to get your manager to try out Erlang, but they are hesitant (probably an understatement) you might be able to get some leverage if you use a relational database rather than mnesia. So, here's a quick sample of how to connect to and query a MySQL database.

First, you'll need to install the MySQL ODBC driver if you haven't already. You can find it at http://dev.mysql.com/downloads/connector/odbc/5.1.html. Then you'll need a database to connect to and a table to query. Here's the SQL script I used to create my test database:

CREATE DATABASE `test`;

CREATE TABLE `test`.`test_table`

(

  `id` INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,

  `first_name` VARCHAR(45) NOT NULL,

  `last_name` VARCHAR(45) NOT NULL,

  PRIMARY KEY (`id`)

)

ENGINE = InnoDB;

INSERT INTO `test`.`test_table`

(

  `first_name`,

  `last_name`

)

VALUES

(

  'Matt',

  'Williamson'

),

(

  'Matt',

  'Williamson2'

),

(

  'Matt',

  'Williamson3'

);

And here's the code to connect and query in erlang:

application:start(odbc).

ConnString = "Driver={MySQL ODBC 5.1 Driver};Server=localhost;Database=test; User=root;Password=ace152;Option=3;".

{ok, Conn} = odbc:connect(ConnString, []).

Results = odbc:sql_query(Conn, "SELECT * FROM test_table").

This is what my interactive session looks like:

Erlang (BEAM) emulator version 5.5.5 [async-threads:0]

Eshell V5.5.5  (abort with ^G)

1> application:start(odbc).

ok

2> ConnString = "Driver={MySQL ODBC 5.1 Driver};Server=localhost;Database=test; 

User=root;Password=ace152;Option=3;".

"Driver={MySQL ODBC 5.1 Driver};Server=localhost;Database=test; User=root;Passwo

rd=ace152;Option=3;"

3> {ok, Conn} = odbc:connect(ConnString, []).

{ok,<0.39.0>}

4> Results = odbc:sql_query(Conn, "SELECT * FROM test_table").

{selected,["id","first_name","last_name"],

          [{1,"matt","williamson"},

           {2,"matt","williamson2"},

           {3,"matt","williamson3"}]}

5>

It's that simple. Post a comment if you want more samples or description.

Erlang docs: http://www.erlang.org/doc/apps/odbc/index.html

Superb Erlang Web App Tutorial

dawsdesign — Mon, 15 Dec 2008 09:10:58 -0500

I came across the most impressive Erlang tutorial I have seen yet. Check it out. Now.

http://www.vimeo.com/2007411

Duplicate an Object in Flash

dawsdesign — Fri, 12 Dec 2008 16:18:07 -0500

I'm working on a tile editor in Flash CS4 and opted to use the TileList class for the palette. Once a swatch is selected, I need to create a copy of that swatch and attach it to the stage. I wasn't quite sure how polymorphism in AS3 worked but I ended up with this little snippet:

var mySwatchClass:Class = mySwatch.source.constructor;

var newTile = new mySwatchClass();

Let me know if it helps!

Add your location to iPod/iPhone Wifi Location Service

dawsdesign — Wed, 10 Dec 2008 11:35:43 -0500

I recently got an iPod Touch and it is the best gadget I've ever had. One of the features of the powerful little machine is the location service. This service uses the wifi radio to listen for wifi beacons, the signal an access point sends out every second or two to let clients know the AP is there. When you refresh your list of wireless networks, it gathers these signals to make the list. Anyway, the utility of this feature is actually quite extensive. You can use it to find nearby restaurants, movie showtimes, and make getting directions a breeze. Unfortunately in my home area, there is no coverage.

I began searching and found an article on MacRumors that mentioned Apple uses a company called Skyhook to provide their database of AP MAC addresses and their coordinates http://www.macrumors.com/2008/01/23/skyhooks-wi-fi-location-technology-f.... Skyhook, according to their website, keeps a database of MAC addresses and associates them with their GPS location. Using this database is claimed to be accurate within 10-20 Meters.

After visiting Skyhook's website, I found the form where you can add your Wifi Access Point to their database. The page is at http://www.skyhookwireless.com/howitworks/submit_ap.php. Unfortunately it can take up to a few weeks to have the data updated with all of their partners. I will repost here if it ends up working for me.

P. S. If you can't find your MAC address, this article might help http://getsatisfaction.com/skyhookwireless/topics/how_do_i_find_the_mac_....

UPDATE 12/22/2008
It worked! Now when I push the current location icon in Google Maps it centers right on my street. 12 days later ain't bad.

Google Maps Circle Overlay

dawsdesign — Mon, 08 Dec 2008 16:17:11 -0500

This was another feature that should have been simple, but just wasn't. This Google Maps extension allows you to use circle overlays. It is really a GPolygon with 45 points, but it looks like a circle. Here is a sample. You can download the source here.

js/gmaps.CircleOverlay.js:

// This file adds a new circle overlay to GMaps2

// it is really a many-pointed polygon, but look smooth enough to be a circle.

var CircleOverlay = function(latLng, radius, strokeColor, strokeWidth, strokeOpacity, fillColor, fillOpacity) {

    this.latLng = latLng;

    this.radius = radius;

    this.strokeColor = strokeColor;

    this.strokeWidth = strokeWidth;

    this.strokeOpacity = strokeOpacity;

    this.fillColor = fillColor;

    this.fillOpacity = fillOpacity;

}

// Implements GOverlay interface

CircleOverlay.prototype = new GOverlay;

CircleOverlay.prototype.initialize = function(map) {

    this.map = map;

}

CircleOverlay.prototype.clear = function() {

    if(this.polygon != null && this.map != null) {

        this.map.removeOverlay(this.polygon);

    }

}

// Calculate all the points and draw them

CircleOverlay.prototype.redraw = function(force) {

    var d2r = Math.PI / 180;

    circleLatLngs = new Array();

    var circleLat = this.radius * 0.014483;  // Convert statute miles into degrees latitude

    var circleLng = circleLat / Math.cos(this.latLng.lat() * d2r);

    var numPoints = 40;

    // 2PI = 360 degrees, +1 so that the end points meet

    for (var i = 0; i < numPoints + 1; i++) { 

        var theta = Math.PI * (i / (numPoints / 2)); 

        var vertexLat = this.latLng.lat() + (circleLat * Math.sin(theta)); 

        var vertexLng = this.latLng.lng() + (circleLng * Math.cos(theta));

        var vertextLatLng = new GLatLng(vertexLat, vertexLng);

        circleLatLngs.push(vertextLatLng); 

    }

    this.clear();

    this.polygon = new GPolygon(circleLatLngs, this.strokeColor, this.strokeWidth, this.strokeOpacity, this.fillColor, this.fillOpacity);

    this.map.addOverlay(this.polygon);

}

CircleOverlay.prototype.remove = function() {

    this.clear();

}

CircleOverlay.prototype.containsLatLng = function(latLng) {

    // Polygon Point in poly 

    if(this.polygon.containsLatLng) {

        return this.polygon.containsLatLng(latLng);

    }

}

CircleOverlay.prototype.setRadius = function(radius) {

    this.radius = radius;

}

CircleOverlay.prototype.setLatLng = function(latLng) {

    this.latLng = latLng;

}

circle.html:

<html>

<head>

    <title>GMaps Circle Test</title>

    <script src="http://maps.google.com/maps?file=api&amp;v=2&amp;key=ABQIAAAAOzwQ7Lh4CqiMto5Mx5BruhS-xyZmcPPoBCehH0LrfEv4pktlHxSKTILYLRlghp_oPDcwTo3STVXADg" type="text/javascript"></script>

    <script src="js/gmaps.CircleOverlay.js" type="text/javascript"></script>

    <script type="text/javascript">

    var circle = null;

    var circleRadius = 50; // Miles

    var map = null;

    var isCompatible = GBrowserIsCompatible();

    //<![CDATA[

    function load() {

        if (isCompatible) {

            // Create Map

            map = new GMap2(document.getElementById("map"));

            map.setCenter(new GLatLng(42, -74), 6);

            // Add controls

            map.addControl(new GLargeMapControl());

            map.addControl(new GMapTypeControl());

            // Create and add the circle

            circle = new CircleOverlay(map.getCenter(), circleRadius, "#336699", 1, 1, '#336699', 0.25);

            map.addOverlay(circle);

        }

    }

    //]]>

    </script>

</head>

<body onload="load()" onunload="GUnload()">

    <div id="map" style="width: 400px; height: 300px; border: 1px solid #666666;"></div>

</body>

</html>

Check if a polygon contains a coordinate in Google Maps

dawsdesign — Thu, 06 Nov 2008 09:16:08 -0500

At work, I've been tasked with a neat little project where a user can select a list of users represented by markers on a Google map by editing a polygon to surround the markers with. However, to my dismay, I could not find any methods in the API to see if a polygon contains a coordinate. I also had a hard time finding an algorithm so that I could implement it. I later learned this type of algorithm is called Point in Polygon (PiP).

First I tried the winding number algorithm (http://en.wikipedia.org/wiki/Point_in_polygon) but failed. I didn't put too much time into it since it is so CPU-intesive anyway. Then I came across an article on MSDN (http://msdn.microsoft.com/en-us/library/cc451895.aspx) which I adopted into Javascript. I extended the GPolygon class with a new method called containsLatLng, just like the GLatLngBounds class. The difference is that the method in the GLatLngBounds class will only get the 4 coordinates of the bounding box, which are the farthest points in the polygon, and see if the coordinate lies within that box. My method, however, does true collision detection to see if the point lies within any shape polygon with as many sides as you want. It worked out perfectly.

Without any further ado, here is the code [js/gmaps.polygon.containsLatLng.js]:

// Create polygon method for collision detection

GPolygon.prototype.containsLatLng = function(latLng) {

    // Do simple calculation so we don't do more CPU-intensive calcs for obvious misses

    var bounds = this.getBounds();

    if(!bounds.containsLatLng(latLng)) {

        return false;

    }

    var numPoints = this.getVertexCount();

    var inPoly = false;

    var i;

    var j = numPoints-1;

    for(var i=0; i < numPoints; i++) { 

        var vertex1 = this.getVertex(i);

        var vertex2 = this.getVertex(j);

        if (vertex1.lng() < latLng.lng() && vertex2.lng() >= latLng.lng() || vertex2.lng() < latLng.lng() && vertex1.lng() >= latLng.lng())  {

            if (vertex1.lat() + (latLng.lng() - vertex1.lng()) / (vertex2.lng() - vertex1.lng()) * (vertex2.lat() - vertex1.lat()) < latLng.lat()) {

                inPoly = !inPoly;

            }

        }

        j = i;

    }

    return inPoly;

};

And here is the sample usage [index.htm]:

<html>

<head>

    <title>GIS Test</title>

    <script src="http://maps.google.com/maps?file=api&amp;v=2&amp;key=ABQIAAAAOzwQ7Lh4CqiMto5Mx5BruhRP7uQ16ovEZegyAfeetOdXUUHv1xQhaNqllcioqRUJo6b7JhhTgJZNyw" type="text/javascript"></script>

    <script src="js/gmaps.polygon.containsLatLng.js" type="text/javascript"></script>

    <script type="text/javascript">

    var isEditing = false;

    var polygon = null;

    var map = null;

    var isCompatible = GBrowserIsCompatible();

    var markers = [];

    var iconIncluded = 'http://maps.google.com/mapfiles/dd-start.png';

    var iconExcluded = 'http://maps.google.com/mapfiles/dd-end.png';

        function editClick() {

                isEditing = !isEditing;

                if(isEditing && polygon != null) {

                        document.getElementById('edit_button').innerHTML = 'Stop Editing';

                        document.getElementById('help').innerHTML = 'Click to add points to your selection.';

                        document.getElementById('reset_button').disabled = 'disabled';

                        polygon.enableDrawing();

                } else {

                        document.getElementById('edit_button').innerHTML = 'Edit Selection';

                        document.getElementById('reset_button').disabled = '';

                        document.getElementById('help').innerHTML = 'Click "Edit Selection" to select users.';

                        polygon.disableEditing();

                }

        }

        // Update all markers with in poly status

        function updatePoints() {

                for(var i in markers) {

                        var marker = markers[i];

                        var point = marker.getLatLng();

                        var inPoly = polygon.containsLatLng(point);

                        if(inPoly) {

                                marker.setImage(iconIncluded);

                        } else {

                                marker.setImage(iconExcluded);

                        }

                }

        }

        function resetPolygon() {

                if(polygon != null) {

                        map.removeOverlay(polygon);

                }

                polygon = new GPolygon([], "#000000", 1, 1, "#336699", 0.3);

                map.addOverlay(polygon);

                GEvent.addListener(polygon, "lineupdated", function() {

                        setTimeout(updatePoints, 50);

                });

                GEvent.addListener(polygon, "endline", function() {

                        setTimeout(editClick, 50);

                });

                updatePoints();

        }

    //<![CDATA[

    function load() {

        if (isCompatible) {

            // Create Map

            map = new GMap2(document.getElementById("map"));

            map.setCenter(new GLatLng(40, -90), 3);

            // Add controls

            map.addControl(new GLargeMapControl());

            map.addControl(new GMapTypeControl());

            // Create Selection Polygon

            var latlng = map.getCenter();

            var width = 20;

            var height = 12;

            var lat = latlng.lat();

            var lng = latlng.lng();

            var x1 = lng - width / 2;

            var y1 = lat - height / 2;

            var x2 = lng + width / 2;

            var y2 = lat + height / 2;

                        resetPolygon();

            // Add 10 markers to the map at random locations

            var bounds = map.getBounds();

            var southWest = bounds.getSouthWest();

            var northEast = bounds.getNorthEast();

            var lngSpan = northEast.lng() - southWest.lng();

            var latSpan = northEast.lat() - southWest.lat();

            for (var i = 0; i < 100; i++) {

                var point = new GLatLng(southWest.lat() + latSpan * Math.random(), southWest.lng() + lngSpan * Math.random());

                var marker = new GMarker(point, {bouncy: true, title: "Customer " + i, autoPan: false});

                map.addOverlay(marker);

                marker.bindInfoWindowHtml("This is customer info " + i);

                markers.push(marker);

            }

            updatePoints();

        }

    }

    //]]>

    </script>

</head>

<body onload="load()" onunload="GUnload()">

    <div>

        <span><button id="edit_button" onclick="editClick()" type="button">Edit Selection</button></span>

        <span><button id="reset_button" onclick="resetPolygon()" type="button">Reset Selection</button></span>

        <span id="help" style="color: #666;">Click "Begin Editing" to select users.</span>

    </div>

    <hr />

    <div id="map" style="width: 640; height: 480; border:1px solid #333;"></div>

</body>

</html>

Sample: http://dawsdesign.com/Samples/GIS/
Download: http://dawsdesign.com/Samples/GIS/GIS.zip

Erlang gen_server OTP Demo

dawsdesign — Sun, 10 Aug 2008 18:04:35 -0500

This is a short video I put together. It shows how to use emacs to generate an Erlang gen_server to greet a client. It also demonstrates Erlang's hot code-loading capabilities.

Erlang/OTP gen_server demo from Matt Williamson on Vimeo.

Really high quality at http://dawsdesign.com/video/video-gen_server_demo

Hello, Robert

dawsdesign — Wed, 06 Aug 2008 07:00:45 -0500

It never gets old.

ErlFS Distributed Storage System

dawsdesign — Tue, 22 Jul 2008 11:08:59 -0500

I've begun work on ErlFS, a distributed storage system which uses distributed Erlang at it's core. The idea is that it splits files into chunks and copies them, redundantly over multiple disks on multiple machines. This system should scale nearly linearly-- that is, the more machines/disks you add, the more storage, throughput, response and reliability you get.

Each node keeps track of where the file chunks are located and will load balance automatically. I hope to have an Erlang, HTTP, FTP and Bittorrent interface for the system, probably using YAWS/erlyweb.

Google code is hosting the project for me. You can view it at http://code.google.com/p/erlfs/.

I will update here when an alpha is ready.

Update: We just registered http://erlfs.com, so we will have a home for it.

$40 Off Web Hosting

dawsdesign — Mon, 09 Jun 2008 09:31:21 -0500

Get $40 off Dreamhost hosting for any billing cycle. That's 1/3 off of the price for one year of hosting!

Just visit the site at dreamhost.com, click "Sign Up Now" and enter the following promo code in the form.

Promo Code:
DAWS

Buy My Stuff Now!

dawsdesign — Mon, 09 Jun 2008 07:23:33 -0500

If you are looking to buy, sell or trade your stuff for free, please check out buymystuffnow.com. It is a free classifieds site where you can post your listings and browse and search for listings.

When you sign up, you will get some credits. Credits are currency which you can spend on attaching images and videos to your listings. Check out buymystuffnow.com to see how many you get for free. If you need more credits, don't worry, they are very affordable.

Buy My Stuff Now also doesn't take any of your profit, they only make money of off credits, which you get some fore free anyway. Because they don't skim off the top, you are free to do things such as barter or give away things for free!

Feel free to check out the website and post any comments about it you have here. http://buymystuffnow.com