Jaime Gil de Sagredo

Booby 0.5: Introducing the inspection api

2014-01-04T00:00:00+00:00

Exactly a year ago the first Booby release was announced. Today with the release of the 0.5.0 version I’m very happy to say that Booby has continued to be developed keeping its simplicity and isolation from other libraries and providing a thin and robust abstraction layer for data modeling and usage.

The path taken

During the past two versions I focused on fixing bugs and taking into account some use cases without adding so much features, and provide a pythonic api that would not interfere nor create undesirable side effects in the end-user application code.

The idea is to be as non-intrusive as possible and provide a toolkit to build your own data models, validators, serializators or wathever you need to work with your data.

And the first step to achieve this comes with the most important feature of this version and that names this post: the Inspection api.

Implementing the Inspection api

With the desire of turning Booby into a toolkit for building data-driven applications and libraries arises the need for access some internal data of Booby objects and classes, such as model fields.

A first approach could have been to add a public attribute (let’s name it fields) to the Model class containing the model fields. Several things made me get away from this idea quickly. Some examples are:

What happens if my data model should have a fields field?
What happens if I want to add more public attributes accessing internal Booby data? I will finish with a base Model class filled with lots of attributes, so the first problem would become bigger.

That would taken me away from my goal of a thin and non-intrusive api, so I decided to look for other people’s approaches for this kind of functionalities. I didn’t need to look far to find two good examples: the Python inspect and the SQLAlchemy inspection modules.

I took some ideas from both modules to build the Booby one. From the SQLAlchemy module I took its name because I thought it would be less likely to collision with the Python module. From the Python module I took its functional api (although with some changes in the way functions are named) because it seems more explicit than the SQLAlchemy common inspect function.

All this has resulted in the booby.inspection module, that for now defines the get_fields(model) and is_model(obj) functions.

Take a look to the inspection module documentation for more information about how these functions work.

Future

The future of Booby is linked with the goals exposed above: build a toolkit to model, validate, serialize and whatever you need to work with your application data, always trying to be the more pythonic and least intrusive as possible.

Some of the changes I have in mind for the next releases are:

Decouple validations and fields. As Booby fields are not intented only to validate data I want to extract validations from fields. Of course it will be possible to validate fields, but I want to make it more decoupled.
Extract more generic base classes (Object <- Validator <- Model) giving support for a more wide range of use cases.
Take into account data serialization/deserialization.
Extend the inspection api with more functionality.

And finally don’t doubt to share your ideas here or opening issues at Github. And of course download and try Booby.

Booby: data modeling and validation

2013-01-04T00:00:00+00:00

Today I’m glad to announce the first public release of Booby, a standalone data modeling and validation library written in Python and licensed under the Apache2 license. This is an initial version of the data modeling layer that will be used for defining API models in Finch.

See the following example to get an idea of what Booby is intended to do.

Models should be data source independent

Although resources definition is one of the main Finch features, I’ve decided to release it as a separated project to keep these two parts as isolated as possible and that they can evolve independently. In the end, data modeling is a very important point in software design and development, and it should not be dependent of the data sources. Furthermore, the idea is that Finch will be completely independent of the resources definition layer sometime.

Using Booby

During development I tried to keep Booby very simple for easy integration with other libraries and systems. Currently Booby is under active development and it is not a complete data modeling library yet, and documentation isn’t too complete, but I’m going to add new field types, validators and serialization formats.

By the time, although there is so much work to do, you are encouraged to install and try it, report issues and contribute your code through Github.

A python RESTful API consumer

2012-12-26T00:00:00+00:00

Web APIs, and more particularly RESTful APIs, have become very popular in the last few years by the hand of large sites like Fakebook, Twitter or Github, who give developers the opportunity to extend their services with a wide variety of applications and services.

This huge growth in the development and use of REST APIs has been mainly because of the ease of consuming data from different services to create applications, tools or whatever you imagine, such as another APIs! on nearly any device, operating system or programming language. And what has made this possible? The answer is the HTTP protocol.

Consuming REST APIs

Right, let me show you how easy is to start playing with, for example, the Github API, using Python and the awesome Requests http library.

It’s so easy! We only have needed to make a GET request to the user repositories endpoint and wait for an OK response to display a list of repository names and programming languages.

We also could add a new repository to the list making an authenticated POST request to the repositories endpoint and including some data.

These two previous examples shown very simple use cases, but serve to show the simplicity of consuming REST APIs.

But unfortunately, in the real world, client applications tend to become more and more complex and begins to be necessary to write some boilerplate code for stuff like prepare requests, validate data and parse responses.

Introducing Finch

Finch is an asynchronous RESTful API consumer I’m developing for Python.

The idea is to develop a general purpose, asynchronous http API consumer, specially focused on remove all of this boilerplate code and provide a high level abstraction layer on top of any API.

Metadata driven clients

In general, a REST API client can be divided into two different parts: resources definition and http related stuff. This separation let us put http code in a high level abstraction and resources details in application metadata.

“Put Abstractions in Code, Details in Metadata” The Pragmatic Programmer

See the Finch code example below to understand what I’m talking about.

Here, we have only described the Repo model and Repos collection. All of this code is only metadata, declaratively defined using Python, that Finch will use to perform all the operations needed to get a list of repositories from Github.

I think this is really interesting because you can dedicate exclusively to define the peculiarities of the API that you are going to consume and your business logic, and leave Finch to do all the repetitive work.

And do it asynchronously

The last example shows you Finch working synchronously. That’s fine when your application does not make an intensive use of API request, like in our example.

But imagine you have to build a highly scalable service that combines several services behind an unique API. You should not have to wait for each response to perform the next request. Here is where asynchronous programming comes into play.

To allow asynchronous requests, Finch will be initially built on top of the Tornado asynchronous HTTP Client. This way you could do something like repos.all(callback).

Let’s see the example where we added a new Github repo, but now using Finch asynchronously.

Sounds good, I want to start using it!

Sorry, not yet. Finch is still under active development. Although the examples shown here already work, there are some things I want to finish before release the first public version. For this first release I want to provide at least complete CRUD support, resources definition, authentication and asynchronous requests. I think I can have this version for the beginning of 2013.

And finally, if you are interested in the project you can track progress on the Github repo, on this blog or through my Twitter account. And of course, I would like to know your thoughts ;)