DamnHandy

Handy-URI-Templates 1.1.7 Released

2012-11-21T05:47:48-08:00

Yesterday Handy URI Templates 1.1.7 was released which fixed a few uritemplate-test issues as well as some URI encoding fixes. Hat tip to tekener for those fixes.

With that out of the way, I’ll be focusing on 1.2 which will change the API a bit, but will finally add reverse mapping so that you can use it to match request URLs to a URI template pattern. It turns out that this is a bit more complicated than I first imagined. A number of folks have pointed to the excellent wo-furi project as this already does reverse mapping. However, it only handles level 1, and maybe level 2 templates. Things get hairy when you start reverse mapping level 3 and level 4 templates.

Handy Aspects Moved to GitHub

2012-08-07T05:12:28-07:00

A few years back, I dabbled a bit with Aspect Oriented Programming and dorked around with JBoss AOP and AspectJ. I created a few aspects an threw them up on Java.net. Over the years, I never really kept up with maintaing the project. Since then, Java.net migrated projects and Handy Aspects was removed. Since then, I have received a few requests for the code and I planned on moving it to GitHub.

But rather than simply throwing the code up there “as-is”, I decided to bring the project up to date a bit. For example, I moved the build from Ant to Maven and brought the dependencies up to the latest versions. I also removed JBoss AOP version and everything is now based on AspectJ. With that said, you can follow and fork the new project Handy-Apsects project here.

I want to like SCXML, but executable XML is a bit of a two-bagger

2012-05-31T05:18:52-07:00

Over the past few days I have been reading up on the State Chart XML spec. Ever since reading some of Stu Charltons ideas on a RESTful Hypermedia Agent and listening to his WS-REST keynote presentation, I’ve taken more of an interest in hierarchical state machines and began taking a more in-depth look into SCXML.

The design of SCXML is interesting. I like the ECMA Script functionality and XML structure feels clean at first. It all seems fine until you get to the section on executable content. There’s a bit of debate around “executable xml” and executable XML frameworks like Jelly. And even folks poking fun at the idea of an XML programming language. As a Java guy, I have grown to dislike build tools such as Ant due to the fact that one can express relatively complex conditional logic in XML. I like tools like Maven better since it’s XML model is more declarative rather than executable (profiles are conditional). When I need more complex build operations, I’d be looking at tools like Gradle. Don’t get me started on XSLT.

I really like the concept of SCXML, but I’m not sold on the design. I don’t really have issue with the use of XML in general, I get it. However, the executable XML content bit is really hard to get past. Expecially when a scripting evironment is available to the SCXML environment. I can debug JavaScript code with a number of tools. Executable XML content? Not so much. For me, the executable content bit is the technical equivalent of a two-bagger.

Handy URI Templates 1.1.2 Released

2012-05-21T18:28:28-07:00

After a few weeks of tweaking, I put out a new release of Handy URI Templates. What’s important about version 1.1.2 is that it is now being tested against the uritemplate-test suite started by Mark Nottingham. Most importantly, it is also now passing all tests. Additionally, this release also marks the introduction of expression validation as well. If you’ve been using the 1.0.x versions, I’d highly recommend moving up to 1.1.2.

Fun with URI Templates and the Facebook Graph API

2012-05-03T06:56:10-07:00

URI Templates can make interacting with the web APIs like the Facebook Graph API a little easier. Here, I’ll show you how you can use URI Templates to create request URIs into the Facebook Graph API using the Handy URI Templates library. The URI Template examples described should be usable by any RFC6570 URI Template processor, but I’ll be focusing on how to use the Handy URI Templates API in Java. If you’re using PHP, have a look at Michael Dowling’s excellent Guzzle Framework which has great URI template support.

URI Template Basics

We’ll assume you have some familiarity with URI templates. Basically, a URI template is expression where replacement variables are identified by the text between ‘{‘ and ‘}’, like so:

https://graph.facebook.com/{id}

Where {id} is the name of the variable. This is similar to how paths are expressed in JAX-RS, and OpenSearch URL template syntax . The RFC6570 spec provides a lot more details on the URI template standard syntax, so we’ll focus on how to use URI templates in conjunction with the Facebook Graph API.

Facebook Graph API Basics

For the most part, most URIs in the Graph API follow the basic pattern of hostname + id. In a URI template, this pattern can be expresed as:

https://graph.facebook.com/{id}

When the template variable is expanded, it will allow you to create URIs that can be used request resources like so:

Users: https://graph.facebook.com/btaylor (Bret Taylor)
Pages: https://graph.facebook.com/cocacola (Coca-Cola page)
Events: https://graph.facebook.com/251906384206 (Facebook Developer Garage Austin)

And so on. Facebook also requires you to supply the access_token in a query parameter on the request URI, so now we end up with the expression:

https://graph.facebook.com/{id}{?access_token}

Please check the Facebook documenation on how to get the token.

Using the Handy URI Templates API, you can create a URI from an expression like so:

String expression = "https://graph.facebook.com/{id}{?access_token}";
String uri  = 
   UriTemplate.fromExpression(expression)
              .set("id", "bgolub")
              .set("access_token", System.getProperty("fb.access_token"))
              .expand();

This will give you the following URI:

https://graph.facebook.com/bgolub?access_token=your_fb_access_token

Because the {id} variable can contain sub paths, we need a way to express that. If we have want to express a URI template that gets a users information or the users photo albums, we need additional path segements. We could use multiple path segments with more variables, but this can make the template more complicated. One option is to modify the expression so that {id} can accomodate a single path segement or multiple path segments by rewriting the expression as:

https://graph.facebook.com{/id*}{?access_token}

This does a few things:

By putting the path ‘/’ operator in the variable expression, we’re stating that the values in this variable path segements. By default, if the variable values is an array, Collection or Map, the values will be ‘,’ delimited.
The * modifier means ‘explode’. With an array or Collection plus the explode modifier, the values will be delimited by the ‘/’ operator.

Now if we change the code a little bit:

String expression = "https://graph.facebook.com{/id*}{?access_token}";
String uri =  
    UriTemplate.fromExpression(expression)
               .set("id", new String[] {"bgolub","albums"})
               .set("access_token", System.getProperty("fb.access_token"))
               .expand();

You’ll note that the value we pass to the id variable is an array of strings and the processor will render valuse as two strings separated by a /.The resulting URI is now:

https://graph.facebook.com/bgolub/albums?access_token=your_fb_access_token

Now lets see how we can make more advanced types of requests.

More Advanced Requests

The Graph API has a number of query parameters that modifiy the request. All of these are defined in the Facebook Graph API documenation so I won’t detail them here. With all of the query parameters collected, you end up with the following URI template expression:

https://graph.facebook.com{/id*}{?q,ids,fields,type,center,distance,limit,offset,until,since,access_token}

You can pretty much use this template express for the majority, if not all, of the Facebook Graph API. We’ll use that expression for rest of the examples.

Search Request URIs

With the URI template, we can create URIs that map to Graph API search requests. Using the “coffe” example from the Facebook documenation:

String expression = "https://graph.facebook.com{/id*}{?q,ids,fields,type,center,distance,limit,offset,until,since,access_token}";

String uri = 
    UriTemplate.fromExpression(expression)
                .set("id","search")
                .set("q", "coffee")
                .set("type","place")
                .set("center", new float[] {37.76f,-122.427f})
                .set("distance", 1000)
                .set("limit", 5)
                .set("offset", 10)
                .set("access_token", System.getProperty("fb.access_token"))
                .expand();

This will give us the URI:

https://graph.facebook.com/search?q=coffee&type=place&center=37.76,-122.427&distance=1000&limit=5&offset=10&access_token=your_access_token

Note how the variables until and since are not included in the expanded URI.

FQL Request URIs

Even things URIs with FQL queries can be expressed in a template. Give our code:

String expression = "https://graph.facebook.com{/id*}{?q,ids,fields,type,center,distance,limit,offset,until,since,access_token}";

  String uri = 
    UriTemplate.fromExpression(expression)
               .set("id","fql")
               .set("q", "SELECT uid2 FROM friend WHERE uid1=me()")
               .set("access_token", System.getProperty("fb.access_token"))
               .expand();

Wrap Up

Hopefully this gives you a good idea on both how to use URI Templates in general, and a good insight into how you can use teh Handy URI Templates API. If you want more exmaples, have a look at the code on GitHub here. There are examples for Facebook, Twitter, and GitHub.

A URI Templates Implementation for Java

2012-04-22T16:48:56-07:00

In my copious spare time, I have managed to cobble together an initial implementation of the URI Templates spec (RFC6570), for Java. The project is up on GitHub here:

http://github.com/damnhandy/Handy-URI-Templates

The majority of the documentation is available in the README file, so have a gander at that for details on how to use it. It’s in the initial phases right now, but it’s a good time to kick the tires and provide feedback.

RESTful URIs, Unicorns, and Pixie Dust

2011-12-29T07:33:50-08:00

It often pains me to hear people talking about so-called “RESTful URLs”. If you’re using that term, or your spending the majority of you application planning designing URI structures rather than your media types, then chances are you don’t really get the concepts in REST.

Frequently, I see developers sit down and start doodling a “REST API” by mapping out a bunch of URI templates like so:

/users/{userId}

/users/{userId}/stuff

/users/{userId}/stuff/{stuffId}

I’ll admit that I’ve been guilty of taking this approach myself. For one thing, it’s easy to communicate on paper. Most folks in business roles are used to seeing site maps where the content layout and URL structure are usually one in the same. By laying URI templates, you’ve kind of accomplished the same thing. Folks can visualize a high-level structure of your application, but you end up backing yourself into a corner that is difficult to get out of. Stu Charlton perhaps summed it up best in one of his more insightful posts:

If one is thinking of “how to methods map to my URI structures”, we’ve basically lost the plot as to why we have HTTP, URIs and media types in the first place. The goal is to *eliminate* the API as we normally think of it. (It’s not about a library of calls over a network).

The problem is that in approaching application design with the URI structure first is that you’re doing things bassackwards. Some people do this because they’ve followed some debatable advice and identified all of the “nouns” their application and started to work out a series of URL patterns that map to these nouns. As they create these URLs, they’ve followed some questionable advice as to what constitutes a “RESTful URL.” Subbu has another nice post dispelling some of those claims, so I won’t get into it here. The problem with doing all of the URI structure work up front is that you end up create a set of type resource URLs end up becoming fixed. Clients now end up coding to a specific set of URI patterns and/or conventions that are only discoverable from your documentation. The URI says nothing about what the data looks like or how the client should interpret it.

Imagine for a moment that you’re a DBA and you’re designing a set of database tables. Which of the two activities are you likely to spend more time on:

The structure of your primary keys
The schema of your database tables

A few of us would opt for something like MySQL’s auto-increment function and we’d be spending the majority of our time on describing the schema. In designing a RESTful application, you should be focusing on the design of your media types rather than what your URLs look like. To be more blunt: you must be focusing your efforts on what the hell your data looks like to consumers of your application. In addition to that, you need to think about how you are expressing links to other resources within your application or resources that are external to it.

This isn’t to say that URL design should be arbitrary and delegated to your web framework of choice. Of course not, you should still have URI strategy. The point is that the specific URI structures are not what consumers of your API will have to deal with directly. It is bad form to make a client rely on “out-of-band” information to construct a URI in your own special little way in order to get into your application. Take this blog post for example. You likely followed a link posted somewhere else. It could have been from an Atom feed, a search result in Google, or a Bit.ly link off of Twitter. It is highly unlikely that you had to type in the URL yourself and figured out the WordPress permalink structure that I have enabled on this site. If you did do that, well then, you’re awesome!

The fact is that clients will enter your application from some entry point or bookmark you’ve defined (.well-known is looking promising), or somewhere else on the web. Unless you’re a major player like Amazon, Facebook, or others, these clients won’t know that you have documentation that painfully detail your APIs URI structures. These clients will simply follow a link into your application. They didn’t type it in according to your fancy-schmancy URI template scheme that is only found in your documentation. The URL is only a means to identify an locate a resource on the web, it does not define how the resource is represented or make suggestions as to what it’s about (remember that URIs are opaque?). At the end of the day, the client is going to have to be able to understand the media type that is retuned by requests made that URI. If you spend all of your time up front mapping methods to URI structures, you’ll end up introducing a coupling that you can’t easily break free from.

URL vs. URI: URLs as Queries and URIs as Identities

2011-12-28T08:15:06-08:00

Continuing my with my ranting about the URL vs. URI bit, I thought I’d continue on given my renewed interest in this topic thanks to Ora. In our LEDP position paper, we made the observation that URLs represent queries while URIs are identifiers. If you’re wondering why you should care about this subtle distinction, please read on.

URLs as Queries

We’ve stated that URLs are queries, but what does that really mean? Those of you familiar with blog software such as WordPress, know that the default URL pattern might go something like this:

http://www.damnhandy.com/?p=399

Here, the URL forms a query for a blog post using its internal identifier. In this case, the URL is asking the WordPress database for a post and related items using the primary key of a row that represents the post. For most, it’s pretty obvious that the query parameter “p” refers to the internal identity of the post.

As we mention in the paper, there are many other ways to construct URLs to the same post. For example, we can embed the ID into a path segment:

http://damnhandy.com/archives/399

In all of these cases, the server application is interpreting the URL and using elements of the URL to internally resolve the information the client requested. While I’ve only singled out WordPress, this pattern is quite common among several web application frameworks.

Internal Identity vs. Global Identity

When folks put information on the web, the content they publish usually has two identities:

An Internal or local identity. This maybe the name of a file (i.e. “me.jpg”) or a the primary key of a row in a database.
An external identity which is the ID of the information you’ve published. On the web, this is the “global identity” exposed by the URL of the content

Often, people don’t tend to think about either much. The global identity of a resource is usually an after thought and is determined by the underlying framework driving the application. With web servers serving up documents, we’re usually exposing the the local file name of the of the document. With database driven applictaions, we’re exposing the primary key, or some alternate key, of a row in a database. Quite often when web applictaion changes frameworks, we see the global identity change too and the URL patterns change (i.e. .NET to Java’s JSP, to Ruby, etc.).

Using the previous WordPress example, we know that the internal blog post ID is “399″, but this internal ID really isn’t suitable as a globally unique, unambiguous identifier. Another blog using WordPress, running the same exact version of the software, could also have a blog post ID of “399″. This does not means that the two sites have the same content, it only means that the two instances happen to have a post with an internal ID of “399″. As you might have noticed, the value “399″ isn’t a suitable web-scale identifier. We need something else.

Globally Identifiers

Subbu Allamaraju had an interesting post a while back about Resource Identity and Cool URIs. Subbu asserts that there is a distintion between the identity of a resource and its location and his key point is spot on:

URIs uniquely identify resources but a URI used to fetch something is not always a good candidate to serve as a unique identifier in client applications.

And this is where I feel that the core confusion with URLs and URIs: identity vs. location. If we look at his initial example, he desribes the following:

<accounts xmlns="urn:org:bank:accounts">
  <account>
      <id>AZA12093</id>
      <link href="http://bank.org/account/AZA12093" rel="self"/>
      ...
  </account>
  <account>
      <id>ADK31242</id>
      <link href="http://bank.org/account/ADK31242" rel="self"/>
      ...
  </account>
</accounts>

In his example, the internal identity of each account is being expressed through a path segment in the href attribute of the link element. This approach is functional and is similar to that of the previous WordPress example.

The problem with this approach is that the ID values are only unique within the domain “bank.org”. There’s no reliable way to assert that two sites are referring to the same account if we have to rely on the value of a path segment or query parameter. As stated earlier, if we take WordPress as an exmaple again, blog post “399″ might talk about the Kardashians, or something else. There’s no gurantee that the two URLs refer to the same information if they share the same internal identity. Most likely, they don’t.

We could make the link and the ID one in the same:

<accounts xmlns="urn:org:bank:accounts">
  <account>
      <id>http://bank.org/account/AZA12093</id>
      <link href="http://bank.org/account/AZA12093" rel="self"/>
      ...
  </account>
  <account>
      <id>http://bank.org/account/ADK31242</id>
      <link href="http://bank.org/account/ADK31242" rel="self"/>
      ...
  </account>
</accounts>

You might wonder what the hell is going on here since it looks pretty much the same as the first example. The difference is that we’re saying that the ID and the link are identical. That is, the identity of the account is the URI. That URI also happens to be a URL that can be dereferenced. This works, and it’s considered basic principle of Linked Data.

However, there are some problems with this approach too. As Subbu rightly points out, URIs are not always cool URIs. We are all aware that URIs do change at some point. If Bank.org is acquired by BiggerBank.com, what happens to the ID since we tied the ID to a host name that is likely to be retired soon?

One solution is to follow good web practices and maintain the bank.org domain and either redirect requests to the older URLs to the new ones. Adobe does this with links to FurtureSplash and Macromedia Flash locations. These URLs all resolve to the Adobe Flash product pages.

This strategy allows us to keep the original identity but the link is changed to accommodate the new domain. We can expand on this strategy and change the value of the link:

<accounts xmlns="urn:org:bank:accounts">
  <account>
      <id>http://bank.org/account/AZA12093</id>
      <link href="http://biggerbank.com/accounts/?id=http%3A//bank.org/account/AZA12093" rel="self"/>
      ...
  </account>
  <account>
      <id>http://bank.org/account/ADK312423</id>
      <link href="http://biggerbank.com/accounts/?id=http%3A/bank.org/account/ADK31242" rel="self"/>
      ...
  </account>
</accounts>

Yes, it looks ugly and weird, but it’s valid and it works. If you look closely, it’s not much different from the initial WordPress URL. The only difference is that we’ve replaced a numeric identifier for a URI. It’s a URI that references another URI, but it is valid. For some reason, people just don’t like URLs that look like this.DBPedia does this since they’re describing data on Wikipedia:

http://dbpedia.org/snorql/?describe=http%3A//en.wikipedia.org/wiki/BMW_7_Series_%2528E23%2529

The big difference with this approach is that it’s clear that the identifier is globally unique. There’s significantly less ambigutity about the ID: http://en.wikipedia.org/wiki/BMW_7_Series_(E23) than the ID: BMW_7_Series_(E23). Because no one else can mint valid URLs within the Wikipedia domain, you can have greater confidence that multiple applications are referring to the same thing. URIs as identifiers are globally unique.

The global identity of information resources shouldn’t change as frequently as it does. It drives my wife apeshit that all of her recipe bookmarks change everytime MarthaStewart.com updates their site. Part of the problem I believe is that most folks doing Information Architecture don’t take identity into consideration and that a fair number of web frameworks do very little to assist in quality URI/URL design. But this post is long enough, so I’ll save that for another post.

W3C Linked Enterprise Data Workshop and more about URLs vs. URIs

2011-12-15T08:21:44-08:00

Last week, I had the pleasure of attending the W3C workshop on Linked Enterprise Data Patterns. My colleague Ora Lassila and I gave a presentation on the isues of identity in Linked Data. You can read our position paper and view our presentaion here:

Identity Crisis in Linked Data

View more presentations from Ryan McDonough

It would seem that many share the same frustrations that Ora and I have with the confusion between URLs and URIs, and the idea that URLs are queries and URIs are identifiers. Readers of this blog will note that over the past few years I have been trying to clarify the difference between a URL and a URI. I think it’s safe to say that even after multiple, detailed posts on the topic, the distinction between the two remains murky for some. Hopefully we can start to clear things up.

Amazon Makes the Kindle Fire an Easy Target for a Thief

2011-11-26T07:55:00-08:00

Having recieved a Kindle Fire this year, I was really surprised at the unpacking process and minimal security. While the device is very nice, the whole unpacking process made me realize that the Kindle Fire is a package thief’s dream. Here’s a few reason why:

Packaging
One thing that makes stealing a Kinlde Fire easy is the packaging, Amazon makes it quite clear what is in the box. When it arrives to your door, or your bulding’s mail room, the package is clearly marked with the words “Kindle Fire” all over it. In my case, the package sat in our building lobby clearly advertising what’s inside:

This makes it really easy to identify which package has the 7-inch tablet inside.

Automatic Sign-In
When I first opened the package and turned on the Kindle for the 1st time, it skipped past the sign-in screen since my wife’s credentials where already entered and she was automatically logged into Amazon. At first, this seemed like a nice touch for folks like my grandmother who don’t quite get sign-in procedures. However, this approach has far more cons than advantages.

1-Click Enabled by Default
The real kicker is that 1-Click is enabled by default. When you purchase content, you are never prompted for a password. This gives potential gift takers an added bonus: the ability to purchase Amazon content on your dime. And if you have kids, they can snag games as often as they want.

So if you’re considering getting an Amazon Kindle for someone this year, make sure you check the box that says “this is a gift” as this will make sure the device doesn’t sign you in automatically when you get it. Even if you’re going to get it for yourself, this is probably a good thing to do. Second, if you live in an apartment building or condo, you might want to consider shipping the box to your work or someplace where the box isn’t in plain sight.