What is YQL and what is it good for ?

The Internet as we see it (web sites, games, videos) is only a shop window to the thing that really drives it: data. All the things you see are based on information one provider offers another provider. Even you are a data provider – if you comment on a video on YouTube or you add a person to a photo on Facebook you create a data set. All this data is made available behind the scenes to different people – this could be developers in the same company or partners or – for example in the case of the Yahoo Developer Network – other developers.

The information comes in different formats. For example the photos on http://www.flickr.com/photos/tags/cat are available in “JSON” format at http://api.flickr.com/services/feeds/photos_public.gne?tags=cat&lang=en-us&format=json.

To you this might look like nothing much but developers have a lot of fun with this information. It gets especially interesting when you mix and match different information sources to get a new picture. You can for example map the news onto the world to give them geographical context.

All of this sharing of information follows certain conventions much like normal conversation does. You need to ask the right question to the right person in the right format to get the correct information back. In the case of data on the Internet this gets even more confusing as all the data providers speak different languages and give you information back in formats that you might not understand. Imagine going shopping for clothes and each shop assistant speaks a different language and each shop has different size charts for the same type of clothing.

In normal communication amongst people with different languages we have translators – sometimes even simultaneous ones. And this is what YQL is. YQL is a simple to understand language to allow you to get information from any source on the web and gives it back to you in a language you understand. Every resource on the web can get a simple name and developers can call this name up to get to the data it offers. So for example getting photos from flickr becomes

select * from flickr.photos.search where text='cat'

Translating a text to French becomes

select * from google.translate where q='hubcap' and target='fr'

You can also mix and match different sources, for example you can get the latest headlines from yahoo’s top stories and translate them to French:

select * from google.translate where target='fr' and q in (
 select title from rss where url='http://rss.news.yahoo.com/rss/topstories'
)

This language is very close to another language called SQL which is the standard for accessing information in databases. So, in essence, YQL turns the internet into a massive database where developers can access information and remix it to make it easier for end users to see relationships between different pieces of information or even develop interfaces that make it possible for users to get access to the information.

For example this great lecture on YouTube can be listened to by blind users using the EasyYouTube interface that uses YQL to get to the information of the video.

YQL can be accessed on a pure programmatic level but the easiest way to get a glimpse of its translation and access powers is to use the console.

YQL is a great communicator - it allows you to speak to all kind of data sources in the right language, ask the right questions for you, find the right internal phone numbers to get to even other resources needed to get to a certain piece of information and give back to you only what you asked for and not a whole mass of information you will never need. And it is amazingly fast in doing so as Yahoo built it to deal with exactly the same problem of data communication inside the company.

Last Thursday I took the opportunity of being in the Silicon Valley to give a talk about giving technology to the world, pointing out mistakes we made in explaining our services and APIs, what works well and how some competitors do a great job at explaining complex technology in an easy to understand fashion.

It was a great opportunity to explain the concepts of developer evangelism to an internal audience who hadn’t yet read anything about the matter of seeing developers as an audience.

Check the slides on SlideShare and the audio on archive.org:

Listen to the Audio of the talk on archive.org:

• Giving Technology to the World – MP3 46MB
• Giving Technology to the World – OGG 37MB

Do you know any great API sandboxes and documentation? I’d be happy to have more positive examples!

There is actually no magic to it – I am not a superhuman developer. The main trick to build products like that really fast is to use building blocks that work, separate concerns of the application (APIs for the backend returning HTML, JavaScript for behaviour and Ajax and CSS for look and feel).

Without further ado, here is the slide deck for you to go through:

The audio recording

The audio of the talk is available on archive.org

The preview in the slides

One new demo I built for the talk was Flickr Collector which allows you to quickly collect photos from flickr and get the source code for them easily add them to blog posts and articles.

The source codes

• Flickr Collector Source Code on GitHub
• CSS charts on GitHub
• GooHooBi on GitHub (and as a live build screencast)
• Geomaker on GitHub

The talk was filmed and will be available on the YUI Theater soon.

All you need to do to use the script is get your own Google Maps key and embed the script in the page you want to analyse, giving it the ID of the main text element as a parameter:

<script src="http://github.com/codepo8/geotoys/raw/master/addmap.js"></script>
<script>
addmap.config.mapkey = 'YOUR_API_KEY';
addmap.analyse('content');
</script>

You can customise the look and feel by writing your own CSS for the generated HTML and setting the addmap.config.width and addmap.config.height properties to resize the map.

Under the hood here is what happens:

• The script grabs the innerHTML of the element and runs it through the open YQL table of Yahoo Placemaker.
• It then takes the data, assembles the HTML and puts together the URI for the map
• The map gets displayed using the Google static map API

The code of addmap.js is available on GitHub.