Identi.ca is a microblogging service brought to you by Control Yourself, Inc.. It runs the Laconica microblogging software, version 0.4.1, available under the GNU Affero General Public License.

Unless otherwise specified, contents of this site are copyright by the contributors and available under the Creative Commons Attribution 3.0. Contributors should be attributed by full name or nickname.

It is based on a new V0.1 open microblogging protocol from Evan Prodromou. In general, the tech blogosphere is excited about identi.ca and the open microblogging protocol it is based on

• Identi.ca: May A Million Twitters Bloom by ReadWriteWeb
• Federating identi.ca? by Dave Winer who is a big fan of federated microblogging.

identi.ca supports OpenID, that’s cool because I don’t have to remember yet another username and password. But the real cool thing is that the code is open source, get it here, and already a couple of people have installed their own instances of it. Here’s detailed instructions from one developer from his successful installations (2 of them) and another supposedly simpler set of installation instructions here on a shared host. I haven’t tried them yet so use them at your discretion. Also, it has attracted the attention of other developers that are going to start tweaking the code to make it more scalable, such is the power of open source especially since Laconica’s GNU Affero license means any changes made on anyone’s server needs to be released back as well. Developers are also starting to experiment with federating different instances of Laconica where a message posted on one instance of Laconica, e.g., http://laconica.kamleitner.com/, shows up on another Laconica instance, e.g., http://identi.ca and vice versa.

Since the launch yesterday, the site was up and down several times, so hopefully it will get better over time, I sure would hate for it to suffer the same dreadful Twitter scalability issue. It was probably due to the buzz in the tech blogosphere that generated a ton of traffic. Even with the site being inconsistently available, a nice feature is the XMPP support. I was able to interact with it via my Pidgin / Gtalk setup even when the site isn’t available and that’s very handy.

I am excited about this for data portability because with it being open source, developers can add support for relevant data portability technology such as XFN, microformats, FOAF, RDF, RDFa, etc. If this really takes off, it can mean serious trouble for Twitter. Even without identi.ca, the tech blogosphere have talked about migrating to FriendFeed from Twitter due to its scalability issue. To get an idea how interested the blogosphere is in identi.ca, it has 10,000 registered users in 3 days from launch.

On a side note, I am spending less time on this blog and more time on my Ruby on Rails blog as I ramp up development for my next startup.

IMHO, this is huge for data portability, in this case, OAuth support for all Google Data APIs, everything from Gmail contacts to Google Calendar to Docs to YouTube. Bottom line, users no longer need to give up their confidential Google accounts username and password to 3rd party services in order for the 3rd party services to access their data on Google services. I suspect that Google is doing this because it helps them become the service provider of choice using an open and standard means of authentication hence channeling even more traffic through Google servers that they can figure out how to monetize later, much like their Friend Connect effort.

This is a major win for OAuth, in fact I would say that OAuth has now become a bigger player than OpenID in the space of data portability technologies. Given the recent history of big players announcing back to back support of similar features, I predict (ok I hope) that Facebook and Microsoft will follow suit.

MySpace is taking a much more interesting approach than Google, which controls data sent to third party sites via an iframe. MySpace is actually streaming data to these sites, which allows for true integration between the services, not just a bolted-on social tool.

My initial reaction is awesome, now I (as a 3rd party service provider) can consume the open user data but reading further into the article

Since actual data is being streamed out of MySpace, they have a strict terms of use policy that forbids third party sites from storing or caching the data, other than the unique MySpace user id of the user. Each time a page is rendered the third party must re-request the data from MySpace via a set of APIs. That means any changes by the user to their MySpace profile data or friends list will be instantly applied across third parties who access the data.

So basically MySpace TOS forbids me to do anything more than what is currently allowed by Google Friend Connect. Granted that there is a technical difference between the two, Google Friend Connect uses an iframe and MySpace actually lets the data out, there is no inherent difference in the 3rd party service provider ability to consume the data. In fact I would argue that it is more work for the 3rd party service provider to provide a UI page to render the data rather than just sticking in an iframe and letting Google do the heavy lifting.

Saying that MySpace’s data availability solution solves the problem of constant syncing of data so that the users remain in control is like Facebook saying that they are blocking Google Friend Connect due to user privacy concerns. IMHO the real reason is to maintain control and quoting the user privacy concern is merely a convenient PR front for both companies. I am surprised that Arrington is buying into MySpace’s PR spiel especially since he called Facebook on their user privacy concern blocking Google Friend Connect.

I come across such privacy discussions frequently but nowhere have I seen any consensus on privacy standards other than broad sweeping statements or wishes such as (quoting from the above article)

I have a policy which I assert over my stuff that I control, which is this protected zone in the cloud. And I have some real control over how I define policies over that thing and who gets access to it and on what terms. And I get to audit that access in a coherent way.

As I said in a comment on the above post, this is a loaded statement or perhaps the devil is in the details. It’s not like people haven’t tried, I know for a fact that there were several such discussions at dataportability.org that never reached any consensus (check out the policy group threads), plus numerous posts by Michael Arrington, Robert Scoble, Marc Canter, Steve Gillmor, etc. My personal take on it “user privacy is a personal and individual thing and is also dependent on the context of usage“. I attempted to define some levels of privacy controls that make sense but it is far from complete. Marc Canter has a similar idea and calls it dynamic privacy.

I understand that this is a difficult area to tackle but I think it is time to define open privacy standards much like the approach taken by OpenID and OAuth. Let’s start with something small and tangible for 1.0 version and build on it based on real-world usage on what works and what doesn’t work, essentially going it the agile way.

• Social Networking Gets a Sanity Check by Om Malik
• Social Networking: Taking Off or Taking a Dive? by Sarah Perez

In particular, Malik thinks that the market has transitioned onto niche social networks, in his words

The way I see it, the market has shifted its focus onto niche social networks, such as those dedicated to sports, music, automobiles and pets. You know, sites like Dogster! They have focused, engaged communities, which means they can attract a higher amount of advertising dollars.

In the very first comment on that post,

Niraj said:

While it would be easier to monetize, I can’t see the niche site idea working very well because users would have to sign up at several sites to cover their different interests. Facebook and Myspace’s broader appeal give them a much larger base, and it seems like a tradeoff: easier to monetize versus easier to gain users.

Like Malik, I am seeing a shift to niche social sites because it is getting incredibly harder and harder for horizontal play sites to gain mass traction, I am also speaking from personal experience with my video aggregation site, JiggyMe.

But Niraj does have a point that it is a pain for users to repeatedly sign up and re-enter the same profile information and recreate the social graph over and over. And this is what data portability advocates have always said, a user shouldn’t have to do that time and again. The real value-add for social sites should be the services they provide, not the user and social graph data. I am seeing this happening already now with projects like DiSo, Google Friend Connect, Open Social Foundation, Google Social Graph API, and stealth startup chi.mp. These startups and projects aim to make user and social graph data more open and accessible so social sites can instead focus on providing value-add services instead of keeping users data walled in. Mind you, this is just the start and there is plenty of work to be done to make the user experience seamless, but it is a good start.

• DataPortability.org May Progress report, click on either the HTML or PDF link to see the actual report.
• Trent Adams, an active DataPortability.org contributor and founder of MatchMine, interviewed me on my in the wild posts for his regular podcast series. You can listen to my podcast interview here. While you are there, check out his other podcast interviews with people like Plaxo – Joseph Smarr, Chief Platform Architect, John McCrae, VP Marketing; Google – Kevin Marks, Developer Advocate for OpenSocial, Robert Scoble, Managing Director of FastCompany.tv, etc.

As a side note, I plan on changing the focus of my startup, JiggyMe (video aggregation site), to feature only technology videos, so feel free to add your technology videos there.

DiSo Project Background
Straight from the DiSo Google group About page

Social networks are becoming more open, more interconnected, and more distributed. Many of us in the web creation world are embracing and promoting web standards – both client-side and server-side. Microformats, standard apis, and open-source software are key building blocks of these technologies.

DiSo (dee • zoh) is an umbrella project for a group of open source implementations of these distributed social networking concepts. or as Chris Messina puts it:“to build a social network with its skin inside out”.

You can also listen to an interview by Chris Messina on Chris Messina about DiSo.

At this stage, DiSo plugins only work on self-hosted WordPress blogs which means if you have a blog on wordpress.com, you are out of luck. Also, all DiSo plugins currently are written in PHP, WordPress’s choice of language. Visit the WordPress site to get instructions on how to host your own WordPress blog and install plugins.

Application Overview
The DiSo WordPress profile plugin has the following main features

• When a user signs up for a WordPress account, the plugin makes it easier to import the user’s profile information via hCard and XFN (if available)
• Once a user has signed up for a WordPress account, the plugin makes it easier for the user (now blog owner) to publish their own profile with standards like hcard, XFN
• Supports permission features allowing blog owner to restrict access to his information based on predefined relationship, e.g., I can’t see his phone number but friends of him who login with their OpenID and are present on his authorized list of friends can see his phone number
• There is a sidebar widget that displays names/avatars of most recently logged in visitors

Technology
The key technical pieces are hCard, XFN (rel-me, rel-contact), PHP5, and standard WordPress plugin architecture. The plugin should work on WordPress 2.0 and above, and has been tested on 2.3 through 2.5. Currently the plugin mimics SGAPI functionality without the FOAF bit. Also, FOAF was considered but not implemented, another item for the future perhaps. He plans to add Google Social Graph API (SGAPI) support, but it wasn’t available when the plugin was written, so it is something to consider for the future – Steve Ivy wrote a PHP wrapper for SGAPI.

While the plugin works with OpenID, it does not require OpenID. There is a button to import profile and can fetch profile information if it is not an OpenID URL. OpenID profile extraction for XFN and hCard is automatic upon registeration and login. For OpenID feature to work, it needs the WP-OpenID plugin. No other libraries or plugins are required, in fact the import button works fine if the WP-OpenID is not installed. To display the user’s profile, the user needs to add a WordPress template tag. There is a page token for rendering on a WordPress page and a PHP function for addition directly to the template (documented on the plugin page). So far, most people don’t use it as a sidebar widget and instead display their profile information inline in the blog.

For an example of the plugin in action, check out Stephen’s blog, it powers the top half of his main page and the avatars of recent visitors in his sidebar.

Lessons learned
Some people have hCard on their OpenID pages via OpenID delegation usually or directly on the page. A large number of people have rel-me links going to their main profile somewhere else. In his opinion, the biggest hurdle is still HTML parsing in PHP which is surprsing to me since PHP is such a popular web development language. Event though PHP has excellent XML support but if the HTML is broken or incomplete as it is often the case in the world wild web, there is no library to handle that. An option is to fix it with HTML Tidy but most shared service providers (like DreamHost) do not have HTML Tidy installed. Without HTML Tidy, the plugin has to run the page through W3C remote tidy proxy which can be slow. Another option is to use HTML Purifier which is a re-implementation of HTML Tidy in PHP.

The current plugin user base is primarily DiSo developers and he has not gotten any feedback from non-DiSo developers. He noted that there is a goofy WordPress thing where the permissions model is based on the contacts list but WordPress only supports one blogroll list, so everyone on that list has the same permission. This is not a problem for most blogs but it could be a problem for multi-authors blog. There is no affiliation with WordPress other than it is a WordPress plugin.

Application Overview
Basically with floe.tv we started out just wanting to try out some video ideas we had. We started out with a simple playlist of videos on the internet linked together at a site, which evolved into a full own video editor. After a number of months of development we hit a point where the team sat down with some users and did a testing session, asking questions and gauging responses to see how well we were hitting our marks. We came to the subject of data storage, local hardrives, and getting media online, and just as a thought exercise I asked “well, what if floe.tv just knew about all your online media by your login name, and referenced it automatically in your libraries the first time you logged in — just as if it was an app installed locally on your hd?” and immediately both of them became excited and one asked “can I do that right now? when can I use that?” and I knew from experience that the market was speaking very loudly and clearly in my direction, and that I had better listen very closely.

The very next meeting I posed this question to our team:

What if our app was “inherently installed” in the internet? What if someone logged in, and the app just acted like a desktop app that “knew” about your flickr images, your youtube videos, it knew about your myspace friends, facebook friends, and automatically treated them as one logical database, one logical social graph? And someone started right into an app tutorial right off the bat with their contacts, files, and assets already referenced (but fully respecting privacy, control, etc)?

So the next question naturally becomes “that all sounds really great, but … how do we get there?”

From there we really began to push a “what if this/that” scenario, and drew up our ideas into a document entitled WRFS and from that we began to re-engineer the floe.tv app towards a truly linked data experience.

Technology
We are currently using FLEX/as3 for the editor and player with ASP.NET for the server technology. Discovery is a big deal with how I view next gen web apps — dynamically finding data at runtime without having to go through “Big Data’s” walls to get to it. How I see this happening is

1. the user logs in with an identity url, be it an openID or a XRDS file location. floe.tv is an openID only application, but user’s can map multiple openIDs to one account with it.
2. the app authenticates the user, and then uses multiple fallback methods to find the XRDS-S file if not specified.
3. The XRDS file is parsed, and relevant data types (here image and video) location uris are pulled out
4. Each uri points to a data container, which is then queried via its API for a list of resources the user has stored there.
5. these results are then aggregated back together into a single “recordset” to be returned to the floe.tv application layer

The fallback methods for XRDS discovery (done in the FLEX client) are ordered as:

1. First do basic yadis discovery to see if the identity delegate is the openID provider or a blog of some sort head link delegation setup. Either the openID provider or the XRDS location may here, or both. In some cases, such as the DiSo XRDS plugin, the XRDS file located in the head link tag will have the openID provider location.
2. The secondary method we have kicked around is to query the openID provider for an Attribute Exchange key that points to the XRDS file. This is not well defined yet but has been discussed amongst various groups.
3. Lastly we fallback to having our flex app prompt the user for a XRDS url so that we can “enhance their user experience”.

So although I think our secondary option with the AX key is a little shaky right now, overall we degrade gracefully.

For a quick demo of how some of the data query and aggregation mechanics might work, I’ve built a short demo illustrating the step by step mechanics.

Lessons learned
We aren’t done with this application, obviously, and a lot of work remains to be done – I should note that I am currently at Step 2 of 5 as stated above. However, the application is evolving into the embryo of what I think a linked data application can and will be. What I can share are the places that we are actively looking for solutions, simple decentralized solutions, that solve these issues.

One thing that is slowly changing is the perception of cross domain scripting on the internet. As services get more complex, and require more aggregation of data from multiple sources, we are going to push more data handling duties to the client, as scalability will suffer. For flash crossdomain scripting we need the crossdomain.xml file present on the server of the api would like to call for it to work. This is a trivial thing to setup as it consists of a simple xml file located off of the subdirectory level you wish to grant access to.
Examples:
http://api.flickr.com/crossdomain.xml
http://api.floe.tv/crossdomain.xml
Once this file is exposed, the flash runtime will then allow calls from the client to those servers.

For cross domain client side javascript, things get a little bit trickier. A lot of cross domain tricks, such as widget embedding, is done via iframe embeds. This type of embedding significantly restricts access to the rest of the page, so effectively the widget is isolated in terms of interaction with the rest of the page DOM. Firefox 3 will allow cross domain client side scripting with certain http headers being present on the server response from the remote cross domain endpoint. I’m not sure how future versions of Internet Explorer will address this issue, but I think evolutionary pressure from both Firefox and Flash will yield some effect there. A new development that is supposed to make javascript more secure is Google Caja. I’ve begun to follow Google Gaja but I’ve yet to deeply jump into that project.

We’ve been waiting to see how the discovery wars pan out, as it stands now XRDS-S is looking like the service index of choice amongst the big players (presently Yahoo is endorsing it, hiring Eran Hammer Lahav as their Open Standards Evangelist. How the XRDS resource is discovered automatically without a tremendous amount of user interaction is something that we are taking many approaches towards, as discussed above. For now we’re going to focus on finding XRDS files as our catalog of service endpoints for a user. The DiSo project is going to be publishing its user’s service endpoints in the XRDS format and already has a plugin for it, so I think in the short term we’ll be focusing on consuming that data in terms of an early conceptual demo in runtime linked data.

Once we find the XRDS file, we aren’t out of the woods. How do I set my XRDS file up so that I can tell the floe.tv application that I have “images in flickr”? This is a fundamental question being setup and worked on at the site http://www.xrdstype.net/ by many different groups and people, and has yet to fully be fleshed out.

There’s also the issue of once we find a data endpoint, how do we talk to it? ATOM? Some sort of standard data interop api that spits out RDF? In a perfect world I’d love to see a self organizing web, a linked data web that can find a data endpoint at runtime, find its semantic schema, and wire itself such that it can talk to that api without ever needing user intervention — it simply will understand how to talk to it.

Another key development will be the permission system, possibly using OAuth+Discovery to automate the updating of a XRDS-S file for someone when they add data to a service. I need to learn a lot more about OAuth and the direction its heading. I’d prefer a world where the user doesn’t need to manually go to a site to allow their resources to be used by a third party, but for now, this is how we have to operate.

So really, I think I have more “lessons underway” than “lessons learned”, but sharing this information is key since it sparks interest in other like minded developers who may know a lot more about some of these areas than me. There’s going to be some places that we punt and just hard code some scaffolding code in to just get going, but over time I’d like to evolve towards a linked data web that auto discovers new connections at runtime and self organizes to give a smarter and far more intuitive user experience than we’ve seen so far.

If you are interesting in linking data or trying some data interop experiments, please feel free to email me (jpatterson at floe.tv) or check out the WRFS workgroup or my blog http://jpatterson.floe.tv

Feedback and suggestions are welcomed.

RDF is a powerful, graph-based language for declaring things, and attributes and relationships between things. In a simplistic way, one can think of RDF as the language that allows expressing truths like: Alex IS human (type expression), Alex HAS a brain (attribute expression), and Alex IS the father of Alice, Lilly, and Sofia (relationship expression). RDF is powerful, but because it is highly recursive, precise, and mathematically sound, it is also complex.

On a high level, FOAF has the following categories

• Person’s basic information such as name, nick, title, homepage, email, surname, first name, given name, email (expressed as mbox), etc.
• Person’s extended information such as blog, interest, project, school home page, work home page, friend(s) he knows, etc.
• Person’s online accounts (includes instant messaging)
• Person’s projects and groups
• Person’s documents and images

As you can see, it allows for extensive coverage of a person’s information. If you want the full details, check out FOAF Vocabulary Specification 0.91. Assuming you have a basic knowledge of XML, let’s take a simple example to illustrate how FOAF is actually written.

The above XML snippet describes a person’s name and email, obviously John Doe is not a real name and johndoe@hisemailprovider.com is not a real email but you get the idea. Technically this describes a person, not the “me” identity as defined by XFN rel-me.

Let’s take a more slightly more complex example

Basically the above FOAF says there is a person John Doe, his email is johndoe@hisemailprovider.com, he “knows” Jane Smith, and her email is jane.smith@hercompany.com. Note that “knows” does not necessarily equate to a friend, it simply means that you know someone, that person could be a friend, lover, co-worker, parent, sibling, or someone you met online but not in person, etc. For a more in-depth look at FOAF, check out XML Watch: Finding friends with XML and RDF.

Ok, now that you know how to create basic FOAF, what next. Unlike XFN / hCard / microformats, FOAF is delivered via a file separate from any HTML markup. Creating a FOAF file is not something that a Joe Blow user will ever do manually. However, if you are curious and want to play around with creating FOAF, you can use this handy tool FOAF-o-Matic. For most users, their service providers, the ones that support FOAF, will handle all the intricate details behind the scene. For example, if you have an account at MyBlogLog, you can check out their FOAF feature by appending “/foaf” to the end of your MyBlogLog member URL. If you aren’t a MyBlogLog member, here is Ken Brewster’s MyBlogLog FOAF file, http://www.mybloglog.com/buzz/members/kentbrew/foaf/, and check out his fun FOAF consuming app, picture below

Since FOAF is a separate file, you have to link to it from a HTML page by adding the following HTML code within your <head> tag (using Kent Brewster’s FOAF URL)

In MyBlogLog’s case, that HTML code is added automatically to a member’s profile page. I should note that in MyBlogLog’s case, the FOAF file is publicly available without any authentication allowing any search engine to crawl and index it, in fact, this information is available through Google Social Graph API but beware that there is a current limitation with Social Graph API in that Googlebot does not crawl a FOAF file linked from an HTML page, hence not making it available for the API to parse. In Brad Fitz own words,

I just verified that the problem is the crawl coverage. Historically Googlebot hasn’t cared about FOAF because it hasn’t used FOAF, so why hit your server fetching it just to throw it away? Now that the SGAPI is using it, though, I need to ask Googlebot to go get that FOAF that I know exists but is uncrawled.

A few parting thoughts…

• Chris Messina thinks that FOAF baking in relationship information is a fundamental flaw, I have to respectfully disagree because I think it is a key missing feature from XFN/hCard, at least when it comes to my XFN consuming app. Upon further reflection, I think I understand Messina’s concern about FOAF being potentially convoluted. It is due to the powerful recursive nature of RDF that can lead to circular references and if a FOAF parser doesn’t account for this, it can lead to infinite loops.
• Since FOAF only has a “knows” property, it is less descriptive, hence less useful, than XFN rel values (friend, acquaintance, co-worker, neighbor, sibling, parent, child, crush, date, etc)
• Apparently FOAF does not have properties to specify an address, e.g.
  state/province, country, city, street address, quite odd indeed. However, it can be remedied by adding VCard namespace to RDF.

• The general vibe in the web development community is that FOAF is more complicated than XFN so it is more likely to see XFN in the wild than FOAF
• Firefox Operator plugin does not consume FOAF information
• FOAF information is meant for app-to-app consumption unlike XFN / hCard which are embedded in HTML. Check out this post WordPress, FOAF, OpenID – updated for a WordPress plugin retrieving FOAF profiles when users authenticate a wordpress blog with OpenID.
• There is a recent effort, hFOAF, to combine FOAF, hCard, and XFN