Sappenin on Software

OInvite Draft-3 Released

2010-03-29T20:30:00.000-07:00

It's been a while since I've written about OInvite (or anything for that matter). But a few days ago, I had the chance to spend some more time updating the OInvite spec. Below is an overview of some things that have changed. For more information and to read the updated spec, checkout http://oinvite.net.

OInvite Core

First of all, I've decided to create a base specification called OInvite Core. This is what I'm referring to when I advertise "Draft 3". OInvite Core defines an XML based request/response document format to define the parameters surrounding an invitation. This core spec is meant to be a baseline for extension profiles of OInvite that can work appropriately under different situations. For example, there will be an HTTP profile of OInvite that specifies discovery and other aspects which work well for web applications. At the same time, I'm envisioning it might make sense to have a slightly different profile for XMPP, Google Wave, and posibbly something for regular email (heavy underline the "possibly" in that one).

OInvite Verification Extensions

The core spec provides a brief outline of the process a server should follow before prompting a user that a new invitation has been received. This is known as OInvite verification, but the exact details of how or what to verify are left to extensions to define.

Spam Prevention

You may notice that previous versions of OInvite had a Proof-of-Work scheme defined that worked to reduce and/or eliminate spam invitations. I'm still working to define this, but it will be in the form of an optional extension. Best of all, the core spec includes a way for servers to advertise any verification extensions in a programmatic way. Since these extensions are optional, individual implementors will have lots of freedom to experiment with the best verification mechanisms to protect from spam, verify senders & receivers, and ensure that every aspect of a particular OInvite has what it needs to work with a recipient's systems.

OInvite over HTTP

Next on my list is to create an HTTP profile of OInvite that relies on OpenID+WebFinger for authentication, XRD for discovery, and OAuth for authorization. This latter piece is going to be very interesting. It's starting to look like OInvite could be an automatic way for two users to exchange OAuth authorization tokens, which could open up some interesting new possibilities for private resource sharing across domains.

More to come!

Announcing OInvite

2009-06-03T08:34:00.000-07:00

<Some Context>
A while back, I posed a question to the Diso mailing list wondering if there was any work being done to enable cross-domain "friend requests" in the context of social networks and other systems that have the notion of "friend lists".
</Some Context>

As I soon found out, this is a pretty involved topic, so I collected my thoughts and shared them with the world in a blog post here (I recommend reading that post for a lengthy--if surfacey--background relating to the issues surrounding friend requests, open vs. closed communications relationships, and my general musings on this topic).

At the same time, I formalized some more thoughts surrounding this idea into a spec I'm calling OInvite. Perhaps "spec" is too formal of a word. In reality, I just like to use the xml2rfc tool, so it was more enjoyable to try to hone my ideas in the confines of a specification. To say that this document is anything close to a spec would be an understatement!

:)

At any rate, the idea behind OInvite is to codify an open protocol that helps facilitate the creation of unsolicited "communications relationships" (i.e., friend requests) between various parties in unaffiliated domains without the worry of spam.

Some key attributes of OInvite (see my musings post here for more background on what these terms mean):

Open: Communications Relationship ("CR") requests can be send to users with identifiers in unaffiliated domains or federations.
Bi-Directional: OInvite assumes all CR's are two-way, meaning information can be sent and received by both parties (so long as participants can elect to ignore messages from a particular sender, uni-directional relationships can be simulated in a bi-directional CR).
One-to-One: OInvite always involves a CR's between only two participants (a.k.a., cardinality) because this greatly simplifies the spec, yet still supports one-to-many communications.
SPAM-Free: OInvite provides support for a pluggable mechanism to guard against "first-contact" spam, which is the only type of SPAM available in a white-list type communications environment, such as a social network (in systems like email, this pluggable model is not sufficient to prevent spam due to the implied "all senders are good" principle underlying the design of SMTP).

Check out the specification, and feel free to share your thoughts in the OInvite discussion group or on the Diso list.

The Case for an Open "Friend Request" Protocol to Enable Communications Nirvana 1.0

2009-06-03T08:00:00.000-07:00

A while back, I posed a question to the Diso mailing list wondering if there was any work being done to enable cross-domain "friend requests" in the context of social networks and other systems that have the notion of "friend lists".

There was some interesting discussion surrounding this topic, and I've been meaning to solidify my thoughts around this whole idea. Given that I love to utilize the xml2rfc tool, I decided to just codify my thoughts into a formal specification. However, I quickly realized that I was creating a lot of new terminology, so this blog post is my attempt to provide some background explanation about the "what" and "how" of cross-domain friend requests, as well as to illuminate my new proposed specification, OInvite.

So here goes...

A "Friend Request"?
First, you might be wondering, "What exactly is a 'friend request', anyway." Well, for my purposes I will use the following definition:

Friend Request: A request from an invitor (the sender of the request) to an invitee (the recipient of the request) to enter into a "communications relationship."

OK, there's some more new terminology.

First, the notion of a "communications relationship" is purposefully left quite vague. Perhaps two users (let's call them "John" & "Beth") simply want to email each other. That's certainly an obvious form of communication. However, communications relationships are much bigger than this. Imagine if John wants to allow Beth to see his photographs on Flickr...that's communication, too, and it generally involves some acceptance of this communication.

Thus, a "communications relationship" is the term I invented to describe an agreement between two parties to send and receive information.

In tandem, an "OInvite" is a request to enter into such a relationship.

In Facebook terms, this type of thing is generally referred to as a "friend request"; in twitter terms, a "follow me" request; in OAuth terms, it's "I want to share data with you" request, etc.

One-to-One? Uni-Directional? Bi-Directional?
As we travel down the rabbit-hole of "communications relationships", we quickly encounter the notion of directionality. For example, what if John only wants to share (read: send) information with Beth, but doesn't really care to receive information from Beth? Such a relationship would be considered "unidirectional", because information only flows one way. However, if John wants to both send and receive information to/from Beth, then a bi-directional communications relationship would be required.

In addition to directionality, "friend requests" might be sent to a single individual, or they might be sent to a group of invitees (a.k.a: recipients). This is the notion of Invitation cardinality, and quickly adds to the complexity of a communications relationship. For example, should an invitation be "one-to-one" (an invitation from one user to a different user); one-to-many (an invitation from one user to multiple users); or many-to-many (I'm not even sure if this is possible).

;)

Closed vs. Open Communications Relationships
Going even deeper, there's one last area of communications relationships that should be considered/defined, and that is the notion of an "open" vs. "closed" communications relationship.

On today's Web, we are commonly exposed to "closed" communications relationships. Facebook users can invite other Facebook users to become "friends", thereby allowing communications of various types (messages, photos, activity-streams, etc). Twitter users can only "follow" other Twitter users; MySpace users are limited to communicating with other MySpace users, and so on. These are all considered to be "closed" relationships because a Facebook user (for example) cannot easily "becomes friends" with a user from Twitter, without that Twitter user first having a Facebook account with which to interface to Twitter with.

Conversely, the notion of an "open" communications relationship would allow Facebook users to "friend" users on any other social-network (assuming the two networks could speak the same protocol). For example, John (on Facebook) might invite Beth (on Orkut) to "share information".

We don't see this type of thing today for a myriad of reasons. For one, most social networking sites are closed ecosystems, so they have never bothered to support a "hook" into other social networks (this is beginning to change, e.g., Facebook Apps).

However, even as social networking websites "open up" their content (and access to that content), they still require an account on the "home" social network. In the example above, Beth (on Orkut) would need an account on Orkut as well as on Facebook to have John's Facebook data show up in Orkut.

These "silos" exist for a number of reasons (competitive, economic, technological, etc), but at the core they exist because there is no protocol that will allow Facebook (for example) to control the number of invitations coming from an infinite number of other social networks. In essence, even if the Internet community could overcome the all the hurdles standing in the way of "open" social networking, there would still be the great white elephant in the room: SPAM.

Now, I'm not talking about the kind of spam everyone is familiar with in the email world. Instead, I'm talking about Invitation Spam.

Social Networks don't allow communication between users until each user has given some sort of "approval" to be communicated with (e.g., acceptance of a "friend request"). When a social network provider controls both ends of this interaction, that provider can easily throttle invitations, remove "bad" accounts, and more; thus controlling and eliminating SPAM of any kind.

However, in a truly open world, the ability to control the "bad guys" goes away, demanding some other mechanism that will both 1.) allow a nearly infinite number of cross-domain social networks to interact with users on the local social network (i.e., a completely open Facebook) and 2.) prevent unwanted invitation SPAM from ruining the entire user experience for legitimate users.

Email: Truly (Bad) Open Communications Relationships
O.K., so there's a lot to digest there...but we're not quite done yet. One last thing to consider before delving into the innards of OInvite is the current SMTP-based email system.

SMTP provides for cross-domain information sharing, or "open" communications relationships. In fact, the current email system is perhaps the most successful (and unsuccessful) "open" communications mechanism ever created in the history of the world. For example, messages from an email address in one domain can be freely sent and received to/from email addresses in another domain. As we all know, this is a blessing and a curse, enabling unfettered communication on the one hand, while at the same time enabling mass amounts of SPAM on the other.

Email: Communications Nirvana 0.1
Despite its shortcomings, email is pretty amazing. It has enabled an incredible amount of asynchronous communications, productivity (perhaps that's debatable), and more. However, if there is such a thing as "Communications Nirvana", then email is version 0.1 because it has, despite its open-ness, three significant shortcomings--all of which ultimately have to do with SPAM.

First, email lacks the ability to properly authenticate the sender of a message. I can send an email purporting to be from john@example.com, and it's somewhat difficult to verify whether or not "John" was the actual author of the message.

Editor's Note: The Internet is making progress in this area: SPF, DomainKeys, etc are all positive moves, but they're not universally required nor adopted, making SMTP fundamentally susceptible to this vulnerability.

Second, email puts all of the resource burden onto the recipient of a message, allowing senders to trivially send mass amounts of email at virtually no cost. Bandwidth, CPU resources, and intermediate data storage for email messages is borne by ISP's, and likewise each recipient, making it very inexpensive to send copious quantities of SPAM.

Third, email has no concept of a "friend list". Email was designed from the perspective that a particular user should receive all incoming messages. Today, this principle still holds, unless a message, sender, or domain is specifically blocked by a SPAM filter.

Social Networks: Communications Nirvana 0.5
So, if email is Communications Nirvana 0.1, then social networks are version 0.5. They solve issues #1, #2, and #3 above; but at the expense of open-ness (e.g., we can't send arbitrary messages to a Facebook account from a system in a different domain).

Yet, social networks are still pretty incredible. Virtually no spam; resource burdens borne by the network provider; and the concept of a friend-list that means I'm only receiving information that I elect to receive (for the most part -- I can't say that I really care about Facebook activity updates from 200 'friends', but again -- perhaps the topic of another post).

The Future: Communications Nirvana 1.0
Well, if you're still reading this, then congratulations--I'm now ready to make a point, which is this: If you buy the argument that social networks are an improvement over email, then the last remaining hurdle to getting to a 1.0 version of Communications Nirvana (in my opinion) is to "open" up social networks. This is was the Diso project is working towards, but it seems to me that as long as we sacrifice "open-ness", we'll never it make it to a 1.0 version of communication Nirvana, let alone a version 2.0, 3.0, or 10.0.

To become "open", we require a mechanism to enable cross-domain invitation requests which will allow us to enter into communications relationships with each other. In layman's terms, we need an open "Friend Request" mechanism that solves the problem of first-contact SPAM.

My answer is OInvite--an open protocol to enable cross-domain "friend requests" with a pluggable anti-spam mechanism--and since this blog post is so stinking long, I'm going to discuss the actual protocol in a different post, namely this one.

<Musing>
Why do we express incredulity when remembering the Internet of ~16 years ago, where for a long time people with an AOL email addresses could not send email to people with a CompuServe addresses? Today, we suffer this same abuse when it comes to the communications platform du-jour, namely, social networks.

Imagine 16 years from now when we tell our children how it "used to be":
Dad: "...in my day, you needed an account at every social network provider in order to be able to communicate with people there..."
Daughter: "Seriously? You're kidding, right?"
Dad: "Yep, if I wanted to talk to you on Facebook, I had to sign-up there. Then, if I wanted to follow you on Twitter, I had to sign up there..."
Daughter: "What's Facebook?"
</Musing>

Google (Tidal) Wave is here...with lots of questions.

2009-06-02T10:48:00.000-07:00

If you haven't yet heard, you will -- Google Wave is here. It's a pretty amazing thing, but prompts many questions in its wake (pardon the pun).

This blog entry details just a few of the provocative questions that Wave raises:
http://storm.alert.sk/blog/2009/06/02/Good-Vibrations

Great Synopis of the state of Web Metadata Discovery

2008-12-05T15:31:00.000-08:00

Eran Hammer-Lahav has posted a great synopsis of the state of Web Metadata Discovery.

Update: Eran has blogged about the Discovery Protocol Stack here.

Java-based EAUT libraries released...

2008-11-18T09:36:00.000-08:00

I've finally published a java-based EAUT library (Apache 2.0 license) that can be used to provide Email Address to URL translation in your java applications.

Read more about EAUT here: http://www.eaut.org.

Download the java binaries here (Maven2 supported): http://www.eaut.org/code/

View the the source-code here.

Automatic Email-Address Control Verification with EAUT & OpenID

2008-10-24T12:55:00.000-07:00

I recently read about an idea to accelerate OpenID adoption by turning-on RP's to the notion of using OpenID as a means to automatically verify email addresses.

As one of the author's of the EAUT Specification (EAUT Draft 5), I have some ideas on how to make this work. This blog-entry sums up my current thinking, so give it a read, and let me know if it could work, or if I'm just an idiot. ;)

Here goes....

Email Address Verification vs. Validation
Before diving in, it's important to clarify exactly what I'm talking about here. Please note that the definitions I'm presenting are completely made up by me, so it's possible you will have seen these terms and phrases elsewhere. However, for this blog-post, here's what they mean.

Email-address Verification merely ensures that the user[1] of a particular email address actually controls that email address.
Email Address Validation determines whether or not an email address works (i.e., does it properly receive email messages, and does the user respond). Validation can also provide insight into the "quality" of a particular email address, such as "Is this address controlled by a human or a robot?" and "Does a lot of spam come from this address?", etc.

To be clear, I'm mostly talking about email-address verification (of control), and not validation (though once an email address has been verified, some assumptions can be made about its validity -- more on that later).

Why is Verification Important?
Verification is important for a number of reasons. Most important, however, is that many public-facing websites (a form of Relying Party, or RP) use email to communicate with their users. These RP's know who each user is (based on the user's identifier), because that's how each user registered with the RP website (I should be more accurate here: each RP knows, at least, that the user logging into the RP controls the id used to login with -- who the actual human doing the login is....well, this is a different question).

At any rate, since many websites use email to communicate with their user-base, an email address is almost universally collected at registration. At this stage of the game (directly after the user has supplied an email address), the RP has no way to know that the person controlling the login also controls the email address that was just entered into the RP's system (e.g., a user named Beth might enter the email address 'gwb@whitehouse.gov', which she likely does not control. Most good RP's want to be sure they're sending emails to a person who wants to be receiving those email messages...otherwise it's spam).

Thus, RP's need to ensure with some high degree of certainty that the person who controls the userId used to login to the RP, also controls the email address that was supplied.

The most common form of email verification today is to send an email message with a secret code, and ask the user to either click a link (or enter the secret code directly) to verify that the email address is indeed controlled by the user who entered it. It is important to note that this also tends to semi-validate an email address as well, from the perspective that a successful email address verification also tells the RP that the email address in question can receive email messages (though it says nothing about the quality of the email address).

The Good Stuff: How to Create Automatic Email-Address Control Verification Service

Ok, so how to verify control of an email address? Follow these simple steps.

Start with OpenID.
OpenID was created for a variety of reasons, but one important reason was to allow Relying Parties (i.e., websites that we login to every day) to assert that the person logging into a particular site with a particular OpenID actually controls that OpenID. With OpenID, an RP can always be assured that the person logged-in controls a particular OpenID.
Provide the RP with an Email Address.
This part is not really defined for this article. There are lots of ways to do this, including "ask the user" for their email address, SREG, Attribute Exchange, and others like "make the email address a first-class OpenID". Additionally, there's the EAUT protocol, whereby a user can enter only their email address to login to an RP. The email address is then captured by the RP, transformed into an OpenID URL, and the user logs in via OpenID per usual (click here for an example of this translation in action). With EAUT, at least, the RP automatically "knows" the email address of the user, because that's what was used to login to the RP in the first place.

That said, I'll mainly defer on these options for now, since the manner in which an RP gets a user's email address is tangential to verifying the email address, which is the subject of this blog entry.
Utilize EAUT for Verification.
The EAUT protocol specifies one method for mapping an email-address to an URL. The idea was originally proposed as a way utilize email-addresses in the OpenID authentication process. The rationale for such an idea was (and still is) that it's much more intuitive for a user to enter their email address, as opposed to having the user enter a URL that may be hard to remember, or un-familiar, etc.

Ironically, EAUT is also applicable to many use-cases outside of plain-old-vanilla OpenID Authentication, as is the case here with Email-address Control Verification.

It's key to note that EAUT offers only a one-way mapping, from email address to URL. So, one cannot take a URL and figure out what the email address is. This is a key privacy control.

However, if an RP knows both a user's email address and an URL that the user controls (e.g., an OpenID), then EAUT can be used to verify whether or not that email address maps to the specified URL, and OpenID can be used to determine whether or not the two URL's are controlled by the same user (by checking to see if they match -- see more below).

The basic collory is this: If an email address maps to a URL, and I can show (via OpenID) that I control that URL, then either: 1.) I control the email address, or 2.) the person who controls that email address wants me to also control it, or 3.) the person who controls the email address accidentally mapped it to a URL that I control, thus allowing me to register the email address on an RP site (This latter possibility is unlikely, especially if a mapping service is used).

So, here's the workflow for automatic email address verification:
1. User logs-in to an RP via OpenID (using URLs in this example, but see the "assumptions" section below for XRIs). Note that EAUT could be used to compliment OpenID, and in any case the end-result is that the RP knows that it's dealing with a person who controls the OpenID in question.
2. RP receives the user's email address ("how" this is done is implementation defined - with EAUT-based login, the RP will automatically know the user's email address).
3. Use EAUT to transform the email address to an Openid.
4. If the EAUT-resulting OpenID matches the OpenID that the user logged-in with (or another OpenID that the user has demonstrated control over via OID Auth) then the Email is verified to be under the control of the person who also controls the OpenID.
5. Otherwise, follow the OpenID auth flow on the EAUT-resulting URL. If OpenID auth is successful, then the RP now knows that the user in question controls both the OpenID URL resulting from EAUT, and the email address in question is thus assumed to be under this same user's control (otherwise, why would the email address map to a URL under this user's control?).
6. If it cannot be demonstrated that the currently logged-in user controls the URL that the email address maps to, then fallback on "send me a secret token" email methodology in wide-use today.

Some Examples

Example 1: A Valid User Named Beth

Assumption 1: Beth can send/receive messages with the email address 'beth@example.com'.
Assumption 2: 'beth@example.com' maps (via EAUT) to the url 'http://beth.example.com'.
Assumption 3: Beth can login to an RP with the OpenID 'http://beth.example.com'.
Outcome: Beth very likely controls the email address 'beth@example.com'.

Example 2: A Spammer/Attacker

Assumption 1: BadGuyBob cannot send/receive messages with the email address 'steve@apple.com' (i.e., this email address is not under BadGuyBob's control).
Assumption 2: 'steve@apple.com' maps (via EAUT) to the url 'http://steve.apple.com'.
Assumption 3: BadGuyBob cannot login to an RP using the OpenID URL 'http://steve.apple.com'.
Assumption 3: BadGuyBob can login to an RP with the OpenID 'http://badguybob.example.com'.
Outcome: BadGuyBob logs into the RP, but is unable to assert control over the email address 'steve@apple.com', so the RP does not allow him to enter this email address as his own on this RP.

Example 3: Should Never Happen

Assumption 1: Dennis can send/receive messages with the email address 'dennis@example.com'.
Assumption 2: 'dennis@example.com' maps (via EAUT) to the url 'http://dennis.example.com'.
Assumption 3: Dennis does not control 'http://bill.microsoft.com'.
Outcome: Dennis cannot even login to an RP with Bill's OpenID URL, so he will never even get to an email verification step on the RP's site.

---------------------------------------------------------------------------------------------
Assumptions

The DNS system is not compromised. If DNS is compromised, then the mechanism in this proposal is not guaranteed to be accurate. However, if DNS is compromised, then traditional email verification (i.e., send a message with a secret code and have the user tell the RP what that code is) could be unrealiable as well. Thus, this proposal is no less-safe than the current system (it's arguably more safe), and is much easier on the end-user (email verification is automatic, instead of the manual process it is today).
XRI OpenID's are not Discussed Here. The OpenID Authentication 2.0 spec allows both XRI's and URL's to be used as OpenID's. This article does not address how to verify email address control when an XRI is used as an OpenID, but EAUT supports XRI's, so it would seem trivial to use this method with XRI's as well.

-------------------------------------------------------------------------------------
[1] Owner/User/Controller is a tricky term, too. Who owns 'beth@big-web-co.com'? Does Beth own it, since it's the email address she uses to communicate with? Does big-web-co own that email address, since it sits in their domain, and is ultimately controlled by them and they're legal department? For purposes of this article, let's use the term "User", and take it to be Beth for the email address 'beth@big-web-co.com'. At some level, she both uses and controls this email address (at least until big-web-co tells her she can't).

A Very Thorough OpenID Presentation

2008-07-25T09:30:00.000-07:00

It's about a year old, but it's a great presentation on OpenID by Simon Willison and David Recordan.

http://www.slideshare.net/daveman692/openid-bootcamp-tutorial/