There is a pretty good story behind this. That is, how we found and managed the OAuth protocol security threat identified last week. In many ways, the story is much more important and interesting than the actual technical details of the exploit.

For everyone involved, this was a first-of-a-kind experience: managing a specification security hole (as opposed to a software bug) in an open specification, with an open community, and no clear governance model. Where do you even begin?

But right now, I know you want the technical details.

If you are reading this, I assume you have a basic understanding of how OAuth works. If you don't please take a few minutes to read at least the first two parts of my Beginner's Guide to OAuth.

The first part will give you a general overview and second will take you through the user workflow. It is the workflow that is important here. The rest of the guide deals with security and signatures, but the content of these posts, surprisingly, is not involved in this attack.

I'll start with what this attack is not about; it does not involve stealing usernames or passwords; it does not involve stealing tokens, secrets, or Consumer Keys; in fact, no part of the OAuth signature workflow is involved. This is not a cryptographic attack. And it does not violate the principal of a user granting access to a specific application. All that remains intact.

You might think I am stalling, but it is critical for people to understand both what is broken, as well as what is still secure (that is, no known exploits identified). It is important because even after this is made public, there will be plenty of OAuth-powered services up and running and I want you to feel safe using them.

Understanding the exact details of security threats is important in order to prevent exploits and fix the specification. But it is equally important to know what is working well and can be trusted.

For example, many applications use OAuth for 2-legged requests that do not involve user authorization and are unaffected by this threat. As a community effort, a big part of our work is to build adoption and support for the protocol. We need to fully disclose threats, and also make sure the reputation of the protocol is not damaged by misunderstandings and speculations.

The identified threat is a session fixation attack, empowered by a social engineering attack. The basic idea is that an attacker tricks an application using an OAuth API (a Consumer) to give it access to someone else's resources (via an Access Token). The attacker never gets the Access Token itself, just the ability to use the application. But since the application has the Access Token, the attacker can use it to access the victim's resources.

The OAuth authorization workflow consists of three steps:

1. The user initiates the flow by requesting the application to access his resources. This triggers the application to request a Request Token from the provider and redirect the user to the provider for authorization.
2. The user signs into the provider and grants access to the application. If the user is educated about phishing attacks, he can verify that he is in fact giving his password to the provider and no one else.
3. After granting access, the user is redirected back to the application to complete the flow and enable it access to his data.

The only thing that binds these three steps together is the Request Token in the authorization URI and callback URI. In both cases, the call is not signed and easy to guess. The problem is, there is nothing at all to enable the consumer or provider to ensure that these three steps are performed by the same user. While the application can use cookies and other session management tools to ensure that the first and last step are performed by the same user, it has no way to make sure the middle step is the same as the first or last.

A normal OAuth Flow looks like this:

An attacker starts the process by using an application account and asking to initiate the flow, which results in the application obtaining a Request Token and redirecting it to the provider. At this point, the attacker stops and takes note of the Request Token included in the redirection URI. The attacker may also modify the callback but we will get to that later.

The attacker then uses social engineering to trick a victim into following that link (the authorization URI from the redirection). This can be as simple as a blog post with a review of the application, inviting people to try it out. When someone clicks on that link, they are sent to the provider to authorize access.

Since this is what he wanted to do, the victim will not realize that he should have started at the application itself, and will continue to sign into the provider. Because of how we train people to look for phishing attacks, even an educated user will notice that he is at the right place.

The provider will then ask the user to grant access, identifying the right application. This will increase the comfort level of the user, since so far, everything checks out. The right provider and the right application. But as soon as the user grants access, the attacker can construct the callback URI (from either learning how the application works or from the callback parameter provided in the redirection URI) and return to the application.

At this point, the attacker's application account is associated with a valid and authorized victim Access Token. Remember: the attacker does not have the Access Token or any of the secrets. It was never in possession of the Request Token secret, and it does not make signed OAuth requests. All it has is an application account associated with the victim's resources.

The Attack Flow looks like this (simplified):

Let's talk about callbacks a bit. OAuth includes an optional callback parameter allowing Consumers to set a callback when they send the user to the provider for authorization. Providers can implement callback support in a couple of ways:

1. Allow any callback at all.
2. Allow any callback within a pre-registered (and verified) domain.
3. Allow only a static pre-registered callback and ignore the parameter.
4. Not allow callbacks at all, requiring manual user action.

If the provider uses the first option, the attacker can simply change the callback. Remember, the authorization step is not signed, allowing the attacker to change the callback parameter. This will send the user to a page controlled by the attacker, letting it know when it can go back to the application and finish the job.

If the provider uses the second or third option, this becomes a race condition as to who will make it back to the application first: the attacker of the victim. Now, since the attacker has no way of knowing exactly when the user grants access, it will have to try multiple times until it works. It is up to the Consumer application to handle such brute force attacks properly by voiding the entire transaction. But this requires the application to expect and handle this case.

Another point to consider in the second option is that sometimes domains have an open redirector which will redirect a request based on a given parameter. For example, this is done to count how many people click on ads or outbound links. An attacker can use such redirectors to set the callback to a URI at the pre-registered domain, which in turn will send it to an attacker controlled site.

If the provider uses the fourth option, it is similar to the previous two, but the race condition becomes a lot easier since it will take the victim longer to go back to the application (no automatic redirection back). Also, in manual 'press here when done' cases, applications tend to allow multiple attempts because users make more mistakes. In such cases, the application will simply try to exchange the Request Token with an Access Token and will fail until the access is authorized by the user. If the provider doesn't void the transaction on the first failed attempt, it increases the exposure.

Now, we talked about the attack and we talked about callbacks. But that is not all.

Even if the attacker doesn't get to be first to return to the application or find out when access has been granted, the application may have a critical design flaw. Some applications use the identity of the user who started the flow when they associate the Access Token with the local account. What this means is that even if the victim is the one to return to the application, the application will still attach his Access Token to the attacker account because it will look up the Request Token in the database and use that information to continue.

If we put it all together, even if an application:

• Makes sure to use the account of the user coming back from the callback and not that of the one who started
• Checks that the user who starts the process is the same as the one who finishes it
• Uses an immutable callback set at registration

It is still exposed to a timing attack trying to slide in between the victim authorizing access and his browser making the redirection call. There is simply no way to know that the person who authorizes is the same person coming back. None whatsoever.

And that's pretty much it.

To get you started thinking about a solution, a few community members are thinking about fixing this by adding an unpredictable callback parameter generated at the Service Provider to the callback URI on return. The Consumer will need this new extra parameter to exchange the Request Token for an Access Token, ensuring that the real user has to return to the application to complete the flow.

We'll meet back here tomorrow to continue talking about ideas on how to fix it with a protocol revision. Meanwhile, think about this for a bit, join the conversation on the OAuth mailing list, talk about it with your coworkers, or blog your thoughts.

Be safe.

Brian Eaton, Dirk Balfanz, and Chris Messina provided valuable feedback to this post.

It has been a year since I decided to put my startup Nouncer on hold and join Yahoo!. It has been fascinating to witness Twitter's renewed media attention and recent growth, and it has inspired me to go back to my old posts about trying to run such a business.

The following are three posts on the subject:

Waiting for Google

Early Thoughts About a Microblogging Business

Playing with Numbers: A Microblogging Business Model

This was a light blogging week with mostly short posts. They included an invitation to attend the upcoming Internet Identity Workshop, clarifications to help those implementing OAuth on the server side, announcement about OAuth being nominated for a Webware 100 award, call for action in the Open Web Foundation, and some advice on versioning specifications.

I will be in traveling to New York, New Jersey, and Virginia over the next 10 days. Hope to see a lot of my New York friends while I'm there.

Next month will be a year since the work on creating the Open Web Foundation started. It has been 8 months since the initiative was unveiled at OSCON 2008, and six months since the legal entity was created and work began on the legal framework for specifications.

We accomplished a lot during this time. Not so much in hard deliverables, but in building industry-recognized momentum and starting to change the way individuals and companies think about community-driven specifications and light-weight standards. Over the past couple of months, members of this community have been invited as experts to talk to well respected and established standards bodies, and explain this work. This is not a small accomplishment.

Most of the current work is focused on delivering a legal framework communities can use to write specifications in a way that will allow large and small companies to use them. Adoption, after all, is what specifications are all about. We formed the Legal Affair Committee with individuals representing a wide range of positions, some their personal views while other of their corporate employer. The committee has been focusing on producing the first Open Specification Agreement document and we have a working draft that we hope can be finished in the next few weeks.

I strongly believe that in order for this organization to have a respected voice, it has to have some form of open government, that is more than just 8 self selected individuals. The Open Web Foundation board needs to vote before any legal document can be published as an official document. This means that as soon as the Legal Affairs Committee is ready to submit a document to the board, we need to have an elected board ready to ratify it.

It is also time to rethink the purpose of this organization and consider better aligning it with some of the expectations the community had when it was announced. I am not suggesting anything radical, but there are obvious needs that can be accommodated without betraying the original goals. I think the two needs: to put in place an elected board, and to realign the purpose of this entity, can be accomplished with one plan, by moving forward and establishing a foundation membership.

As the name suggests, this organization is about the Open Web, the platform the web is built upon. This includes specifications, technologies, ideals, ideas, and people. We have been focusing on a subset of the ground floor: specifications. But there are other areas where we could expand such as evangelism, education, resources, and building a home for people to discuss these topics.

We have a chicken and egg challenge. How to select the right membership before the goals are clear, and how to define the goals, without a larger membership. I much rather begin by increasing participation than having 8 people make decision. I trust that smart dedicated individuals coming together will figure out the common cause, and if they don't fit with the majority's direction, will step aside un-offended.

I am going to propose a plan, and after some discussion on this list, will ask the board to vote on it. I think it is absolutely essential that we debate this out in the open in the most inclusive way possible. However, since we have a legal organization with well-defined bylaws, it is important that we use that framework to cast votes and make decisions. It is also important we act quickly.

Please join the discussion.

Its never good when specifications are read like the bible, where people find in it what they want.

Over the past few months I have been getting many questions about requirements in the OAuth Core 1.0 specification, as well as finding significant issues with existing implementations. I agree that the specification isn't clear on many of these issues, but until we have a replacement, I hope this list would help.

Nonce

Consumers are required to generate a unique nonce value for each request with the same timestamp:

"The Consumer SHALL then generate a Nonce value that is unique for all requests with that timestamp." (Section 8).

However, there is currently no requirement for the Service Provider to check or enforce the nonce:

"When verifying a Consumer signature, the Service Provider SHOULD check the request nonce to ensure it has not been used in a previous Consumer request." (Section 9)

For those not intimately familiar with RFC 2119, SHOULD means:

"that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and    carefully weighed before choosing a different course."

There are plenty of places where APIs don't need to worry about replay attacks, the only attacks solved by checking the nonce. For example, requests over HTTPS or non-confidential read-only requests don't benefit much from using nonces. APIs for ordering pizza, on the other hand, do.

Developers must consider the threat model and security requirements of their applications, and if checking the nonce adds value, they should do it. However, this requirement is likely to change in future versions (making nonce checks mandatory).

Parameter Transmission Methods

OAuth defines three methods for including OAuth parameters in the HTTP request: URI query, form-encoded body, and HTTP Authorization header. Many providers assume that they can pick and choose which methods to support. This is wrong.

Now, granted, the specification does an awful job making this clear. First, it has some contradictions, and second, it does not use normative directives (MUST, SHALL, SHOULD, etc.) to define the required behavior.

The Consumer requirements, even though they lack normative directives, specifically calls for using one of the three methods:

"OAuth Protocol Parameters are sent from the Consumer to the Service Provider in one of three methods" (Section 5.2)

This implies two things: the Consumer must only use one method at a time, and can pick any of the three and expect it to work. If the server offers additional methods, they Consumer may use those as well:

"In addition to these defined methods, future extensions may describe alternate methods for sending the OAuth Protocol Parameters."

But then the specification contradicts itself:

"It is RECOMMENDED that Service Providers accept the HTTP Authorization header."

So what we have here is an implicit requirement for Consumers to use one of three defined methods, and allowing Consumer the freedom to use either one (with an order of preference defined), but allowing the Service Providers to omit HTTP Authorization header support. This is broken.

This problem has been made worse by the OAuth Discovery specification which made the methods a matter of configuration. Note that listing which methods are supported is not listed in the configuration documentation requirements of the core protocol.

For lack of better guidance, I would offer this:

Providers must support all three methods. Consumers should use the first method available to them from the list (in the order listed). If a Consumer request fails due to unsupported method, it should attempt making the request with another method. Providers should avoid coming up with new parameter transmission methods until this issue is resolved.

I expect future versions of the specification to require a basic set of methods from all servers, and would personally like to see the Authorization header as the primary method.

OAuth Parameters

During the specification development process, we had several discussions about parameter names and restrictions. OAuth parameters start with the 'oauth_' prefix, while server specific parameters don't.

The two main areas of confusion are:

• What prefix should extensions use?
• What should providers do with unrecognized parameters?

The specification offers a few hints:

"OAuth Protocol Parameters" are defined as "Parameters with names beginning with oauth_." (Section 3)

"Service Provider specific parameters MUST NOT begin with oauth_." (Section 4.2)

"OAuth Protocol Parameter names and values are case sensitive. Each OAuth Protocol Parameters MUST NOT appear more than once per request, and are REQUIRED unless otherwise noted." (Section 5)

Putting this all together, the specification allows extensions to define new 'oauth_' parameters. It only restrict server-specific parameters from using the prefix. In other words, if the behavior isn't server-specific (i.e. plays well with others, interoperable), it is acceptable and allowed.

However, such extensions are still restricted in their use of 'oauth_' parameters:

• Parameters names and values are case sensitive
• They cannot repeat
• Are required unless explicitly noted otherwise.

So to answer the first question, extensions should use the 'oauth_' prefix (and not 'xoauth_' or other such proposals). Individual services should not use it to define their own, incompatible, proprietary extensions.

This provides us with the answer to the second question. Providers should ignore any 'oauth_' parameters they do not recognize. They should not fail the entire request just because it includes an unknown 'oauth_' parameter. They should however, include such parameters in the signature and verify they have not been tempered with.

Want more proof that this is the specification's intention? Section 10 defines a few error conditions:

• Unsupported parameter
• Unsupported signature method
• Missing required parameter
• Duplicated OAuth Protocol Parameter

Note that the first three errors are generic and talk about parameters in general, not just 'oauth_'. The last error is specific to 'oauth_' parameters. This is intentional. Note that there is no error for unknown OAuth Protocol Parameter.

OAuth intentionally includes a protocol version. This version is meant for two primary objectives:

• Define the meaning of 'oauth_' parameters listed in the specification
• Define the signature workflow

Any parameter undefined by OAuth is not allowed to change how the protocol works. It may add additional functionality but it will not affect how the core protocol is implemented. This is important because it means providers can safely ignore unknown parameters without introducing security issues within the core protocol.

The reason why I want servers to simply ignore unknown 'oauth_' parameter is because I expect new extensions to introduce just such parameters. If providers hard-code which parameters are valid today, they will create problems when new Consumer libraries add such support.

I am working on some new designs for this blog, trying to transition it into a more persistent online resource for the subjects I care about: OAuth, Discovery, Open Web, and Microblogging. This will include a new look, custom pages for each topic, and the usual house cleaning.

The first step was to review every blog post I wrote over the past two years and recategorize them. The idea is to make the categories a useful tool to stay up-to-date with the topics you care about. On the right side of this blog you will find the 'Categories' tool which shows (in font size) how prominent this topic has been over the past two years. It also provides a link to the posts in each category.

Here are some of my categories:

• Discovery
• Microblogging
• OAuth
• Open Web
• OpenID
• Startup