esjewett.com

Introducing jsglue

esjewett — Wed, 06 May 2009 23:03:01 +0000

Last weekend I pushed out the very first version of something I'm calling jsglue to Github. It now lives here: http://github.com/esjewett/jsglue/tree/master

jsglue is in essence a framework for implementing web-connective applications a la Yahoo! Pipes and Tarpipe. Currently it is at best a compliment to those programs and at worst totally useless. In the future I would like to see it or something like it become an alternative to these tools, for a few reasons that I'll eventually get into in later posts.

jsglue does three things:

It allows you to register a handler to a path.
- The handler consists of a path and two pieces of javascript - one that constructs a response to a request sent to that path, and one that constructs one request (and in the future multiple requests, optionally) to another URL.
It accepts HTTP requests to paths with registered handlers.
- When this happens, it creates a response using the handler javascript for this purpose, and it adds a job to a stack that will be processed later.
It provides a program that can be run periodically to process the stack of jobs that has built up, sending off new requests as specified by the javascript in the handler associated with each job.

That's it.

Why do I care?

Well, hopefully that will become clear of its own accord. But the key is that the full contents of the original request are exposed to the javascript processing script that constructs the new request. As such, you can do pretty much any kind of processing you like within this handler code, which is user-defined.

So why do you care? Let me count the ways:

1. Receive a request in JSON and spit it back out multi-part form-encoded (in fact, right now this is pretty much the only thing you can do). Ever tried to connect up Yahoo! Pipes with Tarpipe? It doesn't work. With this, it can.

2. Webhooks are great. Webhooks are the facility to have a web application issue an HTTP request to an arbitrary URL when some event happens in the web application. That sounds boring, but it's actually awesome. Webhooks are great, except that no one speaks the same language so every webhook-based solution is bespoke. Bespoke is great in a suit or a coffee mug, but it's bad in web infrastructure. Yahoo! Pipes can't understand webhook calls. Tarpipe usually can't understand them. Most other web applications can't understand them. There needs to be some sort of middle-person.

3. If I'm going to run a ton of my personal data through some middleman web application, you should have the option of running that web application yourself. I'm not saying you will, but I think it would be nice if you could.

Okay, 3 ways is enough for now. We'll get to more later.

Why do you *not* care?

Well, there are lots of reasons for that too.

1. This is dorky. No, there is not a UI. No, it doesn't do much of interest. It's an infrastructure prototype more than anything else. The idea is really that we need infrastructure for building applications that can do this sort of thing. I don't have a lot of time to spare, so I'm willing to just put a framework out there, and maybe a REST-only web-application if I can get the components running on a hosting service (harder than it sounds). I'll leave it to someone else to put the UI on top of it. I'm not convinced that the "pipe" metaphor is correct (I'm partial to "tubes" myself), but I don't have a better idea, so someone else will have to have that idea.

2. This code sucks. Yes it does. I urge you to fork it, improve it, or throw up your hands in disgust and start over. I just want something that does this. I don't really care if it's written by me.

3. There's no way this execution model will fly on a public site, and no one is going to run this on their own server. This is sort a feature of this design that allows your users to execute arbitrary javascript on your server. As such, I think this will primarily find use on private servers, or as a back-end engine for a public site where the inputs are carefully cleansed. Not a recipe for ultra-popularity, I'll grant. But that's not really the point either.

So what's it made out of?

Currently, there are only four main ingredients:

Ruby is the implementation language. It's role in jsglue is to serve as duct-tape for the other components.
Datamapper is the database interface, allowing you to use pretty much any supported database (I'm using SQLite at the moment).
Sinatra for the HTTP web-service interfaces. These interfaces are pretty much a direct mapping onto the database. (REST-ful? Maybe.) (Incidentally, how is it that a minuscule Ruby web-framework beats out FRANK SINATRA in the Google rankings?)
Johnson for the Javascript processing.

That's it. It's a couple hundred lines of code. I haven't really counted, or put it on Ohloh.com for that matter, but it can't be more than that. It's got some unit tests. It's going to be changing quickly as I make it more multi-purpose.

I'll document and post examples as they become available.

Trackback URL for this post:

http://www.esjewett.com/trackback/41

A tour of testing with an SAP focus (in the end)

esjewett — Tue, 28 Apr 2009 02:43:16 +0000

As might have been assumed from the my post on automated testing in SAP systems a couple months ago, I've been delving into testing in the SAP landscape. I'm beginning to put together a series of presentations and workshops on the subject, the first of which I was delighted to deliver last week.

Being the first in the series, this presentation focuses on an overview of the leading edge of the field. It would be nice to be able to emulate the sort of testing techniques we can use in Ruby in SAP BI and EPM application development. Nice, but not necessarily realistic. I can dream!

Testing Sap: Modern Methodology

View more presentations from Ethan Jewett.

Trackback URL for this post:

http://www.esjewett.com/trackback/40

How to get Johnson built

esjewett — Fri, 03 Apr 2009 02:38:05 +0000

Johnson is a Ruby wrapper of the SpiderMonkey JavaScript interpreter. In practice, this means you can use Johnson to evaluate JavaScript statements within a Ruby program.

It works really nicely, but there is no released gem and building the development gem is a bit of a headache. The instructions in the readme at the Johnson Github site appear simple but have several prerequisites that must be fulfilled before the whole process works. Here's what I did on Leopard. I'll try recreating on Windows tomorrow.

[sudo] gem sources -a http://gems.github.com
[sudo] gem install jbarnette-johnson

Multiple errors occur - if you see gems missing (like hoe, for example) then install them.

[sudo] gem install jbarnette-johnson

Errors occur, probably due to a failure to compile the native gem. Usually something about a missing rake/extensiontask.

[sudo] gem install rake-compiler
[sudo] gem install jbarnette-johnson

Yet another error occurs - a missing Manifest.txt file. Download from Github at http://github.com/jbarnette/johnson/tree/master and put it into the directory in the error message.

[sudo] gem install jbarnette-johnson

Success!

[Apr. 4, 2009 - Fixed typo, changing "rake-compile" to "rake-compiler"]

Trackback URL for this post:

http://www.esjewett.com/trackback/39

OAuth Q&A Part 2

esjewett — Thu, 26 Mar 2009 23:08:52 +0000

This the continuation of my ongoing OAuth Q&A, now with fewer links and more editorial commentary. See Part 1 as well.

If I have access to an authorized token and the consumer key and secret, I can make "authorized" API requests galore. So OAuth is no more secure than the username/password pattern.

That's not a question. Way to start off part 2 with a big fail!

[Ed. - Ahem . . . You're asking the questions as well as answering them.]

[Me - I deny that. But okay, let's try again at a real answer.]

This is only partly correct.

First, OAuth provides the a pattern for providers to scope access much more granularly than when a single username/password pair gives total access to an app. For example, most OAuth providers do not allow access to administrative functions when authorizing with an OAuth token, so a rogue client cannot change a user's password. Because of this, the user's access to the app can be guaranteed and damage can be limited to the scope of access granted to the client.

Second, because the user's access can be guaranteed and because client access can be managed at the level of the token, authorization for a given token can be revoked in a self-service manner. The scenario here is that a user grants access to a malicious app that vandalizes the account. The user realizes this, logs in to the provider, revokes the consumer's access, and repairs (hopefully) the damage. Because the malicious consumer cannot change the password on the account, the provider does not have to become involved in the initial response to the vandalism. Follow-on activities that the provider may need to become involved in (like account restoration or consumer key revocation) still exist, but the need for instantaneous response is lowered.

Hey, you just said "consumer key revocation", but back in Part 1 you said that the provider can't assume a consumer key uniquely identifies a particular class of consumers. What gives?

It is correct that the provider can't make this assumption based on the OAuth specification. However, the provider can impose requirements around the consumer key on consumer developers, and the provider can reserve the right to revoke a consumer key for whatever reason they choose. This could be required in the case of denial of service attacks or malicious consumers using a particular key. Providers should work closely with consumer developers to clearly lay out what actions the provider will take in these cases and what options consumer developers have.

Providers should recognize that the case of desktop and web apps are significantly different in the case of key revocation and they should plan accordingly. In the case of web apps, it should be relatively easy to replace a compromised consumer key and secret. In the case of desktop apps, the consumer developer may face the task of updating thousands or millions of installed applications in the case of a key revocation.

Let's assume that a user has installed a malicious desktop app that wants to use the API of my web app. What's the point of requiring OAuth, since this malicious desktop app already owns the user's system and will have access to anything the user types?

I'm not sure where this assumption comes from. Well, let me amend that. I know exactly where this assumption comes from, but it is still incorrect. Let me assure the reader that there are many platforms where the installation of a malicious application does not translate into ownership of the host system. The reader will find this sort of behavior in any properly configured Unix-like operating system including Linux, on the iPhone, in any modern browser, in Mac OS X, and even in some locked down versions of the Windows operating system.

On these platforms, even if the user is tricked into installing a malicious client or going to a malicious website, the user may still be careful to verify that they are only entering their password into the real site of the provider. There are a lot of options for verifying this, but the best way is for the user to make sure that he or she always enters there password into their provider's site using a modern browser with anti-phishing technology and using an https connection.

Trackback URL for this post:

http://www.esjewett.com/trackback/38

Background processing in Web Dynpro for ABAP

esjewett — Fri, 13 Mar 2009 22:47:56 +0000

This week I was writing a Web Dynpro ABAP interface for a long-running process. In general it is probably not a good idea from a performance perspective to run intensive processes synchronously in the context of a Web Dynpro view. It's also a really bad idea because it makes the user wait for the process to finish. If the process runs longer than the server's time-out setting then you get a nasty error message.

I knew that there was a way to handle this in ABAP, but I don't really live in the language, so it took me a little while to figure it out. Here's the skinny:

The answer is to write a function module to carry out the processing for you. Enable the function module for RFC (and by extension, set all of the parameters to "Pass by value"). You then call the function module from a Web Dynpro controller method in the background. This registers the function module for execution as an RFC function call in the background.

Once you've done all of your "CALL FUNCTION xxxxx IN BACKGROUND TASK." statements, use the "COMMIT WORK." command to trigger the execution of these functions. You end up with a code snippet that looks something like:

call function zsample_function in background task
* exporting
*    parameter =
* importing
*    ret =
    .

commit work.

Now your function will start running and control will be returned to the user in the Web Dynpro UI.

One important thing to remember is that this method provides no feedback to the user by default. Usually it will be a good idea to at least return a message explaining that the task has been started. If possible further feedback should be provided for long-running jobs. I was able to do this by querying a status table that my job populated, but other options are available.

Help documentation - http://help.sap.com/saphelp_nw70ehp1/helpdata/en/8f/53b67ad30be445b0ccc968d69bc6ff/frameset.htm

Trackback URL for this post:

http://www.esjewett.com/trackback/37

OAuth Q&A - Part 1

esjewett — Sun, 01 Mar 2009 00:58:01 +0000

OAuth Q&A - Part 1

I've been seeing a lot of misinformation about OAuth in discussions lately. Mostly, this is because there has been a lot of activity around OAuth due to the announcement that the Twitter API will be supporting OAuth and eventually (probably, and less officially) moving to OAuth-only authentication and dropping support for basic auth entirely. Generally this misinformation is rebutted somewhere else on the web but it can be somewhat inconvenient to track down all of the call-and-response blog postings on the topic.

As such, I thought an O'Grady-style Q&A might be in order, based on my notes and recollections. Said Q&A turned out to be a little longer than expected, so I'm doing it in a number of parts that will be determined by the cessation of misinformation or the end of my attention span, whichever comes first. If there are any inaccuracies included in these posts that I become aware of (On the web? About OAuth? From a non-expert? Impossible!), I'll post corrections inline.

What was OAuth designed to do?

OAuth was designed to do delegated authorization, which is a fancy way of saying that it was designed to ensure that a user will never have to give the password for a service to any entity other than the service provider. If the user wishes to use an application that is not the provider itself in order to access the provider (for example, using Twhirl to access Twitter, or Plaxo to access your Google contacts), then that application can use OAuth to request authorization, which is then granted in a transaction that occurs directly between the user and the provider and does not involve giving the password to the application.

You can read lots about it on the OAuth site, the OAuth wiki, the OAuth mailing list, on Eran Hammer-Lahav's site, and the Google identity and authorization site.

OAuth doesn't work in desktop applications because there is no way to protect the consumer key or secret, so there's no point in using it here, right?

No, OAuth does what it was designed to do just fine in desktop apps.

But doesn't OAuth rely on a consumer secret and key to identify applications that are allowed to access protected resources and services? How do I protect that in a desktop application?

OAuth neither requires nor assumes that the consumer key and secret are hidden from other apps (though there are some nice payoffs if you can manage this). The consumer secret is simply not sent over the wire. Providers should not assume that consumer keys or secrets identify specific consumers or classes of consumers. Note that this may require providers to treat desktop consumers differently than web apps where requests come from a single IP address and the consumer has more control over the key than is the case with desktop apps.

My company has a web app and we also provide a desktop client. Because we provide the client, we can just use username/password authentication in the client and that's safe, right?

No, and there are two reasons for this. First, doing this teaches your users that responding to the password anti-pattern is acceptable behavior and makes your users more susceptible to malicious API clients that ask for their password. Second, this client behavior necessarily results in the user's password being stored by the client in a recoverable format, which is a significant security risk.

OAuth is really just a standard for token negotiation and request signing. My application already generates API access tokens and I have a scheme for sending the token along with the request, so why do I need OAuth?

The most important reason is that there are a lot of things that can go wrong with API access tokens, especially when used over an insecure transport (http instead of https). Using a public, peer-reviewed standard lets you rest easy knowing that lots of experienced thought has gone into avoiding security issues.

Well, okay, but I've actually got the best security minds in the business working on my access token scheme, so I'm pretty comfortable there. What are the other reasons?

The next biggest reason is interoperability. There are lots of OAuth [client libraries] that make it relatively easy to inter-operate with systems that use OAuth as their auth delegation protocol. For example, Shindig (OpenSocial, Google Gadgets, iGoogle, etc.) has built in support for OAuth API access for gadgets running on the platform. I expect that within a year there will be at least one web "glue" application (like Yahoo! Pipes, or Tarpipe) that will support reads from and writes to arbitrary OAuth endpoints. If you use a custom scheme, you will miss out on this interoperability.

Trackback URL for this post:

http://www.esjewett.com/trackback/36

Twitter -> Pipes -> ESME

esjewett — Sun, 22 Feb 2009 22:19:24 +0000

This is a quick how-to, showing how to route your Twitter feed through Yahoo! Pipes to filter and cleanse it, then consume it using ESME actions.

I've created a parametrized Yahoo! Pipe here where you can input your Twitter username and get back a feed of all mentions of "ESME", "esme", or "Esme" from your Twitter timeline.

(Pipe address: http://pipes.yahoo.com/pipes/pipe.info?_id=fJZ29qrr3RGjLPJI6icw5g)

Once the "Run Pipe" button is pressed, the filtered time-line displays. I've also removed the username from the beginning of the Tweets, for better display in ESME.

Feel free to use this pipe with your username, or "Clone" it and modify it as you like. The structure of the Pipe as it is now is available as an image here.

Once the pipe has been run, it is possible to retrieve the output as an RSS feed. Copy the link for the RSS feed for this pipe, as we will need to consume it using an ESME action.

Now navigate to your ESME server. We need to use a relatively recent version of ESME, specifically a version including the recently merged actions branch. I'm using http://esmecloudserverapache.dickhirsch.staxapps.net/ in this example.

Log in to the ESME server and create a new action. Fill in the action name as desired. For the test, use the "every N mins" test. I'm chosen 5 minutes as my interval, which means that the pipe feed will be queried every 5 mins for new entries.

For the Action, use the rss: action type, followed directly (no space) by the URL of the Pipes RSS feed that we copied earlier. Your action form should look like this.

Click the "Add" button and your recent Tweets containing the word "ESME" will start showing up on the ESME server. One neat aspect of this is that any hashtags (#esme for example) in your Twitter updates will be converted to ESME tags automatically.

This only works for public Twitter updates, as Yahoo! Pipes doesn't support authentication for feeds. You could query your private Twitter feed directly from the ESME server, embedding your Twitter username and password in the URL, but I strongly recommend against this as your username and password will be stored in the ESME server database, which you do not control.

Trackback URL for this post:

http://www.esjewett.com/trackback/35

A RESTful ESME API

esjewett — Sun, 15 Feb 2009 05:20:39 +0000

The Enterprise Social Messaging Experiment (ESME) project grew out of a collaboration in the SAP world and is now a project in the Apache Incubator. The project itself is an interesting and inspiring demonstration of the organizing power of so-called web 2.0 tools, which allowed the project to go from an idea to an Incubator project in about half a year through the efforts of a wide-spread group of individuals, many of whom have never met. I've been on the sidelines of the project, looking on, but I've been fascinated by some conversations that have taken place around the API.

The ESME API is often described as a "REST" API or a "RESTful" API. "REST" refers to the design principle of Representational State Transfer. Wikipedia has a good overview. REST is often seen as an alternative to RPC APIs, or "Remote Procedure Call" APIs. At root, the difference as described by Wikipedia is that RPC is about telling an application to do something while REST is about changing the state of the resources of an application.

The upshot is that RPC can be thought of as interfacing in verbs, while REST can be thought of as interfacing in nouns. To grossly oversimplify, let's pretend that I'm updating my location in the application http://www.foobar.com/.

In RPC I would do something like

POST HTTP request http://www.foobar.com/api/update_location?lat=1234&long=5678

In the context of a REST API, I would do something like

PUT or POST HTTP request http://www.foobar.com/location?lat=1234&long=5678

A subtle distinction to be sure, but let's look at the difference. In the RPC version, we see the verb in the URI. There is a separate URI for every possible action we might want to take with regards to a location (update_location, get_location, create_location, delete_location). In a REST API, the resources of the program are addressed directly (that is, we send the request to the same URI that we would use to display our location in a web browser), and the "verb" that we want to apply to the resource is embedded in the HTTP request. REST is a very HTTP-oriented design approach, but it is an approach that makes sense because HTTP is a protocol for handling resources. The Wikipedia page has more on this, and may very well contradict me, as I am by no means an API expert!

To get to the point, what would an ESME REST API look like? Let's first look at the current ESME API, which is described as "REST", but which I think we can safely conclude is actually RPC.

http://code.google.com/p/esmeproject/wiki/REST_API_Documantation

Note how each API command is a verb. This is the hallmark of an RPC API. (ESME is part of a tradition here. The Twitter "REST" API is also primarily written in an RPC style, where the verb is part of the path of URI and is not assumed based on the HTTP method. The Twitter API does assume that a GET HTTP request maps to a read except when they have also have a specific "show" verb, but now I'm just nit picking.)

There is nothing wrong with this, and RPC is actually more in line with enterprise API design standards than a REST API, but I'd like to get at what a real ESME REST api would look like. I provide here the current ESME API method along with the REST equivalent that comes to mind.

I'm listing arguments here as URL-encoded, but they could easily be form-encoded, XML, JSON or all of the above. On the REST side, where a portion of the URL is in all caps, this would be substituted by the unique ID of that particular resource. This is, of course, not a well-thought-out proposal, but rather a suggestion to illustrate what a more RESTful API might look like.

Current (RPC)	REST
GET /api/status	GET api/sessions (It might make sense for this to return only the current session, but in theory it would return all sessions that the current session is allowed to access, so for an administrator, it might return all open sessions. An individual session would be accessed at GET api/sessions/SESSIONID.)
POST /api/login token=API_TOKEN	POST api/sessions?token=API_TOKEN
GET /api/logout	DELETE api/sessions/SESSIONID or DELETE api/sessions?session=SESSIONID (get SESSIONID from api/sessions)
GET /api/get_msgs	GET api/users/USERID/messages (get USERID from api/session)
GET /api/wait_for_msgs	GET api/users/USERID/messages (long-poll?)
	GET api/messages/MESSAGEID Gets a particular message.
POST /api/send_msg message=messagebody via=client tags=tags metadata=XML_data replyto=message_id	POST api/messages?message=MESSAGE_BODY&via=CLIENT&tags=TAGS&metadata=XML&replyto=MESSAGEID
	PUT api/messages/MESSAGEID (payload the same as POST)
	DELETE api/messages/MESSAGEID
GET /api/get_following	GET api/users/USERID/followees
GET /api/get_followers	GET api/users/USERID/followers
POST /api/follow user=id_of_user	POST api/users/USERID/followees/USERID2 or POST api/users/USERID/followees?user=USERID2
POST /api/unfollow user=id_of_user	DELETE api/users/USERID/followees/USERID2 or DELETE api/users/USERID/followees?user=USERID2
GET /api/all_users	GET api/users
GET /api/get_tagcloud numTags=optional_no_of_tags	GET api/tags (This doesn't really seem like an appropriate API method. It should really return all of the tags, or user-specific tags (GET api/tags/USERID) and let the front-end decide what to do with it.)
GET /api/get_tracking	GET api/users/USERID/tracks
POST /api/add_tracking track=text	POST api/users/USERID/tracks?track=TEXT_TO_TRACK
POST /api/remove_tracking trackid=id_of_tracking_item	DELETE api/users/USERID/tracks/TRACKID
GET /api/get_conversation conversationid=Conversation_id	GET api/conversations/CONVERSATIONID
GET /api/get_actions	GET api/users/USERID/actions (Actions probably don't make sense outside of the context of a specific user.)
POST /api/add_action name=name test=trigger action=action	POST api/users/USERID/actions?name=NAME&test=TEST&action=ACTION
POST /api/enable_action id=action_id enabled=true\|false	PUT api/users/USERID/actions/ACTIONID?enabled=true\|false (This is actually a general outlet to update any attribute of an action, including whether or not it is enabled.)
POST /api/delete_action actionid=action_id	DELETE api/users/USERID/actions/ACTIONID

One point to note is that most HTTP clients do not currently support
the "PUT" or "DELETE" methods, so these have to be simulated
through POST methods with an extra parameter. I think that because of the close mapping to resource verbs, is worth using these methods in
the specification and defining the simulation method for the entire API
separately.

The above is based on a rough object hierarchy as follows:

ESME API instance (api/)
- Sessions (api/sessions)
- Users (api/users)
  - Messages posted by a user (api/users/USERID/messages)
  - Users followed by a user (api/users/USERID/followees)
  - Users following a user (api/users/USERID/followers)
  - Trackers belonging to a user (api/users/USERID/tracks)
  - Actions belonging to a user (api/users/USERID/actions)
- Messages (api/messages)
- Tags (api/tags)
- Conversations (api/conversations)

Each of these bullets represents a set of objects. The resource representing an individual object lives at api/objects/OBJECTID. For example, api/sessions/SESSIONID. As much as is reasonable, one would expect to be able to GET (read), POST (create), PUT (update/amend), or DELETE (delete) any individual member of each of these object sets. Going through each of these objects to ask what it would mean to create, read, update, or delete that object may reveal holes in the existing API, some of which I have filled in above.

Trackback URL for this post:

http://www.esjewett.com/trackback/34

Automated testing in SAP systems

esjewett — Fri, 30 Jan 2009 19:55:14 +0000

This is another in my series of notes to myself that might somehow come in handy to others, so I'll post them to the blog. This is more of a brainstorm than anything else, but some thought has gone into this area. In terms of actual testing going on in the SAP world, it probably displays my ignorance more than anything, but I'd love to be corrected. If you aren't interested in SAP or in testing methodology, then I recommend you stop reading now!

Based on what I've seen in my admittedly shortish time in the SAP world, the state of automated testing is sadly lacking. This is not to say that the facilities for testing don't exist, but perhaps that they are not adequately ergonomic or well understood. The result is that most implementations don't have a mature testing automation strategy, which is a key component of a modern approach to software development.

Unit testing

In the unit test arena we're doing alright as we have most of the standard Java capabilities on the Java development side and we have ABAP Unit on the ABAP side of the house. Though, I'll note that you've got to be a bit careful in designing your ABAP Unit tests to handle more than one failure per execution. There also isn't much of a facility for handling alternative testing strategies such as specification derived tests or behavioral testing. Part of this is due to the limitations of ABAP as a (mostly) statically typed, strictly object-oriented language. Part of this inflexibility is simply due to the fact that the necessary facilities have not been developed.

I'll also note that while Java isn't my cup of tea, apparently setting up automated unit testing using JUnit on the Java side of the SAP world isn't straightforward. The SDN blog "Test-driven development with Composite Application Framework (CE 7.1)"looks like a decent overview of the process of getting JUnit testing set up.

With SAP's focus on positioning itself as a full-featured development platform for the enterprise and the growth of agile, modern development methodologies in the enterprise, I'm surprised that unit testing and test-driven development aren't getting more attention from SAP as an integral part of the application lifecycle.

Performance

In the performance testing area a lot of work has been done to work out transactional systems under load. However, I'd argue that we need to understand the limits of this sort of testing. Usually it is focused on a single transaction or report, which is fine for testing the maximum performance of that piece of the system, but the tester must have a significant knowledge of the underlying architecture of the system in order to really test for table write contention or the performance of specialized processing logic under load.

Performance testing gets really worthwhile only when the level of knowledge of the system architecture reaches a level where a significant number of these trouble spots can be identified and tested under load in a test environment. As it happens, we're mostly doing this type of testing in production in the SAP world, which is the situation we are ostensibly trying to avoid by doing "performance testing" in a pre-production test system.

Jonathon Kohl writes knowledgeably of some of the complexities inherent in performance testing. I think we would be wise to admit the limits of performance testing in an environment as complex as most SAP implementations. This is not to say that we shouldn't do it, but simple load testing isn't getting us very far and we might be better served by focusing on replicating real system use or focusing on a more exploratory version of performance testing.

Functional/Behavior

Up until this point we've had cursory automated testing occurring in most implementations. But when we get to functional testing things start to fall apart. There is a lot of potential contention over the right place to interface into the application to do functional testing. Most MVC frameworks provide an interface into the controller level to mock view requests and model contents. Some believe that the view/interface level is a good place for this kind of testing, which makes a certain amount of sense if the framework doesn't have a clear delineation between the user interface and the application logic or won't commit to the delineation that exists.

The options for functional testing in delivered SAP systems are pretty much eCATT or an external testing tool that hooks in at the UI or API level. eCATT is not an ideal functional testing tool on some reads of "proper" functional testing but should be usable. I'll be looking into some eCATT good practices and testing strategies in a future article. There are also some options from vendors like Mercury (now HP).

It should be possible to hack together a working external automated functional testing system for your organization's SAP landscape and chosen development method, but it would be expensive to maintain across significant upgrades. For example the recent Netweaver 7.0 enhancement package 1 upgrade would probably break most browser driven testing of WebDynPro ABAP/Java applications because of the shift from full server round-trips to a more AJAX-y user interface programming approach. Multiple levels of abstraction in the test harness would be necessary to limit breakage to predictable and manageable layers of the testing stack.

I still think it is worthwhile to pursue this strategy if the built-in support for functional testing from SAP does not satisfy the needs of an organization. The automated approach done well can provide for better coverage, accuracy, and speed, while saving a bunch of un-fulfilling person-hours spent manually exercising a system while struggling to stay awake. There may be room for a vendor to build up and maintain standard APIs across versions of SAP systems and I'm sure that vendors like Mercury are working on this, at least in transactional systems. Heck, there may be room for an open source project to do the same using a browser driver like WATIR. Hmmmm...

BI and performance management applications still seem to be pretty much an open field here.

Why automation

The real power of automated functional testing starts to show up when we consider doing things that we basically don't do in SAP projects. Coupled with an automated system build process (yes, we probably need to talk about this too), automated test suites allow for

Full nightly functional regression tests
Running the full test suite on a contemplated change or note application
Full regression testing after upgrading
Behavior driven development (BDD) or business applications, where the specs are the business requirements
Continuous integration for every level of the development and maintenance landscape

The potential gains in productivity and reductions in risk that can be gleaned from engaging in a disciplined testing methodology (be it focused on coverage, TDD, BDD, etc) appear large. Certainly worth more thought and study.

Trackback URL for this post:

http://www.esjewett.com/trackback/33

Workflow change costs estimation

esjewett — Sun, 18 Jan 2009 16:43:17 +0000

Originally posted as a comment by esjewett on The Inquisitr using Disqus.

These are excellent points, but it only gets at half of the equation. As I see it, the two main questions when working on my personal workflow are effectiveness and flexibility. My impression is that when people are thinking about their workflows there is far too much emphasis on the stability of the platform (which is your point, and a good one) because of an inflated estimate of the costs associated with a workflow change.

The basic question is this: Is a change to my workflow that takes X hours to execute worthwhile?

The way a lot of people think about this question is by asking "Do I have X hours to spend on this or would I rather spend it on something else?"

The question people should be asking contains another variable - the number of hours they will save overall through this change. Let's call this number of hours "Y". The question is: "Do I have X-Y hours to spend on this or would I rather spend it on something else?" When the number of hours saved (Y) becomes greater than the switching cost (X), the cost associated with a workflow change becomes negative.

I actually switch tools quite often, so my estimate of the total productivity gain for a given tool is necessarily limited by a short time horizon and my tolerance for switching costs should be correspondingly lower, but I still find that a switch is of tools is justified for a pretty small daily productivity gain (even 5 minutes saved per day makes a tool a clear winner).

If Google Reader shut down tomorrow and I had to switch to a different feed reader, would I come out ahead in productivity over the last year or two as compared to the next best solution? I have no doubt in my mind that I would. As such, I have no problem "entrusting" this part of my workflow to the web, as long as the switching costs (X) are less than the productivity gain.

I find that the "web-iness" of a tool doesn't factor much in the estimate of switching costs. The OPML export from Google Reader happens to make the switching cost very low, which makes my decision to use Google Reader very easy.

Original article: http://www.inquisitr.com/15368/so-you-want-to-trust-your-workflow-to-the-web-good-luck-with-than-plan/.

Trackback URL for this post:

http://www.esjewett.com/trackback/32

esjewett.com

Introducing jsglue

Trackback URL for this post:

A tour of testing with an SAP focus (in the end)

Trackback URL for this post:

How to get Johnson built

Trackback URL for this post:

OAuth Q&A Part 2

Trackback URL for this post:

Background processing in Web Dynpro for ABAP

Trackback URL for this post:

OAuth Q&A - Part 1

OAuth Q&A - Part 1

Trackback URL for this post:

Twitter -> Pipes -> ESME

Trackback URL for this post:

A RESTful ESME API

Current (RPC)

REST

Trackback URL for this post:

Automated testing in SAP systems

Unit testing

Performance

Functional/Behavior

Why automation

Trackback URL for this post:

Workflow change costs estimation

Trackback URL for this post: