Quick Tech Notes

Atlassian Stack: FishEye (auto start)

2010-08-05T12:32:00.000-04:00

Today's ticket is to rig up FishEye so that it starts at boot time. I scoured the FishEye docs and googled but came up empty. There is a Mac OS X solution and one for an older version of FishEye and CentOS. Nothing for Karmic Koala. My solution? Emulate what was done for Jira. I copied /etc/init/jira.conf and created /etc/init/fisheye.conf. My resulting file looks like this:

# fisheye

description     "Atlassian FishEye - Source control viewer"

start on runlevel [2345]
stop on runlevel [!2345]

kill timeout 30

env RUN_AS_USER=fisheye
env BASEDIR=/opt/fisheye

script
    LOGFILE=$BASEDIR/fisheye.out.`date +%Y-%m-%d`.log
    exec su - $RUN_AS_USER -c "$BASEDIR/bin/start.sh" >> $LOGFILE 2>&1
end script

The solution does not appear to be perfect, however. If I issue sudo service jira stop, I get this: jira stop/waiting. If I issue sudo service fisheye stop, I get this: stop: Unknown instance. Not good. I don't want to just kill the process when I shutdown the box. Who knows what type of data corruption will occur? I need to figure out what is wrong with my script and fix it. One difference is that the other Atlassian pieces bundle their own copy of Tomcat and that is what we invoke in the start scripts. FishEye uses something else.

I googled about Upstart and decided that I won't ever fully understand how it works. So, I've decide to make two upstart scripts. One that controls starting up FishEye and boot time (fisheye-start.conf) and one that controls shutting down FishEye during shutdown (fisheye-stop.conf). Perhaps not the most elegant solution but it appears to work. All I'm doing is calling start.sh in one and stop.sh in the other. Nothing fancy.

How Do I Use Spring Security To Secure Spring MVC and Spring BlazeDS In The Same Application?

2010-08-05T10:04:00.000-04:00

Context: I have a single web application that is based on Spring 3. It serves up RESTful resources using the new Spring MVC annotations and it works fine. After some head scratching and research, I finally figured out the configuration required to lock down the Spring MVC calls with Digest Authentication via Spring Security 3. I then wanted the ability to provide access to the same set of services to our Flex clients using AMF so I added Spring BlazeDS Integration to the mix. Remoting a service via Spring is almost trivial. Add an annotation or two and you are good to go. Adding security to the mix is almost as easy. If you follow the directions on how to lock down your AMF channels, it'll work fine. The problem I ran into is that set of security filters set up by the Spring BlazeDS Integration directions interfere with the set of filters needed by Spring MVC. The solution? Watch things in the debugger in a working environment, reverse engineer the required filters for Spring BlazeDS and then specify them by hand in your Spring Security set up. Spring Security uses a chain of filters to apply authentication logic to servlets. Spring MVC and Spring BlazeDS get their own instance of DispatcherServlet in the application. You then apply the required filters to the appropriate servlet. My solution was to break up the Spring Security beans into three files:

common-security-context.xml - holds beans that are common to both Spring MVC and Flex authentication
mvc-security-context.xml - holds the beans specific to authentication of the RESTful API
flex-security-context.xml - holds the beans specific to authentication of the BlazeDS calls

What you should end up with is the ability to invoke the service in two ways: one as a RESTful resource and one as a Flex Remote Object. In each case, providing the same set of credentials should authenticate you. In summary, relying on the Spring Security namespace to set up your security environment does not work if you are combining Flex and Digest authentication: you need to set things up by hand.

Here are my versions of the files in question. I hope that it saves you some time and effort.

Atlassian Stack: FishEye

2010-08-02T13:04:00.000-04:00

Today's mission: install FishEye, the source control repository analysis tool. Quoting the online docs: "FishEye unlocks Subversion, Git, Perforce, Clearcase, CVS, and Mercurial with real-time notifications of code changes plus web-based reporting, visualisation, search and code sharing." I'll be installing 2.3.5 Standalone edition.

create a new user account and group named fisheye
add the fisheye account to the subversion group -- it needs read only access to the Subversion repository
extract the fisheye zip into /opt
make a softlink from the new installation directory to /opt/fisheye
change ownership of the installation directory over to fisheye:fisheye.
mkdir /opt/fisheye-home
change ownership over to the fisheye account
export FISHEYE_INST in /etc/environment so that it points to /opt/fisheye-home
copy /opt/fisheye/config.xml to /opt/fisheye-home
logged out and then back into the account to make sure that the environment variable was set correctly
as the fisheye account, start fisheye: sudo -u fisheye /opt/fisheye/bin/run.sh
connect to http://localhost:8060/ and run through the wizard
had to do something odd, in order to get the server id I had to start with the "obtain evaluation license" option and then back out and do the "enter existing license option"
Added the Subversion repository by specifying svn://localhost/opt/svn

The initial installation is done. Next step, integrate it with the rest of the stack.

Spring Security 3 and Digest Authentication

2010-07-30T15:14:00.000-04:00

This was a tough one so I thought I'd take a few moments and share what I found. Context: I'm using Spring 3 as the basis for a set of RESTful services. The new MVC enhancements makes supporting a RESTful API pretty simple. What I wanted to do what to lock down the API a bit and turned to Spring Security 3 to make that happen. Not wanting to enable HTTPS, at least not yet, I turned to Digest Authentication which provides a reasonable authentication mechanism without having to hassle with digital certificates. Getting Spring Security enabled and, at least initially, authenticating with Basic Authentication was a breeze. Once I tried to enable Digests Authentication, things got weird.

Luckily, I was able to locate two resources that steered me down the correct path. One is the book, Spring Security 3 and another was an article on DZone titled "Using Spring Security to Enforce Authentication and Authorization on Spring Remoting Services Invoked From a Java SE Client". The upshot is that you cannot rely on the simple namespace configuration that is almost always talked about in the Spring Security 3 literature. It assumes a JSP type of environment with form login screens et al. In my case, I'm using just enough of the web stack to allow for HTTP based manipulation of the server-side resource -- no gui. This means you have to configure things by hand so it'll suit your needs. Here is the Spring context file I used to enable Digest Authentication for my RESTful services.

Since this was a prototype, I'm relying on the in-memory credential store but you should be able to wire up some more suitable for a production environment. What I wanted to do was to authenticate at the web layer but authorize at the services layer. The authentication is handled by the Digest Authentication pieces and the authorization is handled by annotations in service interfaces. For example:

public interface EchoPort
{
    @PreAuthorize( "hasRole( 'ROLE_ADMIN' )" )
    EchoResponse echoMessage( final EchoRequest request ) throws EchoError;
}

I used Apache HttpClient 4.0.1 for testing and was pleasantly surprised to find that it handled both Basic and Digest authentication for me. All I had to do was to provide the credentials prior to making the call the HttpClient handled the rest.

theClient.getCredentialsProvider().setCredentials( AuthScope.ANY, new UsernamePasswordCredentials( defaultUserName, defaultPassword ) );

I hope this saves you a bit of time.

Atlassian Stack: Confluence (integration)

2010-07-29T13:44:00.000-04:00

Today's goal is to integrate Confluence with Jira and Crowd. In the Jira admin doc I found a section on integrating Jira and Confluence. I'll run through that.

configured quick search to point back to the Jira URL
configured track back, had to also go into Jira
user Confluence gadgets in Jira (I those to use trusted applications instead of oauth). I had to configure trusted application on both the Jira and Confluence sides. After reading the instructions, it became clear that adding gadgets would be a pain -- you have to add them individually. I've decided to add gadgets only when/if I decide they might be useful. Enabled the atlassian-gadgest-shared plugin in Confluence
enable Jira linker plugin by entering the download URL of the plugin into the Confluence plugin manager screen
after a bit of searching I found the Confluence section that describes how to integrate Confluence into Crowd. I did not build a new directory. I just reused the one I made for Jira instead. I made the required confluence groups and made one of the users a member of both groups. copied the jar files and edited the configuration files as instructed. restarted confluence. Logged in with a crowd user and enabled External user management so that you have to manage users via Crowd.
add the "userful extendsions" described in the instructions. Of the 3 suggested, I could only install the Jira Confluence portlet because the other two wanted me to authenticate for access to accounts I don't have. Oh, we..

Not as a clean integration as I would've liked. Much of it appears to be plugin driven, which have to install yourself.

Atlassian Stack: Confluence (auto-start)

2010-07-28T12:36:00.000-04:00

I had to search on "confluence auto start" to find the page that describes how to auto start Confluence under Linux. Like the other products, the note from Daniel Harvey describing how to configure things for Karmic are the steps you want to follow. Make sure to make adjustments based on the link so that Crowd starts before Jira or Confluence. For me, that meant a trip back into the crowd.conf file. I rebooted the machine and verified that Confluence had started for me. Easy as pie.

Atlassian Stack: Confluence

2010-07-27T13:17:00.000-04:00

GreenHopper says that I have to install Confluence so that is what I'll be doing today. Dragging the Confluence card over from To DO to In Progress gets me started. I'll be following the Confluence installation guide using the standalone directions.

download and extract the standalone version into /opt
make a soft link to the new directory so I can get to it via /opt/confluence
the docs says that for Debian based Linux you might need to grab some X11 libraries. Just to be safe, I'll follow their advice and install them: sudo apt-get install libice-dev libsm-dev libx11-dev libxext-dev libxp-dev libxt-dev libxtst-dev. Installation failed because I already have the libs installed (I'm running the desktop version of Ubuntu so the X11 stuff is already available).
created a confluence user and group
created a directory to hold the Confluence data: mkdir /opt/confluence-home
changed the owner ship of that directory and the confluence installation directory to be owned by the confluence account
edited the confluence-init.properties to point to /opt/confluence-home
Jira is running on ports 8005 and 8080 which is Confluence's default port, so I have to edit the server.xml file to use a new port. I'll go with 8000 and 8090.
next, I've got to wire up Confluence to our mysql instance
I chose not to modify the MySQL instance to specify InnoDb for fear of corrupting the existing databases.
using Webmin, I created an new MySQL user named confluenceuser and gave it full privs.
using Webmin, created a new database named confluence making sure to select UTF-8 as the character set and utf8_bin as the collation order.
start up Confluence, connect to http://localhost:8090/ and run through the wizards
I elected to start with an example site instead of an empty one

Everything went off without a hitch. I did realize, however, that a task needs to be added to have Confluence auto-start at boot time. The question is, what is the best way to add that story in Greenhopper? Specifically, how can I make the task dependent? I decided to go into the "install Confluence" story and make a sub-task. That new task ended up in my To Do list. Easy. I found a workflow issue: I was allowed to resolve the master task despite having two open sub-tasks. There must be a setting somewhere that prevents that.

Atlassian Stack: Subversion and Mercurial Installation

2010-07-26T13:39:00.000-04:00

I've got a few moments so I figured I would take an easy task and install Subversion and Mercurial. I'm not sure if the Atlassian stack supports Mercurial but I'm going to install anyway, just in case. In Greenhopper, I moved the two tasks from the to-do list into my in-progress list by simply dragging and dropping the two cards from one column to the other. I was in the Task Board view at the time. Using Synaptic, I installed the subversion, subversion-tools, mercurial and mercurial-server packages. The mercurial server installation failed with a post-install trigger failure. Since I'm not even sure if I can use that server, I decided not to worry about the problem. I created a subversion user and group who will own the Subversion repository area. I made my local account a member of the subversion group. As my local user, I created the Subversion repository: svnadmin create /opt/svn. I then changed the ownership over to the subversion account and made the group bit sticky: sudo chown -R subversion:subversion /opt/svn followed by sudo chmod -R g+s /opt/svn. Lastly, I modified a copy of the /etc/init/jira.conf so that it would start Subverion in daemon mode at startup. I tested the server by using Subversion over SSH: svn list svn+ssh://localhost/opt/svn. I created a mercurial user and group and added my local account to the mercurial group. I then created a directory that will hold the various mercurial repositories making sure to set the ownership to mercurial:mercurial and making the group bit sticky. I'll admit that I didn't do any real testing but I think that when we discover an issue in the future, I'll be able to use the bug tracking features in Greenhopper. I went into GH, logged the amount of time I worked on each issue and the resolved the task.

Atlassian Stack: GreenHopper Configuration

2010-07-26T12:38:00.000-04:00

I'm following the GreenHopper 101 document to make sure that the basic configuration is correct.

install the recommended Labels plugin so I can make use of Epics
shutdown Jira: sudo service jira stop
copy the Labels plug into Jira: sudo -u jira cp jira-labels-plugin-2.4.jar /opt/jira/plugins/installed-plugins/
restart jira: sudo service jira start
follow the directions for configuring Jira for Scrum
create a new project category:Installation
create a new project: Atlassian
set the project to use the Scrum template
add Kanban constraints to the project (I don't see the view that the instructions say I should have). I noticed that the Task view wasn't coming up and the error message said it was because the project didn't have a release date. I added a version to the project and gave it a release date. Now I can get into the Task view but I don't see the Compact (Kanban) they say I should see. I looked and Googled but never found anything. Eventually, I gave up. I might take advantage of my support and see if they can help.
I had to create a new dashboard based on the default dashboard before I could add the Agile gadget indicated by the directions
I opened up GHS-1194 to see if Atlassian can help with the missing Compact (Kanban) menu selection.
Going back into the server after Atlassian commented on my ticket, I found that the missing menu selection was available. I'm guessing I had to reboot the server to make everything appear.
I was finally able to configure the constraints as described in the document. Multiple columns can be constrained so I just followed the example and did one.
I should comment that while I was waiting for Atlassian to get back to me, I created 5 stories for Sprint 1 of the project. I'm not sure if that, plus the server reboot, caused the Kanban menu selection to appear but I wanted to note it none the less.

Atlassian Stack: Crowd and Jira Integration

2010-07-20T16:56:00.000-04:00

Followed the Crowd Administration Guide, specifically the section on adding an application
create a Crowd directory for Jira. I called mine Jira
create three groups in Crowd: jira-users, jira-developers, jira-administrators and add them to the Jira directory
create at least one user who is in all 3 groups
define the Jira application in crowd following the instructions provided
copy the Crowd client libraries into the Jira application. I stopped the jira server. Removed the existing crowd client jar, copied the new one from /opt/crowd/client and set the permissions so that the file was owned by the jira user and group.
replace Jira's cache configuration file in /opt/jira/atlassian-jira/WEB-INF/classes crowd-ehcache.xml with the one in /opt/crowd/client/conf and reset the file ownership
edit /opt/jira/atlassian-jira/WEB-INF/classes/osuser.xml and add the Crowd specific section described in the instructions
edit /opt/jira/atlassian-jira/WEB-INF/classes/seraph-config.xml as instructed
start jira
log into Jira using the new Crowd user
enable external user management as instructed

Done. A lot of steps but Crowd now controls user accounts. Next up: GreenHopper configuration.

Atlassian Stack: GreenHopper

2010-07-20T12:54:00.000-04:00

grab a copy of GreenHopper, standalone version
follow the installation guide
shutdown jira - sudo service jira stop
copy the GreenHopper jar to /opt/jira/plugins/installed-plugins. I also had to change permissions on the file to be owned by the jira user and group.
start jira - sudo service jira start
hit http://localhost:8080/ and log in as the administrator
follow the directions for entering in the GreenHopper license

That is it. Tomorrow I'll configure GreenHopper because I intend to use it to help track the installation of the rest of the stack.

Atlassian Stack: Jira (autostart)

2010-07-20T12:24:00.000-04:00

I found these directions for configuring Jira to auto-start. The main recipe is for the old System V mechanism but a reader left a note on how to set things up for Upstart, which is what my Ubuntu installation uses. I'll use his configuration file which I dropped into /etc/init. All I did was edit his configuration to match my directory names, rebooted and then verified that I could hit http://localhost:8080/. Worked like a champ. Onto the next application.

Atlassian Stack: Jira

2010-07-19T13:21:00.000-04:00

Next up is Jira, the task/bug tracking portion of the stack.

Grabbed the standalone version and unpacked into the /opt directory.
Made a soft link to the install so upgrades will be a bit easier: ln -s atlassian-jira-enterprise-4.1.2-standalone jra
edit /opt/jira/atlassian-jira/WEB-INF/classes/jira-application.properties so that jira.home points /opt/jira
used Webmin to create a user just for the Jira process. I called mine, wait for it...jira.
changed ownership of the jira installation directory to be owned by the jira user
start up jira as the jira user: sudo -u jira /opt/jira/bin/startup.sh
I got a PermGen switch warning which pointed me to the troubleshooting guide. Decided not to worry about it until I actually do run out of PermGen space
I realized that the instructions did not walk me through how to connect Jira to my database so I shutdown the instance and followed those directions.
using Webmin, I created a new db user named: jirauser and gave it all permissions
using Webmin, created a new database name jira making sure to set it to use UTF-8 encodings
copy the MySQL JDBC driver to the /opt/jira/lib directory. Once I did that, I saw that a JDBC driver already existed in that directory, although a slightly older one, so I removed mine and left things as is.
run the Jira configuration tool: sudo -u jira /opt/jira/bin/config.sh . I had a problem doing this because the jira account isn't meant for logging in and doesn't have an environment. To work around this, I reset the permissions on the installation directory to my account and re-ran the tool. Once I made the proper configuration choices, I reset the files back to being owned by the jira account.
started jira again using the command in step 6
connected to Jira via the web browser: http://localhost:8080/
ran through the configuration wizard. Decided to not accept the default directories and use /opt/jira-home instead. Figured I wouldn't lose data to upgrades if I did a stand alone directory.

That's it. Seems to work. There are many configuration options that I'll need to plow through but I'll have to first learn how to get Jira to autostart at system bootup, like I did for Crowd.

Atlassian Stack: Crowd (auto-start)

2010-07-18T15:39:00.000-04:00

One final tweak I need to make to Crowd is to get it to start automatically at system start up. I'm using the Setting Crowd to Run Automatically and Use an Unprivileged System User on UNIX directions to make that happen. Here are the basics of the steps I ran:

using Webmin, create a crowd user and a crowd group, making sure to not allow the crowd user the ability to actually log in
changed ownership of the crowd installation directory and the crow-home directory so that they they were owned by the new crowd user and crowd group: sudo chown -R crowd:crowd /opt/atlassian-crowd-2.0.6/ and sudo chown -R crowd:crowd /opt/crowd-home/ (the provided script in the instructions did not appear to change all the permissions needed).
used the /etc/init/crowd.conf provided by a reader in the notes (current Ubuntu uses a slightly different startup/shutdown mechanism: upstart) instead of the Sys V script provided in the doc
issued sudo service crowd start from the command line to start up crowd
verified crowd had started by hitting http://localhost:8095/crowd/

Java Testing and Design - 2.4 The Web Rubric

2010-07-17T00:00:00.060-04:00

As with many things, clearly stated objectives and criteria are useful in determining the health of your system. A rubric has certain advantages:

assessment is more objective and consistent
helps the testers focus on clarifying criteria into specific terms
clearly describes to the developer how her work will be evaluated and what is expected
provides benchmarks against which the developers can measure progress

Here is an example ruberic:

Criteria assessed through system use	Level 1 Beginning	Level 2 Developing	Level 3 Standard	Level 4 Above Standard
Basic features are functioning	Few features work correctly the first time used	Many features do not operate. Some missing features required to complete work	Most features operate. Workarounds available to complete work.	All features work correctly every time they are used.
Speed of operations	Many features never completed.	Most features completed before user lost interest.	Most features completed in 3 seconds or less.	All features completed in 3 seconds or less.
Correct operation	Few features result successfully without an error condition.	Some features end with an error condition.	Most features complete successfully.	All features complete successfully.

Ruberics can take many forms but typically offer the following:

focus on measuring stated objectives (performance, behavior or quality)
use a range to rate performance
contain specific performance characteristics arranged in levels indicating the degree to which a standard has been met.

A ruberic's job is to help remove subjectivity when assessing the health and quality of the application.

Java Testing and Design - 2.3 Web-Enabled Application Measurement Tools

2010-07-16T00:00:00.011-04:00

The best measurements of health for a web application include:

meantime between failures in seconds
amount of time, in seconds, for each user session, sometimes knows as a transaction
application availability and peak usage periods
which media elements are most used (HTML, Flash, JavaScript, etc.)

Developing criteria based on these elements is difficult but author promises to offer help in upcoming sections. One question I had when I read this section is how to accurately determine what a failure is? If you are trolling through server logs looking for error messages, you are likely to find lots of them. What do I have to do in my application to make it easier to identify a real problem from a benign one?

Atlassian Stack: Crowd

2010-07-15T21:43:00.000-04:00

I finally got off my lazy butt and shelled out a couple dollars and purchased starter licenses for the entire Atlassian stack. I've been a user of a few of their components for a few months now and liked what I've seen so far. I'm of the opinion that most of the power comes when everything is fully integrated. I'm always writing Java programs, mostly to learn new frameworks, but have come up with an idea for something on a larger scale. I'm thinking I can use the Atlassian tools to help guide me through the specification and development process. I'll blog my journey from installation to day-to-day use. Wish me luck.

I've used the free, personal-use licenses that Atlassian used to provide and always ran into a problem with their licensing mechanism: it is tied to a machine. If I wiped my laptop to install the latest version of Ubuntu, I would have to contact Atlassian in order to get a new key. Hoping to avoid that pain, as well as providing a "time machine" I'll be installing the stack into a virtual Ubuntu server machine based on Virtual Box. Here are the basic steps I've used to get started:

created a 20 GB Virtual Box VM based on Ubuntu 10.04 Server edition
added the Webmin repository and installed Webmin. The box will be headless and I'll need a tool to help me manage the box remotely.
updated the box so it is patched to current levels
downloaded and installed the current Sun Java 6 JDK taking care to export both JAVA_HOME and JDK_HOME in the /etc/environment script. In that script, I also appended the path to point to the JDK bin directory.
installed Virtual Box Guest Additions using these instructions
using Webmin, installed mysql-server and mysql-client (I cheated and looked ahead at the installation directions and knew that a database was needed)

That gives me a basic Java enabled machine to work with. The plan is to take a snapshot prior to the installation of new server giving me the chance to rollback in case something really goes wrong.

Crowd is a single-signon solution and I figured it makes sense to be the first service to install. The documentation says that deploying their applications into a single Apache Tomcat instance is not supported so I'll use their standalone versions instead.

download and unzip Crowd. I placed mine in /opt
created /opt/crowd-home and edited /opt/crowd/crowd-webapp/WEB-INF/classes/crowd-init.properties to point to it
using Webmin, I created a MySQL user named crowduser and gave the account all db rights
using Webmin, I created the Crowd database, named crowd, taking care to specify UTF-8 as the character encoding
changed the isolation level on MySQL: sudo vi /etc/mysql/my.cnf and added transaction-isolation = READ-COMMITTED into the [mysqld] section
using Webmin, restarted MySQL so the setting would take affect/li>
download the JDBC driver and copy the JAR file to /opt/crowd/apache-tomcat/lib
start crowd via start_crowd.sh
connected to Crowd via http://localhost:8095/crowd
followed the steps in the setup wizard

That was it. I have no idea how to set up the application but a quick glance at the admin screens makes it look at though adding users and roles will be pretty simple.

Java Testing and Design - 2.2 Defining Criteria for Good Web Performance

2010-07-15T00:00:00.009-04:00

The author asserts that three criteria must be checked if you are to determine if your web application is healthy:

are the features working?
is the performance acceptable?
how often does it fail?

Create a short list of the most important features of your application and create automated tests for them. Consult real users and find out how long they will wait for a feature before abandoning it and moving on to something else. Consult your server logs to find out exactly how often the system fails. Once you know that, you decide if the failure rate is too high for your situation.

Java Testing and Design - 2.1 Just What Are Criteria?

2010-07-14T00:00:00.038-04:00

Web-enabled applications need to be constantly tested, monitored and watched, primarily due to the fact that the user's expectations of the application will change over time. At first, they might care that the application worked. Later, they might care about how fast it works? Even later, they might care if it does exactly what they need. You can use test agents to test your application daily. If you build your agents based on archetypes you can gather meaningful data that can help you gauge whether or not your application is meeting your user's needs.

The more time spent on defining archetypes, the easier it will be for your team to understand exactly what the system needs to do.

Archetypes make an emotional connection between the goals of a prototypical user of the application and the software team. Team members will have an easy-to-understand example of a person that prototypically will use your application. Rather than discussing individual features in the application, the team will be able to refer to functions that the archetype will want to regularly use to accomplish their personal goals.
Archetypes bring discussions of feature, form, and function in the application from a vague general topic to a focused user goal-oriented discussion.
Archetypes give the team an understanding of where new functions, bugs, and problems need to be prioritized in the overall schedule.

Archetypes make it easier to know what to test and why because the test agents, modeled after the archetypes, will be testing in a way that is similar to your actual users. Archetypes can also be helpful in identifying when performance becomes a problem.

Java Testing and Design - 1.5 Testing Methods

2010-07-13T00:00:00.079-04:00

This section describes the following testing methods:

click-stream testing
unit testing (state, boundary, error, privilege)
functional system testing (transactions, end-to-end tests)
scalability and performance testing (load tests)
quality of service testing

Click-stream testing is when you keep track of the clicks a user performs as she navigates your sites. The thinking is that more clicks equals more advertising revenue. This type of testing doesn't tell you if the user was able to achieve his goal when visiting your site and isn't very useful when trying to determine the quality of your software.

Unit testing is when a developer tests an individual software module by providing it various inputs and comparing its against expected outputs. Although useful to developers, module based testing doesn't tell you how useful or healthy the entire web application is. Don't rely solely on unit tests when assessing your system's health.

Functional system testing takes the perspective closer to that of the end-user in that they check the entire application from end-to-end. A system test will start at the browser, travel through the web servers, individual software modules, the database and then back. When system testing, you have to understand your system well enough to know exactly what front-end actions trigger interactions between the various back-end components. Bringing up a page which is a simple static HTML isn't much of a test because it doesn't trigger software modules to fire that access the database and other integration points.

Scalability and Performance testing tries to understand how the system behaves with concurrent users accessing the system. Typically, the functional tests can be used to drive the system and obtain loaded system performance metrics.

Quality of Service testing attempts to show that the application is meeting service level agreements. Functional tests should be used to monitor the web application's health. Done over long periods of time, you can get a picture of what is normal and what is not.

A test agent is a framework or process that allows for automation of the system's test. Typically, an agent must provide the following:

checklist - defines the conditions and states of the test, such as being able to log into the system
test process - lists the system transactions that are needed to perform the checklist
reporting method - records the results after the process and checklist are completed

The more automated a testing agent is the easier it is to certify the health of the application.

You can use your automated test agent to test for scalability and performance. Scalability describes the system's ability to serve users under varying levels of load. Run your tests when the system has 1, 100, 500 and 1000 concurrent users and measure how long the tests take to respond and you have an idea of how scalable your system is. Performance measures the system's ability to deliver accurate results under load. If the system responds quickly (scales) but returns incorrect results or errors it isn't performing very well. Taken together, performance and scalability can tell you how your system behaves under load.

Creating functional tests can be difficult because it can be difficult to know exactly how the tests should interact with the system. One idea to try are Archetypes. Construct several testing "personalities" which describe different types of system users and capture the way they interact with the system. For example, a field agent might be somebody who does lots of reads and relies heavily on the querying capabilities of the system. A manager might be more of a power user who uses the reporting portion of the system to generate his monthly status reports to his boss. The more detailed archetype you can create the better your test agents will be and the better tested your system will be. In practice, automation of the test agents is the only practical way of running the tests in any consistent and meaningful way.

Java Testing and Design - 1.4 A Concise History of Software Development

2010-07-12T00:00:00.003-04:00

This chapter shows how computer programming grew from very small and tightly controlled environments to the large, free-for-all that is today's web environment. The interesting part of the discussion revolve around the modern software development processes. A flow that the author has termed The Internet Software Development Lifecycle looks like this:

Specify the program from a mock-up of a Web site
Write the software
Unit test the application
Fix the problems found in the unit test
Internal employees test the application
Fix the problems found
Publish software to the Internet
Rapidly add minor bug fixes to the live servers

The web-style of software engineering makes use of many interconnected systems living all over the network. The same decentralization can also be applied the software components themselves as many parts of an application are implemented using shared software. The author calls to our attention the current "experiment" with open source software development and distribution as well as agile programming.

Next up, Testing Methods...

Java Testing and Design - 1.3 Why Writing High-Quality Software Today Is Hard

2010-07-11T00:00:00.045-04:00

In this section, the author offers 5 prime reasons why creating software is difficult:

The Myth of Version 2.0 Solidity - this is when people believe that scrapping the 1.0 product and starting from scratch will create a superior product. A rewrite is a major overhaul and in the author's experience a more moderate approach has proven its worth. There is more stress to get the rewrite to the same state that the 1.0 is at and there is no "down time" for developers to recharge their batteries. A smaller, iterative technique allows for constant improvements without the burnout.
Management’s Quest for Grail - this is when management lays down the law and says all projects and all teams shall use methodology X. The author contends that there are different programming styles which require different degrees of architectural direction: Users, Super Users, Administrators, Script Writers, Procedural Programmers, Object Programmers, Interoperability Programmers and Orchestration Programmers. A team must select a methodology that matches the ability and style of the team members. Forcing a team of Script Writers to adopt SCRUM might not be a wise decision.
Trying for Homogeneity When Heterogeneity Rules - trying to standardize the entire company on a single platform is not practical because you need to exorcise the "impure" parts of the existing solutions, which never happens in practice. Solutions were built upon technologies that made sense at the time and provide useful functionality so deal with it.
The Language of Bugs - instead of using bug numbers as the vocabulary to describe if a system is working correctly, why not use a test agent that is modeled after a single user? Choose just one user, understand their goals, and learn what steps they expect to use. Then model a test against the single user. The better understood that individual user’s needs are, the more valuable your test agent will be.
The Evil Twin Vice Presidents Problem - this is the scenario where development and test is run by one person and the data center is run by another. This promotes sniping between the two groups where cooperation would be more productive. The author recommends providing a common testing framework that is used by development and test to run functional, scalability and performance tests. The IT guys can leverage that framework to verify the health of the system and provide QoS reports.

Java Testing and Design - 1.2 Web-Enabled Applications, the New Frontier

2010-07-10T00:00:00.055-04:00

In this section, the author describes how software has been architected in the past. The interesting point is how he sees the new emerging architecture, namely one where the desktop contains both presentation and some business logic but much of the business logic lies elsewhere, out on the web somewhere where it accessed by RESTful and SOAP APIs. He believes that the driving force behind the new architecture are a set of "firsts":

This is the first time in the information technology (IT) industry that software developers, QA technicians, and IT managers agree on interoperability architecture.
This is the first time that developers, QA technicians, and IT managers agree that security needs to be federated.
This is the first time that developers, QA technicians, and IT managers agree that scalability in every application is critical to achieving our goals.
This is the first time that government regulation is part of our software application requirements.
This is the first time that interoperability tools and technology is delivered with integrated development environment (IDE) tools
This is the first time that Java and other interoperable development environments and open source projects have reduced operating system and platform dependencies to a point where, in many cases, it really does not matter any more.

In summary, now that we have standards, tooling and infrastructure that allow applications to inter-operate, so we'll be doing lots more of it. I use a nifty little program called XBMC to serve up media files in my home. One of its nicest features is its ability to show me information about the movie, TV show or song I'm thinking about playing. It can do this because it can use the web, and its standard protocols, to mine the internet for information. XBMC wouldn't be nearly as useful if it couldn't leverage the data contained in other applications.

Java Testing and Design - 1.1 The First Three Axioms

2010-07-09T00:00:00.016-04:00

I'll be reading Java Testing and Design: From Unit Testing to Automated Web Tests by Frank Cohen and summarizing the chapters. After skimming it I think it'll be an interesting read.

The author briefly describes his first Basic program and progresses to a more modern programming language. The upshot is that he learned the three axioms of development:

even though a program contains only a single line of code, it needs to be maintained
every piece of software inter-operates with other software
there is more to delivering high-quality software than writing code

It is these axioms that help the author to deliver quality, web-enabled Java software. Some of that software is TestMaker, an open-source test automation utility for which the author is the architect. Hopefully, these axioms will allow the author to show us some techniques for improving our ability to write better software.

Prefactoring: Plan Globally, Develop Locally

2010-07-08T00:00:00.015-04:00

Plan Globally, Develop Locally. Incremental implementations should fit into a global plan.

The main idea here is that as you go through your use cases and iterations, take few moments examine your objects to see if their structure makes sense, in broad terms, with the existing use cases as well as the planned ones. You can't know everything you'll need to know before you start typing code but you can look at your planned implementation for the sprint to make sure you won't have to do a major refactoring later on. If possible, try and keep changes as small as possible. Changing the implementation of a class is less painful than restructuring an entire inheritance hierarchy.