mikereedell.com

Back to Basics

2012-08-21T00:00:00-04:00

Back to Basics

21 August 2012

I’ve spent entirely too long being an “enterprise” developer. Too much time putting a UI on a database with a dash of proprietary ‘secret-sauce’ to justify 6- and 7- figure per annum license agreements. Too much time stringing together an ever-growing collection of frameworks, containers and libraries to build these applications in the name of ‘enterprise Java’.

These are precisely the reasons why I’m going back to basics and learning low-level programming. What do I intend to do with such knowledge? Who knows. Maybe a job building development tools, data-stores, web-servers, platforms, etc. or maybe not. Either way I’ll be learning something new and thinking about concepts that I don’t get to think about in my professional setting.

I don’t know about you, but when I hear ‘low-level programming’, I think of good, ‘ol-fashioned C. So that’s where I’m starting, getting back into the land of structs, pointers, header files and other things that give most Java developers nightmares. From C, I intend to re-kindle my brief affair with Go and possibly Rust, learning lower-level programming in a language built for modern multi-core machines.

So far, I’ve dusted off my old copy of Practical C and have been burning through the early chapters. From there I’ll go through Learn C The Hard Way since I had a wonderful time picking up the essentials of Python with Learn Python the Hard Way.

Any useful tips for getting into lower-level/system programming?

Conditionally Ignoring Test Cases With JUnit Rules

2011-12-15T00:00:00-05:00

Conditionally Ignoring Test Cases With JUnit Rules

15 December 2011

With JUnit 4.x, the developers introduced the concept of annotating tests with @Ignore so developers could mark tests that the test runner should ignore. What wasn’t added was the ability to specify conditions for which the test should be ignored, for instance:

Ignore this test when running locally:

@IgnoreWhen(env == Env.LOCAL)
public void testSomethingReallyCPUIntensive() {
	...
}

A bit of google’ing will yield a lot of pages that purport to have the solution to this problem or discuss ways to work around it.

While there’s nothing wrong with any of these solutions technically, I didn’t want to write a custom JUnit test runner nor have code that executed after the class- and method-level setup’s were run. In other words, I wanted a conditional short-circuit that would exit the test as soon as possible without running any code the test-to-be-ignored depends on.

That leads us to using JUnit’s @Rule feature. By using Rules, we can introduce just the type of short-circuit mechanism mentioned above by creating a rule that applies to all test cases that will determine whether the test case should execute or not.

The case we had to solve was specifying the test data sets that certain tests could run against. If one test had test data available in data set QA01 but not in QA02, we didn’t want that test to run, and fail, when the suite was run against QA02.

To solve this, we created a method-level annotation called DataSets:

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface DataSets {
	String[] value();
}

which we then used to annotate our test cases with the data sets that they should be executed on:

@DataSets({"QA01"})
@Test
public void someDataIntensiveTest() {
	assertTrue(....);
}

To ‘enforce’ the DataSets annotation, we created a JUnit Rule that would inspect the annotation along with the JVM argument ‘dataset’ to see if the test should be executed.

public class DataSetRule implements TestRule {

	@Override
	public Statement apply(Statement statement, Description description) {
		DataSets annotation = description.getAnnotation(DataSets.class);
		if(annotation == null) {
			return statement;
		}
		String testDataSet = System.getProperty("dataset");

		List<String> allowedDataSets = Arrays.asList(annotation.value());
		if (allowedDataSets.contains(testDataSet)) {
			return statement;
		}

		return new Statement() {

			@Override
			public void evaluate() throws Throwable {
				// Return an empty Statement object for those tests 
				// that shouldn't run on the specified dataset.
			}
		};
	}
}

This rule checks the value of the annotation, and if the ‘dataset’ JVM argument is contained in the list of allowed data sets (or the annotation isn’t present at all), passes the test (Statement object) unaltered to the next step in the execution sequence. If the ‘dataset’ JVM argument is not in the list of allowed data sets, the rule creates a new, empty Statement object which then gets executed in place of the actual test case.

To put the DataSetRule into effect, we need to decorate our test classes with one more annotation:

@Rule
public TestRule dataSetRule = new DataSetRule();

This snippet lets JUnit know that there’s a rule that needs to be executed before each test case gets run. There’s also a ClassRule interface that can be implemented to create JUnit rules that are executed before any @BeforeClass methods are run.

While this solution isn’t perfect: it still shows the tests as being run and passing, albeit with an execution time approaching zero, it does allow JUnit tests to be decorated with custom annotations that control the execution of the test

For more information on JUit rules:

Using MyBatis to Insert Collections Into MySQL

2011-03-21T00:00:00-04:00

Using MyBatis to Insert Collections Into MySQL

21 March 2011

I recently updated one of my Java projects from and old iBatis version to the latest and greatest MyBatis 3.0.4. The conversion part was fairly straightforward so I decided to see what the new library could do to help with performance.

Since the application I’m writing is an ant task that process XML documents into list data that needs to be saved to MySQL, I wanted to see if MyBatis could help out with inserting Collections.

If you google around for ‘mybatis insert list’ you’ll end up on this page which shows the following (incorrect) XML snippet:

INSERT INTO tbl_name (A,B,C) VALUES 
<foreach collection="_parameter" item="parm" open="(" close=")" separator=",">
	#{parm.val1},#{parm.val2},#{parm.val3}
</foreach>;

Which will generate SQL:

INSERT INTO tbl_name(A, B, C) VALUES (1, 2, 3, 1, 2, 3, ...)

After downloading the source, finding the unit test that covered this scenario and adding an additional test for the case I was seeing I discovered that this is the snippet that generates correct MySQL SQL for inserting a list into a table:

<insert id="insertList" useGeneratedKeys="true" parameterType="java.util.List">
	INSERT INTO tbl_name (A,B,C) VALUES (
	<foreach collection="_parameter" item="parm" open="" close="" separator="),()">
		#{parm.val1},#{parm.val2},#{parm.val3}
	</foreach>
</insert>
)

Which will output SQL like:

INSERT INTO tbl_name(A, B, C) VALUES (1, 2, 3), (1, 2, 3), .....

POST'ing Form Data to CSRF-protected Forms With Cocoa

2010-12-12T00:00:00-05:00

POST’ing Form Data to CSRF-protected Forms With Cocoa

21 December 2010

While writing MacGist, I had to engineer my way around GitHub’s lack of API support for gists. The gist API doesn’t allow for much, just simple GET’s around the gists metadata and content, for public gists only. The only way around these limitations that I saw was to programmatically access the web interface of gist.github.com and POST the data in the HTTP headers.

Sounds easy enough.

And it was, but my lovingly crafted HTTP POST’s didn’t actually do anything. Some quick Firebug’ing uncovered that Github uses CSRF form fields to protect themselves from, well, CSRF attacks. Without this uniquely generated key present in the POST data, the requests weren’t going to do anything.

Here’s the solution I came up with:

Login to github using standard username/password authentication (BASIC auth over https).
Construct the URL of the form you want to POST to, in my case the URL has the form https://gist.github.com/gists/$GIST_ID/edit.
GET the form URL.
Get the “authenticity_token” hidden form field from the HTML content.
Use the value of the “authenticity_token” in the POST data.
POST the form.

Getting the authenticity_token from the HTML form data is pretty straight-forward:

+ (NSString *) getAuthenticityTokenForUrl:(NSString *) url {
    ASIHTTPRequest *request = [ASIHTTPRequest requestWithURL:[NSURL URLWithString:url]];
    [request startSynchronous];

    NSError *error = [request error];
    if(error) {
        NSLog(@"Error getting authenticity_token from github for url: %@ with error: %@",
            url, [error localizedDescription]);
    }

    NSString *textData = [request responseString];
    NSRange location = [textData rangeOfString:@"authenticity_token" options:NSLiteralSearch];

    if(location.location != NSNotFound) {
        location.location = location.location + 41;
        location.length = 40;
        NSString *formData = [textData substringWithRange:location];
        return formData;
    }
    return nil;
}

In this solution, we’re essentially grep’ing the HTML for the string “authenticity_token” then returning the next 40 characters. There’s more than one hammer for this nail: NSScanner, regular expressions, parsing the XHTML and walking the DOM, but for something quick and easy, this one works just as well.

Sunrise-WS - A Super Simple Sinatra Wrapper for RubySunrise

2010-08-11T00:00:00-04:00

Sunrise-WS - A Super Simple Sinatra Wrapper for RubySunrise

11 August 2010

Just did a push to github of sunrise-ws, a super simple Sinatra wrapper for the RubySunrise gem. The concept is simple: call a web-service to get the sunrise/sunset data for a given location in a given timezone. Since the requirements were so simple and there is no database or security, it made sense to use Sinatra instead of rails. First off, Sinatra is easy. I know I’m not breaking new ground here, but as a 10-year professional java developer, getting ANYTHING done in that little code is next to impossible. I certainly appreciated the ease of setup of the Sinatra app and how easy it was to test it using RSpec.

Like the title says, this app is SIMPLE. It only exposes three web-services: get the sunrise/sunset data for today, a specified date, or for the entire year. That’s it. And instead of using XML as the response, everything is in JSON.

For more information on how to use it, go to the project’s README on github.

If you’d like to try it out without having to run it locally, it’s been deployed to Heroku. Hit the following links to try it out:

To get a better visualization of the JSON output, install the JSON bundle for TextMate or use this JSON Visualization page.

RubySunrise Gem

2010-03-15T00:00:00-04:00

RubySunrise Gem

15 March 2010

As a follow-up to the Java sunrise/sunset calculator I released early last year, I’m releasing a ruby gem that does essentially the same thing. Just in A LOT less code, but with the same attention to test goodness.

To install and use the RubySunrise gem just:

$ gem install RubySunrise

or download from RubyGems.

The current version as of today is 0.2, which requires the tzinfo gem to perform the timezone offset lookups.

If you want/need the lastest, just grab it from the github repo:

$ git clone git://github.com/mikereedell/sunrisesunset-ruby
$ gem build RubySunrise

At this time the RubySunrise gem covers astronomical, nautical, civil, and official sunrise/sunset calculations for every location that experiences a sunrise and sunset every day (Sorry Alaska!). I’m currently working on the strangeness that is causing my Alaskan test suite to fail with known-good data.

Example time:

require 'solareventcalculator'

date = Date.parse('2008-11-01')
calc = SolarEventCalculator.new(date, BigDecimal.new("39.9537"), BigDecimal.new("-75.7850"))

utcOfficialSunrise = calc.compute_utc_official_sunrise
localOfficialSunrise = calc.compute_official_sunrise('America/New_York')

puts "utcOfficialSunrise #{utcOfficialSunrise}"
puts "localOfficialSunrise #{localOfficialSunrise}"

Which yields:

utcOfficialSunrise 2008-11-01T11:33:00+00:00
localOfficialSunrise 2008-11-01T07:33:00-04:00

The source is available under the Apache License, Version 2.0 at Github

Maven with Ruby Part II

2009-03-20T00:00:00-04:00

Maven with Ruby Part II

20 March 2009

Want to add something to your projects Maven builds (like code coverage…) but don’t want to CTL-C/CTL-V your way to boredom? Use the below script to lend a hand. Substitute the XML in the SNIPPET variable to the artifact/plugin/dependency to be injected (leave the ‘inject’ tag in place) and use the command:

$>ruby mavenInjector.rb /xpath/to/element

Here’s the code. Check the git repository for the latest.

In the beginning of the ‘for’ loop in the inject method I remove the read-only bit from the file. This is for those of us afflicted with ClearCase and can be removed for those fortunate enough to use a sane SCM.

#/usr/bin/env ruby
require 'rexml/document.rb'
require 'fileutils.rb'

class MavenInjector

  def inject(path)
    if(path == nil)
      puts 'Must provide the root path of the node to inject under.'
      exit
    end

    poms = Dir[Dir.pwd() + "/**/pom.xml"]

    for pom in poms do
      if(!File.writable?(pom))
        FileUtils.chmod 0644, pom
      end

      pomXML = REXML::Document.new(File.new(pom))
      element = pomXML.get_elements(path)[0]

      if(element == nil)
        parentElement = pomXML.get_elements("/project/build")[0]
        element = REXML::Element.new("plugins", parentElement)
      end

      for plugin in getElementsToInject() do
        element.add_element(plugin)
      end

      formatter = REXML::Formatters::Default.new(4)
      formatter.write(pomXML, File.new(pom, "w+"))
    end
  end

  def getElementsToInject()
    elementXML = REXML::Document.new(SNIPPET)
    return elementXML.get_elements("/inject/plugin")
  end
end

SNIPPET = <<head
<inject>
  <plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>cobertura-maven-plugin</artifactId>
    <configuration>
        <formats>
          <format>html</format>
        </formats>
    </configuration>
  </plugin>
</inject>
HEAD

MavenInjector.new.inject(ARGV[0])
<inject>
  <plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>cobertura-maven-plugin</artifactId>
    <configuration>
        <formats>
          <format>html</format>
        </formats>
    </configuration>
  </plugin>
</inject>
HEAD

MavenInjector.new.inject(ARGV[0])

Any questions/problems/enhancements are welcome in the comments.

Using Ruby to Help Maven

2009-03-13T00:00:00-04:00

Using Ruby to Help Maven

13 March 2009

Maven projects can become a wasteland of neglected, old dependencies: huge, unmanageable .pom files and a local repository that’s catalog of every Apache project in the past 10 years. Who knows if your project still requires that old version of Xerces?

This script takes the name of a dependency and removes it from every pom.xml in any sub-directory from where it’s called, removing the dependency element for the given jar file from each one. The script also saves the dependency tree in the file $artifactname.xml in case you need to put it back.

#/usr/bin/env ruby
require 'rexml/document.rb'

if(ARGV[0] == nil)
  puts 'Must provide the name of the artifact to remove.'
  exit
end

artifact = ARGV[0].chomp
poms = Dir[Dir.pwd() + "/**/pom.xml"]

for pom in poms do
  pomXML = REXML::Document.new(File.new(pom))
  element = pomXML.get_elements("//dependency[artifactId='" + artifact + "']")[0]

  if(element != nil)
    temp = File.new(artifact + ".xml", "w+")
    formatter = REXML::Formatters::Default.new()
    formatter.write(element, temp)
    puts "Removing the dependency: " + artifact + "\nfrom " + pom
    element.parent.delete(element)
    pomXML.write(File.new(pom, "w+"))
  else
    puts "No artifacts with the name:'" + artifact + "' were found."
  end
end

Licensing Your Code the Ruby Way

2009-01-31T00:00:00-05:00

Licensing Your Code the Ruby Way

31 January 2009

Do you have an existing project you want to open-source? Does the thought of manually adding the license comment block to EVERY. SINGLE. SOURCE. FILE. sound not-so-exciting? Here’s some help, ruby style:

class Licenser

    def licenseFiles(dir)
          filenames = Dir[dir.chop + "**/*.java"]
	  for filename in filenames do
              contents = ''
              File.open(filename, 'r') { |file| contents = file.read }
              contents = HEADER + contents
              File.open(filename, 'w') { |file| file.write(contents) }
          end
    end
end

HEADER = &lt;&lt;HEAD
/*
 * Copyright 2008-2009 Mike Reedell / LuckyCatLabs.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

HEAD

Licenser.new.licenseFiles(ARGV[0])

This script was inspired by the the check_license_headers.rb script in the Apache SVN repo. Couldn’t get it to work on my setup, so I rewrote the parts I needed.

This can be found in the GitHub scripts project. Look for updates to handle more source types and to check for existing license headers.

Sunrise/Sunset Java Library

2009-01-27T00:00:00-05:00

Sunrise/Sunset Java Library

27 January 2009

I just wrote a Java library that computes the sunrise / sunset using GPS coordinates and a date. To use, simply download the tar or zip file from the GitHub page, or use git to pull the source:

$>git clone git://github.com/mikereedell/sunrisesunsetlib-java

Once you have the source, use the command:

$ ant all

To compile the source, build the bin, src, and test jars, execute the unit tests and generate the JavaDoc for the library.

This code snippet illustrates the main usage pattern:

// Location of sunrise/set, as latitude/longitude.
Location location = new Location("39.9937", "-75.7850");

// Create calculator object with the location and time zone identifier.
SunriseSunsetCalculator calculator = new SunriseSunsetCalculator(location, "America/New_York");

Calendar date = Calendar.getInstance();
String dawn = calculator.getCivilSunriseForDate(date);
String dusk = calculator.getCivilSunsetForDate(date);

The list of supported time zone identifiers is given by:

//Returns String[] of supported tz identifiers.
TimeZone.getAvailableIDs();

Any feedback? Comments? Features? Bugs? Comment below.

Released under the Apache License 2.0.