Nodal.info

How to start over in a Github repo

2012-01-28T12:42:00.000-08:00

So I am cleaning up some source code and putting it onto Github. I made some mistakes when I made my first commit, so I needed to back that out. The rest of this short post describes how I did it.

But first, a warning: If other people are using the code in your repo already do not do this. It will completely mess them up and should only be done when you are first committing a project, or at least only when no one has forked the project.

First you scrap your commits in your local repo...


git init
git add .

Next make sure there are no extraneous files...


git status

If there are files you don't mean to add the remove them...


git rm --cached FileToRemove

Next do the commit and force it onto the master...


git commit -a -m 'first commit'
git remote add origin git@github.com:Owner/Repo.git
git push origin +master

Being More Productive

2011-11-27T08:44:00.001-08:00

The start of a personal productivity plan

Some background

I'm already very productive at work in my day job but my side work has not been as productive as I'd like. So I've decided to adapt the techniques I use during the regular work day to my side business and personal life.

As an incentive I decided to be open about it and publish as much as I can about my path to better personal productivity. This post is the first in that series, all of which will be tagged #ProductivityMeasures

First Steps

So the first thing is to establish the measures. In the System Engineering world there are two main types of measures. There are Measures of Effectiveness (MoEs) and Measures of Performance (MoPs). To paraphrase the DAU Dictionary definition a Measure of Effectiveness (MoE) is a measure of how a system changes its environment, i.e. the effect it has. A Measure of Performance is more granular measure than a Measure of Effectiveness and is also more quantitative. Right now I am going to focus on MoEs, later I may decompose them into MoPs

To provide a context for my effort I need to specify the environment affected, which I'm defining as an combination of available Open Source software, my own closed source software, my social network, the body of knowledge around the technologies I am interested in (mobile apps, Node.js, social software, semantic technologies, and more), humanity, and family. That is just the initial definition of the environment, I expect to revise it as I go forward. Humanity is included because while a positive effect on humanity is not going to make me more productive immediately it will likely make me a better person and a more connected person. Family is included because my family keeps me grounded, balanced, and sane - and having a positive effect on my family maks me a better person. Of course I am the system under measurement.

So how can I affect this environment?

Here are some ways:

Write a blog post that increases the technical body of knowledge (at least I'd like to think so)
Make a piece of Open Source Software
Create a document describing an idea for a piece of Open Source software to create
Create a document describing an idea for a new blog post to publish
Improve a blog post draft
Publish a blog post
Make a piece of closed source software
Join a new social network
Add info that will enhance my discoverability to my profile on a social network
Add a contact to one of my social networks
Help someone in my social network
Strengthen my ties by having an authentic conversation with someone in my social network. What do I mean by an authentic conversation? For that see E.E. Kim's great post on "Authentic Relationships and Networking"
Do one of the things that don't directly make me more productive, but that make me a better person and better people are more productive. These things are:
- Watch something with my wife or daughter and have an authentic conversation about it.
- Make something with my daughter
- Have an authentic conversation with my wife or daughter
- A family trip
- Teach my daughter something in depth
- Volunteer service
- Give a gift (and I don't mean Farmville)

We've described possible effects, still need measures and goals

The previous section describes the how I can affect my environment in ways that will make me more productive. Describing these in itself won't help me be more productive. To do that I need to take the descriptions of effects and determine how to measure them, then decide on my goals, then measure my effects against these goals and alter my behavior accordingly.

So the next step is to determine how to measure these effects. That's actually easy - since I organized the list into discrete activities the unit of measure is one activity. That's not saying writing a complete web-app is the same as a single Java class, but it's a place to start.

threshold: The lowest level, this is the bare minimum that's considered to meet the goal
objective: This is the level of effect that is expected and planned
ideal: The stars align, everything works for the first time, and this level of effect is achieved. This is what I strive for.

So thinking about the above list I came up with the following areas of effect: family, blog, standards work, software, social network.

I need to have a positive effect on each at least once a day, so there is my threshold

For my blogging objectives I'd like to write 3 good blog posts a week, with an ideal of one a day

For my social networking objectives I'd like to add three new contacts a week, ideally one a day. These a meaningful contacts mind you, not the 'I like your smile' kind (see E.E. Kim's article). I'd also like to add one new social network a quarter, ideally one a month. I'd also like to post 3 useful links/comments each week to LinkedIn, Facebook, and StackOverflow, ideally one per day.

To be finished later today...

Add goals for family here

Add goals for standards work here

Next steps

Next I need to figure out a schedule to keep in order to meet my objective goals and the tools I'll use to track my efforts, but that will be a different blog post.

Installing and testing the DITA Open Toolkit

2011-06-27T15:06:00.000-07:00

Installed the DIT Open Toolkit with...



$ cd /code

$ wget http://sourceforge.net/projects/dita-ot/files/DITA-OT%20Stable%20Release/DITA%20Open%20Toolkit%201.5.3/DITA-OT1.5.3_full_easy_install_bin.tar.gz/download

$ tar zxvf DITA-OT1.5.3_full_easy_install_bin.tar.gz

Next we need to test it by trying to build just the web version of the sample files...



$ cd DITA-OT1.5.3

$ export DITA_HOME=/code/DITA-OT1.5.3

$ ./startcmd.sh 

$ ant samples.web -f build_demo.xml

Seems to have worked, let's see what it produced...



$ find out/samples/web

out/samples/web

out/samples/web/commonltr.css

out/samples/web/commonrtl.css

out/samples/web/concepts

out/samples/web/concepts/garageconceptsoverview.html

out/samples/web/concepts/lawnmower.html

out/samples/web/concepts/oil.html

out/samples/web/concepts/paint.html

out/samples/web/concepts/shelving.html

out/samples/web/concepts/snowshovel.html

out/samples/web/concepts/toolbox.html

out/samples/web/concepts/tools.html

out/samples/web/concepts/waterhose.html

out/samples/web/concepts/wheelbarrow.html

out/samples/web/concepts/workbench.html

out/samples/web/concepts/wwfluid.html

out/samples/web/dita.list

out/samples/web/dita.xml.properties

out/samples/web/image

out/samples/web/image/carwash.jpg

out/samples/web/index.html

out/samples/web/tasks

out/samples/web/tasks/changingtheoil.html

out/samples/web/tasks/garagetaskoverview.html

out/samples/web/tasks/organizing.html

out/samples/web/tasks/shovellingsnow.html

out/samples/web/tasks/spraypainting.html

out/samples/web/tasks/takinggarbage.html

out/samples/web/tasks/washingthecar.html

out/samples/web/toc.html

$ open out/samples/web/index.html

Next lets build the PDF version...



$ ant samples.pdf -f build_demo.xml

out/samples/pdf

out/samples/pdf/Configuration

out/samples/pdf/Configuration/OpenTopic

out/samples/pdf/Configuration/OpenTopic/cfg

out/samples/pdf/Configuration/OpenTopic/cfg/common

out/samples/pdf/Configuration/OpenTopic/cfg/common/artwork

out/samples/pdf/Configuration/OpenTopic/cfg/common/artwork/hand.gif

out/samples/pdf/Configuration/OpenTopic/cfg/common/artwork/warning.gif

out/samples/pdf/Customization

out/samples/pdf/Customization/OpenTopic

out/samples/pdf/Customization/OpenTopic/common

out/samples/pdf/Customization/OpenTopic/common/artwork

out/samples/pdf/Customization/OpenTopic/common/artwork/README.txt

out/samples/pdf/dita.list

out/samples/pdf/dita.xml.properties

out/samples/pdf/image

out/samples/pdf/image/carwash.jpg

out/samples/pdf/sequence.pdf

$ open out/samples/pdf/sequence.pdf

How to build the OpenXDI server

2011-06-27T14:30:00.000-07:00

So first we have to get the code...



$ cd /code/

$ svn checkout https://svn.gluu.info/repository/openxdi/code/ oxserver

Next we try to build (I am using Maven 2.1):



$ cd oxserver

$ mvn install

The result? I got a bunch of errors on incompatibility with -source 1.3. Probably because they build using their own settings file on the U: drive, according to mvn_ox.bat.

So I added the following in the pom.xml:



 16  <build>

 17     <plugins>

 18       <plugin>

 19         <groupId>org.apache.maven.plugins</groupId>

 20         <artifactId>maven-compiler-plugin</artifactId>

 21         <version>2.3.2</version>

 22         <configuration>

 23           <source>6</source>

 24           <target>6</target>

 25         </configuration>

 26       </plugin>

 27     </plugins>

 28   </build>

This got me past the Java syntax errors, but the following two tests fail:

initializationError(org.xdi.graphmodel.TestGraphBuilder)
initializationError(org.xdi.graphmodel.TestGraphBuilder)

So I check the error log...
$cat /code/oxserver/GraphModel/target/surefire-reports/org.xdi.graphmodel.TestGraphBuilder.txt | pbcopy

and I see...

-------------------------------------------------------------------------------
Test set: org.xdi.graphmodel.TestGraphBuilder
-------------------------------------------------------------------------------
Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.008 sec <<< FAILURE!
initializationError(org.xdi.graphmodel.TestGraphBuilder) Time elapsed: 0.003 sec <<< ERROR!
java.lang.Exception: Test class should have exactly one public constructor
at org.junit.runners.BlockJUnit4ClassRunner.validateOnlyOneConstructor(BlockJUnit4ClassRunner.java:143)
at org.junit.runners.BlockJUnit4ClassRunner.validateConstructor(BlockJUnit4ClassRunner.java:132)
at org.junit.runners.BlockJUnit4ClassRunner.collectInitializationErrors(BlockJUnit4ClassRunner.java:121)
at org.junit.runners.ParentRunner.validate(ParentRunner.java:269)
at org.junit.runners.ParentRunner.<init>(ParentRunner.java:66)
at org.junit.runners.BlockJUnit4ClassRunner.<init>(BlockJUnit4ClassRunner.java:58)
at org.junit.internal.builders.JUnit4Builder.runnerForClass(JUnit4Builder.java:13)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:57)
at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:29)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:57)
at org.junit.internal.requests.ClassRequest.getRunner(ClassRequest.java:24)
at org.apache.maven.surefire.junit4.JUnit4TestSet.<init>(JUnit4TestSet.java:45)
at org.apache.maven.surefire.junit4.JUnit4DirectoryTestSuite.createTestSet(JUnit4DirectoryTestSuite.java:56)
at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.locateTestSets(AbstractDirectoryTestSuite.java:96)
at org.apache.maven.surefire.Surefire.createSuiteFromDefinition(Surefire.java:209)
at org.apache.maven.surefire.Surefire.run(Surefire.java:156)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:345)
at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1009)

initializationError(org.xdi.graphmodel.TestGraphBuilder) Time elapsed: 0.001 sec <<< ERROR!
java.lang.Exception: No runnable methods
at org.junit.runners.BlockJUnit4ClassRunner.validateInstanceMethods(BlockJUnit4ClassRunner.java:177)
at org.junit.runners.BlockJUnit4ClassRunner.collectInitializationErrors(BlockJUnit4ClassRunner.java:122)
at org.junit.runners.ParentRunner.validate(ParentRunner.java:269)
at org.junit.runners.ParentRunner.<init>(ParentRunner.java:66)
at org.junit.runners.BlockJUnit4ClassRunner.<init>(BlockJUnit4ClassRunner.java:58)
at org.junit.internal.builders.JUnit4Builder.runnerForClass(JUnit4Builder.java:13)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:57)
at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:29)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:57)
at org.junit.internal.requests.ClassRequest.getRunner(ClassRequest.java:24)
at org.apache.maven.surefire.junit4.JUnit4TestSet.<init>(JUnit4TestSet.java:45)
at org.apache.maven.surefire.junit4.JUnit4DirectoryTestSuite.createTestSet(JUnit4DirectoryTestSuite.java:56)
at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.locateTestSets(AbstractDirectoryTestSuite.java:96)
at org.apache.maven.surefire.Surefire.createSuiteFromDefinition(Surefire.java:209)
at org.apache.maven.surefire.Surefire.run(Surefire.java:156)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:345)
at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1009)

Oh well, Open Source is not called bleeding edge for nothing. By all accounts Yuri does good work so hopefully it is just a minor fix. Time to breakout the workhorse program:

$ emacs GraphModel/test-src/org/xdi/graphmodel/TestGraphBuilder.java

Looks like everything is static and this is a singleton helper for the tests. Problem is that it is named TestGraphBuilder so Junit thinks it's a test since it starts with Test, which is reasonable.

So I renamed it to SampleGraphBuilder.java and reflect that name through the test code (include that file itself), then see what happens...

Sidebar: If you are trying to use sed on a Mac you need to know you are using BSD sed. Why? because using a command like:
sed -i "s/TestGraphBuilder/SampleGraphBuilder/g" GraphModel/test-
src/org/xdi/graphmodel/xri/SynonymsPatternXri20110317.java

Will give you a cryptic error about missing characters. BSD sed requires a backup extension after the -i option, like so:

sed -i.bak "s/TestGraphBuilder/SampleGraphBuilder/g" GraphModel/test-
src/org/xdi/graphmodel/xri/SynonymsPatternXri20110317.java

So that said, how do we do all the replacing with one command line...
grep -R TestGraphBuilder GraphModel/test-src | grep .java\: | awk -F':' '{ print $1 }' | xargs -I {} sed -i.bak 's/TestGraphBuilder/SampleGraphBuilder/g' {}

That gets us passing tests, but the jars do not get produced. Instead we get...

[INFO] [jar:jar]
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Error assembling JAR

Embedded error: basedir /code/oxserver/GraphModel/bin/production does not exist

That's a problem, since as far as I can tell there is no assembly file anywhere, named production or otherwise.

I see a separate build.xml so maybe we can switch to ant to produce the jar file:
ant archive

Unfortunately this produces the following error output:

BUILD FAILED
/code/oxserver/build.xml:67: /code/oxserver/libs not found.

Looking at the directory more I see a weird directory: GraphModel/bin\\production

Looks like Windows specific artifact issue, so let's fix the directory paths with:



$ mkdir GraphModel/bin

$ mv GraphModel/bin\\production GraphModel/bin/production

I also see a GraphModel/bin\\test\\GraphModel so fix that in the same way:



$ mkdir GraphModel/bin/test

$ mv GraphModel/bin\\test\\GraphModel GraphModel/bin/test/GraphModel

And a...

$ mvn clean install

Only to get another error, this time Mac related:

[INFO] [jar:jar]
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Error assembling JAR
Embedded error: /code/oxserver/GraphModel/bin\production/.DS_Store isn't a file.

Fixed this with:



$ find . -name \.DS_Store | xargs rm -rf

This gets us close, but still gives an error:

[WARNING] File encoding has not been set, using platform encoding MacRoman, i.e. build is platform dependent!
[INFO] Compiling 51 source files to /code/oxserver/WebClient/bin\production

Looks like another Windows artifact, so fixed the same way:



$ mkdir WebClient/bin

$ mv WebClient/bin\\production WebClient/bin/production

Still getting errors...

WARNING] POM for 'com.ibm.icu:icu4j:pom:3.4.4:compile' is invalid.

Its dependencies (if any) will NOT be available to the current build.
[WARNING] POM for 'org.slf4j:slf4j-jcl:pom:1.0.1:compile' is invalid.

Its dependencies (if any) will NOT be available to the current build.
[WARNING] POM for 'jug:jug:pom:1.1.2:compile' is invalid.

Its dependencies (if any) will NOT be available to the current build.
[WARNING] POM for 'xml-security:xmlsec:pom:1.3.0:compile' is invalid.

Its dependencies (if any) will NOT be available to the current build.
[WARNING] POM for 'xalan:xalan:pom:2.7.0:compile' is invalid.

Its dependencies (if any) will NOT be available to the current build.
[WARNING] POM for 'net.sf.ehcache:ehcache:pom:1.2.3:compile' is invalid.

Its dependencies (if any) will NOT be available to the current build.
[WARNING] POM for 'org.apache.httpcomponents:httpclient:pom:4.0.1:compile' is invalid.

Its dependencies (if any) will NOT be available to the current build.
[WARNING] POM for 'org.apache.httpcomponents:httpcore:pom:4.1:compile' is invalid.

Its dependencies (if any) will NOT be available to the current build.
[WARNING] POM for 'commons-codec:commons-codec:pom:1.4:compile' is invalid.

Its dependencies (if any) will NOT be available to the current build.
[INFO] [compiler:compile]
[WARNING] File encoding has not been set, using platform encoding MacRoman, i.e. build is platform dependent!
[INFO] Compiling 51 source files to /code/oxserver/WebClient/bin\production
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] error: error reading /Users/wolf/.m2/repository/com/ibm/icu/icu4j/3.4.4/icu4j-3.4.4.jar; cannot read zip file
[ERROR] /code/oxserver/WebClient/src/org/xdi/webclient/server/impl/MessageRequestor.java:[4,22] package org.apache.http does not exist
[ERROR] /code/oxserver/WebClient/src/org/xdi/webclient/server/impl/MessageRequestor.java:[5,22] package org.apache.http does not exist
[ERROR] /code/oxserver/WebClient/src/org/xdi/webclient/server/impl/MessageRequestor.java:[10,30] package org.apache.http.message does not exist
[ERROR] /code/oxserver/WebClient/src/org/xdi/webclient/server/impl/MessageRequestor.java:[68,19] cannot find symbol
symbol  : class HttpResponse
location: class org.xdi.webclient.server.impl.MessageRequestor
[ERROR] /code/oxserver/WebClient/src/org/xdi/webclient/server/impl/MessageRequestor.java:[51,14] cannot find symbol
symbol  : class HttpResponse
location: class org.xdi.webclient.server.impl.MessageRequestor
[ERROR] /code/oxserver/WebClient/src/org/xdi/webclient/server/impl/MessageRequestor.java:[71,38] cannot access org.apache.http.message.AbstractHttpMessage
class file for org.apache.http.message.AbstractHttpMessage not found
            final HttpPost httppost = new HttpPost(p_url);
[ERROR] /code/oxserver/WebClient/src/org/xdi/webclient/server/impl/MessageRequestor.java:[74,23] cannot find symbol
symbol  : class NameValuePair
location: class org.xdi.webclient.server.impl.MessageRequestor
[ERROR] /code/oxserver/WebClient/src/org/xdi/webclient/server/impl/MessageRequestor.java:[74,69] cannot find symbol
symbol  : class NameValuePair
location: class org.xdi.webclient.server.impl.MessageRequestor
[ERROR] /code/oxserver/WebClient/src/org/xdi/webclient/server/impl/MessageRequestor.java:[75,35] cannot find symbol
symbol  : class BasicNameValuePair
location: class org.xdi.webclient.server.impl.MessageRequestor
[ERROR] /code/oxserver/WebClient/src/org/xdi/webclient/server/impl/MessageRequestor.java:[78,31] cannot access org.apache.http.entity.StringEntity
class file for org.apache.http.entity.StringEntity not found
            httppost.setEntity(new UrlEncodedFormEntity(nameValuePairs));
[ERROR] /code/oxserver/WebClient/src/org/xdi/webclient/server/impl/MessageRequestor.java:[80,29] cannot access org.apache.http.HttpHost
class file for org.apache.http.HttpHost not found
            return httpclient.execute(httppost);
[INFO] 12 errors 
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Compilation failure

error: error reading /Users/wolf/.m2/repository/com/ibm/icu/icu4j/3.4.4/icu4j-3.4.4.jar; cannot read zip file
/code/oxserver/WebClient/src/org/xdi/webclient/server/impl/MessageRequestor.java:[4,22] package org.apache.http does not exist
/code/oxserver/WebClient/src/org/xdi/webclient/server/impl/MessageRequestor.java:[5,22] package org.apache.http does not exist
/code/oxserver/WebClient/src/org/xdi/webclient/server/impl/MessageRequestor.java:[10,30] package org.apache.http.message does not exist
/code/oxserver/WebClient/src/org/xdi/webclient/server/impl/MessageRequestor.java:[68,19] cannot find symbol
symbol  : class HttpResponse
location: class org.xdi.webclient.server.impl.MessageRequestor
/code/oxserver/WebClient/src/org/xdi/webclient/server/impl/MessageRequestor.java:[51,14] cannot find symbol
symbol  : class HttpResponse
location: class org.xdi.webclient.server.impl.MessageRequestor
/code/oxserver/WebClient/src/org/xdi/webclient/server/impl/MessageRequestor.java:[71,38] cannot access org.apache.http.message.AbstractHttpMessage
class file for org.apache.http.message.AbstractHttpMessage not found
            final HttpPost httppost = new HttpPost(p_url);
/code/oxserver/WebClient/src/org/xdi/webclient/server/impl/MessageRequestor.java:[74,23] cannot find symbol
symbol  : class NameValuePair
location: class org.xdi.webclient.server.impl.MessageRequestor
/code/oxserver/WebClient/src/org/xdi/webclient/server/impl/MessageRequestor.java:[74,69] cannot find symbol
symbol  : class NameValuePair
location: class org.xdi.webclient.server.impl.MessageRequestor
/code/oxserver/WebClient/src/org/xdi/webclient/server/impl/MessageRequestor.java:[75,35] cannot find symbol
symbol  : class BasicNameValuePair
location: class org.xdi.webclient.server.impl.MessageRequestor
/code/oxserver/WebClient/src/org/xdi/webclient/server/impl/MessageRequestor.java:[78,31] cannot access org.apache.http.entity.StringEntity
class file for org.apache.http.entity.StringEntity not found
            httppost.setEntity(new UrlEncodedFormEntity(nameValuePairs));
/code/oxserver/WebClient/src/org/xdi/webclient/server/impl/MessageRequestor.java:[80,29] cannot access org.apache.http.HttpHost
class file for org.apache.http.HttpHost not found
            return httpclient.execute(httppost);

It looks like two things.

First, the dependency with group com.ibm.icu, artifact icu4j ,and version 3.4.4 appear missing. I've run into this before because icu is not in the central repository.

Second, the build seems to be creating the weird Windows specific directories. Looking into the pom.xml files in the modules GraphModel and WebClient that seems to be the case. Using \ characters in path works perfectly fine on Windows, but if you go to a flavor of Unix then it gets treated as a special character. On the other hand the / character is handled properly by Maven on both platforms.

So I fired up emacs and changed the offending '\' characters to '/' characters, and then rebuilt.

Still getting the missing icu jar, which was expected

Spending more time on this than expected, and icu4j is not a direct dependency which means fixing it could be a pain, so taking easy way...



$ wget http://dev.mapfish.org/maven/repository/com/ibm/icu/icu4j/3.4.4/icu4j-3.4.4.jar

$ mvn install:install-file -DgroupId=wrap:mvn:com.ibm.icu -DartifactId=icu4j -Dversion=3.4.4 -Dpackaging=jar -Dfile=icu4j-3.4.4.jar

This got me past the icu4j issue, but still going to have same issue for several jars so this is where I am calling it, since I am most interested in the server (i.e. GraphModel) portion anyways and I don't have the time to debug the rest of the Maven issues. I may revisit this later.

Node, sqlite, and Locker - oh my!

2011-03-25T19:41:00.001-07:00

This article is the first in the Lions, Tigers, and Bears - Oh My series, how-to articles involving getting several pieces of software (usually with node.js in the mix) to work together.

This week the software is:

node.js : the fast server side javascript platform built on the Google V8 engine
node-sqlite : the node.js bindings to the sqlite library
Locker : The Personal Data Store (PDS) project from Jeremie Miller

npm, the node package manager, was installed as well in the course of installing the above software.

The operating system this was installed on was Ubuntu and running inside VirtualBox.

The Ubuntu iso used was ubuntu-10.10-server-i386.iso.

Script started on Fri 25 Mar 2011 05:37:14 PM EDT

tester@whitestar:~$ uname -a

Linux whitestar 2.6.35-22-generic #33-Ubuntu SMP Sun Sep 19 20:34:50 UTC 2010 i686 GNU/Linux

tester@whitestar:~$ apt-get update

tester@whitestar:~$ sudo apt-get install git

tester@whitestar:~$ sudo apt-get install build-essential

tester@whitestar:~$ git clone git://github.com/joyent/node.git

tester@whitestar:~$ cd node

You can go with the distribution standard packages for OpenSSL:

tester@whitestar:~/node$ sudo apt-get install openssl

tester@whitestar:~/node$ sudo apt-get install libssl-dev

Or alternatively, if you want openssl-1.0.0d...

tester@whitestar:~/node$ mkdir ../downloads && pushd ../downloads

tester@whitestar:~/downloads$ wget http://www.openssl.org/source/openssl-1.0.0d.

tar.gz

tester@whitestar:~/downloads$ tar xzf openssl-1.0.0d.tar.gz

tester@whitestar:~/downloads$ cd openssl-1.0.0d

tester@whitestar:~/downloads/openssl-1.0.0d$ ./config

tester@whitestar:~/downloads/openssl-1.0.0d$ make

tester@whitestar:~/downloads/openssl-1.0.0d$ sudo make install

tester@whitestar:~/node$ ./configure

Checking for program g++ or c++ :/usr/bin/g++

Checking for program cpp :/usr/bin/cpp

Checking for program ar :/usr/bin/ar

Checking for program ranlib :/usr/bin/ranlib

Checking for g++ :ok

Checking for program gcc or cc :/usr/bin/gcc

Checking for program ar :/usr/bin/ar

Checking for program ranlib :/usr/bin/ranlib

Checking for gcc :ok

Checking for library dl :yes

Checking for openssl :not found

Checking for function SSL_library_init :yes

Checking for header openssl/crypto.h :yes

Checking for library util :yes

Checking for library rt :yes

Checking for CLOCK_MONOTONIC :yes

--- libeio ---

Checking for library pthread :yes

Checking for function pthread_create :yes

Checking for function pthread_atfork :yes

Checking for futimes(2) :yes

Checking for readahead(2) :yes

Checking for fdatasync(2) :yes

Checking for pread(2) and pwrite(2) :yes

Checking for sendfile(2) :yes

Checking for sync_file_range(2) :yes

--- libev ---

Checking for header sys/inotify.h :yes

Checking for function inotify_init :yes

Checking for header sys/epoll.h :yes

Checking for function epoll_ctl :yes

Checking for header port.h :not found

Checking for header poll.h :yes

Checking for function poll :yes

Checking for header sys/event.h :not found

Checking for header sys/queue.h :yes

Checking for function kqueue :not found

Checking for header sys/select.h :yes

Checking for function select :yes

Checking for header sys/eventfd.h :yes

Checking for function eventfd :yes

Checking for SYS_clock_gettime :yes

Checking for library rt :yes

Checking for function clock_gettime :yes

Checking for function nanosleep :yes

Checking for function ceil :yes

Checking for fdatasync(2) with c++ :yes

'configure' finished successfully (6.158s)

But it says OpenSSL not found. Actually it was because configure test for openssel libs

in two ways. The first is to see if the OpenSSL package is detected as installed, and in

this case it is installed but it's not detected so that test fails. The second way is to see

if it can find the openssl dev library and if that library has a certain function SSL_Library_init.

This test passes, so we are ok to proceed.

So standard steps now to make and install node...

tester@whitestar:~/node$ make

tester@whitestar:~/node$ sudo make install

tester@whitestar:~/node$ cd ..

Now we clone, make and install npm, the node package manager. Think of npm

as apt-get for node.js and you wouldn't be that far off.

tester@whitestar:~$ git clone http://github.com/isaacs/npm.git

tester@whitestar:~$ cd npm

tester@whitestar:~/npm$ sudo make install

tester@whitestar:~/npm$ cd ..

Now to build Locker, Jeremie Miller's interesting Personal Data Ecosystem

project.

First we clone the repo.

tester@whitestar:~$ git clone https://github.com/quartzjer/Locker.git

Now here's where things get a little funky. If you built Locker now then

it would use the publihs 1.0.3 version of the node-sqlite package. That

would be ok as long as you didn't have a different version of sqlite installed

anywhere on your system include path. If you do, like all Mac OS X users,

then the node-sqlite dependency build will break preventing Locker from

being built.

I went through this and created a modified version of node-sqlite that has

updated sqlite dependencies (to 3.7.5) and fixes the include problem.

So lets get that version downloaded from my repository:

tester@whitestar:~$ git clone https://github.com/billbarnhill/node-sqlite.git

Now we have to go into Locker and tell it to use the local version of node-sqlite:

tester@whitestar:~$ cd Locker

tester@whitestar:~/Locker$ sudo npm install ../node-sqlite

We're at the point now where we can build Locker...

tester@whitestar:~/Locker$ sudo npm install

tester@whitestar:~/Locker$ node locker.js

[18:53:07][Locker] Spawned dashboard pid: 20329

[18:53:07][Locker] locker running at http://localhost:8042/

[18:53:07][Locker] generating id private key

[18:53:07][Locker] generating id public key

[18:53:07][Locker] id keys loaded

[18:53:07][Locker] dashboard stdout: http://localhost:18042/

Now you're ready to open up the dashboard by pasting the following

url into your browser's address bar and going to that location:

http://localhost:8042/

Useful Erlang snippet to find calling function

2008-03-25T14:40:00.000-07:00

The following snippet is the erlang equivalent of generating an exception and getting the backtrace info. Still need to parse out the function name. Working on integrating this into a debugging module.

1> B = fun () -> erlang:process_info(erlang:self(), [backtrace]) end.

2> A = fun () -> B() end.

3> [{backtrace, Bin}] = A().

4> io:format("~s",[binary_to_list(Bin)]).

Program counter: 0x00000001005d6de0 (unknown function)
CP: 0x0000000101a975f8 (erl_eval:do_apply/5 + 2608)

0x00000001010142a0 Return addr 0x0000000101a944e0 (erl_eval:expr/5 + 15888)
y(0) []
y(1) []
y(2) [{'A',#Fun},{'B',#Fun}]

0x00000001010142c0 Return addr 0x0000000101acf9b8 (shell:exprs/6 + 736)
y(0) none
y(1) {cons,1,{tuple,1,[{atom,1,backtrace},{var,1,'Bin'}]},{nil,1}}

0x00000001010142d8 Return addr 0x0000000101acf138 (shell:eval_exprs/6 + 160)
y(0) []
y(1) []
y(2) [{'A',#Fun},{'B',#Fun}]
y(3) {value,#Fun}
y(4) {eval,#Fun}
y(5) 15
y(6) []
y(7) []
y(8) []

0x0000000101014328 Return addr 0x0000000101acecc0 (shell:eval_loop/3 + 600)
y(0) []
y(1) []
y(2) [{'A',#Fun},{'B',#Fun}]
y(3) <0.30.0>
y(4) Catch 0x0000000101acf228 (shell:eval_exprs/6 + 400)

0x0000000101014358 Return addr 0x000000010017b7a8 ()
y(0) 15
y(1) <0.30.0>
ok

A method for visualizing progress against requirements

2008-03-23T14:10:00.000-07:00

This post describes a novel way I've come up with to visualize a development team's progress against a set of requirements. It should work equally well on an agile-style backlog. For the method to work the requirements must be categorized and each requirement either marked as satisfied or not satisfied. A paper is in the works to publish this and I'll update this post with a link to that when it's published.

So the steps:
1. Create a radial graph
2. Divide it equally into sections matching the categories, with boundaries being lines radiating out from center
3. Mark these axis with their category names
4. Figure out max displayable radius of the graph
5. Divide that by the # of requirements in the largest category to get tick gap
6. Give ticks on each axis spaced tick gap apart
7. Draw lines from total requirements ticks to each other, making an irregular polygon, and shade a light color
8. Do same for completed # of requirements ticks, and shade a dark color.

Some alternate approaches:
1. Assume each category is equal despite # of requirements, so normalize. E.g. if you have 3 categories A,B,C with completed/total ratios of 3/8,2/4,3/4 then the tick gap in B and C would be twice the tick gap of A
2. Adjust tick gaps based on importance of requirement

I still need to do some research to make sure this is not re-inventing the wheel, but I haven't seen this approach yet, so am hopeful.

How to...run Yaws embedded, Pt. 2

2008-02-24T12:32:00.000-08:00

Ok, so now we are able to run Yaws from a script. Next up on my todo is running yaws from within a gen_server application.

It gets a little different and during the process I unrolled of the yaws functions to streamline things. The code below works for both starting and stopping. Make sure you follow the previous post's steps on building yaws.

In addition you need a project dir layout like this:

Project root
3rdparty
- yaws
ebin
src
include

You'll need an Emakefile in your src directory. Something like this:
{'*', [{outdir,"../ebin"},{pa,".."},{i, "../include"},{i, "../3rdparty/yaws/include"}, {warn_format,0}]}.

Then put all your .hrl files in ./include, your .erl in src, and build with


    rm ebin/*.beam
    pushd src
    erl -noshell -s make all -s init stop
    popd

The source for the gen_server code is:


-module(atom_app).
-behavior(application).

-include("yaws.hrl").

%% application callbacks
-export([start/2, 
         stop/1]).

start(_Type, Args) ->
 start_atom_yaws(Args),
 error_logger:info_msg("Starting atom_app~n",[]),
    atom_sup:start_link([]).

stop(_State) ->
 stop_yaws(),
    ok.

start_atom_yaws(Args) ->
 error_logger:info_msg("Starting embedded yaws~n",[]),
 Id = "yawstest",
 Debug = true,
 ok = application:load(yaws),
 ok = application:set_env(yaws, embedded, true),
 ok = application:set_env(yaws, id, Id),
 application:start(yaws),
        DefaultGC = yaws_config:make_default_gconf(Debug, Id),
        GC = DefaultGC#gconf{tmpdir="./temp/yaws",logdir="./temp/logs"},
        yaws:mkdir(GC#gconf.tmpdir),
        yaws:mkdir(GC#gconf.logdir),
        SC = #sconf{port = 8080,
  servername = "ubuntu01",
  listen = {0,0,0,0},
  docroot = "www"
 },
 Result = yaws_api:setconf(GC, [[SC]]),
 error_logger:info_msg("Finished starting embedded yaws~n",[]),
 Result.

stop_yaws() ->
 application:stop(yaws).

How to...run Yaws embedded

2008-02-24T10:33:00.000-08:00

Yaws is the primary Erlang web server and can be run as part of an application or script, not just standalone.

I've found some good tutorials out there but they didn't quite get me what I wanted:

What I wanted that was missing was:

No yaws command line args
Debugging in Yaws on
I didn't want to create a full app to embed yaws
Temp dirs are made by script if needed
Set temp dir for where logs go, and for the yaws working dir
Run yaws from a subdir rather than globally installed

Steps:

In you project directory make a new '3rdparty' subdir
Checkout and build yaws with
pushd 3rdparty svn co https://erlyaws.svn.sourceforge.net/svnroot/erlyaws/trunk/yaws cd yaws ./configure make popd
You don't need to do install as we're using it from local dir
Enter the source at the bottom of the post as an example. I've put this in the project root for simplicity, but you can modify to be in src by modifying the include directive in the source
Compile with erlc -pa 3rdparty/yaws/ebin test.erl
or..use an Emakefile (another topic for a future post)
Start Erlang shell with erl -pa 3rdparty/yaws/ebin
In shell enter test:main()
You should see =INFO REPORT... Yaws: Listening to nnn.nnn.nnn:nnnn under ./www

The source:


-include("3rdparty/yaws/include/yaws.hrl").

main() ->
  Args = [],
  yaws:start_embedded("./www", sconf_list(Args), [gconf(Args)]).

gconf(Args) ->
  Id = "yawstest",
  Debug = true,
  GC = yaws_config:make_default_gconf(Debug, Id),
  yaws:mkdir(GC#gconf.tmpdir),
  yaws:mkdir(GC#gconf.logdir),
  GC#gconf{tmpdir="./temp/yaws",logdir="./temp/logs"}.

sconf_list(Args) ->
  SC = #sconf{port = 8080,
              servername = "ubuntu01",
              listen = {0,0,0,0},
              docroot = "www"
      },
  [SC].

First Post

2008-02-24T09:55:00.000-08:00

I've been absent from the world of blogging for a while now. Seems a good time to start again given my recent efforts in promoting Communitivity, XRI, and XDI. More on those efforts in future posts, but for now I'll just explain the concept of communitivity.

No not the math term. I am using communitivity is a social networking context based on some word-play.

First think of the word 'community'. That is central to social networks, and in many ways central to us as human beings. Look it up in a dictionary..'community' is a noun.

Nope, don't think so. Have you ever read Spider Robinson's Stardance? If not I highly recommend it. The main characters in Stardance treat the word 'gravity' as a verb, not a noun, because for them it is a choice and a choice that transforms their lives.

What would it mean if we as human beings start treating 'Community' as a verb? For one we'd be more aware that our actions impact our connections with others, and that improving or degrading those connections is a conscious choice. I like to think the world would be a better place.

The English language allows adding -ive to the end of a verb to describe tending to or performing the action that verb describes. Applying this we get 'communitive' to describe doing community.

We get then use the rules for the -ity suffix to get communitivity, which we're using to describe how much you're doing community.

The goal of Communitivity.com will be to enable non-profits, socially-conscious businesses, and government organizations to increase and monitor their communitivity. We'll do this through teaching how that has a significant ROI, teaching communitivity best practices, communitivity consulting, and Open Source software written primarily in Erlang.

Stay tuned for more...