I got to review the O'Reilly book about AngularJS, which was nice since I was planning on using it for a recent project. I followed through most of the examples, which went OK-ish. Yes, I'm very impressed with how easy it seems to be to make very dynamic and interactive web apps. But as soon as I tried to apply it to my own project, I had quite some problems getting it working. Especially with concepts like directives or getting functions from multiple modules work together, I couldn't get the required knowledge from the book to get them working.

This book is probably more recommended for somebody who's already familiar with Javascript MVC frameworks, looking to brush up on AngularJS, rather than a Javascript rookie like myself.

Background

• Human Movement Scientist
• Specialized in gait analysis and pressure measurements
• Pythonista, Matlab-survivor and reluctant R-user

Normally pressure measurements are done on humans using a pressure plate. In this case we used a 200x36 cm pressure plate, it has 256x64 sensors which measure at 126 Hz. When you walk over it, each sensor measures the force applied to them and after interpolation, you get these nice looking pressure measurements.

But I needed a version that worked on animals too. The regular software doesn't work on animals, due to several limitations (not detecting the paws being a major one).

So I created Pawlabeling, see Github: https://www.github.com/ivoflipse/pawlabeling

Its called Pawlabeling, because its used to label paws. Who ever said naming hard to be hard?!?

Scientific Python stack

The application uses a lot of libraries from the scientific python stack:

• PySide (GUI)
• NumPy (array operations)
• SciPy (scientific calculations)
• PyTables (storage)
• OpenCV (computer vision)
• Matplotlib (plotting)
• Scikit-Learn (machine-learning)

For installing I recommend Anaconda (Continuum Analytics) or Canopy (Enthought).

Interesting problems

While working on this project I ran into lots of interesting problems and like any junior developer: you go to Stack Overflow! Luckily the Python community is awesome and especially Joe Kington's answers helped me a lot (all his Matplotlib answers too BTW).

• Detection and tracking of paws

• Labeling the paws

• Detecting toes

Focus of the talk

Today, I'll just be focusing on the problem I'm working on right now: Labeling the paws.

Given the data of a paw, predict its label (LF, LH, RF, RH)

So how can we solve this?

Let's use Machine Learning

Gael Varoquax (scikit-learn developer):

Machine Learning is about building programs with tunable parameters that are adjusted automatically so as to improve their behavior by adapting to previously seen data.

Today we'll focus on Supervised Learning:

Supervised learning consists of learning the link between two datasets: the observed data X and an external variable y that we are trying to predict, usually called target or labels. Most often, y is a 1D array of length n_samples.

Like finding a line that separates these black and white points

Or predicting the digit given an small image of a digit, like in the MNIST dataset:

If you have no clue what algorithm to use, you should read the documentation first or just use the Scikit-Learn cheat-sheet:

Source: Andreas Müller

Starting at Start:

• Yes, I have more than 50 samples
• Yes, I'm trying to predict a category
• Yes, I even have lots of labeled data (its called Pawlabeling for a reason d'uh!)
• Darn, I don't have more than 100k samples
• I could use a Linear Support Vector Classifier, but why stop here?
• No, I don't have text data
• Meeh, I don't want to use KNeighbors Classifiers
• So we end up at Ensemble Classifiers (ignoring the Support Vector Classifiers)

From the Ensemble Classifiers, I randomly picked the Random Forest classifier

Random Forests

Why use Random Forests:

• Easy to use
• Used for regression (predicting values)
• Used for classification (predicting categories)
• Gives you good scores on (entry-level) Kaggle competitions :-)

I thought I'd add this Quora question as a note, that explains Random Forests in laymans terms:

Suppose you're very indecisive, so whenever you want to watch a movie, you ask your friend Willow if she thinks you'll like it. In order to answer, Willow first needs to figure out what movies you like, so you give her a bunch of movies and tell her whether you liked each one or not (i.e., you give her a labeled training set). Then, when you ask her if she thinks you'll like movie X or not, she plays a 20 questions-like game with IMDB, asking questions like "Is X a romantic movie?", "Does Johnny Depp star in X?", and so on. She asks more informative questions first (i.e., she maximizes the information gain of each question), and gives you a yes/no answer at the end.

Thus, Willow is a decision tree for your movie preferences.

But Willow is only human, so she doesn't always generalize your preferences very well (i.e., she overfits). In order to get more accurate recommendations, you'd like to ask a bunch of your friends, and watch movie X if most of them say they think you'll like it. That is, instead of asking only Willow, you want to ask Woody, Apple, and Cartman as well, and they vote on whether you'll like a movie (i.e., you build an ensemble classifier, aka a forest in this case).

Now you don't want each of your friends to do the same thing and give you the same answer, so you first give each of them slightly different data. After all, you're not absolutely sure of your preferences yourself -- you told Willow you loved Titanic, but maybe you were just happy that day because it was your birthday, so maybe some of your friends shouldn't use the fact that you liked Titanic in making their recommendations. Or maybe you told her you loved Cinderella, but actually you really really loved it, so some of your friends should give Cinderella more weight. So instead of giving your friends the same data you gave Willow, you give them slightly perturbed versions. You don't change your love/hate decisions, you just say you love/hate some movies a little more or less (you give each of your friends a bootstrapped version of your original training data). For example, whereas you told Willow that you liked Black Swan and Harry Potter and disliked Avatar, you tell Woody that you liked Black Swan so much you watched it twice, you disliked Avatar, and don't mention Harry Potter at all.

By using this ensemble, you hope that while each of your friends gives somewhat idiosyncratic recommendations (Willow thinks you like vampire movies more than you do, Woody thinks you like Pixar movies, and Cartman thinks you just hate everything), the errors get canceled out in the majority. Thus, your friends now form a bagged (bootstrap aggregated) forest of your movie preferences.

There's still one problem with your data, however. While you loved both Titanic and Inception, it wasn't because you like movies that star Leonardio DiCaprio. Maybe you liked both movies for other reasons. Thus, you don't want your friends to all base their recommendations on whether Leo is in a movie or not. So when each friend asks IMDB a question, only a random subset of the possible questions is allowed (i.e., when you're building a decision tree, at each node you use some randomness in selecting the attribute to split on, say by randomly selecting an attribute or by selecting an attribute from a random subset). This means your friends aren't allowed to ask whether Leonardo DiCaprio is in the movie whenever they want. So whereas previously you injected randomness at the data level, by perturbing your movie preferences slightly, now you're injecting randomness at the model level, by making your friends ask different questions at different times.

And so your friends now form a random forest.

A Forest of Decision Trees

Random Forests combines a 'forest' of decision trees each trained on a random subset of the training data

So what's a decision tree, we'll here's an example:

Then you train a whole 'forest' of these decision trees on random subsets of your data so they don't all learn the same features and they don't overfit

Source: Citizennet.com

Random Forests in Action

Most machine learning algorithms implemented in scikit-learn expect a numpy array as input X. The expected shape of X is (nsamples, nfeatures)

Supervised Learning also needs as input an array Y, the labels of the data

We'll also split the data in a training and testing set.

Never ever ever ever learn on test data

The classifiers learn the patterns in your data and if you test how well the classifier works on the data you used to train it, off course its going to work well. But then you present it with new data and suddenly your performance can go down the drain. Even better would be to make split your training data even further (check out cross-validation) and take the model that performs best on all the different splits of your data. Seriously, go read the docs on this.

Source: Scikit-Learn Tutorial

Here are Dropbox links to the two files and the notebook this was created in:

• IPython Notebook
• classify X_train.csv
• classify X_test.csv

``` import pandas as pd

# Let's load the data from some CSV files I prepared
# The training data contains the data of (only) 4 dogs
X_train = pd.read_csv("classify X_train.csv", index_col=0)
# The testing data contains the data of the remaining 29 dogs
X_test = pd.read_csv("classify X_test.csv", index_col=0)

# My Y's are simply a column called "label" inside my X's

n_samples, n_features = X_train.shape
print("Training Samples: {:<4}, Features: {}".format(n_samples, n_features))

n_samples, n_features = X_test.shape
print("Testing  Samples: {:<4}, Features: {}".format(n_samples, n_features))

Training Samples: 486 , Features: 32
Testing  Samples: 3236, Features: 32

```

Normally you can take 50-50 splits or 80-20, especially if you want to do Cross Validation, but this seemed to work just fine.

As you can see, there are 32 features which basically describe the following:

• max_force = the peak of the force that paw applied to the ground. The ratio between front and hind paws seems to be about 60:40 so this should give it a way to estimate whether its a front or hind paw
• max_surface = the peak of the contact surface area of the paw. Again, the front paws are almost always a bit larger, probably because they also have to bear more force and are compressed more, so again a nice feature for separating front and hind paws.
• max_duration = the number of frames the paw was in contact with the ground. This isn't particularly useful, but it might put the difference in frames between paws in perspective

Then for each paw, we look backwards and forwards 2 paws and calculate the same features (f, s, m) but we also add the distance in x (width), y (length) and z (frames). This basically tells you where the other paw was located relative to the current one. Given that the pattern is highly repeatable, I have high hopes this will work well.

For the first two and last two paws, there may not be 1 or 2 paws in front or behind it, so I'll fill those numbers with NaN's for now.

cols = ['max_force', 'max_surface', 'max_duration', 
        '-f2', '-s2', '-m2', '-x2', '-y2', '-z2', 
        '-f1', '-s1', '-m1', '-x1', '-y1', '-z1', 
         'f1',  's1',  'm1',  'x1',  'y1',  'z1', 
         'f2',  's2',  'm2',  'x2',  'y2',  'z2']

The features

So for the yellow encircled paw, we calculate the x and y distances. The white line connecting the paws shows the chronological order.

Here I added a plot of the total force (for the whole paw) for each frame. Red are the right front paws, green are the left front paws. For now, we'll just use the peak value.

from sklearn.ensemble import RandomForestClassifier

# So we create a class instance, without any hyperparameters, we'll get to that later
rf = RandomForestClassifier()
# We fit a model on the training data, using only the feature columns and we drop any columns that contain NaN's
# Because of I have to drop the NaN's I have to make sure I also drop them from my labels column, so it looks a bit clunky
rf.fit(X_train[cols].dropna(), 
       X_train[cols+["label"]].dropna()["label"])  

score = rf.score(X_test[cols].dropna(), 
                 X_test[cols+["label"]].dropna()["label"])
print "Score: {:.2f}".format(score)

Score: 0.93

Ways to improve performance

We scored 93%! Which is most likely better than the interns that labeled the data for me! Apparently there are such strong patterns in the data that it already works really well on such a small data set.

Since the Random Forest is indeed random, the score I get above here seems to vary slightly each time I run it. There are probably flags to prevent this behavior, but its more fun this way :-)

Since my example data has a lot of samples (~800, so about 25%) that contain NaN, so we'll replace them with the mean using Imputation. You could get more fancy and use regression to predict what values I should replace them with, but we'll keep it simple. Actually, the way I'm imputing the values is wrong, because I'm replacing it with the mean along all the columns, even though there are huge differences based on the size of the dog. A better way would be to group the dataframe by dog and then impute the values, but again: let's keep it simple.

We also ran the Random Forest without hyperparameters, so we'll use better settings. I got these using a Grid Search that checked 216 different combinations of hyperparameters on a slightly bigger portion of the data. Grid Search can also perform Cross-Validation, so I'm relatively confident that these hyperparameters will give a better result

%matplotlib inline
from sklearn.preprocessing import Imputer
from sklearn.metrics import confusion_matrix

# We use the Imputer class to replace NaN's with the mean, calculated along the columns
imputer = Imputer(missing_values="NaN",
                  strategy="mean", axis=0)

X_train2 = X_train.copy()
X_train2[cols] = imputer.fit_transform(X_train[cols])
X_test2 = X_test.copy()
X_test2[cols] = imputer.fit_transform(X_test[cols])

# Notice that there are now a bunch of hyperparameters in here
rf = RandomForestClassifier(bootstrap=False, min_samples_leaf=3, 
                            min_samples_split=3, criterion='gini', 
                            max_features=10, max_depth=None)

# We fit and score again and get a new (high) score
rf.fit(X_train2[cols], X_train2["label"])
score = rf.score(X_test2[cols], X_test2["label"])
print("New score: {:.2f}".format(score))

# Compute confusion matrix
y_pred = rf.predict(X_test2[cols])
cm = confusion_matrix(X_test2["label"], y_pred)
print(cm)

import matplotlib.pyplot as plt
fig, ax = plt.subplots(figsize=(10,10))
cax = ax.matshow(cm)
fig.suptitle("Confusion matrix")
fig.colorbar(cax)
ax.set_ylabel('True label')
ax.set_xlabel('Predicted label')
ax.set_xticks(range(4))
ax.set_xticklabels(["LF","LH","RF","RH"])
ax.set_yticks(range(4))
_ = ax.set_yticklabels(["LF","LH","RF","RH"])

New score: 0.95
[[812   1  36   0]
 [  5 710   3  17]
 [ 28   3 845   0]
 [  3  61  16 696]]

Evaluating the results

This time we score slightly higher: 95%, but on the whole 100% of the dataset. It seems it systematically get's certainly samples wrong, so tuning the parameters and imputating values won't really help here.

By looking at the confusion matrix, we see on the diagonal that it did a kick ass job of predicting most right. Then interestingly enough, there are some superdiagonal which also have 30-60 errors. It turns out, sometimes when you give it a Right Front (RF) sample, it gets labeled as a Left Front (LF) sample or vice versa. The same thing happens for the hind paws. So clearly, it would help to have better features that distinguish between left and right.

I'm also curious how well it performs on each separate dog in our testing set, perhaps there are some outliers.

import numpy as np

subject_scores = []
min_score = 1.0
min_measurement = ""
min_subject = ""
group = None
print("Subject id\tWeight\tSamples\tMean +/- STD")
for subject_id, subject_group in X_test2.groupby("subject_id"):
    scores = []
    weight = subject_group["weight"].unique()[0]
    samples = len(subject_group)
    for measurement_id, measurement_group in subject_group.groupby("measurement_id"):
        score = rf.score(measurement_group[cols], measurement_group["label"])
        scores.append(score)
        if score < min_score:
            min_score = score
            min_measurement = measurement_id
            min_subject = subject_id
            group = measurement_group
    arrow = ""
    if np.mean(scores) < 0.9:
        arrow = "<-"
    print("{:<10}\t{:<2}\t{:<3}\t{:.2f} +/- {:.2f} {}".format(subject_id, weight, samples, np.mean(scores), np.std(scores), arrow))
    subject_scores.append(np.mean(scores))

print("Worst score: {:.2f} for {} in {}".format(min_score, min_subject, measurement_id[:5]))

Subject id  Weight  Samples Mean +/- STD
subject_0   30  86  0.96 +/- 0.07 
subject_10  9   178 1.00 +/- 0.00 
subject_12  50  83  0.91 +/- 0.14 
subject_13  51  74  1.00 +/- 0.00 
subject_14  21  101 0.97 +/- 0.11 
subject_15  5   193 0.95 +/- 0.04 
subject_16  16  123 1.00 +/- 0.00 
subject_17  31  84  0.99 +/- 0.05 
subject_18  9   199 0.98 +/- 0.03 
subject_19  26  111 0.95 +/- 0.09 
subject_2   59  61  1.00 +/- 0.00 
subject_20  64  57  1.00 +/- 0.00 
subject_21  22  82  0.98 +/- 0.05 
subject_22  17  103 0.98 +/- 0.04 
subject_23  25  89  0.96 +/- 0.10 
subject_24  4   229 0.94 +/- 0.04 
subject_25  41  97  0.98 +/- 0.04 
subject_26  4   178 0.79 +/- 0.13 <-
subject_27  6   154 0.84 +/- 0.13 <-
subject_28  2   141 0.80 +/- 0.11 <-
subject_29  69  59  0.87 +/- 0.24 <-
subject_35  5   189 0.97 +/- 0.04 
subject_36  43  80  0.95 +/- 0.08 
subject_4   7   158 0.98 +/- 0.03 
subject_5   57  64  0.92 +/- 0.17 
subject_6   54  50  1.00 +/- 0.00 
subject_7   37  69  1.00 +/- 0.00 
subject_8   62  52  0.98 +/- 0.07 
subject_9   34  92  1.00 +/- 0.00 
Worst score: 0.29 for subject_29 in svr_3

For each dog, I printed out their weight and their mean/std score. This was calculated on each measurement, perhaps there are just some measurements where it performs poorly.

If the score dropped below 90% I added a small <- behind it and as we can see there are at least 4 dogs where it performs more poorly. However, in three of these dogs it has to classify between 141-178 samples, so getting at least 80% right of those is still pretty impressive. On our biggest (or at least heaviest) dog it got 87% right, but out of only 59 measurements. This dog is probably too big for the plate, so you get much less samples out of each measurements. Also, a lot of his steps are on the edge of the plate, which I didn't try to predict in this analysis.

I also looked up the worst measurement, but again randomness made sure that it didn't align with the example I cooked up below. Because it has some incomplete steps, it was a bit of a hassle to show that one, so instead we'll show one of his other measurements instead:

import tables
import sys
sys.path.append("C:\Dropbox\Development\AnalyzingDogs")
import data_mangling

table = tables.open_file("C:\Dropbox\Development\Experiments\data.h5", "a")
contacts, contact_dict, measurements = data_mangling.get_contacts(table, subject_id="subject_29")

measurement_id = "ser_2 - 19-4-2010 - Entire Plate Roll Off"
measurement = measurements[measurement_id]
contacts = contact_dict[measurement_id]
group = X_test2[(X_test2["subject_id"]=="subject_29").values & (X_test2["measurement_id"]==measurement_id).values]
prediction = rf.predict(group[cols])


fig, ax = plt.subplots(figsize=(16, 6))
valid_index = 0
x = []
y = []
for index, contact in enumerate(contacts):       
    minx = contact["min_x"]
    maxx = contact["max_x"]
    miny = contact["min_y"]
    maxy = contact["max_y"]
    x.append(np.mean([minx, maxx]))
    y.append(np.mean([miny, maxy]))
    ax.plot([minx, minx, maxx, maxx, minx], 
            [miny, maxy, maxy, miny, miny],
            color=["#11F05F","#008000", "#FF0000","#800000"][prediction[index]],
            linewidth=5)
    ax.plot([minx-1, minx-1, maxx+1, maxx+1, minx-1], 
            [miny-1, maxy+1, maxy+1, miny-1, miny-1], 
            color=["#11F05F","#008000", "#FF0000","#800000"][contact["contact_label"]],
            linewidth=5)

ax.plot(x, y, linewidth=3, color="w")
ax.imshow(measurement["data"].max(axis=2).T);
ax.set_xticks([])
_ = ax.set_yticks([])

The white line again connects the paws in chronological order. The color on the inside it was the algorithm predicted, the color on the outside is what it should have been. It basically just tries LH, RF everywhere, so by chance it got that right at least once.

The problem seems to be that the dog walked diagonally, which causes left and right to cross over. His previous and next right front paw don't even hit the plate, because his strides are that long, so its not hard to imagine why the classifier got it wrong. Obviously, adversarial samples like this can give hints to what features I should incorporate to make it more robust.

Visualize the Random Forest

Finally, we can export the tree and visualize its structure

import StringIO
import pydot
from sklearn import tree
from IPython.core.display import Image

# Visualize one of the trees
dot_data = StringIO.StringIO()
tree.export_graphviz(rf.estimators_[0], out_file=dot_data)
graph = pydot.graph_from_dot_data(dot_data.getvalue())
image = graph.write_png("./images/random_network.png")

The questions it asks are basically:

• Is the second next paw less than -3.75 in front of me?
  • Left: Then is the next paw less than 28.75 to the right of me?
    • Left: Is that second next paw less than 1.5 to the right of me? (You're probably a Right Front)
    • Right: Or is the previous paw less than 10 to the right of me? (You're probably a Right Hind)
  • Right: Or is the next paw less than 3.75 ahead of me?
    • Left: Is the previous paw less than 0.43 in front of me? (This branch is the deepest, mostly right paws)
    • Right: Is the next paw less than 26.75 to the right of me?
      • Left: You're a Left Front
      • Right: If the previous paw is less than 2.5 ahead of me, you're Left Hind

If I had to write some kind of state machine to decypher it, I'm not sure if this would be what I would come up with

# Here's the index (used in the image of the tree), with the feature name and importance
for i,j,k in sorted(zip(range(len(cols)), cols, rf.feature_importances_), key=lambda x: x[2], reverse=True):
    print "{:<2} {:<12} {:.4f}".format(i,j,k)

12 -x1          0.2180
19 y1           0.2179
25 y2           0.1708
18 x1           0.1153
13 -y1          0.1084
7  -y2          0.1035
24 x2           0.0188
6  -x2          0.0151
10 -s1          0.0061
1  max_surface  0.0051
11 -m1          0.0027
20 z1           0.0024
22 s2           0.0024
14 -z1          0.0022
16 s1           0.0020
2  max_duration 0.0017
5  -m2          0.0017
3  -f2          0.0015
15 f1           0.0010
26 z2           0.0008
0  max_force    0.0008
23 m2           0.0007
8  -z2          0.0005
9  -f1          0.0004
21 f2           0.0002
17 m1           0.0000
4  -s2          0.0000

Take home message

Machine Learning isn't going to bite you. While there are plenty of mistakes you can make and things that can go wrong, as I've hopefully shown you can get a really good performing classifier with relative ease.

Check out the Scikit-Learn docs and just try it at home!

Questions?

I recently decided to try and create a web app and picked Tornado as my web server, because it is also being used in IPython. I like learning new tools by reading books about them, so I got my hands on a copy of Introduction to Tornado and got started.

The book is pretty thin, which I think in this case is a good thing. Its not meant to exhaustively describe all features Tornado has to offer, but rather a gentle introduction. The book covers all of the important elements to get you started:

• Creating templates
• Extending templates with Javascript and CSS
• Interacting with databases (MongoDB in this case)
• Making your web app asynchronously
• Basic security features and authentication
• Signing in with Twitter and Facebook's OAuth

The book features several nice examples, like a shopping cart for a bookstore, asynchronously keeping tabs on how many items are remaining. A simple Twitter client displaying your latest tweets, a Facebook client showing your timeline, both dealing with authentication. Most examples worked pretty well, though I had some issues getting the Twitter client working, because of errors I made in the callback url on localhost. I didn't get the Facebook example working for the same reason, but its not a big issue.

Overall, I found it a pretty useful book. While I was already somewhat familiar with Web Apps through Udacity and Coursera courses, it was good to get a bit more formal explanation about topics like routing, handlers and templates. I also liked the way they explained what each part of the code did, instead of assuming you had already figured it out. So while its a short read, I think its a nice introduction to Tornado to get you going.

Some of the things I’d like to fix are:

• Refactoring my early code to be more up to date based on my current experience. This applies to much of the main panel and how the GUI is controlled by PubSub.
• Make the code more self-contained. Even though I tried to apply MVC by separating the GUI from the calculations and the database, when I started out a lot of database functions would pass along a message back to the GUI. You can imagine that later on when you want to reuse the same database functions, the GUI got called as well creating all sorts of nasty side effects.
• Because I didn’t fully understand how to use debugging, I cheated by simply adding print statements to each function. Obviously that doesn’t scale (though it helped me understand my workflow tremendously), so instead I’d like critical functions to send a PubSub message to which I can subscribe. Then I can simply add a Settings option to print these messages from one central location or simply unsubscribe from them.
• Currently when I change the database, I manually change my own MySQL database and then figure out how to replicate it. Up until now I’d advise my one user not to invest large amount of time in annotating contacts, because she’d risk having all that effort go to waste when I decided to update her table. In the future I’d like to make this more easier for the user, especially the non-technical ones, by only dropping tables if its trivial to compute the results again or update the database (rather than dropping it) when it’s an important table.
• Interface several database action/settings and more general settings, like making a backup of the database, tweak the ratios of the color map or change the default location where the measurement data can be found.
• Add exceptions or make sure the code doesn’t have any expectations about the data. A good example was that my results first required each subject to have 4 paws, else the code wouldn’t run. But some dogs or cats are amputees and humans obviously only have 2 feet, so I did my best to remove any of these expectations. However, I’m sure there are still some of these assumptions hidden in the code waiting to come across a trial that doesn’t meet them.
• Document more! This is one major area where I’ve slacked off, telling myself that it’s just a waste of time and didn’t feel like wasting it. However, after spending several days trying to hunt down obscure bugs, because I didn’t fully remember what function triggered what other functions, I’ve definitely changed my mind. Another great advantage I’ve found was that when I write out what I want the code to do, I spot errors in my thinking much faster and get a much better grasp of what I actually want the code to achieve. On top of all this, I want to look into a library that takes all my doc strings and uses that to create proper documentation.

Furthermore, I’d like to keep adding features that either improve the usability or that make the app more useful. Examples of this are:

• Allow the user to edit objects, rather than requiring them to delete it and create a new one. This applies to simple things adding measurements after a session has been saved, tweaking a single zone location or even being able to redo it without having to recalculate all the results.
• Allow the user to drag the zone’s square to the right position rather than requiring to use the directional keys on the keyboard. Not everyone prefers using a keyboard, so they shouldn’t be forced to use it.
• Make it easier to annotate the contacts when the keyboard lacks a numpad. While the numpad is definitely the fastest way to annotate them, laptops generally lack one and pressing Fn while trying to find the keys isn’t as easy. I used to have a version where you could click on the average contacts, but the event required to do so (EVTCHILDFOCUS) had the nasty habit of not being very reliable and getting called more often. One ‘shortcut’ would be to make a button and display the imshow() image as the face of the button.
• Create a function to manually override the paw detection in a given slice of the entire plate, so if the automagic detection fails enabled the user to try and fix it.
• Allow different shapes, sizes and number of zones. This should make the application more flexible for measuring other kinds of data, such as humans, horses or elephants (you have to think big!).

There are also features I’d like to add that allow for more data exploration and analysis:

• Currently I don’t remove any outliers (other than ignoring incomplete contacts or those that didn’t get recognized properly), so it would be helpful if before calculating all the definitive results you could clean up the results. For example by displaying histograms with the distribution of certain variables or plotting all contacts in one graph. By listing all contacts in a list and allow the user to delete them if necessary they can perform any required data cleaning.
• At the moment you can only analyze one protocol at a time (while the graphs do allow you to plot both at the same time), but obviously comparing them would be very interesting. As I’ve experienced in the past, just displaying two graphs next to each other is not comparing. It takes a lot of experience to interpret the differences especially without a clear frame of reference.
• Another thing that I’d like to analyze are differences on a population level. Even though I already experimented with this, I wasn’t happy with the end result. Simply displaying multiple graphs over one another wasn’t really useful and since the dogs were subdivided into weight groups there was a lot of variety in numbers. Another issue was averaging dogs with a ‘normal’ and amble gait pattern. As you can see in this figure, the step lengths between the left front and right hind paw (bottom right) were either negative (right hind being behind the left front paw) or positive. However, when you calculate an average, you get a value around -10 which doesn’t really describe either pattern. Clearly there’s a need for some more diligence when segmenting the data.
• I don’t only want to display the population data alone, but rather compare any dog with the ‘normal’ values based on the medical history of the dog. That way its results are much easier to quantify, because you get a sense of what they should be, so spotting abnormalities should be a lot easier.

Lastly there are several things that I’d like to work on in the future, but that will me require me to learn a lot more. First on the bucket list is: - Reading more books, first of all finishing Code Complete. I’ve got plenty of interesting books, I just haven’t had the time the past few months to try and read them. On top of that, in most cases its best to have a small pet project to try out all the news things you’re trying to learn. This obviously doesn’t go so well if you’re under any time constraints. - Experiment with OpenGL, because though I have an animation working, that library doesn’t work with wxPython. So I’d love to learn some OpenGL so I can learn how to draw to a Canvas myself, without the need of a library to do the heavy lifting. - I’d love to mess around with Microsoft’s Kinect. Not only because it’s a fun gadget, but also because the kinematic data could be a great asset to the gait analysis. First off, it allows me to measure the angles and make estimations about the moments around the joints. Second, it tells me which paw is where at what point in time, so a synchronized Kinect + pressure measurement would make it much easier to automate the paw detection. But obviously Kinect requires some OpenGL knowledge (for performant displaying), OpenCV knowledge for interpreting the image data and brushing up on my old courses on Inverse Kinematics... So I’m still a long way off ever getting this to work. - Try out if sorting results is any better when using MongoDB. Now I know that NoSQL isn’t the solution to all my database problems, but the problem I have with MySQL is how it requires me to break my data into pieces and stitch them back together when I need them again. I’d much rather leave things as they are and skip tedious parsing loops every time I need a different result. Furthermore, requiring a schema is an absolute pain in the ass when your design isn’t set in stone. I’ve spend nearly as much time writing code to ‘build’ a database as I needed to put things in the database. Off course, I don’t intend to break something that’s already working, so I’m probably going to work on a small pet project to see if I like this any better. - Given that I’m writing scientific software I need to be pretty darn sure whether the results are correct. I’d love to build in a dummy data set that can function as a ‘Mock’ object and allow me to check whether the results are correct. This should allow me to catch errors with my calculations that aren’t detectable with the human eye. Especially when you have highly dimensional data, errors easily slip in without a good way to spot them early on. - Above all else, I desperately want to be able to manage measuring myself! Currently I first need to do measurements in the vendor’s software, without any way to tell if the measurement went ok, then export it from their software and import it into mine. You can understand that most clinicians would find this process far too laborious and decide not to use my app. On top of that, the new drivers allow for continuous measuring in contrast to the 2 second limit of the current software version. This would allow me to greatly increase the pace at which the measurements can be performed and analyzed. However, I first need to convince the vendor to give me this access…

As you can see there’s more than enough work for me in store! On top of all this I have to await what the clinic thinks of my first version and how useful they find it. But I’ll be sure to keep you guys up to date on any progress I’m making. If you have any questions on how or why I did something a certain way, be sure to drop a comment!

The average pressure over time with a standard deviation (N/cm^2). This is basically the sum of all the sensors divided by the surface of all the activated sensors.

Note that the surface is probably an overestimation, because even when a sensor is partially loaded, it will still count the entire surface. There are some ways to counter this, but probably a better way would be to ignore the really low values (<0.1 N/sensor). Interestingly enough the values remain pretty constant once they reach their maximum.

The average force over time (N), which is simply the sum of all the activated sensors.

In humans we generally see an M-shaped curve (purple line), which is due to the ‘rockers’ in the human foot. The first peak is caused by loading the the rear foot (white line), the second peak is caused by shifting the weight towards the forefoot (green line) in order to push off. In the dogs case its just one peak, this most likely has to do with the way quadrupeds walk.

The average surface over time (cm^2). The surface is almost the reversed of the pressure, the dogs tend to put down their paws very flatly (all toes making contact) and don’t take start taking them off until late (> 60%) in the stance phase. The lift off also seems to happen pretty evenly in most cases, where first the central toe, then the medial and lateral toes and lastly the two front toes are lifted off.

The average force (not pressure!) for each ‘toe’ or 2×2 zone (N/cm^2).

Blue is the central/rear toe, then from medial to lateral: green, red, light blue and purple. Because the surface is the same, you can easily compare the forces. The vertical lines depict when the central toe reaches its maximal pressure and when its lifted off. This is analogous to the phases of gait as described by Willems et al (2004, pdf link), which describe very reliable phases during the roll off of a human foot. Starting at the landing of the heel to the first contact of a metatarsal to the contact of all metatarsals to the lifting of the heel and eventually the foot.

As you can see there are a lot of similarities between the left and right paws and also within the paw the force values within each toe are very similar.

The center of pressure plotted on an image of the maximal values of each sensor.

It seems every paw first lands more on the lateral side, only to stabilize somewhere in the middle. In most cases the line is so straight which indicates there’s a very good balance between the medial and lateral side of the paw. Imagine the ankle as a weight scale, where the left and right side are in a constant battle to keep the scale in balance. In order to keep them in balance it requires the muscles on both sides of the paw need to contract with the right amount of force. I expect lame dogs, that may be lacking this balance, will show a completely different center of pressure.

The location of the toes (manually set by me).

The size is fixed at the moment, though technically I could reduce it to 1×1 or scale it up for larger animals. Note that the location in the images may be slightly off, because the interpolation I use (scipy’s map_coordinates) slightly translates the paw, which frustrated me to no end. I haven’t found a solid solution for this sadly.

The paw axis, from the central toe to a point between the two central toes.

In humans I believe the angle is between 2 and 12 degrees, where a positive angle means the paw is exorotated. Now my definition isn’t perfect, especially because I found some large variations in the shape and loading pattern of the central toe. However, I think it will help find extreme outliers in either direction.

The step length, width, duration for each paw compared to the other paws with an image of the relative positions of each paw

I’d love to figure out how to change the image with the relative positions to an animation that shows how the paws are positioned to each other over time as well. Especially for the running trials, it looks like the paws land very close to each other and you can’t really imagine what this means in practice. Still the current image does help visualize how large the steps and strides are, if the dog were lame on one paw it would probably have an asymmetrical step length and easily stand out.

And lastly a dashboard with some useful stats.

For Pressure/Force/Surface I calculated the maximal value, the percentage of the stance phase where the maximum was found and the ratio between left and right. I first used an Asymmetry Index (ASI), but I found the values much harder to interpret and probably only make sense if you can compare them between populations. For each of the forces per zone I calculated the same, with the exception that the percentages at the end aren’t left vs right, but the ratio between the toes (which seemed much more useful).

At the bottom you find the axis in degrees, where exorotation is positive. The ‘Timing’ next to it gives the step length, width and step duration just comparing the left vs right paws (step) or the paw with itself (stride).

I actually already made changes to the version you see above, because I initially had the id’s of the two protocols (running and walking) hardcoded in my calculations so I was sure everything would work. But it turned out to be fairly trivial to made it generic enough to allow any protocol id. The list of measurements on the side is therefore replaced with a list of protocols, so you can easily switch between them.

In my last post I’ll talk about the future changes I’d love to make, so read on!

Now I bet you’re curious what this all looks like! Well I won’t keep you waiting any longer.

The Ribbon

I’m a huge fan of Microsoft’s Ribbon and luckily wxPython had its own version. So I was quick to add one myself, because it allows me to use tabs to switch between logical sections of my app and gives me large icons which make for easier clicking (due to Fitts’Law). While the current 48px are probably a bit overkill I’m still fairly happy with them. The only thing that bothers me is all the stock icons and the depressing amount of duplication. Worse, because I compressed so many functionality into one screen I’m also stuck with a huge amount of buttons.

Basically it reminds me of this:

_{Image from Stuffthathappens.com}

Now this isn’t a fair comparison because I wish you good luck trying to insert subject info or manage a database with just one button, but that’s not to say there isn’t room for improvement.

As you can see, the main tab consists of 4 elements: - Searching the database - Adding subjects to the database - Creating a medical history (anamnesis) based on tags (work in progress) - Creating a session and adding measurements to the session

I think I’m going to reorganize the panel so when you start you basically get a Google like interface: search for a subject and you can get started. If you need to analyze new data then you press a button to add a subject and the other relevant buttons appear (making the ribbon context sensitive like Office) and display the panel to insert the data. Finished inserting a subject? Great we switch to the panel to add measurements. Want to add a more detailed medical history, switch the panels to display just those panels and adjust the ribbon.

Another reason for wanting to change the main tab is that this was literally the first code I wrote, which means it’s horrible. Even though I tried to maintain some sense of a MVC structure, but due to my limited experience I failed pretty badly and a lot of functions need to be untangled.

Processing the measurements

Now on to the more interesting stuff: processing the data.

Again the Ribbon is crazy crowded at the moment; this is because there are so many actions required to allow for a flawless and usable paw annotation. Imagine this:

• Search for contacts
• Refresh loads the average contacts from the database
• Remove any contacts that are ‘incomplete’
• Save the results when you’re done
• Delete contact removes it from the list
• Previous/Next Contact let you switch between contacts
• Undo it if you make a mistake
• Delete all the contacts in case they are saved the wrong way
• Marking the four contacts
• Add a protocol so we can discern between measurements
• My magic eight ball to parse measurement names into protocols
• Cancel protocol set’s all the choices back to default in case you make an error
• Delete all the protocols in case you made a mistake

Then there’s a couple of buttons which could be made context sensitive, because they aren’t needed until you need to assign the zone locations.

• Find zones button was supposed to mark any zones it could find
• Add a zone (moving is done with the keyboard arrows)
• Save the zone locations in the database
• Undo all the zones in case you made a mistake (before assigning)
• Delete the zone

As you can see there’s a somewhat recurring pattern: create -> store -> delete

Perhaps I could ‘streamline’ this process by making the program assume what the user wants to do after certain actions, but honestly I think that’s far too error prone and we’re talking about science here. Shuttles have exploded for errors like this and we don’t want to fit a pair of orthotics based on erroneous data.

Besides, what computer geek honestly uses buttons anyway? I already added several keyboard accelerators to make very easy to be more productive. Ctrl+F to search, CTRL+O to remove incomplete contacts, 7 (Left Front), 9 (Right Front), 1 (Left Hind), 3 (Right Hind) which map to the anatomical order of the paws :

Ctrl+S to save all the annotations to the database and rinse and repeat. In case there’s a contact you don’t want to have stored, simply don’t annotate it and it won’t get saved. When you save a measurement, it will automagically load the next measurement, to reduce any additional redundant key presses.

How can I improve my paw detection?

Off course, I couldn’t have made this feature without the help of Joe Kington, my Stack Overflow hero, who gave this awesome answer that helped me find and sort my paws. While I personally find the second answer more impressive, I didn’t end up using his principal component analysis, because in its current form it doesn’t perform much better than chance. However, based on the results I’ve gathered so far, I should be able to come up with additional heuristics to make the algorithm perform better.

This also shows why it’s so great to have an application wrapped around all the scripts I had in the beginning, I can extend existing functionality without having to redo a lot of work, because a lot of the foundation required is already in place. For example, the GUI allows me to add multiple panels, which let me easily switch between different views of the same data. If I didn’t have a GUI, I would have had to create several figures or switch between them with a command line interface. You can imagine that performing such tasks are very error prone and quite tedious if you constantly have to pass slightly different arguments to display the data you want.

Back to Joe’s code: his find_paws function while dead simple totally did the job:

def find_paws(data, smooth_radius=5, threshold=0.0001):
    data = sp.ndimage.uniform_filter(data, smooth_radius)
    thresh = data > threshold
    filled = sp.ndimage.morphology.binary_fill_holes(thresh)
    coded_paws, num_paws = sp.ndimage.label(filled)
    data_slices = sp.ndimage.find_objects(coded_paws)
    return object_slices

Now it’s not flawless, as you can see with the contact in the middle, it’s larger than the smooth radius used by the uniform filter. The result is that it recognizes them as separate contacts. The problem gets worse with human feet when any midfoot contact is lacking, because then it will split up the foot into a rear foot and a forefoot.

While I could make the smoothing radius scale with the size of the dog (or the weight for instance) this is a slippery slope. When the dog is running fast, the front and rear paws land on nearly the same spot almost at the same time. If the smoothing radius is too large, it might ‘merge’ these two prints. As with most algorithms, when you try to optimize for certain cases it’s bound to perform worse in other cases. The ‘easy’ way out is to make it as easy as possible to manually correct the algorithm and using the adjustments as feedback for future corrections.

When you search for the contacts, every contact gets highlighted by a white square. As soon as a contact is annotated with appropriate paw, its color coded accordingly. On the left we see a list of all the found contacts, which shows the duration of the contact (in frames), maximal force (in Newtons) and the maximal size of the surface (in cm). Originally I thought this list might be useful so you can see the differences between contacts, but I found I never looked at the list.

Instead, I’d use the comparing tool at the bottom. What it does is compare the currently selected contact (yellow square) with average representations of the already annotated contacts. It also predicts what contact it probably is based on a simple subtraction: subtract the selected contact pixel by pixel from the other contacts, the one with the smallest difference is likely to match it most likely. I’m sure there’s a better comparison, like using PCA, compare all the frames, not just the maximal values or comparing multiple values, but honestly I found that after annotating 2/3 contacts the comparison would do a pretty good job and otherwise even I had a hard to ‘guesstimate’ which contact is resembled most.

Something that might be of use, like someone suggested on Stack Overflow, is using Inverse Kinematics because if two paws are in contact with the ground, the next contact can’t be one of those two. This should greatly reduce the amount of options at any given point in time. Furthermore, in a lot of cases there’s a clear pattern in which the paws are placed, i.e. Right Front, Left Hind, Left Front, Right Hind etc. One might even wonder if the measurements where this pattern doesn’t occur was even a valid one. Obviously, this doesn’t hold in all cases, because some dogs inherently walk seemingly random, alternating between a normal and an amble pattern.

Another important feature is the adding of protocols, which is basically a way to tag measurements. For instance when I try to fit someone a pair of running shoes or orthotics, I’d generally measure them both barefooted and shod, walking and running and wearing several shoe models or variations of an orthotic. This means I can’t calculate an average over all these different ‘protocols’, because they have very different results. My current version can only sort measurements based on one protocol and while technically I could add a slew of protocols like ‘Barefoot Walking’ to create what I want, but you can understand the list can become very long if I’d need to make combinations for every type or brand of shoe…

Peak detection in a 2D array

After processing all the measurements, the results get calculated. One of the results is calculating an average contact for each paw with the same protocol. This has several advantages, because loading all 3D arrays for each contact with the same protocol is quite data intensive. Furthermore, I’m mostly interested in average results not in single contacts, so I might as well compute the average up front. Now I understand this is an enormous data reduction, but as you’ll see next it comes with a similarly enormous advantage: I only have to set the toe’s positions (zones) once per paw!

You can image that with 4 paws and 5 zones and 2 protocols, it would become very laborious if I’d have to set the zones for every contact with the same protocol. Now I only need to set 40 zones, whereas else it would have probably been at least 200 per dog(!).

Now there might be a lot of improvements I could make with regards to the averaging. Currently I’m not making any corrections, while slight paw rotations and shape differences effect the average. Rotations mean the toes aren’t aligned and it might increase the average’s values between the toes and lower those of the toes. The shape differences, for instance when the rear toe wasn’t fully loaded, would result in a ‘shorter’ contact. Since the arrays start behind the contact, this means the toes would end up half way through the average contact.

Again though, it would probably be a better decision to ignore any extreme outliers, unless the variation is more the rule than the exception. One possible solution would be to rotate and translate the contacts to have the optimal overlap. An interesting method I’d love to use is this form of shape matching, where any shape is transformed to a time series. I can then figure out how to transform the contact so they get a better overlap, which in this case would be the smallest Euclidean distance. Obviously thinking up ways to do so is easier said than done, but these calculations can be used for other purposes down the line as well, like tracking motion.

Because the story was getting so long, I’ve cut it in two pieces. So read on for part two where I talk about the results!

Starting with the database

I started out with a wxPyton example that allowed me to fill in basic database info, like the subjects name + address.

Then I tried inserting this into MySQL when you pressed the Save button, which required me to learn how to retrieve values from text controls and trigger functions when pressing buttons.

Adding a Ribbon

Next I added a Ribbon, because I just needed to have one. As you’ll see later on, my application has clearly separated parts, therefore it makes no sense for those parts to share buttons. The Ribbon takes care of that.

Moreover, I learned how to add other segments to a Panel (and why a Panel != a Frame) and I got to mess with Sizers to divide everything on a Panel.

Adding a search box

Now that I could add subjects to my database, I needed a way to search through them. So I added a Search control and a list to display them.

I learned a new thing! If you don’t add unique constraints, you’ll get a load of duplicates subjects in your database… I also looked into a tutorial that explained how I could ‘switch’ between panels. Basically you add all panels to the main frame, set one of Show and all the others to Hide. Then based on some input, you switch the other frames to Hide and the next one to Show. In my case, switching tabs on the Ribbon triggers this function. Neat!

Adding measurements to the database

Now my application was already getting more complex. I had several classes for all the different panels, added icons to each of the buttons and moved every button from the panel to the Ribbon. I also added a Session panel, where you create a session and add all the different measurements that belong to that same session.

Small detail: I switched from using list controls to ObjectListViews, because it creates list controls from model objects. Kind of like an ORM I guess. It took some getting used to, but after figuring out how to do things like returning the selected object, I like it a lot.

On to processing the measurements

Oh how I love rapid iterations! I made a first version of the Processing panel that should display a list of all the measurements and an image of the entire plate.

Fast forward, I got added a list with all the contacts that were found in the measurement, I rotated the image so its horizontal so I had space to display average contacts below it. I had some ‘scaling’ issues, because I still didn’t understand sizers 100%. But finally I got more or less what I intended: an average image of the 4 different contacts (LF, LH, RF, RH) and in the middle the currently selected contact.

I also encountered a peculiar behavior, where the Ribbon Panel won’t display any buttons until you actually the ribbonbuttonbar contains at least three buttons… Very strange! While this was easily ‘solved’ by adding a temporary dummy button, a much better solution was adding keyboard accelerators (see snippet).

wx.AcceleratorTable([(wx.ACCEL_CTRL, ord('S'), ID_SAVE), (wx.ACCEL_CTRL, ord('N'), ID_ADD), (wx.ACCEL_CTRL, ord('F'), ID_SEARCH)])

I’ve added keyboard shortcuts for nearly every important function, which also saves me a lot of time when I have to test a new piece of code.

We want to see results!

While the processing panel was far from done, I wanted to visualize the results in some way to see whether I was actually doing a good job on the contact annotating front.

So I added panels to visualize the sum of the pressure over time, temporal and spatial information and an average contact for the four paws.

Work in progress!

After that I went back to work on the processing panel. While there have been some intermediate changes, this is more or less its current state. The currently selected contact is highlighted with a yellow line, the other contacts are white when unassigned, green (LF), cyan (LH), red (RF), magenta (RH) when assigned to a paw.

While in earlier versions you could assign contacts by clicking the average image below. However, the only way I could trigger an event from this was by using a child focus event. This had the unfortunate side-effect that it got triggered several times when focus switched and I couldn’t find a good solution for managing this. In the end I decided to add dedicated toolbar buttons and keyboard shortcuts, which work out fine. Just like in my Paw Annotator 1.0 you can assign contacts by pressing 7 (LF), 9 (RF), 1 (LH), 3 (RH) on the keypad, which maps nicely to the layout of the paws themselves.

Previous iterations only allowed you to annotate contacts, which means that if you made a mistake, you had to start all over! By now, you can undo them (Ctrl+Z) and cycle through the contacts with <—+–> and through the measurements with arrow up and down.

Deleting measurements was tricky at first as well, because a measurement has contacts and measurement data. The contacts have results tied to them. And based on one of the Stack Overflow answers, I started out using MyIsam instead of InnoDB. This means I don’t have any foreign keys to enforce the relation. So deleting meant some double checking before actually deleting things, because else I might end up with rough data in my database!

I also added numbers next to the average contacts, these help give you an idea which paw it might be based on the size, the contact time and the maximal force. It also features a prediction, which suggests what paw it might be. However, this is currently based on a simple equation where the current contacts is subtracted from the average contacts, assuming the difference is that smallest between the contact it belongs to. Obviously this is open for improvement, based on the results I hope to find!

Another thing I added are protocols (and a medical history to the database screen). These are basically ways to label the measurements, so they can be categorized in the analysis. Examples of this are separating walking and running measurements or in humans barefooted vs shod measurements. The current implementation either let’s you pick several labels manually or use a ‘profile’, which is basically a collecting of regularly used labels. Because I’m lazy I added a Magic Eight Ball button, which parses my measurement names and assigns the correct profile. Sadly for my users, I haven’t figured out how to let them tweak this through a simple GUI.

Lot’s of database stuff

Don’t mind the icons, it’s already starting to become too clunky for my liking and perhaps I’ll find alternative ways to manage these functions. I’m also not 100% happy with the current interface, because there’s so many lists and text controls that it get’s confusing to what to do first.

However, compared to the previous version, you have a list of sessions that are attached to a subject. Each session consist of measurements and like the protocols, there’s now an option to add a medical history (anamnesis). This will also be used to categorize the results in a more comprehensible way, though if it get’s more complex, I think it will need a wizard to be really useful in a clinical setting.

I actually ‘wasted’ quite some time to make this as user friendly as possible, by implementing an autocomplete that displays any match with the options from the list. Each list is connected to the one on its left hand side, so Primary problems has six subcategories, System diseases may have yet another amount of subcategories and eventually you get to say how severe it is or where something is located. Please note: this is work in progress!

Again, while making this I ran into some headache inducing problems, because while the list is very complete for orthopedic shoemakers/podiatrists, that doesn’t really make it useful for veterinarians. So again I need to come up with a way to allow the user to alter these lists, back them up and restore them when they update the software… Sigh

We want more results!

While its nice that the application now allows users to add data to subjects and annotate all the contacts, in the end they want to be able to analyze them. So I spent some more time getting the results in better shape. Since currently the results aren’t separated based on their protocol (yet!), these results are an average over all the trials, which well… can give some strange results.

Here’s an example of the temporal-spatial screen, where you get the step length, width and time for each paw relative to itself and the other 3 paws. Below I’ve tried to recreate an entire plate image, based on the step length and width, so you get an idea of the walking pattern.

I tried a different way of calculation the foot/paw axis, based on Friso Hagman’s calculations, since my own experiments didn’t bare any fruit, I turned to Stack Overflow once more: How to calculate the axis of orientation? While Joe did a great job of implementing the calculation, the shape of the paws turn out to be pretty problematic. As Joe puts it: “In other words, a dog’s paw is close to round, and they appear put most of their weight on their toes, so the “back” toe is weighted less heavily than the font in this calculation. Because of that, the axis that we get isn’t going to consistently have a relationship to the position of the “back” toe vs. the front toes.” To spare myself the humilation, I’m leaving my current version out :-P

While my center of pressure calculations are most likely correct, or at least were before I implemented them in my application, but as the following image shows:

apparently I’m displaying certain data upside down. So I’m wondering whether my horizontal axis is really correct and if the center of pressure and average contact even have the same orientation… I’ll probably have to try it on a human measurement, so I have a better idea of what it should look like.

Because the foot axis isn’t working as intended, I haven’t worked on my toe detection yet. I was originally hoping to use the foot axis to rotate the contacts to a neutral position, that way I can make a much better estimation of where the toes should be: two front toes on either side and on around the axis on the rear. Ironically, a better toe detection would allow me to do a better foot axis calculation, so I feel like a dog that’s chasing its own tail.

Power to the population

One the things I had in mind with the results was that I wanted to compare it with ‘normal data’, so you get a better sense of whether the dogs individual results are (ab)normal. While we’ve started to get some feeling for this in humans, this simply hasn’t been done for dogs.

This creates several problems: I don’t have any curated ‘normal data’ yet. Off course that’s the purpose of this project, but that means I’m aiming at a moving target. Another problem comes from differences within the population, there are differences in weight, in walking patterns, the lack of distinguishable toes and several unknown factors that may cause large deviations within my clusters. This causes several issues: Should I normalize the pressures so I can compare small and large dogs or are there other factors that make this comparison useless? Does the difference in walking patterns have a significant impact on the pressure distribution or is the effect neglible? How should I compare contacts with and without distinguishable contacts, should I guesstimate their location for a comparison or not?

While its not a problem to go down each path, calculation these different solutions for highly dimensional data (5 weight groups, 2 walking speed, 4 paws) is quite cumbersome, different for most of my results (1 value, 8 values, 1D arrays, 2D arrays etc.) so you can see I can easily waste a week chasing the wrong idea. Another issue is that I need to display all these results in a sensible way and maintain a usable GUI to switch between the different modes. Its clear that there are still some challenges ahead!

Given that I only added my population based results last week, there’s not much interesting to show you. But here’s an example of the temporal-spatial results, based on the weight categories and walking speeds.

Currently all the population results are being calculated every time I need them, since I haven’t decided on a definitive format yet. I do plan to store the average + deviations for every dog, for each atomic grouping. In this case I would end up with two results for each dog: its walking and its running results, that way I see what the most sensible weight categories are and try to figure out what walking patterns there are.

Conclusion

Please keep in mind that three months ago, I had never worked with MySQL or wxPython before. My only programming experience was basic data analysis, yet here we are with a first rough version of an application. It now consist of little over 100 Python-modules, which allow anyone with a MySQL server to run it fairly easily. I have a lot of challenges still ahead of me, but I’m certain that it will be interesting to see what my application will look like in another three months from now!

But I was determined to make sure all the other researchers out there, don’t have to go through the same pain every single project. So I started learning Python in August 2010 and with the help of several Stack Overflow questions, I worked my way through several books, tutorials and other available resources to get closer to my goal. Now that I had managed to learn enough of the syntax to be even more dangerous than I had ever been in Matlab and got some very promising results, I got at a imaginary crossroad:

Get more projects and just stick to data processing, basically maintaining the status quo or dive into Python even deeper and finish what I started. Choosing the latter means that I won’t have any new projects until I’ve learned enough to create an app that’s good enough that new projects won’t need me around to do the processing.

Now I can already see some more seasoned programmers starting to look worried: here’s someone with no programming experience whatsoever wanting to learn programming in what? A matter of months? Well, true with the one difference that I’m working on it full-time and don’t have any other projects distracting me. So the first task I set my self: learn how to make a GUI!

With some help of my Super User (Fake-Programmer) buddies, I looked at several possible Python frameworks I could use for my GUI. My first goal was to set out and see if I could make the pressure plate ‘roll off’ on the screen. Pretty simple I would say: just draw a window, draw an image inside it and update that with every frame of the plate right? Well yes, but that doesn’t mean there’s a dozen of ways of doing things of which probably 90% is wrong and pretty darn slow!

I tried looking at Qt, but after Nokia decided to collaborate with Microsoft, that’s probably not going anywhere. Besides Qt is heavily C++ oriented, so I couldn’t make any sense of the documentation and any decent documentation for Python seemed to be lacking. I tried some PyGTK tutorials, only to not get the installation of the GTK part not worki... I guess the Linux guys really don’t like Windows all that much! (Note: I did manage to get it working 2 weeks later). I settled on wxPython, because first of all they have an awesome book, filled with ‘working’ examples and these are also all for download, so if something’s not working in your code, you can compare them to figure out your mistakes (and hopefully learn from them!)

While the book is $35 and already from 2006, I can definitely recommend it. It really takes you by the hand to explain the syntax and has clearly written examples. One of the things I appreciated the most is that their syntax is very, very consistent and doesn’t use any ‘fancy’ Python features that obfuscate what the code is supposed to do. Furthermore, it automatically uses the systems native theme, so while your app might look generic, at least it doesn’t look like it originated from Windows 95!

The only complaints I had were some typo’s in the code, which were pretty nasty to locate if you don’t know what the code is supposed to do exactly and the fact that multimedia + graphics were completely under highlighted. Yes, there are code examples that draw bitmaps, yes I can find working examples of an image viewer, but you know what the problem is with those? They’re all static! I want to show off my plate data at 60 fps. I have a fast pc, so the only reason I can’t animate this any faster is because the code I wrote sucks and the book didn’t help me solve that part!

It took me about two weeks to get through the book, because I literally typed out every code example in it and got sidetracked a couple of times when I learned something new and wanted to try that out on my own data. After reading this book, I do feel more comfortable of making a GUI around my code. Now my only worry is the object-oriented part and making sure I design my framework right, so I don’t realize I have to rewrite my code half way through…

To make sure I know what I’m doing, I’m now rereading the chapters on Classes in Learning Python. I can already say that everything makes a lot more sense after having actually used classes in practice. Depending on how confident I am after reading this, I might also read the relevant chapters from Dive into Python or another ‘beginners’ book.

I think knowing classes is vital for developing my app, because though I refactored my code halfway through, I still ended up with one long script that would process all my data at once and stores nothing. You’d almost think I learned nothing… So now I’m planning to properly dissect my code, so I can process the data in a reliable fashion (no crashing half way!) and hopefully also more persistent. Ideally, if all the code is correct, the data would be loaded, processed and the results stored in a database. Every time you’d want to view or analyze the results, you’d simply call the results you needed, without the need of processing every trial of every dog in the process…

So as you can see I, still have a long way ahead of me, but I feel I’ve gotten at least a step closer to my goal!