Prologue

About a month before International Open Data Day was set to commence, I received an email from HacksHackers Nairobi announcing a round of grants for local Open Data Day events. Having attended Open Data Day Nairobi a couple years back, I was pretty stoked at the thought of bringing a similar experience to Arusha.

Without wasting time, I put together a Google Form to identify hot topics and early supporters of the initiative. It soon became clear that most people were interested in financial transparency, so I completed the grant application with the theme “Open Budgets and Spending”, and pressed forward to spread the word about the upcoming event.

Ultimately, our grant wasn’t approved. But fortunately, the kind folks at Habari Node offered to fund us. And since Mobisol gave us use of their facilities for the event, we were officially “open for business.”

Opening Remarks

We kicked Open Data Day Arusha (ODDA) off by thanking our sponsors and supporters:

• Arusha Coders
• Mobisol Tanzania
• Habari Node
• JR Institute of Information Technology (JRIIT)
• Farm Radio’s The Hangar
• AISE - Twende

Then I gave a brief introduction of what Open Data Day was all about.

Open Data Day is an annual celebration of open data all over the world.

— opendataday.org

I explained that ODDA was about identifying key issues in Arusha and solving them with open data. Open data itself is information (usually provided by the government) that is

freely available to everyone to use and republish as they wish, without restrictions from copyright, patents or other mechanisms of control.

— wikipedia.org

I also added that data is typically considered “open” only if it is available in “machine readable” formats such as csv, xls, or json.

The Issues

In order to meet the stated goal of “solving problems in Open Budgets and Spending”, Open Data Day Arusha revolved around discussion, consensus building, and collaboration. The first discussion of the day served to uncover the existing problems.

With a bit prodding, I surfaced a variety of issues from the participants: mobile money, doing business, accessing student records, etc.

Each attendee then voted on their top 3 issues. I used a Python script I prepared the night before to calculate the two most popular. The script applied a preferential voting system technique called Schulze single transferable vote to an input csv file similar to the table below.

For this table, each row represents a single ballot. You can interpret the first row as, “the issues ranked from most to least important are E, B, A.” As you may have noticed in the above picture, the top 2 issues were E (getting in touch with businesses) and F (contacting government officials).

The reason it’s so difficult to accomplish these tasks is that email is not a preferred means of communication for many businesses and government agencies. You are best off calling one (of the up to 3) phone lines most business people have. And to contact a government official, you will most likely have to resort to DHL-ing a letter to their office.

Data Workshop

Now that we were on the same page about the problems to solve, it was time to equip the participants with tools for obtaining and working with data. Our key sponsor, Habari Node, provided cloud storage to which I uploaded relevant data and reports. You can view the list of downloaded files here. Contact me directly if you would like access to our Open Data Day cloud.

In addition to pointing out the uploaded files, I also reviewed several other data sources:

• Tanzania Open Data Portal
• Tanzania Ministry of Finance
• Tanzanian Data Lab (dLab)

To give the participants an idea of some of the things they could do with the data, I create some simple visualizations using the Teachers Service Commission Estimates of the Executive Budget Proposal 2016-2017 Volume II (available here).

The Solutions

Armed with the issues and a basic understanding of how to handle data, the participants next began a discussion of potential solutions. We discussed existing initiatives such as AsokoInsight and investment promotion in Rwanda.

The ideas we came up with ranged from a letter tracking service to a government wide customer service ticketing system.

Following the same voting procedure as before, we identified the top 3 solutions:

• A (an updated business directory)
• C (something to tell people who to contact based on the task)
• D (something to help people write official letters to government agencies)

Now the real fun began. The participants split into 3 groups (1 per solution) to flesh out their respective ideas and prepare a 10 – 15 minute presentation.

Team A tackled the problem of getting in touch with businesses and presented a web and mobile based business directory application. Their app would be continually updated, provide business locations and contact information, and allow the public to comment on the businesses.

Team C presented a similar idea to help citizens locate public services such as schools, hospitals, and banks. Their app allowed a user to pick a region and then drill down by category to find the appropriate contact.

The example user flow they gave was as follows:

Arusha -> Education -> Universities -> Arusha University -> <Contact Info>

Finally, team D presented a web based letter writing assistant. The user would simply select the purpose of their letter, e.g., visa application or letter of invitation, and the app would choose the appropriate template, recipient, and mailing address. The app would then allow the user to customize the letter prior to printing, stamping, and mailing it on her behalf.

After mailing, the app sends email and sms notifications as the letter makes its way to the intended recipient. The app sends its final notification after the recipient responds.

To encourage businesses and government agencies to participate in the letter notification features, the app would enable officials to process letters more efficiently and accurately than their current methods.

Closing Remarks

To bring Open Data Day Arusha to a close, I presented the following ways to maintain communication and stay involved in open data:

• Arusha Coders meetup
• Open Data Arusha mailing list
• Data Summit: March 23 in Scotland
• Africa Open Data Conference: July 17 - 21 in Accra, Ghana
• Data Science Africa: July 17 - 21 in Arusha,Tanzania
• International Open Data Conference: Oct 16 in Europe

Finally, Saad from Mobisol talked about how Data Clubs played an important role in his adolescence and encouraged the students in attendance to start a Data Club at their school.

We ended the day with a discussion on how to improve things for next year. Overall, I think everyone got a lot out of ODDA and are excited to continue their journey into the world of data.

Epilogue

This was my first experience organizing an Open Data Day event and I learned quite a bit in the process. In the frantic weeks leading up to (and including) this day, two things were more challenging than I had anticipated.

First was the amount of effort required to get public sector organizations and government agencies involved. I contacted individuals from over 35 corporations, organizations, and gov’t agencies via email and twitter.

Of those, ~1/3 responded (~1/4 if you remove my personal contacts from the list). And at the end of the day, I wasn’t able to get anyone representing the government or public sector (excluding education) to attend.

Second was attendance estimation. I read that you should expect around half of the people who RSVP to an event to show up. We advertised ODDA as a Facebook event to which around 45 people responded “yes” and another 80 responded “interested”. I also figured the numerous mailing lists we advertised on would draw in a few people who didn’t bother to RSVP. So with an expectation of 30 people (45 * 50% + 80 * 10%) up to 50 on the high end, imagine my surprise with the actual turnout of 15.

Comments

Did you attend an Open Data Day event this past weekend? If so, how did it go? What do you think about the issues and solutions presented in this post? Are you involved in or interested in staring a Data Club? Tweet me @reubano with your thoughts.

Twelve months. That’s how long it’s been since my last client projected ended. So what have I been doing the past year? I hear you ask. Aside from watching my savings dwindle, I’ve put a lot of effort into improving my open source projects, speaking, and revamping my website. Alas, as much as I would love to work non-stop on open source, my GitHub contribution graph doesn’t pay the bills.

Over the last few months, I’ve been preparing myself to re-enter the freelance market. Initially, I applied to a few well known curated freelance marketplaces including Toptal and Crew. Crew rejected my application since I didn’t have enough “client specific” front-end work. Toptal is known for only hiring 3% of all applicants and put me through a rigorous series of code challenges and interviews.

Fortunately, I made it to the final Toptal screening. This involved building a single-page web application (SPA) and back-end system with user authentication and roles. I wasn’t too happy to spend a week (at least) of my time working on an unpaid “toy” project. But I figured Toptal’s name recognition would open up some doors and be worth it in the end.

Unfortunately, I didn’t quite get everything working in time for the interview and and failed the project review. It’s one of those situations where unless you’ve already completed a similar app in the past, it’s difficult to meet their criteria in the given time frame.

To be fair, I didn’t exactly make things easy for myself when I decided to implement the SPA in a JavaScript framework I had never used before (Mithril.js). The reasoning behind this was two-fold.

First, the only SPA framework I had used up until then was the Backbone based Chaplin.js. Chaplin’s last release (v1.1.0) was over a year ago, i.e., an eternity in JS land. And my experience with it taught me that for applications with complex data-binding requirements, I should probably look elsewhere.

Second, I had plans to replace Chaplin.js anyway, so I figured it didn’t make sense to invest even more time with it. And if I wasn’t going to get paid for my time, I decided to use the opportunity to learn something new. In the process, I found Mithril to be an absolute delight and plan to re-apply to Toptal once I have a few Mithril projects under my belt.

Tracking the scent

After all but giving up on freelance marketplaces, I decided to educate myself on marketing my services. I invested in Book Yourself Solid by Michael Port, and am currently reading it. Having recently completed the first module on preparing a “solid foundation”, I’ll now show you how I’ve applied what I learned.

My market

entrepreneurs and product managers of for-profit software companies

My cause (why)

I believe the ability to understand data is key to making informed decisions. Furthermore, I want to lower the barriers to transforming data and extracting meaningful information from it.

My persona

who am I?

I am an international speaker and trainer who uses data to show entrepreneurs and product owners how to make better decisions regarding their operations, customers, and products.

my tagline

Your data can talk. Let me tell you what it’s saying.

My value (how)

I help organizations eliminate inefficiencies and identify how customers are using their products and services.

My service offering (what)

I collect and interpret business data to uncover a path to increased profitability.

Granted, this isn’t nearly as catchy as Michael’s,

I help service professionals get booked solid.

or

The guy to call when you’re tired of thinking small.

But noticed what I didn’t say. I didn’t call myself a programmer, or a developer, or even a data scientist.

I also didn’t say I offer data analysis or web development services. I presented my plan to help companies “increase revenue or decrease costs”. Now, my offerings very well may include data analysis or web development, but that’s an implementation detail.

According to Michael, it took him six months to come up with his tagline. So, I shouldn’t feel too bad with what I’ve come up with in the span of a few days.

Just the beginning

One aspect I’m still not comfortable with is the, “Book Yourself Solid Dialogue”. The premise is that I should introduce myself according my “how” statement from above. So Michael would say,

I’m a small business advisor—I help small business owners get more clients.

Translating my statement, I should say,

I’m a freelancer consultant—I help organizations eliminate inefficiencies and see how customers are using their products.

or something to that effect.

While I’m perfectly comfortable writing that in an email, on my website, or in my LinkedIn profile, I can’t bring myself to actual say it to someone who asks me, “What do you do?” I’m much more inclined to revert to the bland, “I’m a data scientist” or “I’m a freelance software developer.”

Does that mean I haven’t truly nailed down my positioning statements? Maybe. Or I could just need more time to get comfortable talking about myself.

¯\_(ツ)_/¯

I’m already looking forward to the book’s next module on developing a “strategy for creating trust and credibility”. Stay tuned for part II.

What’s your take? Do the above statements give you a clear indication of my value proposition? Does it match what you know about me? Tweet me @reubano with your thoughts.

For those of you keeping track, I’ve now redesigned my website for the second time since its inception. I created my first website with the static site generator (SSG) Octopress. This first site contained a blog, project portfolio, archives, and several pages (home, about, and contact).

Despite Octopress’ popularity and breadth of plugins, the Pythonista in me couldn’t stand its Ruby centric tooling. For the first rewrite, I decide to try a new approach with a Backbone.js based single page app (SPA).

To reduce the required boilerplate and provide my SPA a bit of structure, I used Chaplin.js in combination with the asset compiler Brunch. Using Brunch and two custom plugins, I was able to emulate Octopress’ markdown workflow. In addition, I gained the ability create pages from JSON data sources. I eagerly used this new power to generate a project portfolio from the GitHub API. Keeping with the JSON theme, I also generated a photo gallery from the Flickr API.

Client side JavaScript really shone here. I was able to use cached JSON files in development, but fetch data directly from the APIs in the live version of the site. This meant that any repos I created on GitHub or photos I uploaded to Flickr would instantly appear on the site without additional intervention. I also loved that you could navigate the entire site without page refreshes or server calls after initialization. This meant the site worked offline once it was loaded.

Sadly, all good things must come to an end. Chaplin’s maintainers effectively ceased development on the framework. This, in combination with overall JS fatigue, motivated me to reconsider static sites.

SSG selection

Since I had recently given a talk on SSGs, I had a decent idea of what I was looking for. I was also determined to make the new site as lightweight as possible. This meant I was going to abandon Bootstrap, jquery, Google Analytics, and web fonts.

The contenders

I’ll save the gory details on the pros and cons of each SSG that I tried for a later post. But after trying out Roots, Hexo, Wintersmith, and Hugo, I decided to go with Metalsmith. Metalsmith was the only SSG that gave me flexibility, a comprehensive feature set, and ability to write plugins in a decent language (CoffeeScript in this case).

I briefly considered a few Python SSGs such as Nikola and MkDocs, but found their integration with my frontend stack (stylus and pug, aka jade) to be lacking at best.

This isn’t to say Metalsmith is a panacea, not in the least. What irks me the most is that Metalsmith is very picky about the order you load plugins in, and suffers from slow compile times. Hexo did a great job of alleviating the first issue (and possibly the second) with their plugin registration system.

Slow compilation

Internally, Metalsmith passes the complete list of files to each plugin, and almost every plugin needs to iterate through the entire list in order to do anything meaningful. The following is an example plugin that removes all draft posts from the pipeline.

plugin.coffee

module.exports = (options) ->
  (files, metalsmith, done) ->
    setImmediate(done)

    for file, data of files
      if (data.draft) 
        delete files[file]

The overhead of this architecture becomes present as you scale the number of pages built. Using dumbbench I retrieved the following timings in development mode:

And the following timings are the result from production mode which performs file minification and compression steps:

As you can see in the following graph, the development run displays a time complexity on the order of log(N), while the production run crosses over into linear territory on the order of N. Yet, even with logarithmic growth in the best case, Metalsmith still takes an eternity (almost 15 secs) to rebuild my site. And as I point out below, I have to wait for a full rebuild after each change.

The ugly

Additionally, JavaScript’s inherent mutability is a major source of pain. And despite my best efforts, I was never able to get either the metalsmith-watch or metalsmith-changed plugins to work.

Future work

The ideal SSG would iterate through the list of files a fixed number of times and then pass each file to every plugin in parallel. This would be most effective with immutable data structures like Python’s MappingProxyType or Clojure’s Maps.

I eventually want to look under the hood of Nikola since it may relieve some of issues stated above. Plus with the help of python-stylus and pyjade, I may even be able to reuse my templates.

Ultimately, I don’t think I will ever be completely satisfied without building my own SSG. Something that combines the speed of Hugo, the elegance of Hexo, and the flexibilty of Metalsmith. My stream processing library riko would provide a good foundation on which to build such a framework. If only I had the time…

Do you have any SSG experiences to share? Tweet your story to me @reubano.

We left Sumbawanga the same day and headed back to Mbeya. On Saturday I went to TFA to install the sheller on the bike we had on display. Henry, TFA Branch Manager, informed me that he had gotten several inquiries about the sheller and charger. This was exciting news and on Monday, we made an agreement for TFA to receive shipments for us from Arusha. This way we could take orders from people in Mbeya and direct them to TFA to pick-up the merchandise. With everything in Mbeya straightened out, I headed north to Dar early Tuesday morning on May 25th. I took advantage of my time in Dar to meet with people from MUKPAR, D.light, CNFA, and Solar-Aid. These meetings were extremely helpful and introduced me to several new sales and distribution methods. The most promising meeting was with Karan, MUKPAR Director, who really liked the sheller and charger and gave me the contact info for Ganesh, the MUKPAR Arusha Branch Manager. Karan was open to having us put a bicycle in the Arusha Branch to test out customer demand.

I left Dar on Saturday, May 29th for yet another bus ride north, this time back home to Arusha. Following up with my Dar meetings, I met Ganesh the next Thursday on June 3rd and he agreed to let us put a GCS bike and sheller on display. The next day, Jodie and I met with Laura, D.light marketing manager, to discuss ideas for a marketing collaboration. Since my volunteer visa was set to end on June 8th, I made plans to head (north) to Nairobi the following week. A mix-up with with the terms of my visa led me to make a mad dash packing my belongings in time to catch the 2:00 bus to Nairobi on Tuesday the 8th, a day earlier than I had planned to leave. Luckily, I was able to pack and catch the bus with a few minutes to spare. This was just enough time for me to take a few bites of the ugali and cabbage that Jodie brought with her before I boarded the bus. Due to leaving in such a rush, I wasn’t able to say goodbye to many of my friends and neighbors. But I will hopefully soon be back to Arusha again.

Pleased with our work in Mbeya, we left the sheller and bike at TFA on the 12th as Daniel headed north (with a sheller) to Iringa to visit the remaining TFA’s and I headed south (with a charger) to meet Iddi in Kyela. Kyela is remarkably much warmer than Mbeya (I didn’t need to wear my sweater at night for example), produces much less maize, almost everyone rides a bike (probably due to the flat landscape and few amount of dala-dalas servicing the town), and lacks power in many of the villages –> a perfect target market for the charger. It just so happened that the TechnoServe office was in the same building as PRIDE and Tujijenge Microfinance (TMF), so Iddi arranged for me to meet with the branch managers of each.

Over the next 2 days (Thurs and Fri) I visited several groups of villagers with TechnoServe and TMF to talk about the phone charger. TechnoServe has a cocoa farmers program and the farmers I met with were very eager to begin selling GCS chargers. Similarly, the TMF farmer loan groups (councils) I presented to were just as impressed. Over the weekend, I tried my luck in town by showing the charger to some store owners. It was a tough sell because the charger I had wasn’t working and my swahili still isn’t all that great. Nevertheless, I tried to explain that they wouldn’t have to pay now and landed 6 orders in the end. The feedback I received from the microfinance (mf) groups got me thinking about doing the same thing in Arusha. If we can partner with a few mf organizations, we’ll be able to reach a large number of villagers and likely have an easier time landing bulk orders. This plan will have to wait until next week at the earliest however because I will shortly be on my way to Sumbawanga (via Mbeya). CNFA has secured funding so I will soon be hitting the road with them.

I started out the next day meeting Hillary, the Country Director of TechnoServe/Tanzania. I showed her videos of our sheller and charger in action and then took out the samples I brought with me. She was very intrigued by what she saw and gave me a handful of contacts to follow-up with. The next stop was the Tanzania Investment Centre (TIC) where my plan was to meet with the Director. The Director was away on travel but I did manage to get a meeting with John, the Investment Manager. My videos and samples impressed him so much that he ordered one of each on the spot. John also talked a bit about the benefits that TIC certification would provide us at GCS such as assistance getting residents permits, duty incentives, and greater access to investors. I thanked John for his time and then headed over to FINCA for another impromptu meeting.

One of difficulties of selling products to the rural poor is that many of them can’t afford to purchase them. I was hoping that we could work with FINCA to provide financing to our customer base of agrodealers and maize farmers. Unfortunately, the FINCA President was in a meeting and I was told to come back the next day. The next morning I headed back to FINCA and sat with a Credit Manager named Issa. While not as impressed with our company as John from TIC was, he did see some potential and told me to write a proposal. His main concern was that our sheller is a season product and that a bad harvest to impair the farmers’ ability to repay their loans.

My last stop of the day was the CNFA Convention. At lunch, Philemon and I met with Alfred from KickStart. He gave us a lot of valuable advice on how we should market and sell our products. Towards the end of the Convention, I met Kelly from CNFA who introduced me to a bunch of staff from Dar, Arusha, and Mbeya. She also managed to get Philemon and me a spot at the end of the meeting to show our videos which, after overcoming technical difficulties with the usb drive and projector, went over very well. By the end of the day, we landed over a dozen orders each of both the chargers and shellers. A nice way to kick off the sales trip!

My neighborhood, although just a few minute drive to town, has a very rural feel to it. It’s fairly common to share the dirt roads with a herd of cattle or goats looking for pasture, or a mother hen and her chicks pecking for food. I’ve become quite accustomed to waking up to the crow of our neighbor’s rooster and the occasional flock of chickens that raid our front yard looking for food scraps. However, despite the ruralness of our village, most everything you could possibly need is readily available. Around the corner from our house is a “strip mall” of duka’s that hosts a seamstress, a barber (who also doubles as the electricity bill collector), a video store, and a convenient store. And within a few minutes walk are various bars, restaurants, and other service providers. As I have discovered, most villages are setup in a similar manner making each one a self contained community.

One final observation is our village’s collection of ~5 stray dogs. Each occupies his own area like the bums you see in Central Sq. - Cambridge, MA – and every night we hear them howl in unison. One of them, whom Daniel named Ijumaa (Swahili for Friday), follows us home each day as we pass by the village bar after work. Apparently, we are the few people who don’t throw rocks at him so he’s taken quite kindly to us.

Afterward, we showed the group how they could use the leftover corn husks and cobs to make charcoal in a metal drum. The process is relatively simple: you layer the cobs and husks inside the drum, light it on fire from the top and bottom, burn off the water, cover it, mix it with cassava, and press it to briquettes. The group was very impressed with the charcoal (more so than they were with the maize sheller) and interested in exploring the business opportunities our devices presented. We left a maize sheller with the group and they told us they would do more testing once the maize dried for a few more days.