While communication and the dissemination of information throughout our society has grown exponentially with the spread of mobile apps, there still exist major hindrances to collaboration between local governments. In 2010, Civic Commons was established to provide information sharing amongst governments and institutions creating an infrastructure for the open government. Civic Commons as stated, is a “lot like a community driven civic app-store”. The “Market Place”, provides a platform that “aligns cities around the technologies they buy and build by fostering an engaged community of government decision-makers, organizations, and vendors.” It’s a place for civic engagement tools (and the companies that build them) to connect with governments and municipal authorities.

This past spring, Fulcrum, a mobile data collection application joined the Market Place to provide municipalities and organizations with a dynamic tool to quickly design, build, and deploy micro-apps for all field data collection needs anywhere in the world to any number of users in a matter of minutes. The opportunities for collecting and sharing information to resolve complex municipal problems with Fulcrum is endless and more importantly providing a cost effective solution to community progress.

The API has existed from the beginning of Fulcrum’s development as a communication endpoint for the mobile app, and we’ve added some new functionality to allow complete writing and reading of data to and from your Fulcrum account via the API. We’ve also extended the documentation pages to cover the API endpoints, with examples and descriptions of how to use each.

All that’s required to get started on hacking with the API is to login with your Fulcrum account and go to your user profile page. You’ll see a link at the bottom to the settings where you can generate an API key.

Here’s a short example in Javascript showing how to pull down your list of forms (and form data models) from your account:

And here’s the response returned from the API in the example, a JSON representation of the forms in your account:

Alternately, you could populate Fulcrum forms from another system by sending POST requests containing your data models using Fulcrum’s form definitions. You can even define field conditions (for field visibility or requirement) to build in your own conditional logic.

Here’s another example demonstrating a request for records in a particular form using cURL:

Passing the form ID in the request will yield a JSON response of the data within that form, including attached photos and thumbnails:

Opening up the platform API to developers will allow for tighter integration of Fulcrum with other backend databases like ArcGIS Server, PostGIS, or others by simply writing some connector code to fetch data from your Fulcrum account. If you have any questions about using the API and want to learn more, or have an idea about how you might want to employ it in your own applications, get in touch with us on our support forum.

Since we’re building Fulcrum to capture location-based data in the field from customized forms, I thought it’d be interesting to try building a form to capture POI data in the standard OSM schema, with OSM tags created on export. The main objective was to abstract the form displayed on the mobile to show a human-readable set of classes to describe the points, as opposed to requiring specific OSM tagging, like leisure=pitch or amenity=bicycle_parking. Potlatch does something similar to this by categorizing things a bit more logically with icons, so with my Fulcrum OSM form I built a classification set to reorganize the basic “type” tags into a hierarchy of options to categorize POIs.

So rather than having to memorize or even be knowledgeable about the rather complex Map Features page, the form elements I built into my OSM form are using the “data name” attribute on each field to set the attribute name that’s exported. Instead of having to remember that fast food places should be tagged amenity=fast_food, you can drill into a menu of types to select Restaurant → Fast Food. When the data is exported, it uses the attribute name instead of the label. This abstraction makes it much simpler to enter just the right attributes when you’re collecting data.

I’ve also set up a number of visibility conditions, so that certain fields only appear on my phone when particular conditions are met: e.g. “cuisine” is only shown when the type is “restaurant” or “cafe”, transport tags like stop ID only appear when adding a bus or train stop, and “drive through” is only displayed when adding things like fast food places or ATMs. This makes it much easier to capture the maximum data, but never have to see all fields in the form unless they are relevant to the POI type I’m creating.

Once I’ve captured some data, there are a few more steps to getting the data imported. I export the data as GeoJSON, then use a Python script called ogr2osm to convert my exported data file to the OSM XML format so I can open in JOSM to touch up the tags and do the final import. Once the points are brought into JOSM and any tag corrections are made, it’s as easy as submitting a new changeset with the new POIs. I’ve collected 50 to 100 POIs with this form around town – from restaurants and bars to bike racks, water fountains, and bus stops – all with maximum relevant attributes, since it’s just tapping and selecting from the proper options on the form. I’m still improving the process as I’m testing and collecting, the import stage is still somewhat complex for novice-level users. But this is the first step in understanding the intricacies around building a true user-friendly OSM application.

Keep an eye on the blog to see what’s next with this. Zac have discussed the possibilities of an native, dedicated OpenStreetMap POI collector on iOS powered by the Fulcrum codebase, and we think it can be done very gracefully. Also stay tuned in the coming weeks for this OSM form as a reusable template you can use within your own Fulcrum account to gather POIs on your own. If you have any other questions or ideas about Fulcrum, feel free to contact me on Twitter @colemanm.

I circulated the link to some of my team and asked for input.  After expressing their interest and identifying one of the many potential candidate projects we have in the works, we decide the push ahead and take the plunge to commit and apply for the Accelerator.

While we are not technically a startup (@spatialnetworks has been in business since 2000), we operate very much as a startup and our culture thrives on that.  We chose a product that we have been working on in concept for a number of months.  This idea, when turned into a product will provide Governments and citizenry with at least two, potentially all three of the stated models from the CfA Accelerator program:

1) they provide services on top of open government data;

2) they bring modern web technologies directly to governments;

3) they change the way citizens ask, get, or need services from government.

We see the need in the United States to enable anyone and everyone the ability and access to report things that need to be “fixed” in Government, regardless of whether it is Federal, State, County or Municipal/Local.  There have been some pioneering efforts to bring similar solutions to communities and Governments alike, but we’re looking to push the envelope even farther.  With our product fixitwithme, we intend to provide both Government and Community with great opportunities to engage in dialogue while providing a platform for accountability and transparency.  We’re not stopping there however.

fixitwithme goes beyond simply documenting broken streetlights, potholes, or instances of Government inefficiencies, it reports those in real-time (on a map of course) and creates a permanent page of that issue (that needs to be “fixed”) much like a Facebook fan page.   Those that contribute to documenting the issue, can keep current on the status of that issue (is it fixed yet?) and who in Government is responsible (the Mayor?), how many people have either voted-on, liked, +1 or otherwise added their voices to the problem that needs fixing as a communal voice that, well, at some point, can no longer be ignored.

We’ll take that social input, route that to the responsible Government entities until the issue has a remedy, and will continue to shine a light on the issue until it is resolved to the Community’s satisfaction.  This may also include gradually increasing visibility in social media, search terms, traditional media – basically whatever it takes to get the matter acknowledged and resolved by Government.

With fixitwithme, we’re even going to propose something entirely radical in terms of how things can get fixed.  I’d propose that if Government cannot or will not allocate funding to resolve the issue (ie. fill pot holes or clean up parks, etc.) or that Government is unwilling to become more efficient, that the immediate Community resolve it themselves (think AmeriCorps + flash-mobs) by getting organized and registering to donate time and their own sweat-equity and resources to fix problems in their own community (ie. fix all the potholes, clean up parks, etc.) then those people should be given tax credits for the year they help fix things.  Communities are incented to take ownership and be good stewards of their own environments, Government is held accountable (elections) and the local economy can benefit as well (cleaner, more maintained communities are attractive places to visit and start businesses).  We have this already in some fashion – “adopt a highway” is one such approach, but largely relies on Corporate goodwill and volunteerism.  Nothing wrong with that, but many people suffer from “donor fatigue“, but with fixitwithme, we intend to provide tangible incentives for citizenry to either hold their Government accountable for taxpayer funds, or be given the opportunity to earn a lower tax bill by coming together as a community to fix things that need fixing.

I’ve already been told by many people this “isn’t possible” and “probably against the law” in most Government charters.  Perhaps it is, but just being told “no” makes it that much of a solid candidate for the Code for America Accelerator program – if anyone can bring real change to Government, they certainly can!

The concept of trying to map things (physical or cultural) on the earth over time is challenging at best.  There have been excellent examples of doing this in limited areas over slices of time but I’ve yet to see a truly global effort to provide a continuum of data as a surface.   The closest thing I have seen that even approaches doing this at a global scale just got launched by Chris Tucker at MapStory.  I’m going to attempt to dive into this a bit more in an upcoming talk next month at the Location Intelligence conference in DC with a specific application example that we are working on in the spatial networks labs.

I often use the analogy in conversation, when trying to describe this to colleagues, of the moment when you break through the clouds on an airplane and you happen to have a window seat on a clear day, and you are at eye-level with the top of the clouds.  Looking out, you can see an undulating, irregular and dynamic surface punctuated by pockets (no clouds) or perhaps a boiling thunderstorm rising up in an isolated vertical column.

The turbulence you experience is the impact of the plane slicing through this surface or skidding across patches of it at cruising altitude.  This provides a healthy analogy for continuous mapping of spatio-temporal data across the entire planet.

To do this, we need a way to uniquely index the planet in 4D (x,y,z,t for those of you scoring at home).  My idea (and the latent flaws in it) draws from the combination of these variables, so humor me and let me try to articulate this without additional context or history.

If we look at the maximum practical values for each variable above, we’d have something like this:

Latitude (x) = xx.xxx (ie. 89.999 in DD)
Longitude (y) = xxx.xxx (ie. 142.034 in DD)
Elevation (z) = xxxxx (ie. in meters, every 5m in elevation is approximately 10,000 meters; yes, it’s metric, get over it.)
Date (d) = xxxxxxxx (ie. 31122012, sorry StarWars is cool, but no Star Dates here)
Time (t) = xxxxxx (ie. 23:59:59)

Total Land Surface area: 510,072,000 sq km (thanks @colemanm + Wikipedia)

If we fishnet grid the entire planet, or perhaps just the land area, at 5 meter intervals we get 20,402,880,000,000 points in the grid (give or take).  That’s nearly 20.5 Trillion points.  By any definition, that qualifies as “big data”.  It gets better.

If we concatenate (that word always sounds funny to say out loud) the integer values of the variables above into a single integer (and we stay consistent about it) we get something like:

899,991,420,341,000,031,122,012,245,959 which is more than enough digits to conquer the ~ 20.5 trillion point fishnet across the land surface of the earth.

‘Why bother with all this big math?’ someone countered me when I brought this up last year at WhereCampDC, since we already have UTM to do this?  (I admit, I chose the wrong forum at the wrong time to introduce an imperfect idea and was promptly shot down)  The question is still valid and my response at this point would be, this number (whatever I get around to calling it) provides a truly unique temporal index to just about everything that has ever or would ever be important to us as humans (and the non-human species around us).  It would allow us to reference everything (to the nearest 5 meter coordinate) over time and if that were to happen consistently and uniformly, we could then perform analysis that is only possible in the realm of science-fiction or DARPA (sort of the same thing at times)

Being able to investigate, interrogate or query time & space (true 4D) and have all results returned on-demand and presented in a rich UI (ie. map) via a browser would be, well, pretty awesome.  It leads back to the notion of the top of the cloud deck – that active filtering of spatial-temporal data at 5 m resolution could provide true “art of the possible” capabilities and insight that might just make a difference in trying to understand the world around us and our impacts on it and upon each other, even get crazy and start anticipating if not predicting strategic events.  Or perhaps my idea is just a more modern adventure fashioned after Quixote.

If you thought that was off the reservation, cycle through that math with a 1 meter fishnet and we’re into the quadrillions (510,072,000,000,000,000 to be precise, give or take).  Those numbers venture into the realm of  Jeff Jonas and Terry Busch and that gets truly interesting real quick.

The enthusiasm and passion on display during the entire event had a refreshing sincerity, at least to me, given that I’ve attended countless industry events over the years.  I generally loathe the gauntlet of poorly designed technology demonstrations put on by shallow marketing efforts (elevator pitches) and dissertations masquerading as PowerPoint presentations that most conference attendees are subjected to endure. In truth, I’ve even recently discovered that I’ve become somewhat tone-deaf to our own marketing messages and find them somewhat stale (more to come on this).

The real take-aways for me however can be distilled into three main themes:

1) free & open-source software for geospatial is wicked cool but is lacking in business models for long-term sustainability.

I’ve wondered for a while just how many of the companies I admire earn enough revenue to pay rent, salaries, benefits and seemingly be at every conceivable event giving presentations and evangelizing FOSS4G technologies, including their own “free and open-source” products.  Unfortunately, this was only amplified after this event, particularly after a panel session attempted to address the notion of a business model.  Sophia Parafina posed the question to the panel and while nearly everyone had a response, none of the responses actually answered the question directly or with any specificity.  In an uncharacteristic move, I refrained from opening my big mouth to pursue this line of questioning until satisfied, but well, let’s just say, I was pretty much a n00b and caution got the better of me.  However, in conversation after, I brought this up and several people  commented about the same concerns and scolded me for not asking the question (figures…).  One veteran of FOSS4G, someone that I’ve come to respect immensely, went so far as to comment on the subject in private “I’ll be back in a year or two hiring many of these people after their business’s fail”.  This was a bold and assertive statement that, coming from anyone else, I would have discounted immediately as bluster, but given the personality behind the statement, it slammed me with a healthy dose of reality.

There are emerging notions of a business model from Mapbox and Vizzuality that seem fairly straightforward and even with my limited math skills, makes sense, but there are a lot of devils in a lot of details behind the scenes yet to be played out.  The other most notable exception is OpenGeo, which has products and pricing built on open-source but is a reasonable composite for licensing technology while keeping technology open-source.  Aside from those however, all I can observe is that FOSS4G companies charge for support, training and custom development.  This is a linear equation because all revenue is then tied to hours of people available to work.  This is no different than the business models that have made companies like SAIC, Harris, Booz Allen Hamilton very successful.  I’m not suggesting one business model is better than the other, I am just trying to discern what there is to choose from in the FOSS4G community.  I suspect there was no definitive answer to the question posed during the panel for one of two reasons; no one really wanted to share their secrets (odd given the context) or more likely, there isn’t a real alternative to trading dollars for hours (and that’s ok if that’s the reality).  It would have been truly refreshing to hear that stated clearly and unapologetically from the panel, or at least generate a larger dialogue.

2) the open-source community (geospatial) has a higher percentage of passionate, dedicated and “all in” professionals than any other community or company I’ve experienced, even my own.

I know many of the people in attendance at FOSS4G NA, but the vast majority I don’t know and most have no clue who I am.  (we currently do not have any open-source projects but that is about the change)  I’m constantly impressed by and with the work ethic of the people driving this community forward.  It’s always a humbling experience to be amongst people that I admire and strive to emulate, but to witness the passion and “all in” attitude that each one brings is unparalleled in any other community I’ve been a part of in my professional career and that is truly amazing to me.  Just writing that last sentence hammers home the need to recalibrate my own perspective and that of my company’s.

This is not to suggest that there are no passionate or dedicated people at spatial networks or allinspections, or even within other professional organizations and communities that I participate in.  Quite the contrary in fact – I work with some of the most talented people in Florida, but as a percentage across a community of people, this geoherd (sorry, it fits) enjoys a disproportionately high percentage of people that are driven by the desire to make and do things that matter to the rest of the world.  As a geographer, that makes me proud to be affiliated with such talent.

3) free & open-source software for geospatial will usurp the incumbents within the next 5 years.

I don’t feel that one single company in the FOSS4G community will rise to the top and replace companies like ESRI, Intergraph, GE Smallworld, Pitney Bowes MapInfo, or even Google.  I do however think that the technology innovation of the FOSS4G + OpenStreetMap community will blow by the combined efforts of all the industry giants above and not even look back.  The writing is already on the wall, or more accurately, it’s written on Github.  What comes from the FOSS4G community (not just the annual event) is a steady stream of breath-taking technology innovation and iteration.   For all the resources at the disposal of more traditional GIS software companies, too much is spent on marketing and propping up the hegemony and not enough on solving problems (and yes, UX is a problem that desperately needs to be solved, even in the FOSS4G community).

There was a fair amount of ESRI-bashing (apparently obligatory for FOSS4G) and as I commented on twitter during Peter Batty‘s Ignite talk, it’s not just trash talking for the sake of trash talking – ESRI earns and deserve much of what it gets by the FOSS4G community.  I’m not apologizing for ESRI and nor am I defending them, even if I have no issue with their underlying business model (it’s the behaviors and practices I have issues with) but since I’m among the newest of the excommunicated and deemed “persona non grata” by Redlands, I’m not the most objective person on this issue at the moment.  The point here is that the FOSS4G community has grown up so fast in recent years largely because of ESRI, Intergraph, GE Smallworld, PitneyBowes MapInfo and lately, Google.  That growth in the FOSS4G community is yielding some incredible technology that poses an existential threat to ESRI, Google and the like in terms of dominance over all things “geospatial”.  This is a good thing, if for no other reason than to disrupt the status quo in Redlands and Mountain View and to wake up their customer base to alternatives.

Some additional take-aways for me include a mind-altering discussion on rendering the world in a day by Young Hahn from Mapbox, powerful on-demand mapping with raster images via Javier de la Torre at CartoDB, watercolor maps from Stamen thanks to Nathaniel Kelso and telling map stories with, what else, mapstory.org.

After my quick talk, which centered on promoting more open source technology in higher ed, John Reiser and I had some good discussion on the subject, as he’s well-versed in the issues and shortcomings of academia when it comes to promoting technology. He had thrown out the idea to do a “birds of a feather” session to gather some like minds to brainstorm some ideas on making it easier for universities and instructors to bring FOSS geo technology into the classroom. So for about an hour and a half on Wednesday some folks gathered around, from both academia and industry, to shoot ideas around. Conversation started discussing basic issues – better training and instructional materials, simpler install and deployment, and a more accessible, reliable support structure behind the scenes. Mike Pumphrey from OpenGeo talked about initiatives to help promote their open stack and connect with educators to provide the tools and connections they need to make it easier for students to get comfortable with FOSS applications.

Another facet of the conversation involved the relationship between industry and academia, and how the two can become better connected with respect to what industry wants or requires from potential hiring candidates and what skills academic institutions are promoting and growing amongst their student bodies. Universities and college programs certainly do outreach and connection with companies local to the campuses, and vice versa… But there’s no known, canonical “repository” for sharing both aspects of this – “here’s what we want from hires” and “here’s what we’re teaching students to prepare”. Ideas I’ve tossed out before as possible “benchmarks” for measuring candidates’ fluency with software and technology are things like checking out a student’s contributions to OpenStreetMap, or possibly their interactions and involvement with GIS StackExchange. Both of these are public, tangible resources to identify engagement with geo technology and the community in general (both qualities that are of huge value to us, and probably many other mapping companies).

It was great discussion all around, and it actually continued out into the hallway and onto the social event afterwards – so fantastic to have such a passionate crowd attending the conference and truly wanting to get behind issues and make the improvements that would be good for all of us. It’s not always easy to herd the cats, but it’s often worthwhile.

I’ve included the slide deck from my talk above for anyone interested. John also took copious notes throughout our discussions on ideas tossed around and shared them online. Certainly if you feel passionate about these or related issues in geo technology, get in touch with me on Twitter and let’s chat! Thanks again to John for proposing the session, and to the others involved for the interesting discussion.

This week some of our mapping team will be in DC for FOSS4G-North America, meeting all sorts of folks from the FOSS arena, chatting about how we leverage an array of open source tools and data, and maybe even getting into some code hacking later in the week.

The schedule is jam-packed with good stuff – talks about open data, analysis, and brand new software releases. It’s great to see such a healthy breadth of topics with such passionate people behind them. A few that I’m particularly looking forward to: Paul from FlatRockGeo talking about open source for emergency response, Young Hahn’s talk on rendering the world into the cloud with Mapbox Streets, and the latest goings-on with CartoDB from Javier de la Torre.

After the social hour on Tuesday there will be an Ignite Spatial session at 6:30pm, and I’ll be there presenting my thoughts on how to improve the way we teach GIS and mapping tools to up-and-coming students readying themselves to enter the market. I’m no educator, and certainly not an academic – so the talk will cover my opinions on the matter, and what I see as a ground-level observer of industry needs, the types of skillsets and techniques being taught today in higher ed, and why open source should be embraced.

Our software projects include many of the open source mapping tools out there – from mapping heavy-lifters like Mapnik and GDAL to more baseline technologies such as MongoDB, node.js, and Rails – so it’ll be great to catch up with many of the creators and inventors responsible for such awesome stuff. If you want to catch up and hear more about what we’re working on, track down me, Zac, Patrick, or Tony during the event.

Fulcrum Pro gives the user access to unlimited projects, records, forms, users, and map uploads.  The Pro subscription also adds shapefile and GeoJSON to the list of pre-existing data export formats (CSV and KML). With a Pro account, the account owner has the ability to invite others via email to join in on data collection projects, giving other users access to contribute and modify data with the Fulcrum mobile app. The organization’s members can then join in on collaborative collection efforts and share data between one another in the field, while administrators can view active fieldwork in real-time back in the office or from a mobile laptop from the web app, or using the live Google Earth View.

To upgrade your account and add get your team members added, log into Fulcrum, go to the Subscription Settings and convert to the Pro plan for just $20 per user per month.

Utilizing the available features and capabilities of Fulcrum as your mobile data collection platform provides you with a unique set of features and capabilities.  The Fulcrum platform equips you with a unique means of collecting data online or offline, secured cloud data storage, and dynamic data export formats.

One of my assignments was collecting fire hydrant data using a handheld Trimble unit. The department I worked in does not have any sort of equipment for GPS collection. The Trimble I used was brought to City Hall from the utilities campus. Since my supervisor was not familiar with the equipment, a utilities worker was the one who instructed me on how to use it. The device is simple, but I had to build the data forms in the unit, which is not as user friendly as typing up the forms on a computer like with Fulcrum. Collecting the data in the field with the Trimble is not as simplified and logical as it is with Fulcrum. The Trimble unit I used had a designated laptop with specific software for exporting and preparing the data for import into ArcMap. Hooking up the equipment and running the program is more complicated and takes longer than Fulcrum’s export process does. With Fulcrum you don’t have to tow around a special laptop so that you can actually use the data you collected. You just need your phone and any computer. No wires, no expensive software to learn.

Shortly before I left the internship, the department was preparing to do a survey of the historic buildings in the city. I had to do some of the preparation for the project. This required me to use ArcCatalog to build ‘forms’ (tables) for the survey, which is a frustrating process because one does not simply move columns, change titles, etc in the tables. From my understanding, the plan was to take a laptop into the field to fill out the tables, while separately collecting GPS points of each building. After this, the information would be merged together when back in the office. Fulcrum does all of that at the same time and with pictures, which would be very useful in a historical buildings survey. Plus you don’t have to mess around with table creation in ArcCatalog!

It is important to note that not everyone in the office has an iPhone. I would say a majority of the employees with smartphones had Androids. This is why a Fulcrum release on Android is important and on the horizon. With Fulcrum, every department can have the ability to collect data. When data is needed, city employees won’t always have to depend on a county GIS department, unpaid interns, or outsourcing data collection projects to private companies; they can easily collect it for themselves.