Geospatial news elsewhere...

Some YouTube-related Forms

If you ever see a video, and you’re like “Why is this watch next recommendation from YouTube so crappy?!”, then you should fill out this form for bad watch next suggestions.

If you ever see a video, and you’re like “Holy crap, it’s like I’m watching a dithered display from 2002 in here; these blocky artifacts are terrible!”, then you should fill out this form for bad video quality.

Feel free to pass these on.

Hawk vs. Drone: Fair Use and Copyright

So, I have a popular YouTube video. (It’s not that popular anymore, but it was once.)

YouTube recommended I watch a video: Drone Crash, Fail & Win Compilation 2015, Part 3/3 (watch in 1080p).

So, I watched it, and was somewhat surprised to see a video I recognized in it: mine.

Now, this particular video is not licensed under a Creative Commons license. (The reason for this is that I have seen plenty of evidence that people misunderstand what creative commons *means*, and don’t bother to attribute — sometimes properly, sometimes at all — and I didn’t want to have to argue about it with people who were ripping off the video for their own personal profit.) This means that if someone wants to use the video in a way that isn’t allowed by fair use (or some other portion of copyright law), they need to obtain a license. Some people have obtained a license. In other cases, I’ve been able to convince people who were hosting the video themselves to instead embed the YouTube video. (All of the ad proceeds from the video are donated to the Mass Audubon Society, so this matters to me more than it otherwise would: I feel a responsibility to maximize the money that is donated because of this video..)

In the comments on the compilation video, someone said: “He has just stolen all the clips from other peoples videos without even asking them first. Only Creative Commons videos are free to use so its copyright theft!”

Now, this misunderstands the way copyright works a little bit — Creative Commons videos aren’t ‘free’ to use in the sense of freedom, they require specific requirements that still need to be met — but it’s generally reasonable: Reusing someone’s copyrighted content without a license *may be* copyright theft. But it isn’t when such use is allowed under fair use. (There are other cases too, but I think fair use is the most straightforward claim in this case.)

So, to that end, I replied with the following analysis of whether the 12 seconds of my video used in this compilation was a violation of my copyright. I welcome comments or feedback on ways in which you disagree with or support this analysis.

In the US, 17 U.S.C. § 107 defines a 4-prong test for determining whether something is fair use. In particular:

Purpose and character of the use: ‘whether the art aims to only “supersede the objects” of the original for reasons of personal profit.’ I don’t think that putting all these clips together creates a meaningful work of new art, and that it would fail the test for Purpose and Character of the use. This is not an educational video; there is no commentary; it’s done in no way other than to get eyeballs. (If there were no ads on the video, one could at least claim it wasn’t commercial in nature, which would act in its favor, but there are ads.) That said, the combination of all of these shots together certainly puts some pieces in a different light; that is, there is some creativity being applied. I don’t think it crosses the line

Nature of the copyrighted work: This is not documentation or news footage; it is not the case that the nature of the work is special in a way that would allow it to be meaningfully protected under fair use, as far as I can tell.

Amount and substantiality: This is the real question, and varies a lot depending on the video. For the Burj Khalifa video, there’s only ~2 seconds of video, and I would argue it could be reasonably said that this isn’t the substantive portion of the work. However, for the Hawk vs. Drone video (which is mine), the 12 seconds of selected video very much are the substantive portions of the video — in particular, that specific chunk of the video was exactly the same 14 second chunk of video that a national Nightly News station paid $850 for a license to. Since the Grand Upright Music, Ltd. v. Warner Bros. Records Inc. case in 1991, there has been a massive move towards licensing for music sampling, and this fits a similar role: I think that for some of these videos, there is clear evidence that the amount and substantiality prong is failed by this video.

Effect upon work’s value: I think this one is probably the one that this video succeeds in the most. Specifically, I think that this compilation video does not infringe upon the value of any of the works it has taken from in a serious way. In particular, I don’t believe that anyone who is a likely viewer of my video would say “Oh, I’ve seen the hawk video as part of this compilation; now I’m done.” That said, I think this prong could be minimized even further by including links back to the source videos in the comments: If the creator had done that, I think that the expectation would be that the overall effect would be even less substantial. The most significant video might even be mine — it is one of the longer clips, and it is certainly one that contains the most of the relevant content. (The original source video was only 30 seconds long in the first place — and the other 18 seconds were pretty boring.) With that being the case, I think that it could be argued that this video does not have a meaningful effect upon the work’s value, and might be acceptable under fair use for that reason.

Taken in sum, I think that the video is right on the edge of a Fair Use claim; I think that this could be further improved by the creator by removing ads, and linking to source videos in the description of the video. As is, I have personally considered and rejected the idea of submitting a DMCA claim for my copyrighted content which is contained in this video, because I think that content of this type should be allowed under fair use, even if our current courts would take a protectionist view towards this content in particular. Compilations like this rarely detract from the value of the original work: they create new value in their combination.

Of course, reasonable people may disagree on several of these points.

FAA Enforcement Could Cost You

The FAA’s ability to enforce civil penalties against safe flyers of UAV craft has not yet been tested in court, but don’t take this to mean that flying drones commercially is without risks: even in cases where the FAA isn’t assessing fines, handling the legal side of an FAA investigation can be a long, drawn-out process that can cost you lots in attorney’s fees.

With the advances in unmanned remote controlled craft over the past couple years, a massive industry of new photographers has sprung up. Even with a strongly stated position from the FAA that commercial flying is against Federal Aviation Regulations (FARs), there are many businesses in the US that continue to use drones for photography. Some do this knowing that there is risk, while some eschew the risk, taking the approach that safe flight is not within the realm of enforcement of the FAA for remote control model aircraft.

Even if the FAA has no power to enact penalties for a particular operation, that doesn’t mean that the flight is risk free, from a business perspective. In the ongoing case of “Michael Huerta, Administrator, Federal Aviation Administration v. Skypan International Inc.”, the FAA is pursuing Skypan for failing to meet the terms of a subpoena (original petition, exhibits). Unlike in any other case that has been publicized, however, this case is one where the FAA has no evidence that the flight was in any way unsafe, other than it took place inside New York City’s Class B airspace (approximately 5 miles from LaGuardia airport).

Over the past year, Skypan International has filed more than a half dozen motions related to responding to this subpoena, regularly producing documents that they feel are meeting the FAA’s demands, and being repeatedly called to task by the FAA for failure to meet with the demands of the subpoena. In the latest bout, the FAA claims that Skypan International has failed to the extent that they are in contempt of court for their failure to produce documents associated with the subpoena. Overall, more than a year has passed, with presumably dozens or possibly even hundreds of hours of legal time spent both on complying with, and responding to requests from, the subpoena from the FAA, from an organization which felt it was cooperating with the FAA!

As part of the continuing case, Skypan did attempt to claim that since the FAA had no regulatory power to assess the fines it was considering, it was unreasonable to request the documents as part of the subpoena; this was not successful, as the FAA (reasonably) argued that as part of an investigation, requesting documents to figure out whether anything *did* take place under their regulatory authority is part of their job.

A number of people flying under the “Fly Safe, and Keep Flying” approach. This is a reasonable approach for most people: the risk of FAA enforcement action in most cases is pretty slim. However, even if flying safely, as a commercial organization, there is still risk to any flight for commercial purposes of FAA investigation — which, even without penalties, can be a costly endeavor.

There is one upside at the moment. The pending case in DC Circuit Court against the FAA for their overly broad “Interpretation of the Special Rule for Model Aircraft” is currently held in abeyance at the request of one of the petitioners — which is itself a problem, slowing the potential response from the FAA to this legal challenge. However, as a component of the abeyance, the court has ruled that: “In the event the agency starts enforcing the rule challenged in this proceeding – “Interpretation of the Special Rule for Model Aircraft,” 79 FED. REG. 36,172 (June 25,
2014) – before considering and disposing of the comments it solicited, petitioners may request appropriate action.” (abeyance order). In short: If the FAA chooses to pursue further activities under the interpretation from June, the abeyance can be set aside, and the case against the FAA will proceed. This is hardly an ironclad agreement, but it seems unlikely that this is in the FAA’s best interest, and it is unlikely that they would choose to do so absent extreme cases.

Fly safe, but be aware: No amount of commercial flying is without risk of potential investigation from the FAA.

(One of the best sources of kibbitzing on drone-related legal topics that I have found is the UAV Legal News and Discussion Facebook group (somewhat to my surprise). In this group there are many of the experts in the field, from lawyers to journalists investigating the FAA’s actions. I highly recommend joining the group if you’re interested on keeping up with legal-related drone developments.)

Learning new things

One kind of sad thing about learning new things about software while working for Google: It is unlikely anything that you learn about specific tools outside of Google will be relevant to the tools you use for work inside, and vice versa.

In general, Google has a massive set of awesome tools for everything from deployment to monitoring; and a broad codebase with libraries that can do everything under the sun. However, that means that when you go outside of the Google environment, you’re suddenly stuck a bit out in the cold — the tools that you use inside Google can’t be used outside, so you have to have a completely separate infrastructure (both literal infrastructure, and code infrastructure).

This means that, contrary to prior experiences, where tools I learned on my own time could be useful to me at work, and vice versa, that isn’t true anymore.

In most cases, this is overall a positive thing. At the moment, I’m learning about ansible and vagrant, tools for spinning up VMs locally and provisioning them. However, my work at Google can’t really use this — Google’s toolchain and build/deployment process are totally different. In the other direction, I’ve been setting up monitoring of my services inside Google, but the massive monitoring infrastructure investment that that work is based in is completely unavailable outside of Google.

It’s not a big deal, but it does make the motivation to learn new things outside of Google a bit lower. It also likely contributes to the well-known artifacts of most people who disappear into Google no longer generating as much code outside of Google — once you live in a completely different environment which has some nice things going for it, the idea of learning new tools to replace what you’ve already got seems kinda silly, doubly so since you can’t re-use those tools for work.

On the “scale of things to be upset about at your job”, of course, this is a pretty minor complaint. It’s just something I realized while doing some experimentation with new tools: sharing my “new shiny” at work tomorrow isn’t really going to have the same effect it might have in the past.

Initial Warlords of Draenor Thoughts

Today, I started playing WoW: Warlords of Draenor.

After playing for two hours (just hitting level 91), I have to say that it certainly feels compelling: there was essentially no point in the story where I felt like the action stopped. I moved from task to task — helping defeat a massive army, unlocking some members of the Shadow Council, then moving on towards building a Garrison, and establishing a foothold in Shadowmoon Valley — without ever feeling like there was really a stopping point in the action. The quests almost never *felt* like the “Collect N items” variety — there was no point where I was running around lost in the woods looking for that one last kill.

The new quest markers — with world-visible highlighted objects to click on, instead of the old-school sparklies — was an interesting change, as is the outlining of targeted/hovered interaction targets. The Garrison building is interesting.

I will say one thing though: I still have no real clue what’s going on. Part of this comes from the way that I play — I’m pretty rarely one to read quest text, though I will typically try harder than average on my first play through. Still, even more so than usual, I’m a bit lost… like, I came through a portal, and then it was destroyed… but who are all these people who are already here? Are they just people who live here, or also people who came through the portal? Why are we setting up a base — is our goal to get back to the portal, or … what?

I think that I could make reasonable guesses at this, but as is often the case in WoW, I feel like I’m missing some key plot elements here that would explain what is actually going on.

Mechanics-wise, the Garrison functionality seems an interesting change, though I’m having a little bit of trouble following the mechanics. Marksmanship hunters seem like they’ve had a few changes, but so far the only real change I noticed is that somehow my keybinds got reset, so most of my buttons didn’t work; since I was kitted out in reasonably high-level gear before the expansion, killing things is mostly too easy to care about hitting multiple buttons, so I’ll have to learn those things at some point, but I haven’t yet. Also, I think my health is a lot smaller now? And the amount I hit for? But I assume that those are side effects of some kind of numbers change, and not based on any real relative performance change. (I heard an ‘ilevel squish’ was coming, but I really have no idea what happened, since I never paid attention.)

Anyway, I think it’s interesting enough to keep playing, but I wanted to write down my initial thoughts in the meantime.

Net Neutrality: What it Needs to be About

I recently read through a long series of comments on Net Neutrality between two people at different ends of the political spectrum — most of them about traffic shaping, which is where a lot of the net neutrality debate has centered over the past year. I wrote a long response about traffic shaping policies, but as I wrote it, I realized that Net Neutrality really shouldn’t be about whether the next startup has to pay Comcast a fee to get the same network access that Netflix gets today. It needs to be about protecting US internet users from harmful practices by monopolistic ISPs. Here’s my thoughts:

Sufficient competition in the ISP space would be awesome. There isn’t sufficient competition in the US. The definition of broadband doesn’t include an explicit definition of latency, but the notion that satellite provides a competitive experience in a world where milliseconds of latency make drastic amounts of difference in user experience is silly. The lack of competition in the US ISP market is absolutely what creates the need for alternative approaches to protecting consumers from what are essentially monopoly providers.

If you don’t accept that for most users, the ISP choice is “Whoever happens to serve where I live”, then of course strict regulation under something like Title II/common carrier status seems unreasonable. Let’s take it as a given that 30% of the country has only one provider which meets the government mandated description of “broadband”, and ignore the traffic shaping arguments for the time being, because that’s not the most important thing out there. Instead, let’s look at some other practices that Title II could regulate, under the notion of ensuring ISPs don’t “make any unjust or unreasonable discrimination in charges, practices, classifications, regulations, facilities, or services.”

Injecting ads into content streams at the network level (Ars Technica on Comcast Javascript Injection)
Tracking users by injecting cookies into their HTTP streams at the network level. (EFF on Verizon’s Cookie Injection)
ISPs blocking content for content reasons (porn filters, for example): As it stands today, there is no reason that ISPs couldn’t decide to implement content quality standards — which may not match those of their captive customers. (UK politicians push for mandatory porn filter — though this isn’t a corporation, it’s clearly demonstrating the potential for competing interests to consider blocking internet content to be a good thing.)
ISPs breaking the internet by replacing no-such-domain responses with links to their own, ad-filled pages (Verizon’s DNS Assistant)

If there was open competition, all of these would theoretically be fixed by “the market”. But “the market” has left 30% of Americans with only one fixed broadband internet option (BGR story on FCC internet report) - and 67% with two or fewer; a duopoly of two major providers, oftentimes both partaking of the same borderline illegal practices.

Preventing ISPs from blocking content was deemed by the court to be outside the scope of Sec 706 powers for the FCC. The notion that it could then act on these other practices which impact user experience seems unlikely to me. With that in mind, I would say that there are much more important arguments than protecting startups from being unable to compete due to traffic shaping policies: Instead, the importance is the protecting of free speech and communication over the internet, unimpeded by the whims of monopolistic corporations which provide the sole lifeline of so much of the world’s information to so many users.

Traffic shaping and how it shapes up is certainly of personal interest to me, because a chunk of my job is monitoring traffic shaping across a big hunk of the content that flows over the internet. It’s just not the most concerning thing that ISPs are doing in the absence of strict regulation and a true competitive marketplace.

Hawk attacks Drone: The Hype Dies Down

It seems like my 15 minutes of fame are mostly coming to a close. I’m still getting some 6000 hits/hour, but it’s nowhere near what it was — it peaked at over 130000 hits/hour right after the Daily Mail published their article. All of the other metrics have died down as well — Twitter posts, news articles, textual abuse at the hands of Joe Rogan and Anthony Cumia fans…

As some of you have seen, it’s my intent to donate advertising revenues from the video on YouTube to the Massachusetts Audubon Society. (Just seemed like a good cause to me; I don’t generally believe in advertising for padding my own pockets, but I’m happy to use advertising for good — and frankly, I’ve always wondered how advertising on YouTube worked, so this seemed like as good of an opportunity as any.) As you may have seen in my tweets, a couple of bogus ContentID claims have resulted in some of that revenue/earnings being missed out on: when a video is being disputed under ContentID, *nobody* can run ads, which means that the revenue is lost.

All the stats below are based on rough estimates of views from the YouTube views analytics dashboard combined with a timeline of activity; none are based on any AdSense data.

A brief summary of my monetization history:

First million views: Only display ads in-page, starting around 100k views. These are the ads on the upper right corner of the page. Since much of my content was in embeds, rather than visits directly to YouTube, these were not particularly lucrative, but due to the overall popularity, I did have something like 800k monetized visits on the watch page (this excludes around another 250k of non-monetized visits due to ContentID.) These ads are the only ads that have been on for the majority of the time.
At 1 million views, I enabled overlay ads. In total, I believe that I have had something like 1.5M monetized views with overlay ads enabled. (Another 800k have not been monetized due to ContentID claims.) Towards the end of this (at around 1.8M) there was a content ID claim, which was later cleared up.
At 2.2 million views (after my first ContentID dispute was resolved), I enabled Trueview ads. There have been approximately 600k monetized playbacks with TrueView enabled.

I don’t actually have any real earnings statistics yet, because those things can take up to 7 days to process. (I also don’t know if I could share them if I did have them; a quick read of the ToS suggests probably not.) A quick Google search points to this article on YouTube CPM, but explains that it varies significantly depending on what your ratios are of videos getting pre-rolls. (With the vast number of embeds, and short content, I think it’s probably fair to think I’ll have a lower TrueView/skippable ad ratio than most videos would, but again — no numbers available yet.)

This is by far the most common question I get about my experience: “How much money are you making?” With the numbers I see reported on the internet, I think the answer is that you can expect to see about $2 in revenue for every 1000 views. (55% goes to you, 45% goes to YouTube.) So if I had been running ads on my channel for the entire duration, and had had no ContentID claims, it would be approximately $3500 in ad revenue thus far. With the actual reality of the situation, I’m probably significantly less than that, due to not advertising aggressively at the start, the type of video I actually published, and other issues.

Overall, this is just one component of what has been an eye-opening, thrilling, and exhausting experience. I’ll be doing more write-ups about how things have gone over the coming days and weeks — I can certainly say I’ve learned a lot about what having a viral video looks like, and as might be clear from the opening paragraph, it hasn’t all been sunshine and roses.

Ask the Right Questions

If you’re about to embark on a requirements drill or needs assessment focused on “web GIS,” it is important to be sure to answer one question as you proceed: Do you actually need any specialized mapping server at all? If … Continue reading →

Hawk Attacks Quadcopter!

This morning, while out flying, a hawk attacked my quadcopter, while flying along the Charles River, in Cambridge.

You can see the video — one of my most popular ever — on YouTube:

Of course, because it’s popular, it also means you can see it in a lot of other places — LiveLeak being a ‘primary’ source of the copyright infringement. (I expect this is LiveLeak’s fault as much as it is YouTube’s fault when random people upload copyrighted crap.) I did find that on LiveLeak, the video claimed to have over 9000 views, compared to the 2000 or so on YouTube.

I’m getting a quick lesson in what happens when you create really popular content: People steal it left and right. I’ve already filed two copyright claims against people who reuploaded to YouTube and put ads on it — I guess I’ll also be learning about YouTube’s copyright infringement resolution process.

In the mean time, if you want to check it out, I encourage you to share, like or watch on YouTube, rather than on one of the many clones out there There’s only one real thing!

Social Networks and Business Plans

Like everyone on the internet, I’ve seen a lot about Ello in the last week or so. While I’m not convinced Ello is the next big thing, more recently, there have been articles about how Ello must be planning to sell you out, because their proposed business model can never work, and all Venture Capitalists require an exit strategy. Regardless of how true the latter may be, I am not convinced the former is true at all.

My initial forays into the online world were based on GeoCities and Tripod, like many other people of my generation. In my transition to college, LiveJournal became my home on the internet. It was my first work with an open source project. It was where I made friends, and it was even the website where I met my wife. It was also a website which was run, for years, based on a funding model which was entirely ad-free, at a time when banner ads were the way of the internet.

When the website started, in the early 2000s, “No ads, ever” was the mantra of the site (like Ello). The site was originally invite-based, so that growth was somewhat limited (like Ello). The site didn’t collect and sell your information to advertisers (like Ello). The site was funded by users who paid for additional features (like Ello); for LiveJournal, features included things like more user pictures, the ability to make posts by making a phone call, domain forwarding, advanced customization options for look and feel.

LiveJournal functioned as a business this way for a number of years; from at least 2002 - 2005, when it was bought by Six Apart, LiveJournal seemed like a functional business from the outside. It was a small business run by a small number of employees and supported by a dedicated volunteer base who worked to run areas like user support. There was enough of a business here to result in a sale to Six Apart in 2005; while no details of the deal were ever published, it seems reasonable to assume that it was considered to be a viable business at the time of the sale.

Now, LiveJournal was never started to be a business. It was started as a way for the creator to keep in touch with his friends. It was run as a semi-business, but as with many things started by people in their idealistic years during and shortly after college, sometimes they lose the ability to maintain the dedicated interest necessary to keep them going. (See also: Most of my early software.) After the sale to Six Apart, the “No ads on LiveJournal” policy slipped somewhat, and a number of social shifts caused a bit of a fall from grace in the somewhat utopian ideals that LiveJournal had. (Not the least among them that ads probably became significantly more profitable and effective…)

But LiveJournal isn’t the only social network that had this policy. After LiveJournal’s sale to SUP, some of the volunteers from LiveJournal decided that the things LiveJournal stood for were good, and that the system it had was workable, but it needed a bit more realistic business approach, and started Dreamwidth; like LiveJournal, the site is funded through people who purchase additional features for their accounts, rather than advertising. (One of the site’s Guiding Principles is “We won’t accept or display third-party advertising on our service, whether text-based or banner ads. We are personally and ideologically against displaying advertising on a community-based service.”)

Dreamwidth was founded in 2008, and opened to the public in 2009; it started with invite codes and later was able to move away from them. The site has more than 2M registered accounts, and although it’s not going to be the next Facebook, it’s probably reasonable to assume that the site isn’t losing money hand over fist. (It has been around for 5 years, and shows no signs of unhealthiness that I can see from the outside, though I have no inside knowledge.) It is funded by people who purchase additional features for their accounts.

The idea of free accounts being paid for by people who want additional features is not new. The claim from some that “…no one has ever tried it as a central business model, at least not in social” is clearly false. Some people have tried it. It has even, to some extent, been successful. And although it may be that Ello is not planning to do what they say they’re going to is possible, it seems entirely more likely that Ello is trying to follow in the footsteps of those who have gone before it and created social networks that millions called home in the earlier days of the internet.

If what Ello wants to make is a “sustainable business”, as they’ve claimed, then there is no reason to think that they can’t do it by following exactly the funding model they have proposed. I hope all goes well for them, and that they’re able to hold onto those principles. If they’re not, and you’re still looking for that ad-free, friendly environment that you miss from the earlier internet… there’s always the comfort of Dreamwidth.

Creating Sculptures of the World with Computers and Math

The world around us is a complex place. Sometimes you just want to hold a tiny piece of it in your hand — and with some relatively low cost technological investment, you can do so. Using a $500 quadcopter, I have successfully captured images of a building, converted those images to a 3D model, and 3D printed that model — creating a small model of Cambridge City Hall that I can hold in my hand. The process requires no special skills — just some financial investment and time.

In March of this year, I purchased a Phantom FC40, a $500 everything-you-need quadcopter. This device is easy to fly, comes with a built-in GPS, on-board camera (with a mount for a GoPro), and a remote — everything you need to start doing some amateur aerial photography. (You can see some of my videos in the FC40 Videos and One Minute Onboard to see some of the aerial photography I’ve done.)

Capturing Photos

With quadcopter in hand, this weekend, I ventured to Cambridge City Hall. While there, despite the gusty winds, I captured approximately 20 minutes of video, attempting to film the building from as many angles as possible.[1] I was using the GoPro Hero 3+ Black I recently got, but for the purposes of this excercise, the FC40 camera would probably have been sufficient. I shot most footage in Narrow or Medium mode, to reduce the fisheye effect of the very wide angle GoPro lens; for the one section of video I shot in wide-angle, I removed the wide angle aspect using GoPro Studio before using the video.

Once I had the videos, I reviewed them, doing manual frame-grabs from the video to get coverage. On average, I took one shot for about every two seconds of usable video. (Usable video excludes video where the quadcopter is taking off, where it is facing the wrong direction, where it is flying to get to a different part of the building, where it is occluded by trees, etc.) Another option would be to simply use a program like ffmpeg to extract one frame every second:

ffmpeg -i ~/Documents/input-movie.mp4 -r 1 -f image2 ~/output/project%03d.jpg

The reasons not to do this are:

When flying the quadcopter, some portions (even in a sub-second window) are better than others. Motion blur is a non-trivial problem, even with 60fps capture rates; targeting manual screengrabs at slower motion, or during a more steady period makes a small but noticable difference.
Many of the shots were in the exact same coverage — largely due to the available landing space being all in front of the building. This means that extracting regular shots would have extracted many very very similar images, which would have increased processing time without noticably increasing quality of results.

Instead, I simply opened each video in VLC, and snapshotted the images that seemed to improve coverage of the building. (Option-Command-S on Mac; in the Video menu.)

Building the Model

Once done with this, I loaded the images into a program called PhotoScan, the workhorse of this operation.

PhotoScan is an amazing tool. I say this, having tried a number of other tools — including commercial products like Autodesk’s 123d Catch and open source tools like VisualSFM. Nothing combined the ease of use and functional output of PhotoScan by a long shot. I’m currently using PhotoScan in 30 day trial mode, but despite the relatively steep price tag ($179 for single-user ’standard’ license) for what is only a hobby, I’m pretty well convinced I’m going to have to buy it, because the results are simply amazing.

With my 328 photos in hand, I added them to a chunk of a PhotoScan workspace, and set up a Batch Process (Workflow -> Batch Process).

Job Type: Align Photos. Change Point Limit to 5000, due to relatively small image size (1920 x 1080); further experiments show that this number ends up creating a better model than either 10000 or 20000 points, in a significantly shorter time window.)
Job Type: Build Dense Cloud.
Job Type: Build Mesh. Ensure that the Source Data is “Dense Cloud”.
Job Type: Build Texture

Kicking off the build for these 328 photos uses all of the CPU on my laptop for approximately 1 hour. The majority of this time is spent matching photos via the “Align Photos” step. (An attempt with 20000 points took about 4 hours instead of just one.)

This produces a textured model, fully visible in 3D. In this particular case, anything other than City Hall is pretty … ‘melty’, as I like to call it, since it was only captured incidental to the primary flight objective (city hall itself). From here, you can save the model as a .obj file to use in your favorite 3d program. You can also share it via the web: once exported as a .obj, you can zip the resulting files (including the texture) up, and share for free on Sketchfab: Cambridge City Hall on Sketchfab.

My final goal is a physical version of the centerpiece of this model: City Hall. To achieve this, my next step is Meshlab. Meshlab can open the “Wavefront Object (.obj)” file I saved from Photoscan without a problem. Using the “Select Vertices” tool and the “Delete Vertices” tools, I am able to remove the extraneous parts of the model, leaving behind only City Hall itself. Using the “Export Mesh As” functionality, I can export this as a .stl file — the file format that my 3D printer uses.[2]

Printing the Model - aka ‘hacking it to work’

The next step is to load up the STL file. Since I don’t actually know how to rotate my model, I’ll load it into Repetier-Host, so I can do rotation in my plating process. Playing around with the angles, I take my STL file, and find that a rotation of 204 degrees in the X direction, -5 degrees in the Y direction, and -15 in the Z direction appears to give me a reasonably sane looking model. However, it’s still floating a bit above the bottom, thanks to a small portion of the model that is particularly warped due to low photo coverage. I choose to slice the model anyway, using Slic3r to generate gcode.

As expected, the model has generated some pretty bogus first couple layers. However, judicious use of copy paste can help me: Using the Repetier jump-to-layer buttons, I remove the first 3 layers of the model, then duplicate the g-code for the 5th layer (The first ‘real’ layer with more than a few spots of actual content), replacing the Z index with the correct height for the first, second, third, and fourth layers.

With these relatively minor modifications made, my model is ready to print; I copy it to my SD card, and send it off to the printer. An hour or so later, I have a 3D sculpture that matches my model pretty well.

[1] This can be a challenge in an area where your building is occluded by many trees; shooting shots from the ground can help with this, but I didn’t do any of this for this particular project.
[2] The model that I produce from Meshlab is frankly pretty crappy. A lot of people with experience in this space could probably trivially improve on what I’ve got; I just don’t know much about 3D Model work. Whenvever I open blender, I start with a cube, and end up with something that looks more like a many-tentacled one of Lovecraft’s imagining than reality. As such, the 3d printing process can be a bit … fraught.

File Geodatabase Schema Compare Tool

In my work supporting various aspects of geospatial data modeling, I’ve spent a lot of time delving into concepts around configuration management of such data models. We’ve been able to apply a core tool set to perform various functions such … Continue reading →

New Map Layout

New GeoNames Map Interface

After many years without major changes to the GeoNames Map Interface it was time for an update. The new Map Interface makes full use of the screen for the map view. Other information is displayed on top of the map. The GeoNames gazetteer layer can now be displayed together with the wikipedia mashup. Boundingboxes and polygon boundaries are displayed where available and when associated with a GeoName feature. The bounding box is derived from the polygon information or from bounding boxes of children features. For populated places without polygon information the bounding box is derived from population number and feature code.

Thanks to Christophe for the implementation.

Thanks to our sponsors:

GeoJSON From ArcGIS Server

A while back, I posted about my desire to see GeoJSON supported as an output format from ArcGIS Server. I found myself needing that capability so I recently completed, and posted to GitHub, a first cut at a server object … Continue reading →

OGC Abandons the Web

Those are my words, not theirs. It came to light today that OGC has decided to withdraw the GeoServices REST Specification, also known as the “ESRI REST API,” as a proposed standard. I will not take up the relative merits … Continue reading →

Open-Source GIS Bootcamp at Salisbury University

Thanks to LinkedIn, I saw that Dr. Art Lembo of Salisbury (Maryland) University is leading an “Open Source/Enterprise GIS Summer Bootcamp” at the university from June 3 – 7, 2013. All of the salient details, including contact information, can be … Continue reading →

Simple Tile Viewer

We do a lot of tiles for various customers at Zekiah. Tiling is as much art as science and sometimes things go wrong so we have a range of utilities that we use to perform various kids of QA. Because … Continue reading →

When Is a GeoPortal Not a GeoPortal?

When it’s really a desktop application. Over the past few weeks, I have been reading with conflicted agreement the posts of Brian Timoney and Bill Morris about the nature of geo-portals and what they should or should not be and … Continue reading →

SharpMap 1.0 RC1 Released

Over on Google+, Diego Guidi let me know that the SharpMap 1.0 Release Candidate has been released. There was a time when I worked with, and wrote about, SharpMap a lot. During that entire time, the stable version of SharpMap … Continue reading →

Early Bird Registration for FOSS4G-NA Closes Soon

David Bitner sent out a reminder that Early Bird reagistration for the FOSS4G North Americaconference closes on 1 April 2013. After that, the price goes up by $50 US. You can register online at EventBrite. The preliminary program (PDF) for … Continue reading →

Light Housekeeping

Just a quick note to tidy up some loose ends related to recent posts… First, regarding the post ”A #LazyWeb Compendium of Python Resources for Beginners,” the University of South Florida PyBulls Python interest group, as promised, compiled a list … Continue reading →

The Best Thing I Saw at TUGIS 2013

I spent the day yesterday at Towson University attending the TUGIS 2013 conference. The new one-day format was a firehose that showcased the diversity of geospatial work occurring across the State of Maryland. The keynote by Learon Dalby was well-received … Continue reading →

Off to TUGIS

Tomorrow, I’ll be heading up to the Towson University GIS (TUGIS) conference with 500 or so of my closest Maryland geo-friends. It has been restructured into a one-day event and the program seems to be very content-rich as a result. … Continue reading →

A #LazyWeb Compendium of Python Resources for Beginners

A friend who is in the midst of a career change and moving into the GIS world asked me for some pointers to resources for getting started with Python. I threw the question out to Twitter (with a similar variation … Continue reading →

Arc2Earth: Choose Your ‘Cloud’

For various reasons, I can’t attend today’s inaugural FedGeoDay at the Woolly Mammoth Theatre in Washington, DC, though I’ll be watching the hashtag with great interest. Jack Flood of Arc2Earth, however, has already posted his slides to SlideShare: Fedgeodays 022013 … Continue reading →

Yes, You Need to Code

Over the past year, I’ve been involved in searching for GIS analysts a number of times. As a result, I’ve noticed a few patterns: There are a lot of analysts out there looking for jobs. Every time I run an … Continue reading →

Improved SRTM Digital Elevation Data

GeoNames is switching from the original SRTM data provided by NASA to the processed srtm v4.1 data provided by cgiar. The extract files now contain srtm data where available and gtopo30 elsewhere.

SRTM is digital elevation model from 56° S to 60° N in a three arcsecond (90 m) resolution. The original data provided by NASA contains small holes of no data over water bodies, mountainous regions and desertic regions. Andy Jarvis and Edward Guevara of the CIAT Agroecosystems Resilience project (CGIAR), Hannes Isaak Reuter (JRC-IES-LMNH) and Andy Nelson (JRC-IES-GEM) have further processed the original DEMs to fill in these no-data voids. This involved the production of vector contours and points, and the re-interpolation of these derived contours back into a raster DEM. The Consultative Group on International Agricultural Research (CGIAR) is a global partnership that unites organizations engaged in research for sustainable development to reduce poverty and hunger, improve human health and nutrition, and enhance ecosystem resilience.

GeoNames has been kindly granted permission to use the processed srtm data. In exchange CGIAR has been given complimentary access to the premium web services. Thanks to Andy Jarvis and all the others involved.

srtm3 elevation for N46 E8 in the Swiss Alps - void areas in red

srtm3 v4.1 N46 E8 in the Swiss Alps - no data voids filled by cgiar

Thanks to our sponsors:

South Sudan Country Codes

ISO has published the country codes for South Sudan. As expected in our blog posting it is SS. The letter codes for Sudan remain the same, whereas the numeric code for Sudan was changed from 736 to 729.

	ISO Alpha-2	ISO Alpha-3	ISO Numeric	US BGN*
Sudan	SD	SDN	729	SU
South Sudan	SS	SSD	728	OD

* US BGN=Formerly FIPS PUB 10-4

Thanks to our sponsors:

Republic of South Sudan

South Sudan became an independent state on 9 July 2011.
The new country has received geonameid 7909807. ISO has not yet assigned a country code and we temporarily use the country code XS. It will be changed to the official code as soon as ISO assigns a code to the new country. The ISO code is expected to be SS. The Republic of South Sudan is not the only country in the GeoNames database with a temporary country code. The other temporary country code XK stands for Kosovo.
The features of South Sudan have been updated with the temporary country
code.
Sudan had 25 ADM1 before the independence of the south. 15 ADM1 remain
with Sudan, 10 states are now the ADM1 of South Sudan which formerly composed the provinces of Equatoria (Central Equatoria, Eastern Equatoria, and Western Equatoria); Bahr el Ghazal (Northern Bahr el Ghazal, Western Bahr el Ghazal, Lakes, and Warrap); and Upper Nile (Jonglei, Unity, and Upper Nile).

Simplify SOE Deployment with Self Registration

Registering SOE’s can be a pain, but there is an easier way. Esri suggests having a separate exe that registers the dll, but this needs to be done after running Regasm on it. On top of that, if you’re running Win7, and right click on a .bat file and choose to “run as Administrator”, the […]

Network Trace Add-in for ArcGIS Silverlight Viewer

Today, without any coding, I created this web mapping app that performs network tracing. (Edit: to activate the trace toolbar, click the far right button on the main toolbar). OK, I lie. I did spend a bit of time coding over the past couple of weeks, but the resulting tools: an Add-in for the ArcGIS […]

Historical place names

Improving the way GeoNames handles historical names is a popular feature request. GeoNames is now beginning to address this question.

There are two new flags in the alternate name edit tool:
isHistoric for names of the past that are no longer used.
isColloquial for slang and colloquial names

At the recent Annual Meeting of the Association of American Geographers is was discussed how GeoNames could be extended to improve support for the timeline. Some attributes under consideration:
– fromPeriod (date, year, decade, century or period of usage)
– toPeriod
– source (book or map where the name is used), publication date of the source
– notes

The discussion for these fields is still ongoing and any feedback is welcome.
The two new flags isColloquial and isHistoric are not yet included in the daily extract as we will probably add some more attributes and don’t want to change the extract files too often. Each change in the extact files breaks the import scripts of users. We will extract the records with the two new flags in separate files as a temporary workaround till everything is stable.

Wikipedia Web Services

Over the last couple of weeks a new data extract for the wikipedia web services was implemented and deployed. The major change is certainly the dramatically increased number of geo located wikipedia articles.

A new attribute ‘rank‘ has been added to the xml and json responses. It gives an indication of the popularity or relevancy of an article. The rank is an integer number from ‘1‘ for the least popular articles to ‘100‘ for the most popular articles. It is calculated from the number of links pointing to an article and the article length. The articles are more or less evenly distributed over the 100 ranks.

The ‘elevation‘ field is now filled for nearly all articles, where no elevation could be parsed from the article itself it was enhanced with a reverse geocoded value from srtm3 or aster. The ‘countryCode‘ coverage has also been improved. The attributes ‘population‘ and ‘elevation‘ are no longer set to ‘0’ for unknown values, they are left empty instead.

Indoor GIS with Wp7

Professor Mike Goodchild says the average American only spends 13 percent of the time outdoors. GIS-based services help us find restaurants and hotels, but they offer almost no support for navigating in the complex indoor spaces of shopping centers, hospitals, mines, or airports. I still get disoriented at the Dev summit. I’d like to be […]

Wp7 Photosynth

Michael King, the Gartner keynote presenter at the Dev Summit this morning, mentioned a good use case for a mobile augmented reality app. He described a worker using a mobile device to retrieve annotated virtual reality images indicating what steam fitting to mark up. That brought to mind the photosynth of GasWorks park in Seattle. […]

Iphone vs Android Web Service Usage

Did you ever wonder where GeoNames webservices are called from? We have analyzed 470M reverse geocoding calls from January 2011 and created some heat maps.

reverse geocoding requests in January 2011

Smartphones make about half of the calls. While Android with 150M reverse geocoding calls is leading ahead of the iPhone with 100M calls, the iPhone covers a larger area.

Android

iPhone

Comparing Android vs iPhone we have found that Andoid is leading in Northern Europe, the United States and India. The iPhone is extremly popular on the Arabian peninsula and many megacities.

Android (green) vs iPhone (red)

Thanks to our sponsors:

Crowdsourcing the Certification process

Even though Clay Shirky is making money off his book Cognitive Surplus – Creativity and Generosity in a Connected Age, it is still a good read. He describes how Television in the 90’s did the job that gin performed in London in the 1740’s. He then describes how online communities collaborate to do jobs once […]

Application Identification for free GeoNames Web Services

To cope with the increasing number of smartphones GeoNames has to introduce an application tracking for the free web services. The current system of limiting credit usage per IP address is replaced by a system that tracks usage per application.
GeoNames is now serving over 25 million requests daily, half of which originate from smartphones.

free web service user agents, January 2011

The new domain api.geonames.org will now host the free web services. It requires that the mandatory parameter username= be appended to all requests. You can register a username here.
The domain ws.geonames.org domain will remain operational for a grace period to allow applications to update to the new domain.
We know it is a hassle to have to change the call, but the load and performance problems on the free servers have become unacceptable and need to be addressed.

In order to improve the performance on the free servers we had to reduce the radius upper limit for the findNearbyPostalCodes and the findNearbyWikipedia services. The default radius was also reduced.

Despite the new system we are still looking for Sponsors to help cope with the costs of running the free services. The more sponsors the more requests can be handled.

Thanks to our sponsors:

Visualizations

Stephen Von Worley from datapointed.net has published some beautiful visualizations of the GeoNames database. Each of the 7.5-million geographic features is represented by a single dot, colored additively: blue for water, green for land, and red for manmade structures.

datapointed.net

Also interesting to note, besides their pure beauty, are some artifacts (for instance a band in Ireland with lower feature density) clearly visible on the visualizations. Some of them (like the US-Canadian border) are caused by different data providers (usgs vs geobase) and therefore different feature density or feature type assignment. Others are caused by the data entry process itself. A lot of GeoNames data is coming from paper maps and you can see in some areas the form of the original maps used as data source.

New Countries: Curaçao, Sint Maarten and “Bonaire, Saint Eustatius and Saba”

Three new countries came into being after the dissolution of the Netherlands Antilles in October 2010. Curaçao and Sint Maarten became countries within the Kingdom of the Netherlands whereas Bonaire, Saba, and Sint Eustatius became special municipalities of the Netherlands proper. ISO assigned the code BQ to the three BES islands, which results in the following table:

	ISO2	ISO3	ISO#	FIPS	CURR
Bonaire, Saint Eustatius and Saba	BQ	BES	535	NL	USD
Curaçao	CW	CUW	531	UC	ANG/CMG
Sint Maarten	SX	SXM	534	NN	ANG/CMG

The island of Saint Martin is divided into the French northern part (MF) and the Dutch southern part (SX). The French northern part seceded from Guadeloupe in 2007.

Note: The newly assigned ISO code ‘BQ’ may cause some issues as it was already in use in the past. Till 1979 it referred to the British Antarctic Territory (codes BQ, ATB) before it was merged with Antarctica (AQ).

RFID+Geospatial Relief for Haiti

I recommend FRONTLINE’s Quake documentary which aired last night. They examined a lot of the issues surrounding distribution of supplies following the earthquake in Haiti. They showed supplies piling up at the airport, without any system in place to distribute it. They showed a doctor rushing all over town searching for blood, only to find […]

Time to move on

noreply@blogger.com (Steven)

This is my last post using Blogger.

Thank you Google you have helped me to get GIScussions read by quite a lot of people and to be honest I have nothing but good words to say about Blogger and I would recommend you to anyone getting a blog started. But now is the time for me to move GIScussions to its new home at

Click on the logo to have a look at GIScussions' new home. It will take me a while to work out how to make the most of WordPress so be patient and if you have advice don't hesitate to leave a comment.

Differing views on the OS consultation

noreply@blogger.com (Steven)

I have been following some of the responses to the CLG consultation on freeing OS data and future business models. Some of the reactions to other peoples' responses are almost more interesting than the actual responses.

Before commenting let me be open and say that I chose not to respond to the public consultation for a several reasons - I had already been part of the first phase pre consultation so I had said my bit and I just did not have the time to do any more.

It seems to me that many of the the responses split into a few groups:

Those who say give me as much free data as possible now and fund it through taxes (or even don't bother me with the technicalities of how it is funded) but basically keep OS going as the data collection agency.
Those who say that there is a whole business ecostructure built around OS and that the consultation has been too superficial/quick/limited to find a better model. Oh and perhaps the damage to existing businesses might be larger than the gains from these thrusting newcos that will flourish and innovate on the back of this free data
Those who have a long standing sense of injustice in their brushes with OS and want to see OS dismembered.
Those who are more concerned with derived data than the free availability of a few mid scale data sets
Bits of 1-4 above

I am paraphrasing liberally with tongue firmly in cheek before you start fuming at me. With the exception of 3 I have some sympathy with all of the above views.

More importantly though, if the consultation is to be taken at all seriously I believe that anyone who contributed has a right to express an opinion and for it to be factored into the process and decisions. The responses on twitter and the FOD blog to the letter from the MD's of ESRI, Intergraph, 1Spatial and Cadcorp seems to me to go a bit over the top in deconstructing and critiquing their views. They are expressing a valid concern on behalf of businesses that employ several hundred people that needs to be balanced against social and economic gains that may arise from the changes.

If you were one of the organisations that posted a response that questioned the haste of the process (or any other response) perhaps you would like to let Charles Arthur have a link that he can add to the growing list at Free Our Data and on Ed Parsons' site - the more the merrier.

I don't know how you balance the range of views from individuals and numerous different sized organisations in the public and private sector responding to the consultation. Whatever is decided (and many think the decision is already pretty much made) you can be sure that a lot of people will be unhappy (put the thousand odd people working at OS in that list for a start).

I have a feeling that in a few years time we might just look back on this process and say "if only ...." Babies and bathwater, broken eggs and omelettes come to mind.

So when the decision is announced (and expect it to not be as clear cut as many fear or hope) expect the debate continue. I very much doubt that this will all be over on April Fools Day.

STOP PRESS - you might want to start by reading James Cutler's megarant - hope he does this as a georant at AGI Soapbox this year.

A few BHP's on the Horizon

noreply@blogger.com (Steven)

Fridays have been fun recently, last week wherecamp eu and this week the Horizon theme day in Nottingham. A few of last week's wherecamp crowd were there in cluding Gary Gale, Ed Parsons and Muki Haklay

Horizon is one of the hubs of the Digital Economy Research program

"Horizon will focus on the role of ‘always on, always with you’ ubiquitous computing technology in the Digital Economy.
Building on the Digital Britain plan, Horizon will investigate the technical developments needed if electronic information is to be controlled, managed and harnessed — for example, to develop new products and services — for societal benefit."

There is lot of interdisciplinary research going on here and some exciting ideas bubbling away in the program. I had a lot of fun talking to linguistics and ethics doctorate students on the Horizon program a few months ago and learnt a lot from them (apparently some of them even got a few ideas from me)

Some disjointed observations from my scribblings today:

There are three sectors that they are focussed on - Creative Industries, Transport and Service Industries and three challenges that they want to address - Innovation, Human Factors/Interfaces and Infrastructure. I like the idea that human factors and infrastructure overlap.

With RTK it is now possible to get down to 1mm accuracy with professional GPS, research is now underway to make this level of accuracy available on mobile devices. I don't know why we need that level of accuracy. A few tiresome comments about mobile services being find your nearest Starbucks prompted Gary to have a little rant (he seems to hate 'bucks) but we'll forgive the academics for their quirky sense of humour (I know I am humorously challenged at times). Some interesting thoughts on internal positioning (within buildings) and the need for positioning without infrastructure (need to revisit this and my old article on vector one)

After a lot of slides up comes the guy in jeans with the mac to do a demo of a car sharing application that they knocked together in 6 weeks - "contextual computing and socially mediated real time car sharing" which in English meant hooking up with people at a meeting or in your office to share journeys. Once you understand the challenges in social, human factors and technology plus the combnination of routing and preference/choice algorithms you realise that this is one of those BHP's (the best acronym of the day, read to the end for explanation). And ignoring all advice about doing live demos, particularly of a proof of concept we get a group of people using web browsers, iphones and SMS all joining a car sharing scheme as passengers or drivers and getting routes and pairings suggested. Neat.

The sessions on privacy were some of the best discussions of the subject that I have heard in the last year. I know the cat is out f the bag theory has many adherents but it doesn't cut it for me and this was some sensible thought about trust, and reasons for sharing your personal data and issues about who controls it and ..... I hope that Jeremy Morley gets the slides up on slideshare soon because they really are worth reading through. The tech solution proposed "Personal Data Stores" in the cloud sounded like a very duff idea to me but what do I know?

After lunch we pondered whether Crowds can be authoritative? Muki Haklay updated his research on the coverage and accuracy of Open Street Map with some good motivational research based on a survey. I still have doubts about whether a map that has gaps really is good enough but then if you aren't interested in Newcastle (where apparently coverage is limited) who cares? I wonder whether people will still be contributing in 5 or 10 years or whether we will have found less human intensive ways to gather and maintain the data. Glen Hart of OS finished the session with a somewhat tongue in cheek view of crowd sourcing and its relevance to OS - bottom line is yes but you need to understand what you are getting (I agree with that).

A good agenda, lots of interesting conversations. I hope there are more Horizon events soon.

Oh in case you wanted to know a BHP is a

Bloody Hard Problem - I might be using that one a few times now I know it.

Goes well with a SLAGIATT (answers to @stevenfeldman or via comments here)

MIX10 Notes

I’ve been watching some of the live video feed from Microsoft’s MIX10 presentations. I found these interesting: The Open Data specification looks promising. Netflix open data catalog was used for demonstration. I wish Netflix provided a way to query popularity rank by zipcode for a particular title. This would make it easy to make a […]

Info Porn

noreply@blogger.com (Steven)

By far the most uplifting session that I attended at wherecamp eu was Hal Bertram from ito world talking about his work visualising Open Street Map and transportation data. People sometimes dismiss the geoweb as "only visualisation" well this is visualisation as art and there is no "only" about it.

Info porn? You work it out. Not my phrase, Hal's

Just sit back and enjoy (next time I will take my gorillapod with me so there will be a bit less camera shake and no heads interrupting your view)

Recharging my geobatteries at #wherecampeu - an unconference

noreply@blogger.com (Steven)

Yesterday I was fortunate to have a ticket for wherecamp eu. I say fortunate because this was a free unconference and tickets, which were released in blocks over a few weeks, were sold out within a couple of hours of release. Hardly surprising really if you knew what to expect but I didn't. This was my first unconference and I was a bit puzzled as to how this loosely structured event would work.

After 3 years of chairing GeoCommunity I thought I knew a bit about organising a geoconference - at an unconference there is no programme, you just turn up and stick your name and topic on the wall to show that you have something to say or to talk about and people turn up and join in. Some slides but way less than in the death by PowerPoint or drowning in Keynote days that we have all suffered silently. Spontaneity, improvisation and participation seemed to characterise the day.

There were about 180 people there from across Europe (although to be honest it was largely Brits) enjoying two days of geogoodness ranging from stuff about Open Street Map, location based games strategies, some pretty philosophical stuff about sense of place and capital, map visualisation as art to a heated discussion about the value in Making Public Data Public lead by Eddie Curtis of Snowflake titled "Walking with Dinosaurs" and a presentation called "why metadata is shit" from Charles Kennelly of ESRI. Oh yes and Gary Gale of Yahoo expounded on his "theory of stuff".

I talked (no slides because screen wouldn't connect to my mac) about business models "Without a business model we are all FCUK'd" The basic premise was that to turn your idea into a successful business you need to know who your customers are, what they are buying from you, how much they will pay (and how much it costs you) and why they will buy from you. It was a lively session with people standing up and giving elevator pitches with feedback from the audience. One guy from Google did not see the need for giving thought to a business model he said "Why do you need money?" - turned out that he had a successful bedroom business (a mobile browser for iPhones and Androids that didn't store your history so your girl friend wouldn't know you had been watching porn) that he was running outside of his day job so maybe I had it all wrong, to be honest I didn't know there was that much demand for mobile porn.

These are the slides I would have used if the screen had worked for me

Without a business model we are all FCUK'd

View more presentations from Steven Feldman.

It was a great day and thanks and admiration go to Christopher Osborne, Gary Gale and a load of other people who got the event together. My geobatteries were recharged at the end of the day and it prompted several thoughts about the conventional approach to running a conference - GeoCommunity could borrow a bit from this unconference.

A simple bit of GeoVating, any offers of help?

noreply@blogger.com (Steven)

Louise Campbell at Goodfindr sent me a link to her Craft Nations Unite app that she initiated using Google My Maps. She seems to be getting quite a lot of craft businesses to add their info to this app.

View Craft Nations Unite Map in a larger map

The geocognoscenti may be a little dismissive at the simplicity, lack of search, linkouts to other web pages etc. But for me what is exciting about CNU is that it has been put together by someone who is not a coder but has a passion for craft and encouraging small local businesses and producers to flourish. This was the vision that we had when we launched GeoVation last year.

Anyone out there want to offer Louise a bit of help in adding functionnality and scalability to the site so that she can focus on publicity and reaching the small businesses that CNU could serve? You can find her contact details on the site.

Why customer experience matters Pls RT #custexp

noreply@blogger.com (Steven)

I just signed up for an account with ******* (name withheld, you will see why) so that I could try out their service which I had read about on my favourite social network (begins with t).

To sign up I had to provide quite a lot of information but to be fair the service has a commercial model that probably needs this information. I then waited for my account to be approved, not too long after I get my user name, some quirky meaningless combination of capitals - FAIL 1 - I want to choose my own user name or use my email address not have another one to store or remember.

Move to first try at logging in. Minimum system requirements Windows and IE - Aargh! FAIL 2

OK so that's me off (and about 35% of browser users) but why not warn me about their requirements before I filled in the application form? FAIL 3

Then I thought that if I wasn't going to be able to use their service I would prefer that they did not keep my personal details in their database. Where is the account settings or other link to manage my account and delete if I want? Nowhere (and that is not a pun on my company name) - FAIL 4

So I then sent this mail to their customer services dept:

I registered to use your services only to discover (after taking time to register) that the service only works when accessed from a Windows based machine running Internet Explorer neither of which I have or wish to acquire.

Approximately 35% of internet users now use a browser other than IE and the number of people using operating systems other than Windows is rapidly growing as well. Shame that I won't be able to find out how good your service is or view the quality of ******.

Since I am not going to be using ****** I wanted to delete my account and remove all of the contact information that I provided to you, unfortunately there is no obvious link to manage my account details or to unsubscribe. Could you remove my details from your database and confirm to me that this has been done.

Thank you

Back came this response

Good morning Mr Feldman

I can confirm that the account has been closed

Kind regards

****** *******

Senior Customer Service Executive

Am I missing something here? No apology for any inconvenience caused, no explanation or response to the points made, no regret that I wont be able to use their service or spend money with them. And this person has the title Senior Customer Service Executive! FAIL 5

I sent back this curt response

Thank you

No apology for any inconvenience or explanation of design choices

I get the message

I haven't heard anything else. I won't be using this company, nor would I recommend them to anyone that I know. FAIL 6

You may be wondering why I have not shared the name of the company, I don't want to flame them or cause them undue embarrassment (although maybe they would not care). The reason for posting this is to illustrate how easily a poor customer experience can convert a potential champion into a peeved "detractor" (that's the language of customer experience and satisfaction analysts).

Now if the detractor has a well read blog (not me) or media access or an active social network the company could find itself facing some pretty poor publicity. I don't want that to happen to these guys but I do hope that if they read this blog or a tweet about it (and they might) then they think about retraining their Senior Customer Service Executives.

If you wanted to share this story with other companies who could learn from it then the tag #custexp would enable others to follow the viral spread of the story and learn how important every customer contact is to the success of a business.

UMapper, geotagged tweets and the late Michael Foot

noreply@blogger.com (Steven)

Last week UMapper announced a new feature to their service that makes it amazingly simple to build and customise a map that can show geotagged tweets based on any search you choose.

Today I have been following the flurry of tweets about the late Michael Foot (if you are about my age he might be an old hero).

In Less than 5 minutes thanks to UMapper and Cloudmade I was able to put this together. No doubt many of you could do much more imaginative things with UMapper. If Michael Foot doesn't interest you then just change the search term in the bottom right hand corner of the map and search for tweets of your choice, or go build your own twitter map.

Interesting to see how London centric the tweets are. Pan the map or zoom out to see how the density varies.

At one level I think it is incredible that embedding an interactive map like this with social media real time feed is so simple and accessible to non technical people, who foresaw this 5 years ago or even 2 years ago? However in the same week that UMapper announced this new capability Platial one of the original darlings of community mapping and neogeography announced that they were ceasing operations, presumably because they had run out of money (apparently they had been running on volunteers and thin air for a while. One of the founders said

"Platial was a movement and it will be carried on a thousand fold. It is real and irrevocable. It put the power of maps in the hands of people and lets us see the world as an interconnected tapestry of stories and perspectives.

Sincerely, Di-Ann Eisnor"

What does that tell us? Maybe even movements need a business model.

Maybe someone out there will have a view? I feel a couple of posts on business models and sustainability coming on. More soon

LinqPad Spatial Challenge

To get up to speed with new things in C# 4.0, I’ve been reading C# 4.0 In A Nutshell by Joseph and Ben Albahari. To illustrate examples, the book uses LinqPad, a freely downloadable tool. I recommend the book, and I highly recommend LinqPad. Still, there is room for improvement with LinqPad. The author of […]

PUG Conference

Here’s a summary of my thoughts after attending Monday and (half of) Tuesday at the ESRI Petroleum User Group (PUG) conference. The Interp/GIS Gap In the Operation Database panel, Bill Burroughs described the gap between geologic Interpretation and GIS as being one of the big challenges for developers of software tools. I think a tool […]

The times they are a changin

noreply@blogger.com (Steven)

I have been looking through the feedback from the AGI GI in 2015 survey that we ran as part of the foresight study that will be published soon (just need to finish my third of the editors intro). One comment caught my attention "none of the big 3 software vendors will still be around in 2015"

Not sure who the big 3 vendors are? Well change 3 to 5 or even 8 and you have pretty much swept up all of the players in UK GI software supply. How many of these companies will be around in 5 years time?

To survive let alone flourish GI software providers are going to need to change dramatically to adapt to the challenges of new business and technical models. Some will be too encumbered by their legacies or their management. Will anyone rember them in 2016?

So who do you think will be the winners and losers?

More thoughts on this in the intro to the AGI Foresight Study in March.

- Posted using BlogPress from my iPhone

Reading the runes

noreply@blogger.com (Steven)

If you have a LinkedIn account you can get a weekly update of what your contacts are up to including new profiles, pictures and requests for references.

Sometimes you can correlate bursts of activity from multiple contacts in the same organisation with impending restructuring or downsizing (aka layoffs). Try it, you may be surprised.

Now if someone could write an app to do this on a large scale they might have a market prediction tool.

- Posted using BlogPress from my iPhone

A GeoVating good day

noreply@blogger.com (Steven)

So we finished the GeoVation Awards Programme with a cracking showcase at the wonderful RGS venue in South Ken on Tuesday evening. I got home late and had to pack to head off to for a series of meetings in another time zone first ting Wednesday so I have not had much time to gather my thoughts or write anything up.

I want to write about the whole GAP, what we achieved, what we can improve, whether it is worthwhile and how we should/could move forward but that will have to wait till I get home.

In the meantime Gary Gale has blogged on his views as a judge and shared some pics & Paul Clarke the papperazzi of the twitterati has posted some more pics

Now the hard work starts for the winning ventures and for the judges and GeoVation team who will be supporting them.

.NET Successors to VBA IDE and DOCELL are Needed

ArcGIS 10.0 (formerly known as 9.4) will be the last release to support VBA. Expect to be told to simply download Visual Studio Express for free, and create assemblies containing extensions, commands etc. that can be installed without touching the registry. Still, I find it nice to have code that travels with the mxd. Since […]

Another nut to crack

noreply@blogger.com (Steven)

Just as everyone has been getting excited/delirious/innovative over the launch of data.gov.uk (and of course they should be chuffed) a little spanner appears in the metaphorical works.

Over 2000 people signed this petition to No 10 seeking to get postcodes freed up after the Ernest Marples site was closed down and several useful services that relied upon it were forced to close. Now you might have though that after the PM himself announced in November that he was going to make Ordnance Survey midscale data freely available for use and reuse (even commercial) that post codes were a slam dunk. Here is what Gordon Brown said on 17th November:

And from April next year Ordnance Survey will open up information about administrative boundaries, postcode areas and mid-scale mapping.

No 10 has just responded to the petition on the day after opening up 2,500 data sets many of which have a geographic component and guess what?

As access to the PAF is governed under a condition of licence, Postcomm monitors its practice. Royal Mail’s licence obliges the company to make access to the PAF available on reasonable terms. Postcomm allows the company to make a reasonable specified profit margin and monitors its accounts.

Postcomm has previously undertaken a public consultation reviewing how the PAF was managed. The consultation started in 2006 and finished in 2007. Postcomm took all the diverse uses of the PAF into account before reaching its decision in 2007, announcing more safeguards for the management of the address information held in the PAF with the aim of making sure that the PAF is maintained properly and made available on fair and reasonable terms. The findings of the consultation can be found on Postcomm’s website (www.psc.gov.uk).

If any PAF user or stakeholder feels that Royal Mail is not complying with the terms of section 116 of the PSA 2000 or Condition 22 of its licence, they can either raise concerns direct with the company or with Postcomm. Postcomm would consider the merits of any such concerns in the light of its statutory duties.

I think in English that means "No, no no"

Quite how this response fits with the current spirit of openness in government and the plans to free up some of Ordnance Survey's data products after consultation is beyond me!

Perhaps Sir Tim needs to put a call into his friend Gordon to point out the inconsistency. Oh for a bit of joined up government.

UPDATE: This post on the Ernest Marples blog (which I found after writing this post) sets out very eloquently why the response is so inadequate.

GeoVating thoughts

noreply@blogger.com (Steven)

Less than a week to go to the GeoVation Awards Showcase, starting at 1pm on 26th January at the RGS in London.

There will be 9 ventures pitching. Although I did not have a vote in the judges shortlist session I found most of my favourites seemed to get the judges thumbs up and I am not saying which ones didn't.

It wouldn't be fair to mention one or two of the ventures and not others so I encourage you to pop along to the list of finalists pick your favourites and then register for the Showcase next Tuesday (there are still a few places left) and vote for your choice for the £1,000 Community Award and cheer the winners of the 3 main seed fund awards totalling £20,000.

In some regards the showcase is the beginning of the whole GeoVation program not the end. Once we have made the awards the real fun will start as we try to support the ventures and hopefully see their ideas progress and succeed. I am sure they will welcome offers of advice and help from the wider community - hint hint.

I know some of my friends in the coolest regions of the geoplanet are a bit sceptical about GeoVation, who knows you could be right but you will never find out if you don't take an afternoon to come along to the Showcase and see what people are doing.

Oh and don't forget that if you come along the good folk at MapAction will benefit to the tune of £10 per attendee and they sure need every bit of support that we can give them.

What will UK GI look like in 2015? Your opinions wanted

noreply@blogger.com (Steven)

You may recall me mentioning the AGI Foresight Study that I have been helping to edit.

We are drawing to a conclusion but before we write the final wrap we would like to gather some input from the wider community (a bit late I know but weather, holidays and technology have all conspired to delay us). So hop along to SurveyMonkey and share your wisdom with us.

The survey will be included in the editorial which will be published in February.

And if you can mention the survey to your geofriends or geotweet it that would be appreciated

Thanks

GeoVation Awards Shortlist published

noreply@blogger.com (Steven)

The judges have met (well virtually) - plans have been reviewed, combed, questioned and debated and finally we have a shortlist of ventures for the GeoVation Awards Showcase on 26th January.

You can find the 8 shortlisted ventures here

The short-listed candidates will be invited to present their plans at the GeoVation Awards Showcase, to be held at the Royal Geographical Society on 26 January. Doors open at 11:30 with a sandwich box lunch before the main event which kicks-off at 13:00. £10,000 is up for grabs for the outright winner, and £5,000 each for two runners-up. A Community Award of £1,000 will also be made, as voted for members of the audience on the day.

The GeoVation judging panel includes Steve Coast the founder of crowd-source mapping project, OpenStreetMap; James Alexander, CEO of Green Thing, the online service that encourages people to lead greener lives; James Cutler, CEO of eMapSite and Gary Gale, Director of Geo-Engineering at Yahoo.

It should be an interesting and enjoyable afternoon, so why not come along you might meet a collaborator, an investor, an employer or just a plain old friend.

Registration for the GeoVation Awards Showcase is free (but don't leave it too late as the numbers are limited) and you can also help the good folk at MapAction. GeoVation will be donating £10 to Map Action for each registered delegate who attends the showcase.

See you there hopefully

You can't teach an old dog new tricks

noreply@blogger.com (Steven)

But you can repurpose an old video clip with a new humorous set of subtitles

Enjoy

In the wonderful world of free

noreply@blogger.com (Steven)

In the wonderful world of free you pay (yes pay) London Underground to advertise a free browser. I suppose there must be some logic here, it just bypassed me.

No doubt there is some added value to getting people to click on your ad inventory in your own browser. I wonder what is hidden under the covers of Chrome.

Watch out for the adverts for Maps and mobile directions

Posted via web from Steven's posterous

Civilisation, roaming charges, GPS and holidays (not neccessarily in that order)

noreply@blogger.com (Steven)

Back from a fantastic holiday in Egypt except ....

Roaming rates in Egypt (or for that matter most places) are so extortionate that you can't use mobile internet services. I cannot see how all of the much vaunted location based services like Rummble, Yelp etc are going to reach the travellers who most need them if the cost of using them is £6/MB! There surely can't be a justification for the current level of data roaming charges, perhaps the EU needs to add this to their list of gripes with mobile operators (not that that would help in Egypt, the US etc) because there is no effective competition on international data roaming packages to my knowledge.

Planning ahead I downloaded maps of Cairo using OffMaps only to discover that for some reason the GPS on my iPhone did not work all the time I was away. Mysterious? I know that Egypt opened up to GPS receivers last year, surely they can't be blocking the GPS signal on my phone? Perhaps it isn't the GPS that is doing most of the locating on my phone but a combination of the cell tower locations and Skyhook (I doubt they are too hot in Cairo). So it was OffMaps without GPS to help me work out where we were which isn't easy when most of the street signs are only in arabic script. Good thing we had a nice old fashioned paper map with landmarks etc.

So forgetting all of that what about the civilisation bit. The Step Pyramid (or Djoser Pyramid) at Sakkara was built about 5,200 years ago.

Within a couple of hundred years the more famous pyramids of Giza were being built, the largest is over 130m high.

About 500 years later the Temple of Hatshepsut had been built

Which makes you wonder a bit about so called Western Civilisation. The only thing that I know of that is pre Roman and of any substance in the UK is Stonehenge which is (depending on who you believe) 4,500 years old which is pretty cool but sort of pales into insignificance when compared with the Egyptian stuff

So how do the pyramids fit into the same blog as a rant about data roaming rates and GPS? Not sure really but perhaps the pyramids and temples of Egypt offer a humbling alternative to my (or our) expectations of technological advancement. They managed to build them without GPS, CAD, modern surveying and materials technology but they did have one heck of a lot of people available, apparently they were paid in beer.

Happy new year. More pictures here if you are interested.

GeoSeasonal Greetings

noreply@blogger.com (Steven)

To all my geofriends and geofollowers I want to wish you geoseasonal greetings.

Whatever your faith and geoinclination (neo, paleo, proto or nono) I hope this time of year brings you and yours peace and happiness.

May 2010 be a geotastic and prosperous year for you

Thanks to Gary Gale a geolly good geogeezer for the use of his great geopic

Come to the GeoVation Awards Showcase and we will support MapAction

noreply@blogger.com (Steven)

The deadline is drawing closer for submissions for a GeoVation Award, plans are thudding onto the GeoVation virtual doormat as we speak, the judges are panting with anticipation and the award funds are burning a hole in our e-pocket!

There is still time to submit a venture plan to The GeoVation Awards Programme this blog will tell you how.

All we need now is you to come along to the GeoVation Awards Showcase at the Royal Geographical Society on the afternoon of 26th January. For every registered delegate who attends the Showcase, GeoVation will be making a £10 donation to MapAction.

So come along, grab a geosandwich, listen to some exciting and innovative prospective new ventures utilising geography, vote for the £1,000 community award which will be given to one of the venture teams at the end of the day in addition to the main awards and you will be supporting a very worthwhile geocharity. It should be a lot of geofun.

Have a look at some of the ventures on the GeoVation Challenge to get a preview of what you might see on the 26th.

Registration here is free

New Years Cheer

noreply@blogger.com (Steven)

It has been a tough year for many who work in UK Geo and uncertainty about the future will be clouding the celebrations for some as we approach the new year. So here is a bit of good news for all of us.

The Guardian is asking readers of its Technology section to pick their top technologies for the next 5 years as part of their final print edition, in the article they include the latest Gartner Technology Hype Curve. Just look at "location aware applications"

So we are pushing up the slope of enlightenment and should be mainstream within 2 to 5 years. Now will that be good news for those currently working in the industry or will there be some new entrants into the market who will reap the benefits of "enlightenment" at the expense of established players?

I can feel an end of year poll coming on if I can work out how to do it on the Blogger platform (suggestions?)

Who pays for our new pavement?

noreply@blogger.com (Steven)

It's a decade or more since our street was enhanced with the questionable wonders of cable from Telewest or whoever they were before they became Virgin.

The byproduct was our pavements being ripped and then relaid with broken uneven paving stones, patches of tarmac and shoddy workmanship. Why Haringey allowed them to get away with it I don't know. Then recently Thames have replaced the water mains and had to dig up much of the pavement to make the connections to the houses. Loads more tarmac patches.

At long last and after numerous complaints and probably a few accidents we now have a beautiful new flat pavement being laid. I wonder whether Haringey is recovering any of the cost from Virgin and Thames? Or will this be paid for from my council tax?

What's this got to do with geo? Not sure. Oh yes "someone has to pay to fix the mess"

Correction - Poscodes will not be free

noreply@blogger.com (Steven)

A case of the left and right hands not being connected.

This morning the BBC web site ran an article on the freeing up of the postcode dataset

"Currently organisations that want access to datasets that tie postcodes to physical locations cannot do so without incurring a charge.Following a brief consultation, the postcode information is set to be freed in April 2010.The announcement about releasing postcode data came as part of a much wider plan to use technology as part of the Smarter Government strategy. As part of this push, the government said it would start "consulting on making mapping and postcode datasets available for free reuse from April 2010.""

Within a couple of hours Giles Finnemore, the Head of Mrketing at the Address Management Unit of the Royal Mail sent an e-mail to all the current licensees of the PAF

Dear PAF(r) Customer

You may be aware of a story on the BBC website today that Government is planning to give anyone free access to postcode data.

Access to postcodes is already, and will continue to be, free to every citizen via www.royalmail.com/ postcodes4free.

For the avoidance of doubt PAF(r), the Postcode Address File, remains the intellectual property of Royal Mail and is supplied and used under licence. The new and recently published licences come into effect from April 2010. There are no plans for that to change.

Maintaining a world class postal address file requires significant ongoing investment and it is right that organisations who obtain value from using the file pay to do so.

We are aware of no plans for Government to pay Royal Mail for businesses and organisations to use our address file.

Regards

Giles

Seems that Giles listening to Gordon. I wonder whether he got a call from the Cabinet Office today?

Someone is going to have to do some back pedalling here.

10/12/09 Some useful comments on the Free Our Data blog following up on this

Embedding content into Google maps

noreply@blogger.com (Steven)

Have you noticed the points of interest data that is now incorporated into Google Maps when you zoom in?

I know it has been there for a while but I only recently discovered that it is clickable - could be I just missed it or it could be because the click tolerances for some of the tiny square dot icons are very small or it could be that it is one of those neat tiny changes that google just slips in from time to time (well almost every week actually). No doubt Ed Parsons if he is reading this will chip in and tell us when this feature launched.

In addition to restaurants, hotels and cinemas I found Muswell Hill railway station. Hah I thought caught you out Google! There is no Muswell Hill railway station, the line closed ages and ages ago. But when I found the click zone on the item up pops this useful bit of local history from wikipedia

Pretty neat!

Now you can see how Google will start to drive revenue from all those sites using the API to embed maps or develop applications, the adverts will be embedded in the map. Now can they make them context and user dependent?

To privatise or not to privatise, that is the question

noreply@blogger.com (Steven)

Yesterday Gordon Brown announced a "radical plan to put frontline services first by streamlining government". The headlines have focussed on his comments on the pay of senior civil service employees perhaps masking some inconsistency in the aspirations and the detail of the programme Putting the Frontline First.

"Putting the frontline first: smarter government sets out how Government will improve public service outcomes while achieving the fiscal consolidation that is vital to helping the economy grow. The plan has three central actions: to drive up standards by strengthening the role of citizens and civic society, to free up public services by recasting the relationship between the centre and the frontline, and to streamline the centre of government, saving money for sharper delivery."

So what is wrong with that you may ask? Drive up standards, shift focus to frontline and save money - sounds good but is it realistic and deliverable?

Last month the PM announced that a range of data sets from OS were going to be made freely available garnering widespread praise from all who have argued that geographic information is key to opening up public information and allowing innnovators to create new services and activists to hold public services accountable for their decisions and performance. In yesterday's announcement buried in the detail was

"We are publishing an OEP asset portfolio alongside today's report. This portfolio sets out those state-owned assets which government might seek to commercialise over the medium term. The OEP asset portfolio includes a new framework to govern which government activities should be managed as a business and which should be sold. For those activities which are best managed as businesses in the public sector, we will separate the ownership role from the customer and policy role, with a presumption that they should be incorporated"

The OEP Asset Portfolio can be found here. The introduction explains that

"The Government has today established a new policy framework – summarised in Annex A to this document – to guide decisions on how government activities will be delivered. This will facilitate clearer decision making and faster progress in improving business performance and, where appropriate, pursuing transactions"

One might think that pursuing transactions is a euphemism for privatisation or sell off. The section on the Ordnance Survey has some choice but confusing nuggets (my italics below) including

"On 17 November 2009, the Prime Minister announced that Government proposes to make certain datasets from Ordnance Survey available for free, including information about administrative boundaries, postcode areas and mid-scale mapping. There will be a public consultation on these proposals from December 2009, with implementation of any change from April 2010.

The consultation will also cover other issues around the interaction of Ordnance Survey with the market – particularly the regulatory environment and the governance structure around the free offering.

Market studies have identified significant growth opportunities across the geographic information (GI) market, as data is made more available and new technologies are used to support innovation and greater use of GI data and services. The outcome of the consultation above may affect the opportunities available to Ordnance Survey in some of these growth areas and the alternative asset options outlined below. It may also open up new opportunities to work more closely with other parts of the public sector to realise efficiency savings, for example in local government resource planning and deployment, or working more closely with the Land Registry. Similarly there may be an opportunity to collaborate with local government and Royal Mail to provide a definitive addressing solution for Great Britain."

But then at the end of the chapter under the heading of Private Sector we get

"There are a growing number of commercial market opportunities, particularly around value- added services using geospatial information, which Ordnance Survey is not currently well placed to exploit.
A private sector investor and/or partner might bring expertise, new market access or additional capital for innovation. This could accelerate the development and delivery of these opportunities more quickly and successfully than Ordnance Survey operating alone."

So does this mean that privatisation is on or off? My hunch is off (no inside knowledge though).

I wonder how much of this section on Ordnance Survey had to be rewritten after Gordon Brown's announcement on 17th November. Do the left and right hands know what the other is doing? Maybe that is why senior civil servants are getting those high salaries that Gordon is so concerned about!