.

More creative Maps

noreply@blogger.com (Andrew Zolnai) — Sat, 02 Jun 2012 16:29:00 +0000

I ran across these interesting web-mapping innovations - three on data consumption (multi-modal maps on steroids) and two on data creation (down to earth to outer space).

Planning a cross-continent trip this summer from my paents' in S France to our family@large cottage in Hungary, my wife pointed out www.viaMichelin.co.uk. I'm familiar with paper Michelin maps of yore, that marked chevrons to signal the steepness of slopes - harks back to under powered cars mid last century, but were very handy for cycling which is what I used it for as a teen - or colored lines to signal scenic routes. Not to be outdone the web version marks black spots on roads as well as hotels, in an interactive roadmap on steroids.

click image to enlarge

Luck would have it that Caitlin Dempsey @gislounge pointed out an Ancient Rome travel map from Stanford University. It calculates travel times in the Roman era across the Roman empire by season and with travel costs in the currency of the time. It's deceptively simple but entails a lot of calculations, a multimodal map on steroids if you will. I'm not sure what's more amazing, that what takes 18 hours direct today took 43 days two millennia ago, or that we can calculate that at all!

click image to enlarge

On the data creation side, Google's AtmosphereOnTour show early this week uncovered an intriguing site from the Met Office. While it does provide data online, it's notoriously hard to extract it and make good use of it. Somewhat of a turnaround is their posting of volunteer weather stations.

click image to enlarge

My friend Geo Dailey pointed out the most intriguing app for volunteers to record info on the transit of Venus. On Jun 5 and 6 next week, Venus will swing between Earth and the Sun: measuring the dark speck across the sun disk allowed early astronomers to measure the size of the solar system. This celestial conjunction doesn't happen that often as described my Matt Artz... Geo told me Captain Cook mounted an expedition in the Pacific to measure it in the 18th c.! Well then, contrast that mometous effort with perhaps thousands of volunteers measuring the transit of Venus because:

there's a free iPad app available from Astronomers without Borders and Esri
iPhones have cameras GPS and clinometers built in that allow such measuremts

Captain Cook, Samuel de Champlain and other mariners of the time would sure be surprised to see the equivalent of an astrolabe fit in the palm of your hand! Stay tuned for Geo's web map of that event...

click image to enlarge

And with the Diamond Jubilee celebrations under way in England, how could we not mention this other interactive map of the beacons lit across the country as another set of community events? This is IMHO one beautiful map too from Esri(UK).

click image to enlarge

Cloud Futures #3: Bridging the Gap

noreply@blogger.com (Andrew Zolnai) — Sun, 20 May 2012 15:35:00 +0000

Bridging the gap between desktop and on-line GIS follows the first and second instalment, online vector GIS and spatial data validation. GisCloud introduced a free Esri extension to load features and attributes to its file system. This follows other services such as Arc2Earth and Arc2Google, except in the vector domain. Having both Esri @ home and a private cloud I put this new extension through its paces.

East Anglia Fenlands mashup on arcgis.com looks like this on giscloud.com. But how do I get there? First you download and install the extension, and it's added to ArcMap for you. Pressing the single button in ArcMap pops up this screen:

click on image to enlarge

Log in via your social media or own login, select the layers to export et voilà! layers get uploaded: 9 layers at 1900 features each totalling 58Mb took 9 min. on a 1mbps up / 20mbps down ADSL line on a stock HP Pavilion g6 notebook running ArcGIS 10 for Home Use (read: ArcView with all extensions).

As with online validation, the usual lessons learned are almost as interesting as the process itself... It's so easy, we have to get something more out if it, don't we? Here are mine:

UK Ordnance survey data come in British National Grid (ORD SURV GB) with a defined datum but not ellipsoid, which posts as Transverse Mercator with unnamed ellipsoid and datum, and thus fails the GIScloud import but posts as unprojected
a simple reprojection to WGS84 in the file geodatabase fixed that and posts like so:

You can follow Socium's on-line validation output (triangles where adjacent polygons differ by more than one unit). The best part of it, however, is that it took longer to write this blogpost than it took to upload, fix and re-upload the data! How's that for ease-of-use from Esri and giscloud.com?

Petroleum GIS then and now

noreply@blogger.com (Andrew Zolnai) — Sun, 06 May 2012 17:44:00 +0000

Exprodat published a free eBook: Why use GIS in petroleum?, an excellent state-of-play as well as good industry marketing to augment their impressive blog. Does their Figure 1 not have a certain air of déjà vu, however, compared to Figure 1 of my article in CADalyst written 25 years ago?

click on image to enlarge

Those were however very different days! AutoCAD was a front-end on pre-Windows PCs just entering a Unix workstation market. Just trying to connect with Sun workstations or download data over 9600 baud modems - that's 1.2 kbps or 1000 times slower than a 1 mbps line today - to create a township grid and overlay data in prehistoric mashups for proximity analysis... And Operation Database was still two years away from stating: "Geographic information systems are infrequently used in exploration and production, which represents less than 2% of the total GIS sphere"... And the web? What web?

Mar88CADalyst

As a footnote, the PC in Figure 3 is the basis for the following comparison:

Mega giga

iPad maps

noreply@blogger.com (Andrew Zolnai) — Sat, 31 Mar 2012 16:59:00 +0000

Here is a small selection of mapping tools available on the iPad. Some are from the Appstore, others simply from the web. These are screen shots that I took for those (thanks my readers for how-to tips).

Mapbox has an iPad app downloadable for free from the Appstore. It also has a neat twist in that files like KML & TileMill and feeds like GeoRSS and GeoJSON can be loaded straight from Dropbox. This is a KML of Amundsen's dash to the South Pole I looked up on the centennial to Scott's expedition.

click image to enlarge

The same can be done with Google Earth, better rendered as it's a globe-based iPad app.

click image to enlarge

Giscloud.com on the other hand boasts that with HTML5 you need no App as it works perfectly as-is including fondleslabs. Here's a shot of my project on Medieval Fenlands that includes BGS's onshore and offshore geology as web services.

click image to enlarge

Maps I posted on arcgis.com also show well as-is, for example this real-time DIY weather map for the Middle East: post dust clouds and wind direction from NOAA to see the next storm before it comes!

click image to enlarge

I think the coolest however is the wonderful Kuwait Community Map posted by Nishant Arora from Esri's local distributor, that shows streets and buildings in glorious detail. Available on arcgis.com, I think it shows best on Esri's free iPad application.

click image to enlarge

The upshot is that while there is a variety of apps available for iPads - haven't covered paying ones or custom web maps that can be built for fondleslabs, discussed among others by Esri - web maps appear to work just as well. The advent of HTML5 will especially help render vector maps on smartphone limited bandwidth.

Roundup of web projects

noreply@blogger.com (Andrew Zolnai) — Sun, 25 Mar 2012 14:46:00 +0000

Is it spring in the air or LinkedIn's new (to me) facility to post projects? Here is a round-up of various projects in the past five years as recently posted on my LinkedIn page:

Medieval Fenlands GIS
April 2010 to March 2012
Also: Sue Oosthuizen, Frances Wilmoth
The recent release of UK Ordnance Survey OpenData opened the opportunity to post H.C. Darby's data from The Medieval Fenland and The Drainage of the Fens of East Anglia in the eastern UK. And parishes are the geographic unit that remained constant since the Middle Ages.

click image to enlarge

Captain's ship's logs for the period 1752 to 1855
July 2004 to June 2011
Also: Daniel Schober, George Dailey
This is a sample dataset off the CLIWOC database. This subset tabulates the captain's ship's logs for the period 1750 to 1850, and document the fascinating voyages of exploration in that period. Started on ArcGIS Explorer now on ArcGIS Online.

click on image to enlarge

Teapot Dome 3D GIS
September 2011 to January 2012
Also: Jeanette Buelt, Tom Anderson
Data from RMOTC located on a 10,000-acre U.S. Department of Energy site within Naval Petroleum Reserve No. 3 (NPR-3). Used target formation tops from RMOTC to try new fault gridded contouring in ArcGIS 10

click on image to enlarge

GIScloud
July 2009 to Present
Also: Dino Ravnic, Marko Santic
Early adopter of vector GIS in a personal cloud, with data uplaoded as shape files or to postGIS and even via ftp - HTML5 puts it on fondleslabs with no separate app

click on image to enlarge

Socium On-line Validation
November 2011 to January 2012
Also: Abbie Beckford, Duncan Guthrie
Early adopter of on-line validation service, to test the Medieval Fenlands mash-up of manually enhanced shape files and Ordnance Survey back drop

click on image to enlarge

Personal web maps using free tools
July 2006 to Present
Also: Christophe Staff, Jo Cook & others
Flat files posted online using Openlayers then Google Maps java scripts - updating some from v.2 to 3 Google Maps API is proving to be somewhat of a challenge

click on image to enlarge

Google Fusion Tables
October 2009 to October 2011
Also: Martin Lewis & others
Trying out novel way to map data on-line, largest number of hits on my blog with a little help from my friends

click on image to enlarge

Trending oilelefant.com
August 2009 to October 2010
Also: David Lloyd, Hussein Nasser
Trial social media marketing in aid of a web mapping start-up

click on image to enlarge

And here are a few more mashups:

NOAA dust cloud and wind speed data on arcgis.com as DIY weatherman in the Middle-East, posted once as a live banner map on this blog...

click on image to enlarge

Post-your-own joke by country (old GIS Day project)...

click on image to enlarge

Offshore maritime boundaries as areas of potential boundary dispute...

click on image to enlarge

And show my octogenerian parents an arcgis.com webmap of mine directly on skype!

click on image to enlarge

East Anglia Fenlands wrap-up

noreply@blogger.com (Andrew Zolnai) — Sun, 11 Mar 2012 09:57:00 +0000

It may be time to run an overview, two years on this personal project on East Anglia, the last step of which was reviewed by socium.co.uk:

UK Ordnance Survey releases some geo-data for public consumption
two books chronicle Fenlands geo-history since Domesday (1076)
HC Darby penned no less than 60 maps figures over 510 pages of text
notice that Parishes are a constant geo-element over almost 1000 years
so I extend O|S shapefiles with additional columns quantifying wealth
posted in UK guv and academic repository, giscloud.com and arcgis.com
then add web mapping service onshore and offshore geology shown below
and finally online validation services help quality-control various datasets

East anglia fenlands

The human element in all this are Drs. Oosthuizen and Willmoth at Cambridge University, who are indefatigably extending the necessary work to document the past and future uses of a critical area of East Anglia. They're moving to properly archive the well documented Bedford Levels Corporation: that is in itself a wonderful history of public - private partnership to resolve the critical issue of local drainage. But nowadays, not only will climate change affect the agriculture that is still key in this region, but also tourism that is seen as the next big thing in an era of staycation (staying near home for vacations).

click image to enlarge

The local element is how much one can read in maps and meeting local people on the current geo-history. I repost this intriguing map above this is a sea level rise map - it may be speculative, but mimics exactly the extend of the pre-drainage marshes - the necklace of villages, two of them with the suffix 'beach' (coastline) and Reach with the remains of Roman docks attests to evidence of a watery past. This region offers a superb opportunity not only to document its historic past in a graphic fashion, but also to make it accessible to the public using current technologies, public data and a little bit of work.

More maps R us

noreply@blogger.com (Andrew Zolnai) — Sat, 11 Feb 2012 10:02:00 +0000

Continuing the ongoing (re)discovery of cool maps for the rest of us, here are two I found on Facebook from my friends Christophe Staff in Belgium and Aidos Malybayev in Kazakhstan.

templiers.org chronicles Templar lore throughout Europe and the Middle East. I helped fellow after-hours webster with a simple Google map, which Christophe embedded in his website. Another differentiator is the exhaustive directions for use, which made it helpful even for my octogenarian parents!

zoomed into the Middle East region
click image to enlarge

Former colleague Aidos posted a cool Google clone for Russia, Ukraine, Belarus and Kazakhstan, all of which I visited and perhaps will yet again. I was curious about Tabolsk the old capital of W Siberia, and voilà! here it is, complete photo links and Google Chrome auto-translate. Now if I can just find time to refurbish my ancient webpics to flickr! or Yahoo!Map...

click image to enlarge

On a humouristic note, one may disregard the interpretation of Tabolsk as "tons of bolsk" - it did find Tobolsk after all! - do "corks" ("Пробки") mean plugs or plug-ins (help anyone)?

As I said before, this blog is dedicated to the weird and wonderful ways we find to display information from work through current affairs to the personal, quickly and easily on webmaps... and not unlikely after hours!

Multi-modal maps R us, part II

noreply@blogger.com (Andrew Zolnai) — Sun, 29 Jan 2012 17:27:00 +0000

Last week I reported Google Maps' released of multimodal transportation mapping in the greater London UK area (GLA). I mused that this rendition was more visually appealing that Transport for London's website. The 'granddaddy' for train and tube in London extended beyond GLA to be used from larger surrounding centres - I myself used it regularly from Cambridge an hour train to the north.

click image to enlarge

Not to be outdone, TfL released a brilliant road congestion mapper under Roads Live Travel News, also based on Google Maps! It shows live traffic updates, has a "Use my Location" button and it works as well on my iPad - so surely it's fit for mobile use though I see no mention on their website - perhaps they don't encourage its use, and the moniker you hear on radio show traffic call-ins applies here: "if it is safe and legal to do so..."

So again I couldn't resist reporting this somewhat peripheral topic to what I normally cover, but hey, it's my blog-gage and I'll write if I wa-nt to...

Multi-modal maps R us

noreply@blogger.com (Andrew Zolnai) — Fri, 20 Jan 2012 18:14:00 +0000

Google multi-modal maps are so significant to greater London Area commuters that I cannot pass it up. Ed Parsons posted it on his blog and I immediately tried it: it's just the ticket (pun intended) living near Cambridge about an hour north of London and travelling around London only by public transit.

As I used that map before, I had to clear the cache to have the new mode show up! Also 15 min. on my scooter to nearby Waterbeach station saves me 45 min. bus ride to Cambridge station... plus boarding one station north of Cambridge assures me a seat in rush hour!

click on image to enlarge

Before I went to the otherwise excellent tfl.gov.uk website for timetables. It's also multi-modal with:

spot maps even dynamic map wizard
key: public transit disruption notices

but it doesn't:

allow to enter a complete address only a postcode
show visually when and where to go at what time

click on image to enlarge

Also it shows an hour difference, so there's only one way to find out... But what can I say? That simply isn't as visually appealing!

And as I've been "Pottered" by my daughter, take it away Ernie...

“Au revoir” RMOTC dataset, part VII

noreply@blogger.com (Andrew Zolnai) — Fri, 13 Jan 2012 10:27:00 +0000

“Lucky 7”, this is my closing post on the RMOTC series on subsurface 3D data, to explore reservoir depletion, pipeline routing and gridding/contouring. This also ends for now my tenure in Kuwait, where I prepared this dataset for a training tool. Follow me on LinkedIn to see where I'll go next...

I updated on ArcGIS Online the 3D dataset I gridded and contoured using Esri 3D Analysts new tools added in rel. 9 that work for the petroleum industry. This further illustrates the point of using GIS for data management and quality control as well as classic data visualisation. The faults are extruded to show their relationship to contours and see if the interpretation makes sense.

click on image to enlarge

Again this is not intended to replace full geosciences packages that do a better job with more parameters and geophysical and petrophysical data integration. In fact I posted on a LinkedIn ESRI discussion What Aspects make GIS Projects Unique?:

Location, location, location

context (stacking data from many sources)

data mgmt. (oddities will jump out quickly)

Formats, formats, formats

way to conflate geopatial and other data

best way to show it to the general public

Critical for BI: use location as the premier culling/conflation tool

aha! moments almost guaranteed

see what your neighbour doesn't see

Be keenly aware of crossover from geospatial to pure data and ultra-specialised treatments

Peter Drucker once said: “I look out the window and notice what others don't.”

Gridding and contouring (RMOTC dataset, part VI)

noreply@blogger.com (Andrew Zolnai) — Fri, 06 Jan 2012 09:10:00 +0000

Free geosciences 3D data show GIS helping model reservoir depletion, and displaying it on the desktop and on-line. Then came pipeline routing and now to close the loop is gridding and contouring. Again, this is no replacement for geosciences packages, but rather a tool for triage:

first stack as many data as needed (like basin hydrodynamics or land permitting) for play-fairway analyses
then focus on targets with geoscience apps on specifics (like seismic and petrophysics) for prospects

Starting with ArcMap 9, 3D Analyst extension under “spline with barriers” handles grids with faults like we do in petroleum, no small thanks to “the three Steves”, Kopp and Lynch of Esri and Zoraster ex of Zycor. I posted on Arcgis Online the project resulting from this workflow:

Jeanette Buelt of RMOTC kindly provided tops for the target producing formation
(the original dataset only had measured depths, this had true vertical depth and interpreters)
join the z-aware wells with the table of target formation tops via the common API code
use “spline with barriers” from 3D Analyst for gridding
(tip: make the grid size realistically small, rather than apply smoothing)
use “contour with barriers” from the same for contouring

Comparing the the contours from data provided by Buelt, shows that working with well tops is a good start. But only seismic will show the closure on the target formation top. Yet the new faulted gridding here shows how important it is to consider faults, typically a challenge in play fairway and prospect analyses.

click image to enlarge

Fault data are however plain shape files - not z-aware like well data - so that their 3D treatment is limited here. As this is a relatively shallow play, and it is otherwise known that conjugate faults that cut anticline at a high angle tend to be close to vertical (see slide 8 here), the following extrusion in ArcScene is not an inaccurate picture. A simple geometric depiction helps, for example, to ascertain if the “bulls-eyes” at centre below are real or occur across faults - bulls-eyes are contour depictions, in the absence of other data, of adjacent wells with great depth variation - a fault line running between them may help clean that up, or conversely it may direct geoscientists to look for faults that may have been missed.

click image to enlarge

The most useful aspect here is that one can quickly verify the origin of shallower and deeper tops toward the center of the map. Again GIS is good for overlaying many datasets for data management and triage or clearing, whilst other apps are good for detailed treatment like seismic and petrophysics or reservoir modeling starting this series.

Beyond traditional visualisation, GIS is thus also used for data management.

On-line spatial data validation, part II

noreply@blogger.com (Andrew Zolnai) — Thu, 29 Dec 2011 07:35:00 +0000

Three weeks ago I introduced Socium's Online Validation Service (OVS). I showed the Ordnance Survey vector data QC, on the UK Parishes used as a geographic unit for the Medieval Fenlands project. Two weeks ago Socium kindly created a new rule, to post unexpected variations in adjacent feature attributes. In an essentially agricultural era, economic wealth of the Domesday period was quantified by Darby via the number of plough teams per parish. That is the attribute the rule was written for (it's hard-wired right now and Socium plans to add metadata pick lists in the future).

click image to enlarge

Notice how clean and straightforward the presentation screen is. The result is a set of points along the joint vertices where attributes, scaled from 0 to 5, jump by more than 1 unit among adjacent polygons. That helps find errors in the attributes that were manually entered from Darby map plates, and therefore most prone to errors. A later post will explain an attempt to establish that, or any other correlation with surface geology for example.

click image to enlarge

While this example may appear somewhat trivial, as the colours pop out where there are abrupt transitions, here are a few uses that we'll explore later on:

for very large numbers of polygons (1911 for East Anglia alone)
concatenate the points into barriers and geoprocess that with imagery or lithology
run statistics for the purposes of quantifying the QC applied to a project

Stay tuned for more work on OVS. Thanks again to Socium for deriving this rule so quickly.

Happy Holiday Season

noreply@blogger.com (Andrew Zolnai) — Sat, 24 Dec 2011 06:55:00 +0000

Just for fun - momentary suspension of disbelief required - NORAD Tracks Santa countdown starts now! And follow it on Google Earth.

For serious fun - travel to the other side of the globe - follow the centennial celebration of the British Services Antarctic Expedition. And do it on arcgis.com c/o Esri(UK).

“ 'tis the season t'be 'nclusive, fa la la la, la la, la la!”, so for some of-the-wall fun...

Note: I don't know these folks, but doesn't this seem tongue-in-cheek to you?

How education has changed my life and what I did about it

noreply@blogger.com (Andrew Zolnai) — Thu, 15 Dec 2011 02:33:00 +0000

I consider myself a citizen of the world, as shown in the map below. Born as my parents fled Hungary right after 1956 Uprising. Raised on the expat circuit of an oil company. Emigrated to north America as a student. Lived most of my adult life in north America. Recently repatriated to Europe, yet still work abroad. Education has been a constant thread in my life: first through graduate school, then running continuing education for a local geological society, and now managing GIS in the resources sector worldwide.

Parallel to that, languages have made the most difference in my life. Not only three mother tongues - fluent in all by learning before age six - but also three more from high school. And Latin helped usher my second career from geology I graduated in, to computers I have no formal training in. It's that constant change that fostered my perennial curiosity and made me a good listener, which in turn helped me care for clients. And last but not least, I was more adept at dealing with the multicultural long before globalisation was a fact.

“Andrew has a secret life on the Internet after hours.” That is because real-life people occupy my waking hours (family, community and work). So the virtual world that extends it will occupy my time before breakfast when I'm at home (I'm an early riser), or after dinner when I travel or live abroad (in my hotel room or rental flat). It is important for me to keep that sequence straight: meet the people first, then extend the relationship on-line. While that fosters virtual communities worldwide, relationships will never develop as fully when we only met on-line. It helps to remember that.

Yet a dear friend once said “nous sommes tous des atomes crochus”: we are all hooked atoms. We all affect each other. We interact day to day. “Across the street or across the world” as a mover's packing boxes once said. It is education - in-school (geology) and after-school (GIS) coupled with languages - that helped me navigate among rich and poor nations, or well educated and less educated folk. Is it not the constant and enthusiastic questioning - at the podium in a seminar or in the corridor outside a classroom - that keeps us alert and interested? “Learn something new every day” my grandma said, “and you shall stay young”. That is how education has helped me, and in turn, helped me help others.

This blog post is part of the Vittana "Make a Difference" blogger challenge. The contest invites bloggers from around the world to discuss various ways to make a difference in the world, as well as share stories on who or what has made a difference in their lives.The winning blog post will be the post that drives the most loans to students in need. Please support this cause (and this blog!) by making a loan in my blog's name: "Andrew Zolnai." Be sure to type that in when you reach the checkout page (example screenshot) The more loans you make the more educations get funded and the more recognition and traffic my site gets!Please support this blog and contest by using this special link to tweet about it (You can edit the tweet before it's posted, but make sure this link (http://bitly.com/t8CVlG)and the hashtag #vittanachallenge is part of the tweet or Vittana won't know you tweeted about me!)

Cloud futures #2: on-line spatial data validation

noreply@blogger.com (Andrew Zolnai) — Sat, 10 Dec 2011 08:39:00 +0000

Following vector maps on the web (cloud futures #1), here is a freemium online validation service (OVS) that helps us QC, quantify and clean up spatial data on the web. A speciality of 1Spatial is its spatial validation, an essential first step to setting up proper spatial data infrastructure. They spun off socium.co.uk to offer the same service online.

I put through its paces my Medieval Fenlands GIS project mirrored on giscloud.com and arcgis.com - also posted on sharegeo.ac.uk and linked on data.gov.uk and bgs.gov.uk - in a nutshell, English parishes have been a constant geographic entity since William the Conqueror's Domesday census in 1087. Having transposed HC Darby's economic geographic data on shape files from the UK Ordnance Survey, I wished to test:

how good are vector data in East Anglia, as part of coi.guv.uk data feedback?
how well did I transpose attributes from Darby's map plates into shape files?

OVS offers a set of rules on shape files I report this week and attribute tables next. Provided shape files are under 5 Mb, it's as simple as 1-2-3: 1) upload, 2) select the rules to test vector data integrity, and 3a) view the report for free on line, or 3b) download it for one credit to get another shape file with "data busts" as points with descriptive attributes.

click image to enlarge

Having been in this business for decades, I cannot see how data validation can be made any easier, while maintaining - indeed increasing - data integrity. OVS highlights two of the "holy grails" in matters geospatial:

GIS depict complex data in a simple yet effective way, by placing them in their local context

web services offer two more opportunities:

to post geo-processes on-line that are easy to reach

to offer a flexible pricing plan that helps one and all

Stay tuned for further tests on attribute data. More importantly however, let's see how Socium helps build a community for this oft maligned but absolutely key task of data validation. I wish to thank Socium for helping me get on this service as it started.

Openware inaugural newlsetter

noreply@blogger.com (Andrew Zolnai) — Sat, 03 Dec 2011 13:15:00 +0000

Ask the GIS Expert column on page 14 in the inaugural newsletter of Openware (Esri distributor in Kuwait) echoes the current blog series: use what you already have on your Esri desktop.

I you're stuck on Page 1 then simply type in 14 to get there. And make sure your browser allows access to pops-ups from: www.openware,com.kw

Pipeline routing (RMOTC dataset, part V)

noreply@blogger.com (Andrew Zolnai) — Sat, 26 Nov 2011 17:37:00 +0000

As promised last week here is the update to my second most popular Slideshare post: using ArcGIS Model Builder to plan a pipeline route as a function of topography, slope, land cover and cultural data (roads, rivers, wetlands etc.). As RMOTC is remote, see (pardon the pun) it is uninhabited and land cover is uniformly grass- or shrub-land, which has the same IGBP class of 5 (middle-of-the-road).

The model is to plan a route from a fictitious gathering facility in the north of RMOTC, to another fictitious collecting point by the main road in the agglomeration further north (added Gtopo30 DEM data to extend the project further north). Modelling goes through two stages, details available upon request, but you can get started here:

first calculate the suitability by converting all to rasters and giving them relative weightings and coming up with a suitability grid
second derive a routing along that grid applying weight-costing calculations, these are all functions available in Spatial Analyst

click image to enlarge

I picked a fairly coarse grid of 500 feet, as the 'relief' on the cost-weighting surface is not very pronounced, and more detailed grid yields a rather erratic route. That can be decimated afterwards, or one can simply decimate the data to start with, in other words make a coarse grid. Remember these are simple modelling tools intended to show how GIS can be used out-of-the-box, and not meant to compete with high-end tools.

click image to enlarge

Note in ArcScene, how:

orange squares (high cost) coincide with ridges and rivers
green squares (low cost) coincide with valleys and roads

The highest cost escarpments appear further south away from the route. The entire project will be posted here on ArcGIS Online, if you wish to examine in further detail. The model itself and details will also be posted on Dropbox here, same way as the simple reservoir modelling data were before.

Enjoy! Comments and suggestions are of course always welcome...

Note: Model Builder and ArcScene come with the ArcGIS extensions, Spatial Analyst and 3D Analyst, respectively. Nothing was added to these tools, ArcGIS was used out-of-the-box. And if you don't have it, you can get it here for personal use for $100 per annum.

Model Builder is the single key differentiator from other desktop GIS, and you can load models on ArcGIS Server as Geoprocessing services.

Simple reservoir depletion modelling, part IV

noreply@blogger.com (Andrew Zolnai) — Sat, 19 Nov 2011 15:31:00 +0000

This last in a series shows how to further extend the reach of your GIS analysis across the corporation in full 3D via a free ArcGIS Explorer Desktop. Simply go Add Contents: ArcGIS layers, and to enhance performance go Base Map: Clear basemap. This is a large data set complete with local topography.

click image to enlarge

And the reservoir depletion model foot_feat show best here because of increased performance on large layers (as discussed the model created many many vertices). Of course the layer displays are fixed, but that is a trade off: It helps inexpert users get neither lost nor confused in a complex dataset.

click image to enlarge

Some have asked: why the desktop version of ArcGIS Explorer Online? Well here are the answers:

ArcGIS Explorer Online simply doesn't do 3D, which is the whole point of the exercise here!
ArcGIS Explorer Desktop also consumes the 3D layer packages explained in previous posts

from the your desktop after download, if you have a slow internet connection
directly from ArcGIS Online, if you have a faster connection (over 2-5 mps)

The reason I separated this post out, is that the local DEM now shows properly in ArcGIS Explorer Desktop. As with the Online version everything has to be posted in WGS84, and I simply exported all to a File Geodatabase in the projection/datum. I consequently update the ArcGIS Online posting too.

Note, as commented on last week's blog, that the Esri blog says how to tweak it to view in the subsurface. In addition, to record fly-bys for example, go to ArcGlobe or ArcScene.

Next week will come another Model Builder exercise on the same dataset, one that will update my second most popular slideshare post on pipeline routing (the present discussion having been my most viewed by far).

Simple reservoir depletion modelling, part III

noreply@blogger.com (Andrew Zolnai) — Sat, 12 Nov 2011 10:59:00 +0000

This is to show on the web or with a free desktop GIS the results of the previous two postings. The free data-set from Teapot Dome is a great opportunity to display 3D petro-data in Esri. As the previous posting suggested, data were upgraded to Esri 3D Analyst ArcGlobe here.

click image to enlarge

ArcGlobe is more rigorous in handling projections - in ArcMap or 3D Analysts' ArcScene, even Spatial Analyst's Model Builder, not fully specified projections may give an error message but will still post - not so in ArcGlobe, which in turn allows it to be posted in ArcGIS Explorer Desktop.

[Moved a more complete treatment of ArcGIS Explorer to next week's blog]

The z-aware oil well shape files shows the shallow, medium and deep colour-coding. The swipe tool also allows to show the top 2WCS and bottom Tensleep target geological formations. The reservoir model is far more detailed and is an opaque surface at that scale, but zooming in allows to see the detailed simple flow model discussed in the previous post.

click image to enlarge

Watch next week for another installment on this series. Franco-Belgian comics, bandes dessinées were cartoon serials that ended with: to follow, À suivre...

Simple reservoir depletion modelling, part II

noreply@blogger.com (Andrew Zolnai) — Sat, 05 Nov 2011 17:32:00 +0000

Posted on ArcGIS Online a 3D rendition of the Teapot Dome free 3D GIS dataset by RMOTC and model by me. I used Esri ArcScene from its ArcMap 3D Analyst extension. If you don't have that, then download the free ArcGIS Explorer Desktop, and point to the layer package file here [updated with ArcGlobe]. ArcGIS Explorer Online cannot display 3D packages, furthermore, the drop-down menu on the arcgis.com site will suggest how to access it. You can get ArcGIS Desktop for Home use with extensions for $100 here.

click image to enlarge

The well data are so-called measured (or z-aware) shapefiles that allows to post the TVD (true vertical depth) by colour: note two shallow purple and cyan series, the red one along the 2WCS target formation in green, and yellow ones along the Tensleep target formation in tan (see previous blogs or layer packages for details). Faults are plain (not z-aware) shapefiles draped over the surfaces. Surfaces are step contour maps posted by RMOTC, and a future endeavour may be to find and re-contour the data into smooth surfaces.

click image to enlarge

This has an effect on the simple reservoir model that cannot be circumvented at this time. Here is, however, a portion that depict simple flow patterns - again, please note the caveats from the previous blog post. Please notice how the flow patterns:

go away from the terrace (artifacts of contour images) to the faults or up-slope
ignore the faults that did not create an appreciable step in the formation tops

Stay tuned for the next installement, as Bern Szukalski's blog post says "go through ArcGlobe to layer package 3D data". ArcGlobe is more rigorous than ArcScene on projections, and thus helps properly post data in ArcGIS Explorer for public view.

Simple reservoir depletion modelling

noreply@blogger.com (Andrew Zolnai) — Sat, 29 Oct 2011 14:46:00 +0000

Following on last week's Teapot Dome 3D dataset, here's the first step toward upgrading my most popular Slideshare post: Geoscience class notes have an option to run ESRI Model Builder that comes with the Spatial Analyst extension. Simply reversing reservoir topography and applying a surface run-off model, will mimic the depletion of reservoir of its petroleum content. The same way water flows downstream though gravity, petroleum will flow up-slope through hydrostatic recharge (in other words buoyancy pushes hydrocarbons up on top of denser water and out of a reservoir).

click image to enlarge

The purple lines are the flow lines of the modelled fluid flow on an existing raster surface generated by RMOTC. Notice the pooling at the crest of the reservoir, and how some faults affect some but not all flow lines. This is indeed a simple model inspired by Statoil 10-15 years ago, but as a simple model it had some limitations:

no rock properties such as porosity or permeability, though that is added in the full class notes
surface run-off concentrates the flow lines toward valleys, whereas this model spreads them across reservoir ridges: that could create a very large number of vertices as shown above
and if it appear that some contours acted as barriers, that's an artifice of of drawing vertices

Next steps will be to simply grid and contour in ArcMap and rerun the model [postponed until I find or create measure shape files with tops data]. The point really is:

to show the power of Model Builder to construct, with simple tools that come with ArcMap extensions, not-so-simple fluid flow in subsurface reservoirs
not to replace complete subsurface reservoir modelling packages, only to highlight how much can be done with GIS extension tools out of the box
and to allow GIS / Geoscience experts to pass on models to other users, and thus help promote the use of GIS across the organisation

Again please check my previous blog to find source and background for this nice free dataset.

Guns & Roses, or: 3D GIS anyone?

noreply@blogger.com (Andrew Zolnai) — Sat, 22 Oct 2011 03:18:00 +0000

The Guns part is the colorful history associated with the Teapot Dome. A remote Wyoming oilfield once a sleepy Naval Petroleum Reserve, the Teapot Dome Scandal was, however, the biggest to rock Washington DC until Watergate... or the oil industry until the Enron collapse! Read this introduction to a book excerpt: Marines Invade Wyoming - From the Halls of Motazuma to the Oil Fields of Teapot Dome, for cape-and-sword intrigue from ranchers and oil barons to US senators and the White House...

Its current incarnation is the Rocky Mountain Oilfield Testing Center (on LinkedIn): RMOTC sounds like "remote-see" for short (its location is remote too!). Tom Anderson's early GIS center released a 3D petroleum GIS dataset at the 2005 PNEC conference. That set of 4 DVDs, which have been used as demo data sets by a number of petroleum software vendors, and as web demos by Esri and others. Always on the hunt for good 3D GIS datasets, I used Esri's new layer-packaging to post it on ArcGIS Online, As Is and with the same Terms of Use. Six group layers are thus called: Teapot Dome Six Pack: No ArcGIS? No worries! Buy the home-use version for a 'bill' or $100.

topography, surface geology and quadrangles

subsurface structure, surface facilities and wells
click image to enlarge

The Roses part? Will such a dataset make anyone with a little patience and imagination smell like a rose pushing Esri into 3D space? I started to upgrade my Geosciences class to grid / contouring and simple reservoir depletion modeling - to show what can be done in GIS out-of-the-box, not to compete with full interpretive suites - stay tuned as this will develop...

At almost two daily downloads since its posting two years ago, it's my most popular on Slideshare. Comparing that to 50 hits daily average over the same period for this blog would be apples to oranges, or roses, wouldn't it?

"... with a little help from my friends", part III

noreply@blogger.com (Andrew Zolnai) — Sat, 15 Oct 2011 07:21:00 +0000

25 years ago this week, I left the Natural Resources Canada to start a business with Teknika. I was encouraged (if not pushed) along by a fellow geologist, who had a comprehensive petroleum geocomputing system at Husky in Calgary - his colleague encouraged me at University of Calgary to take a class in computer sciences, the same department where Jim Gosling later on created Java. I teamed up then with a brilliant surveyor who delivered a video-tracing system. These were the DOS days when we used AutoCAD as the graphics prior to Windows. And as my banner note states, he spatialised AutoCAD with a 10Kb DOS kernel that might've given Intergraph and Esri a run for their money, had AutoDesk picked it up at the time.

Can you believe that I received a copy of Windows 1.0 as a bonus, when I upgraded my PC-AT clone from 640KB to a whopping 1MB RAM? This despite the fact that Trey, aka. William H. Gates III, famously quipped who will ever need more that 640K memory!

Mar88CADalyst

Those two years were seminal in many ways:

I called my untitled MBA learning as much in starting up as in winding down a business, though I would never recommend to my worst enemy that they lose their marriage in the process...
introduced me to Metronet, when AGT the telphone company then optically wired Calgary to link up all oil companies, doing it for free as a guv agency thinking they'd get it back in leasing the lines which they did handsomely

on which I built then my first page then call hypertext as the web didn't exist yet as such...

introduced me to those who would 5 years later not only form StrataModel and Argus, later to become MapGuide at Autodesk (no connection I'm afraid)

but also be part of Gulf Canada (later Chevron), who with Finder (later Schlumberger) and Digitech (later QC Data and IHS Energy), founded PPDM whose first met at URISA 91 - GIS and Energy
where I met Charles Fried of Operation Database fame, and later PUG chairman when I was furthering PPDM at ESRI

10 years later joined Munro Garrett whose oil&gas sector of Argus would become OpenExplorer at Landmark, Esri's best-selling ArcView extension

which 5 years later would take me to Esri's industry solutions team as petroleum manager until about 5 years ago

Clouds gathering over the horizon, part V

noreply@blogger.com (Andrew Zolnai) — Sat, 08 Oct 2011 15:06:00 +0000

Will spectacular outages of AWS and now BT cloud services casts a pall over the excitement that has reached even politicians? And now comes this geopolitical issue only hinted at by MSFT's tribulations in Europe over Internet Explorer. This posted in Germany (Google Chrome will translate this page for you) by Ruth Lang - 'SVG queen' to Dino Ravnic 'web vector king'.

Not to be put off by any of this, 1Spatial just launched Socium to provide a web service (a) to help you clean up your data online, and (b) to interact with their 'soon to be your' community of users, where you can propose further tools and formats - that would be the 'jack' in that metaphoric court above - and a freemium pricing model (pay as you use) in the current economic climate will be welcome news for most...

The question I put to them however is the same one I asked giscloud.com, and the same ESRI's private cloud will try to address: what is the best strategy to host, process and keep data on the web, rather than waste bandwidth uploading and downloading data? Outages affect data transactions, not data storage after all! Two contrasting strategies are presented here, and I only use examples I'm familiar with, this isn't exhaustive... and gigaom's Alex Stalker certainly give more perspective!

New kid on the block giscloud.com has a freemium business model, and technically offer to store files for easier storage, or load them in postGIS for actual geo-processing. They completely insulate the user from storage issues, indeed they offer to do away with performance issues by posting vectors online. Being also new they have the least amount of geo-processing tools and least capability to link to other data sources.

Granddad Esri has been pursuing web strategies for a while now, be it through its geoportal do tackle metadata and storage, to its current Explorer rendition of arcgis.com to build geo-communities. In addition to keeping everything online, it offers Portal for ArcGIS to move everything behind a corporate firewall, not unlike Google and Microsoft (and others in the news lately outside 'geo' space).

As mentioned earlier this is key for resources companies and government agencies who both share and protect their data, and where security and access are not insignificant challenges. I can imagine not a few harried web IT managers thinking like so...

Your team is your friend

noreply@blogger.com (Andrew Zolnai) — Sat, 01 Oct 2011 11:41:00 +0000

I am so proud of my teams at client sites and in our office! One team achieved in 6 months at one site what many thought would take years - to integrate surface and subsurface exploration and production infrastructure for an oilfield in 3D+time. Another team created just this week a real-time GIS data capture system that reduces to 4 steps what took 10 on paper - and of those only the first one is manual.

Not only does this reduce the chance of errors creeping in, but it also automates the collection from handhelds in the field to the back-office in real-time. This is crucial to improve business processes in the field on one hand, and to enable (near) real-time situational awareness on the other.

Here is a mock-up of the data capture exercise: Pay particular attention to the colour symbols - each colour represents an intensity reading - spreading across the middle then the bottom of the web map as the operator taps the handheld device for data entry.

Please note that spreadsheet and video are unlisted, meaning you can access them only via their web address as this blog points to, to protect site confidentiality.

[12 October update: Not content with that, here's other application integrating Outlook with staff location for planning and emergency response:

Hr locator

View more documents from Andrew Zolnai]