Terra ETL Blog

MetaCRS - Coordinates & Projections

noreply@blogger.com (Terra ETL) — Sat, 22 Mar 2008 23:27:00 +0000

MetaCRS is a project started by Frank Warmerdam of PROJ.4 fame.

The initial basis for this project is to act as an anchor for an OSGeo Project encompassing several projections, and coordinate system related technologies.

The first technologies or libraries that are being used in this project are:

proj4js (Javascript: Rich Greenwood and Mike Adair )
proj.4
libproj4 (the projection-only library maintained by Gerald Evenden)
OSGSpatialReference (GDAL coordinate system translation classes)
CS-Map (the recently open sourced library from Norm Olsen/Autodesk)

So where do non-programmers fit into this?

Frank has covered that as well and there are areas where a little geodesist like myself can contribute. If you feel you can, by all means join the mailing list.

He states on the wiki the following suggestions:

Common Spatial Reference System or Coordinate Reference System Names and Descriptions
Coordinate System (and CRS related object) dictionaries. Stuff like the EPSG dictionary.
Datum shift lists (towgs84), and datum grid shift files (NTv1, etc).
Transformations, calculations, and algorithms written in pseudocode that can be edited in different languages.
Descriptions of spatial reference systems that can be used by developers in different programming languages.
Notes on transformation from different representations of a CRS (WKT, PROJ.4, GCTP, GML,...).
Test suites with test points in a variety of coordinate systems and their lat/long and WGS84 equivelents).
Articles on spatial reference systems and translations useful for programmers interested in spatial reference system implementations. For example: Understanding The Difference Between National Vertical Datum of 1929 and the North American Datum of 1988

This will be an interesting project to be involved in.

I also hope to explain in my blog a little later, some answers to some of the suggestions above.

MapGuide Open Source 2.0 Released

noreply@blogger.com (Terra ETL) — Sat, 22 Mar 2008 22:08:00 +0000

On February 29, 2008 MapGuide Open Source 2.0 was released.

There are many enhancements to this version, mainly the integration of Fusion, which was developed by DM Solutions.

Fusion was mainly built in JavaScript and is described as a web-mapping application development framework. i.e. the tools that allow you to build cool mapping applications!

Fusion uses “widgets” that provide the interface functionality within Fusion’s modular architecture; therfore developers are able to add, remove, or modify functionality using standard-compliant HTML and CSS. Fusion provides an extensible platform that allows the developer to develop their own widgets.

One advantage of Fusion is that it requires no proprietary browser plug-ins, and it produces applications that work in all major browsers on Windows, Mac, and Linux.

Fusion is bundled with many of the typical functions that are expected in a web-mapping framework. Fusion includes navigation widgets (e.g., zoom In, zoom out, pan, etc.), legend controls (e.g., view, layer management, etc.), GUI widgets (e.g., buttons, menus, tree views, panels, dialogs, etc.), and many more.

It is good to see how two companies can work together to produce a web-mapping application development framework that makes internet mapping what it should be - not difficult!

Way to go AutoDesk & DM Solutions!

ETL: CamptoCamp, FDO and Open Source - What about OLAP?

noreply@blogger.com (Terra ETL) — Sat, 10 Nov 2007 00:01:00 +0000

CamptoCamp & Talend

CampToCamp is introducing (not yet released, but anticipated), a Spatial ETL tool, that works in conjunction with Talend's Open Source ETL product Open Studio.

Once released, I'll begin to work with the product, but CamptoCamp was in Victoria presenting their solution at FOSS4G2007.

More information on their presentation can be found here.

Open Source ETL - Without Spatial

For Open Source ETL, without the Spatial, there are various Open Source solutions. Talend, being one of them.

You can take a look at:

1) Pentaho - URL: http://www.pentaho.com/

2) Clover - URL: http://www.cloveretl.org/

3) KETL - URL: http://www.ketl.org/

Is there another Open Source Spatial ETL Tool?

But we do have another option as well for the Spatial side now that AutoDesk is now working in the Open Source community and they released FDO (Feature Data Objects), which is similiar to FME - but is not an FME.

FDO is a Data Access Technology that was developed to manipulate, define and analyze geospatial data regardless of where it was stored.

FDO was originally developed and included in the Autodesk Map 3D 2005 product during the spring of 2004. In this initial implementation, it was capable of working with the following geospatial types:

Oracle
SDF

The following version introduced ArcSDE.

The third verision implemented more sources, and added providers for MySQL, SQL Server, ODBC, SHP, Raster, OGC WFS, and OGC WMS.

It was then that they decide to take FDO Open Source, but would not release the Oracle version - but being Open Source, there is always an option out there and some ingenious minds to come out with some solutions.

Quoting the OSGeo FDO History site:

"The release of FDO as open source coincided with the release of MapGuide as open source in 2006. It included the SDF, SHP, MySQL, ArcSDE, ODBC, OGC WFS, and OGC WMS providers. "

FDO was now out to the Open Source World - right on schedule with their release of MapGuide Open Source.

But what about Oracle and FDO?

Much of the work for the Oracle side has been developed by SL-King in Slovenia.

King.Oracle is Open Source FDO provider for Oracle.

Through this product, which is Open Source, SL-King is providing a tool that supports Oracle Locator and Oracle Spatial. It is specifically designed for Oracle and Oracle alone and they are designing it in such a way, that it will support full Oracle Spatial functionality.

Currently the latest version (Version 0.7.3) provides:

Support for Oracle 10G, Oracle XE and Oracle 9i
Optimized for Oracle
Using plain Oracle tables and views
Can be used inside AutoCAD MAP 3D to edit and query Oracle data

For a Flash Movie on this, please look here.

Can we convert between FDO different sources?

Yes, of course! This is Open Source.

Coming from SL-King, again, we have another ingenious tool, called FDO2FDO.

FDO2FDO is an Open Source FDO client application which uses the above mentioned Open Source FDO library to manipulate, create, and define geospatial data.

Currently, the software is capable of the following:

Copy data from SHP files to SDF
From SHP to Oracle
Oracle to SDF...

In the end, FDO2FDO allows the user copy and modify any data from any FDO Data Store to any FDO Data Store.

There are three main parts in FDO2FDO and they are:

Fdo2Fdo Api library
F2Fcmd Command line utility
Fdo2Fdo GUI

An introduction to FDO2FDO can be found here.

There are always solutions out there.

Can Geospatial move towards ETL and Data Warehousing?

Yes.

Currently, in order to do web-mapping, you require the following:

database
web server
data

That is all. You can have a web-map up-and-running with MapServer or MapGuide quite easily, definitely within less than a day depending on how complicated and stylish you want to get.

But look deeper and what is it we are after? The data.

Where is the data stored? In a database.

So why don't we work on bringing OLAP into the Internet mapping and GIS worlds?

Aggregations of data can occur anywhere.

You can group data by postal/post/zip codes, populations, etc. - this is prime data for OLAP.

Take industries such as oil and gas - well production is recorded on an hourly basis. This can be summed and aggregated into data marts (OLAP) for daily, monthly, yearly.

The maps are a starting point, but there should be no disconnect between the data, databases, GIS, internet mapping, as we are only working with data and transforming it into much more usable and valuable information.

By bringing OLAP into GIS and Internet mapping, you can add more value to your client's data and this data can be fed into other applications for reporting, etc., etc..

The internet map acts as a portal to a whole other world of information.

Just a few thoughts, as I'm involved in both worlds presently.

What do you think?

Feel free to write and let me know.

Axis Order Confusion: Software, Geodesy & Transformations

noreply@blogger.com (Terra ETL) — Sat, 03 Nov 2007 20:34:00 +0000

Axis Order Confusion

Not simply X, Y, Z or N, E, S, W, but how we interpret them and use them in software, geodesy and navigation - it varies and has led to confusion among many people. This is an overview of the systems, the problem and points out things to take note of.

Over time, since the early days of mathematics, and then moving into software development, GIS and internet mapping, axis play an important role, whether it be in datum transformation or even displaying a map.

Is it X,Y,Z or Z,Y,X?

As we know in mathematics, a coordinate system is a system for assigning n-tuple of numbers or scalars to each point in n-dimensional space.

We are familiar with the following coordinate sytems involved in GIS/Mapping and Geodesy:

Cartesian coordinate system, which may be called "rectangular", where for 3D space, it uses three numbers representing some distance

Polar coordinate systems

Curvilinear coordinates, which are based on an intersection of curves

Delving further into Polar coordinate systems, we see the following subtypes:

Circular coordinate systems, which is represented by a point in a plane, by an angle, and a distance from the origin

Cylinderical coordinate systems which require a point in space, a distance from an origin an a height

Spherical coordinate system, which is represented by two angles and a distance from an origin

In mapping and geodesy, we deal quite often with Spherical coordinate systems and often refer to them as Geographic Coordinate Systems.

Ordered Pairs & Coordinate Systems

In GIS software and mapping software, we have three different perspectives for an Ordered Pair (2-tuple). They are in computer science, mathematics and of course Geographical Coordinate Systems.

Let's take a quick look at how these distinct perspectives see the their world:

In computer science and computer graphics, the axis order is (X,Y), where unsigned values increase to the bottom and to the right.

Mathematics sees this world differently for the same axis order (X,Y), where we have signed values increase to the right and upwards.

In the world, where we are most involved, Geographical Coordinate Systems, the axis order varies, sometimes being (X,Y) or (Y,X). The signed values increase upwards and to the right, based on a spheroid, hence we have -180, -90, 180, and 90.

Rotation Confusion as Well - Which Sign is positive?

There are 2 different conventions in use in the survey and mapping industry for defining rotations.

This too has led to considerable confusion in the GIS and mapping world.

Both are valid when used properly.

The two conventions can be referred to as:

1) Position Vector rotation (Commonly referred to as the Busra-Wolfe)

2) Coordinate Frame rotation

This essentially comes down to the left-handed vs. right-handed rotations (see image above) for the various transforms.

But what does this mean with left vs. right? Well, this is one way of determining orientation of axes and direction of rotations.

Thumb = Positive X
Index up = Positive Y
Middle out = Positive Z

Clifford's Point of View

Clifford Mugnier of LSU, whom I met when I lived in Houston, gives a good explanation and way of handling rotations and coordinate systems. His reply can be found here.

My quote follows:

"Probably the best way to document a rotation method is:

use the accepted terms "coordinate frame" and "position vector".

These terms are also used in other disciplines like kinematics (robotics).

But there are more difficulties.One datum transformation method is laid down in the ISO 19111 standard.

This is an approximated 7-parameter Helmert transformation with position vectorrotation. See also ISO/IEC 18026 - Annex B.

PROJ uses about the same method, only the scaling method differs. PROJ does a scalar multiplication, ISO a matrix multiplication.

With the commonly used parameter ranges, the differences between the scaling methods are less than microns, so not important.

As far as I know, the Bursa-Wolfe transform is an approximation to theHelmert transform. The Helmert transform has sines and cosines in therotation matrices, whereas Bursa-Wolfe (and ISO 19111) use the angles themselves (since sin(a) ~ a, sin(a)*sin(b) ~ 0, and cos(a) ~ 1 for small angles).

If you read section B.6 of ISO/IEC 18026, then you'll notice that aBursa-Wolfe transform can be done with a position vector rotation model OR with a coordinate frame rotation model.

Just what one likes the best; be sure to use the correct sign of the rotation angles.

Therefore:

Bursa-Wolfe is NOT equivalent with this or that rotation model.

A well known expert repeatedly states that the Australians use the same datum transform rotation model as the Americans.

This is NOT true!

The order of the rotations differs (XYZ vs. ZYX).

See the Australian GDATechnical Manual.

By the way, if the rotations are approximated, then the order is not important.

Again, the differences in rotation order for real life numbers are literally microscopic."

He clearly points out, and I agree, that it is "silly to refer to a datum transformation method as an American, Australian, European, whatever regional model. If you want to document the way for instance an application transforms,givethe complete formulae, not just an ill-defined name. Why not referring to the EPSG coordinate transformation method numbers? These clearly define the most used datum transformation methods and projection methods."

A simple solution to a complex problem of rotations.

The key is to document and document and be sure to specify what is being done, what axis are being used and let your users know. Do not assume, ask questions, and your life will be easier when dealing with coordinate sytems and rotations.

Or even simplier, as Clifford points out, refer to the coordinate transformation method numbers referenced in the EPSG data and data model for defined coordinate sytems.

CS-Map, Open Source & FDO - Autodesk Speaks

noreply@blogger.com (Terra ETL) — Fri, 02 Nov 2007 23:06:00 +0000

This article was recently published by the Australian PC World magazine. In it they interview Autodesk's Liam Speden about CS-Map and how it will be integrated into OSGeo.

CS-Map currently supports many projections and over 3000 coordinate systems.

It is an interesting read and shows how Autodesk believes that because there are so many coordinate systems out there, Open Source can make their customers benefit from this technology and how the rapid development and implementation that takes place in the Open Source world provides a sound and stable product used by real users.

Take a look at some of the books I've listed on the side bar. They explain Open Source quite well and how innovation can happen elsewhere.

EPSG & UKOOA - Defining Co-ordinates in Digital Data Exchange Formats

noreply@blogger.com (Terra ETL) — Fri, 02 Nov 2007 19:53:00 +0000

Introduction to the Offshore

UKOOA and the EPSG have been working together for many years. Through their collaboration, they have developed many standards and UKOOA has been open to listening to the industry about positioning in the North Sea.

When seismic data is acquired, whether it be 2-D or 3-D seismic surveys, the shotpoints (energy source, common mid-point, etc.) need to be positioned or referenced on surface.

Over the years UKOOA has developed various formats, named via a version and a year. This is a short introduction to how these files describe positions in the oil and gas industry.

UKOOA P1/90

Information is described in the Header for the file. The Header records following the convention listed below:

Record Identifer "H" Column(s): 1 Format: A1
Header Record Type 2-3 I2
Header Record Type Modifier 4-5 I2
Parameter Description 6-32 A27
Parameter Data 33-80 Varies

Using the above as a basis, let us take a look at how Datum and Spheroid information is described in this file.

Header records H1600 and H1601 are required for Datum Transformation parameters used by the Bursa-Wolfe Transformation.

Reviewing the Bursa-Wolfe Transformation (as vectors), we see the following:

X DX 1 -RZ +RY X
Y = DY + SCALE * +RZ 1 -RX * Y
Z DZ -RY +RX 1 Z

where

X,Y,Z are geocentric cartesian coordinates defined in metres
DX,DY,DZ are the translation parameters defined in metres
RX,RY,RZ are clockwise rotations defined in arc seconds,
but are converted to radians for use in the formula

SCALE = [1+S. (1oe-6)] where S is in parts per million

The Vertical Datum, is identified by Header record H1700. Some examples of the vertical datum, in relation to offshore work are:

LAT - Lowest Astronomic Tide
MSL - Mean Sea Level
SL - Sea Level
ES - Echo Sounder

The units of measurement are specified in H2001. These should be consistent with the position data. The height unit code will be 1 for metres, 2 for any other unit of measure. Header H2002 specifies the Angular unit code to 1 for degrees, 2 for grads.

Projection Data is specified in Header records H1800 to H2509

Currently, in this older format, the following projection codes were defined and used.

001 - UTM Northern Hemisphere
002 - UTM Southern Hemisphere
003 - Transverse Mercator (North Oriented)
004 - Transverse Mercator (South Oriented)
005 - Lambert Conic Conformal with one standard parallel
006 - Lambert Conic Conformal with two standard parallels
007 - Mercator
008 - Cassini-Soldner
009 - Skew Orthomorphic
010 - Stereographic
011 - New Zealand Map Grid
999 - Any other projection or non-standard variation of the 11 listed above

Since this initial positioning file was developed with the help of surveyors, they planned ahead and answered the question: What happens when we cross the Equator?

When a survey crosses from the South to the North, and the whole survey is shot on a Southern Hemisphere UTM Zone, the coordinates will possibly exceed 9,999,999.9. This is not acceptable in the P1/90 format, so Header record H2600 must indicate that 10,000,000 must be added to the co-ordinates.

More detail about this specification can be found here.

As the industry matures, new versions were released, the next in 1994. This was called p2/94 and was derived for raw marine positioning data.

UKOOA P2/94

During this time, differential positioning with GPS was just being implemented and the industry was beginning to rely on this technique more and more for offshore surveying.

This format is based on UKOOA p2/91 and has extended many of the definitions needed for differential GPS.

As operators maintain and store data in these formats, P2/94 also acts as an archiving format and records information such as the satellite ephemeride, ionospheric conditions and weather/meteorological conditions of the survey.

With the move to P2/94, Geodetic information moved to new headers, and are such described as:

H0100 Magnetic Variation - General Information
H0101 Magnetic Variation - Grid Data
H011# Datum and Spheroid Definitions

where # = 1..9 and is the datum & spheroid number

H0120 Seven Parameter Cartesian Datum Shifts
H0130 Other Datum Shift Parameters
H0140 Projection Type
H0150 (Universal) Transverse Mercator Projection
H0160 Mercator Projection
H0170 Lambert Projection
H0180 Skew Orthomorphic & Oblique Mercator Projection
H0181 Skew Orthomorphic & Oblique Mercator Projection cont.
H0190 Stereographic Projection
H0199 Any other Projection

Satellite System Definitions

H600# Satellite System Description
H610# Definition of Differential Reference Stations
H620# Satellite Receiver Definition
H6300 GPS parameter recording strategy
H6301 DGPS differential recording strategy
H631# GPS clock and ephemerides parameters
H632# GPS ionospheric model & UTC parameters
H6330 Meteorological parameters
H65## DGPS differential correction source defn
H66## DGPS differential correction source defn
H67@0 GPS ellipsoid height estimate

Rotation Conventions

Note that 2 different conventions are in use in the survey industry for defining rotations. This has led to considerable confusion in the GIS and mapping world. Both are valid when used properly.

The two conventions can be referred to as:

1) Position Vector rotation (Commonly used in Europe and referred to as the Busra-Wolfe)
2) Coordinate Frame rotation (Commonly used in North America)

I will talk more about these on a later blog, as it comes into play with a lot of GIS and mapping software.

More detailed information about the P2 format can be found here.

UKOOA P5/94

This version came along to facilitate the exchange of position data for pipelines, flowlines, umbicals and power cables offshore.

In these cases, the data required for pipeline positions are the Latitude, Longitude, Easting, Northing, Depth, and Kilometre Point (KP), along with the standard datum and map projection parameters.

Without wanting to bore my readers with more H records, you can found out more about how the pipelines are stored in this format here.

UKOOA P6/98

In 1998, a new version was developed for 3D seismic surveys and binning.

This is quite complex and would make this short blog even longer, so I'll write about this format at a later date.

The main emphasis of this blog, though, is to show how formats can change over time as technology and data sharing increases. It also points out the importance of knowing the format of your data, especially if you are doing historical work over a region - do not always assume a specific data format. This is partly why the OGP has started the Joint Industry Project I mentioned in an earlier post.

UKOOA P7/2000

Well's deviate. With the advent of horizontal wells and sidetracks, and relating to seismic surveys, we enter a whole other story again.

In this story, as well (bad pun!), we have to consider height measures (such as Kelly-Bushing), and the 4 Norths (which I will explain on a later post).

As this file type would make this blog even longer, I'm going to jump ahead to what the EPSG and UKOOA are doing now in defining the Header records for this specific part of the oil and gas industry.

How the EPSG comes into Play

Turning our eyes to the EPSG in P formatted files, we want to enable integrity checking of co-ordinate system definitions in UKOOA P1, P2, P5 and P6 formats, so a provision is made to describe co-ordinate system by reference to the European Petroleum Survey Group (EPSG) database of geodetic parameters. This is the group of codes we see in use throughout the GIS field and in products such as ESRI and PROJ.4.

What this allows UKOOA to do is to adopt an industry-standard name to be quoted where the geodetic co-ordinate system used is a common system. Defining parameters and units are then as given by EPSG and are not strictly required to be explicitly given in the P-format records.

As an integrity check, it is considered good practice also to include the explicit definition .The new records which can be used as extensions within the P1/90, P2/94, P5/96 and P6/98 formats are:

H8000 EPSG Geographic CS Name
H8001 EPSG Geographic CS Code
H8002 EPSG Projected CS Name
H8003 EPSG Projected CS Code
H8004 EPSG Vertical CS Name
H8005 EPSG Vertical CS Code
H8006 EPSG Database Version

As we know, co-ordinate systems may be two- or three- dimensional.

A vertical co-ordinate system is one-dimensional.

For the P1, P2 and P5 formats:

the H8002, H8003 and H8006 records are required when latitude, longitude, easting and northing but no height or depth are given;

the H8002, H8003, H8004, H8005 and H8006 records are required when latitude, longitude, easting, northing and gravity related height or depth are given;

the H8000, H8001, H8002, H8003 and H8006 records are required when latitude, longitude, easting, northing and ellipsoidal height or depth are given.

For the P6 format, the H8002, H8003 and H8006 records are required.

That is the way UKOOA and the EPSG see the offshore world when it comes to positioning and exploration in the North Sea and elsewhere.

Exploration & Production Blog

If you are interested, I also write another blog on the oil and gas industry, mainly describing where exploration is occurring, the technology being used, history of a region, some geology, etc. and some aspects of the UN Convention on the Law of the Seas.

The blog is located here.

Enjoy!

Quarter Degree Grid Cells: Another way of Mapping Africa

noreply@blogger.com (Terra ETL) — Thu, 01 Nov 2007 16:40:00 +0000

Recently, Ragnvald Larsen of the Norwegian University of Science and Technology (NTNU) in Trondheim released some news to the Spatial Data Infrastructure for Africa (SDI-Africa) mailing list.

He has been working on project about creating Quarter Degree Grid Cells for mapping purposes for the African countries on a national level.

But what are Quarter Degree Grid Cells?

Quarter Degree Grid Cells (QDGC) are a way of dividing longitude and latitude degree square cells into smaller squares, forming in effect a system of geocodes. This is similar to the NTS system in Canada (for mapping the Northern areas of Canada) and to the way the North Sea is mapped when determining leases by the various countries involved (Norway, Denmark, UK, Germany, the Netherlands). An example of how the North Sea is subdivided by country follows:

The respective sectors are divided by median lines agreed in the late 1960s.

In the United Kingdom, the UKCS (United Kingdom Continental Shelf) is divided into quadrants of 1 degree latitude and one degree longitude. Each quadrant is divided into 30 blocks measuring 10 minutes of latitude and 12 minutes of longitude.

Norway has a similar model and is divided into quadrants of 1 degree by 1 degree. Norwegian licence blocks are larger than British blocks, being 15 minutes of latitude by 20 minutes of longitude (12 blocks in a quad).

In Denmark, the Danish sector of the North Sea is divided into 1 degree by 1 degree quadrants, and their blocks are 10 minutes latitude by 15 minutes longitude.

Germany and the Netherlands share a quadrant and block grid - quadrants are given letters rather than numbers. The blocks are 10 minutes latitude by 20 minutes longitude. The Dutch sector is located in the Southern Gas Basin and shares a grid pattern with Germany

So the theory of using grid squares has been around for quite some time.

Almost Equal Areas

QDGC represents a way of making (almost) equal area squares covering a specific area to represent specific qualities of the area covered. The squares themselves are based on the degree squares covering earth.

We know that around the equator there are 360 longitudal lines lines. For latitude, i.e. from the north to the south pole we have 180 latitudal lines. Multiplying we determine that this gives 64800 segments or tiles that can cover earth. The form of the squares becomes more rectangular the further north or south we move. At the poles they are not square or even rectangular at all, but end up in elongated triangles.

Each degree square is designated by a full reference to the main degree square.

An Example using Tanzania

Taking from the project web-site, I'll use their example, with regards to Tanzania.

S01E010 is a reference to a square in Tanzania. S means the square is south of equator, and E means it is East of the zero meridian.

The numbers refer to longitudal and latitudal degree.

A square with no sublevel reference is also called QDGC level 0.

This is square based on a full degree longitude by a full degree latitude. The QDGC level 0 squares are themselves divided into four.

A grid at this level is shown as:

AB
CD

Smaller squares are determined by dividing the above squares into 4 again.

So if we divide S01E010 by four again, the new grid would be S01E010AD.

The number of squares for each QDGC level can be calculated using the following formula:

number of squares = (2^d)^2 where d is QDGC level

Putting all the above theory into place, there is code out there that will allow you to compute a Quarter Degree Grid Cell, please follow the link here.

Project Information

For more information on this project and the work done by Ragnvald Larsen, please take a look at QDGC.

The attached image shows QDGC being used for mappings of the fires in Africa in the year 2000.

ETL: Fundamental to Spatial Analysis and Sharing of Data

noreply@blogger.com (Terra ETL) — Mon, 29 Oct 2007 22:55:00 +0000

Extract, Transform, and Load

ETL or Extract, Transform, and Load is a process in Data Warehousing. Data Warehouses are all around us - whether we are querying the Yellow Pages on-line to geo-coding our datasets, there is most often a database or data warehouse behind the scenes.

But how we get data into the warehouse, is called ETL.

This is a short introduction to ETL as it is an important part of data warehousing. I also cover a little about Spatial ETL.

Extract

Extract is where we begin this story. Essentially, in this step, we extract the data from source systems. A source system is where the data originates. It may be a well database, an address database, a MSExcel CSV file or almost anything you can think of.

A data warehouse ends up consolidating the data from different source systems. Each separate source system (in many cases) use a different data organization or format. Common data source formats are relational databases and flat files.

Transform

As our story continues, we run into the transform stage. In this chapter, we want to apply a series of rules or functions to the extracted data from the source to derive the data to be loaded to the end target.

If we are fortunate in having clean data (this rarely happens!) the data source will require very little or even no manipulation of the data. The most common scenario is that one or more of the following transformations need to be done to meet the business and technical needs of the end users.

For example, some of the transformations that may occur are (taken from Wikipedia - a good listing of transformations):

Selecting only certain columns to load
Translating coded values (e.g., if the source system stores 1 for male and 2 for female, but the warehouse stores M for male and F for female), this is called automated data cleansing; no manual cleansing occurs during ETL
Encoding free-form values (e.g., mapping "Male" and "1" and "Mr" into M)
Deriving a new calculated value (e.g., sale_amount = qty * unit_price)
Joining together data from multiple sources (e.g., lookup, merge, etc.)
Summarizing multiple rows of data (e.g., total sales for each store, and for each region)
Generating surrogate key values
Transposing or pivoting (turning multiple columns into multiple rows or vice versa)
Splitting a column into multiple columns (e.g., putting a comma-separated list specified as a string in one column as individual values in different columns)
Applying any form of simple or complex data validation; if failed, a full, partial or no rejection of the data, and thus no, partial or all the data is handed over to the next step, depending on the rule design and exception handling.

Load

After the data has been transformed, cleansed, it is now loaded into the data warehouse.

Spatial ETL

When we apply the above principles to spatial data, we call it Spatial ETL.

As we know, spatial data can suffer tremendously in accuracy, data formats, projections, datum's, etc.

With the advent of WFS and WMS and other Web Services and definitions - knowing how accurate and clean our data is is important - especially since in Web Services we are sharing data between systems.

A common method in the Well-known text (WKT). WKT is a text markup language for representing vector geometry objects on a map. It also relates to spatial reference systems of spatial objects and of the transformations between spatial reference systems. A binary equivalent, known as well-known binary (WKB) is used to transfer and store the same information on databases, such as PostGIS. The formats are regulated by the Open Geospatial Consortium (OGC) and described in their Simple Feature Access and Coordinate Transformation Service specifications.

Frank Warmerdam, the creator and maintainer of the PRO.4 and GDAL libraries of which I've talked about before covers quite well, WKT implementations and things to be aware of in several GIS products (some of the information is out of date - most users are now using Arc 9 and above).

Quoting below (from here):

Oracle Spatial (WKT is used internall in MDSYS.WKT, loosely SFSQL based)
ESRI - The Arc8 system's projection engine uses a roughly simple features compatible description for projections. I believe ESRI provided the WKT definition for the simple features spec.
Cadcorp - Has the ability to read and write CT 1.0 style WKT. Cadcorp wrote the CT spec.
OGR/GDAL - reads/writes WKT as it's internal coordinate system description format. Attempts to support old and new forms as well as forms from ESRI.
FME - Includes WKT read/write capabilities built on OGR.
MapGuide - Uses WKT in the SDP data access API. Roughly SF compliant.
PostGIS - Keeps WKT in the spatial_ref_sys table, but it is up to clients to translate to
PROJ.4 format for actual use. I believe the spatial_ref_sys table is populated using OGR generated translations.

ETL and Spatial ETL - I'll cover more of how data warehousing and GIS and internet mapping can be tied together at a later date.

If you are really interested, take a look at the MapBender project mentioned in earlier posts. It ties together many of the concepts of ETL and data sharing quite well.

If you have ideas for Blog posts that relate to GIS, Datum's, Map Projections, Oracle, Data Warehousing, let me know. Some ideas for future blogs are:

The African Geoid Project
Map Projections of the Middle East
Determining Mean Sea Level
NTv2 files and how to create them

Let me know. Contact me via e-mail.

Joint Industry Project: Geospatial Integrity of Geoscience Software

noreply@blogger.com (Terra ETL) — Fri, 26 Oct 2007 22:10:00 +0000

Devon Energy, Shell, and ExxonMobil and several other major oil companies have started a Joint Industry Project (JIP) entitled "Geospatial Integrity of Geoscience Software".

This project is being financed by the oil majors and is being undertaken with the support and co-operation of the International Association of Oil and Gas Producers (OGP).

As professionals involved mapping, software development, positioning, often the co-ordinate reference systems in the code are taken for granted. This happens in the oil industry in geoscience applications and interpretation packages.

In the past, software provided defaults (such as Clarke 1866 or NAD27 - as it was software developed in North America), but with the movement into global geodetic reference systems (such as WGS84), and many local datum's still being used, software may or may not be upgraded (for various reasons) or further modified (feel that it is working fine), etc., etc., there is the distinct possibility and fact that mistakes have been made due to software errors. The errors may have occurred because of the following reasons:

improperly coded or cartographic algorithms

wrong values for embedded geodetic parameters

poor presentation of user input requirements by software applications

incorrect defaults settings (as mentioned above)

software processes not working as specified (take a look at the Robinson projection discussion and cs2cs and the various work-arounds to account for a spherical representation)

confusing or imprecise terminology (take co-ordinate reference frames and datum transformations for example)

lack of error trapping for user errors

lack of an audit trail

inadequate metadata

inadequate training and documentation for users and of users

There are three main objectives of this Joint Industry Project, and they are:

To transform the management of geospatial data in geoscience software applications to benefit JIP members and improve products and competencies

To develop and disseminate best practice tools for current software applications and future software development

To create a sustainable improvement process in geoscience software applications based on sound geospatial management

By the end of 2007, the JIP has already begun to take a look at Blue Marble's GeoGraphic Calculator. This application and libraries is used in commercial code (such as Oracle) and many oil and gas companies use it on a daily basis.

An example of a possible wrong vertical co-ordinate system happened November, 1999 to Chevron. The article can be found here. I've also included it below:

Chevron Mulls Options After Platform Sinks, Friday, November 12, 1999

Chevron Corp. is assessing the impact on the development timetable of its North Nemba oilfield off the Angola coast after the sinking of the production platform on route from South Korea. The $175 million dollar structure was being shipped by the vessel Mighty Servant 2 early last week when it capsized near the Indonesian island of Singkep with the loss of four crew members. The so-called topside production platform is 230 ft. long, 105 ft. wide, 150 ft. tall and took 24 months to design and build. The vessel was enroute from the South Korean port of Okpo to Angola, having fueled in Singapore, when it began taking on water and sank. Chevron spokesman Fred Gorrell said the company was fully covered by insurance to replace the platform. The vessel was lying in 35 m of water with about 5 m sticking above the surface so recovery was still being assessed. Gorrell pointed out that even if it needed to be rebuilt it would not take as long as the original because design and engineering work was already done. The North Nemba field in the prodigious Block O offshore Angola was due to come into production in the first quarter of 2000. Block O, in which Chevron has a 39 percent interest, produced 510,000 bpd in 1998. Gorrell said he wasn't sure how much North Nemba was due to add to this. Chevron owns 39.2 percent of North Nemba, while the state Angola National Oil Co. owns 41 percent, with Italy's Agip owning 9.8 percent and France's Elf Aquitaine with 10 percent.

Co-ordinates, the software we use, whether in mapping or geoscience software plays a role in many of our decisions.

This Joint Industry Project is a good start and the people involved are knowledgeable in the field (many I've worked with when I was in Houston) and through this project we can hopefully know at the end, that the software we are using is providing accurate information and maintains geospatial integrity.

More details can be found at this website.

Molodensky: A Short History of Man's Contribution to Geodesy

noreply@blogger.com (Terra ETL) — Thu, 25 Oct 2007 23:55:00 +0000

As internet mapping becomes more and more prevelant and GPS is being used more we keep on hearing about the Molodensky Transformation, but who was Molodensky?

A Short History

Mikhail Sergeevich Molodenksy (1909 - 1991) was a promiennt geodesist and geophysicists who many consider a reformer in the theory of the figure of the Earth and the study of the Earth's rotation and oscillations.

He was born on June 15, 1909, in Epiphan, a small town in the Russian province of Tula. He did his initial studies at the Astronomic Department of the Mechanics and Mathematics Faculty of Moscow State University.

He was later invited by F. N. Krasovsky (there is an ellipsoid that bears this geodesist's name - more on a later blog about Russian mapping) to join the staff at the Central Research Institute of Geodesy, Aerophotogrammetry and Cartography (TsNIIGAik). It was here that he worked for over 25 years.

Molodensky's early steps in geodetic research and geodetic surveys date back to 1929 when he did some inital work for the Institute of General Geodetic Surveys (IOGR) and prior to this survey, he was given a proposal from the director of Astronomy - Geodetic Research Institute (AGNII) at State University in Moscow to do geodetic research.

There came a famous decree called the "Soviet of Labor and Defense" (May 6, 1927) which set about the establishment of a general gravimetric survey over the whole country. Lenin was laying out his vision for the country and defining a "Soviet". With this decree, there became an increase in the development of gravimetric surveying throughout the whole country.

Molodensky, who was avidly interested in gravimetry, participated in these surveys with his work at TsNIIGAik. While here and conducting these surveys, a young Molodensky, in 1933, headed up an expedition to the Crimea to perform gravimetric surveys (again under the above decree).

A Rigourous Solution

A year later, in 1934, Molodensky was beginning to make a name for himself by presenting a report at the 7th Baltic Geodetic Commission Conference in Moscow. His topic, which geodesists considered urgent at the time, is the co-swinging influence on double pendulums. Before his presentation, all solutions were seen as impossible because the accuracy of the solutions could not be determined. Molodensky provided a rigourous solution. In turn, this presentation was heard by scientists from Denmark, Finland, Germany, Poland, Sweden, the USSR and the members of the International Association of Geodesy. He essentially turned the world of geodesy upside down concerning astronomic-gravimetric leveling. His final report was published in Helsinki at the 1937 meeting of the Baltic Geodetic Commission.

Molodensky made significant developments to Soviet geodesy and gravimetry, in theory and in practice, especially when developing and applying survey methods combined with the design of gravimetric instruments. In the 1930's the only gravity measuring devices that could be found in the former Soviet Union were those of foreign manufacture. Therefore, the state saw immediately that there was an immediate task facing geodesy in the Soviet Union - the manufacture of gravity measuring instruments. A small batch were initially made, based on the German "Bamberg" instruments and several designs of original instruments had failed. It was not until 1938 that the Soviet Union had developed their own gravity measuring devices.

April, 1943, lead to the appointment of Molodensky as chief of the gravimetric laboratory at TsNIIGAik. He held this post until July 1956 when, against his will, he was appointed director of the Geophysical Institute of the Academy of Sciences (GEOFIAN). At this point he became responsible for the realization of the technical policies related to geodesy in the Soviet Union.

During this time he continued to make significant contributions to geodesy and the state wanted to recognize him for his efforts. In 1946, he was awarded the USSR State Prize, then he recieved the high degree of a Doctor of Technical Sciences and was then elected a member of the USSR Academy of Sciences.

The Geoid and Plumb-Lines

As you may recall in my previous blog, about the geoid, I discussed the "deflection of the vertical", well Molodensky's work played into this realm as well. Molodensky put forward the possibility of using gravimetric survey data for interpolation of plumb-line deflection between astronomic points of astronomic-geodetic networks. The result of this work permitted the integration of isolated sections of astronomic-geodetic networks into the main systems of co-ordinates. This made it therefore possible to map vast areas which had never been surveyed before.

Nowadays a similar process is in progress in Africa and South America and Canada with respects to gravity models which will help in determining Orthometric heights. Through Molodensky, lead to a determination of geoid heights.

In 1957, TsNIIGAik began to change direction in what it saw as important, and focused on solving more complex problems; such as the figure of the Earth, space exploration, defence problems, and the development of triangulation methods for large territories.

Famous Equations

Molodensky, though is more famous for a set of equations, that relate to datum transformation.

As we all know spatial data can have co-ordinates with different underlying ellipsoids or the underlying ellipsoids have different datums. The latter means that, apart from different ellipsoids, the centres or the rotation axes of the ellipsoids do not coincide. To relate these data one may need a so-called datum transformation.

In the early days of satellite surveying, when relationships between datums were not well defined and the data itself was not very precise, it was usual to apply a three parameter dX, dY, dZ shift to the X,Y,Z coordinate set in one datum to derive those in the second datum.

This assumed, generally erroneously, that the axial directions of the two ellipsoids involved were parallel. For localized work in a particular country or territory, the consequent errors introduced by this assumption were small and generally less than the observation accuracy of the data. As we collected more and more information about the shape and form of the Earth, and based on what Molodensky presented, among others, our knowledge and the amount of data that has been built up and as our surveying methods became more and more accurate, it became evident that a three parameter transformation is neither appropriate for world wide use, nor for widespread national use if one is seeking the maximum possible accuracy from the satellite surveying and a single set of transformation parameters.

The simplest transformation to implement involves applying shifts to the three geocentric coordinates. Molodensky developed a transformation which applies the geocentric shifts directly to geographical coordinates. This method assumes that the axes of the source and target systems are parallel to each other.

From a mathematical point of view a datum transformation is possible via 3 dimensional geocentric co-ordinates, thus implying a 3D similarity transformation defined by 7 parameters: 3 shifts, 3 rotations and a scale difference. This transformation is combined with transformations between the geocentric co-ordinates and ellipsoidal latitude and longitude co-ordinates in both datum systems.

The transformation from the latitude and longitude co-ordinates into the geocentric co-ordinates is rather straightforward and turns ellipsoidal latitude , longitude and height into X,Y and Z, using 3 direct equations that contain the ellipsoidal parameters a and e.

The inverse equations are more complicated and require either an iterative calculation of the latitude and ellipsoidal height.

A very good approximation of this datum transformations makes use of the Molodensky and the regression equations, relating directly the ellipsoidal latitude and longitude, and in case of Molodensky also the height, of both datum systems.

Various software uses the formulation put forward by Molodensky, whether used in cs2cs and ESRI software, or even in Oracle, the foundations where laid out by this man who has made significant contribution to our understanding of the shape and form of the Earth.

The Geoid - An Equipotential Description with Gravity

noreply@blogger.com (Terra ETL) — Tue, 23 Oct 2007 05:47:00 +0000

The Geoid is a surface that is not often talked about on blogs or mentioned on the web - so this may be a first.

The Geoid surface is irregular, unlike reference ellipsoids (such as Clarke 1866, Bessel, Hayford, etc.) which have been used to approximate the shape of the physical Earth at a local point. The geoid is considerably smoother than Earth's physical surface.

In looking for a good description that makes sense to many people, I stumbled upon this one on Wikipedia:

"In geodetic surveying, the computation of the geodetic coordinates of points is commonly performed on a reference ellipsoid closely approximating the size and shape of the Earth in the area of the survey. The actual measurements made on the surface of the Earth with certain instruments are however referred to the geoid. The ellipsoid is a mathematically defined regular surface with specific dimensions. The geoid, on the other hand, coincides with that surface to which the oceans would conform over the entire Earth if free to adjust to the combined effect of the Earth's mass attraction (gravitation) and the centrifugal force of the Earth's rotation. As a result of the uneven distribution of the Earth's mass, the geoidal surface is irregular and, since the ellipsoid is a regular surface, the separations between the two, referred to as geoid undulations, geoid heights, or geoid separations, will be irregular as well."

and further it states:

"The geoid is a surface along which the gravity potential is everywhere equal and to which the direction of gravity is always perpendicular. The latter is particularly important because optical instruments containing levelling devices are commonly used to make geodetic measurements. When properly adjusted, the vertical axis of the instrument coincides with the direction of gravity and is, therefore, perpendicular to the geoid. The angle between the plumb line which is perpendicular to the geoid (sometimes called "the vertical") and the perpendicular to the ellipsoid (sometimes called "the ellipsoidal normal") is defined as the deflection of the vertical. It has two components: an east-west and a north-south component."

The reference surface for heights is traditionally taken as Mean Sea Level (MSL).

The geoid, as described above, is a surface of equal gravity potential which closely approximates mean sea level.

With GPS becoming more and more relevant in our daily lives, what is the height measurement we get?

The heights derived from GPS are relative to the GPS reference ellipsoid (WGS84). The separation between the geoid and an ellipsoid is known as the geoid-ellipsoid separation, or N value.

In a mathematical sense, we have the following then:

H = h - N

where H = Orthometric Height

h = Ellipsoidal Height (for example, the height above the ellipsoid WGS84)

N = Geoid-Ellipsoid Height (this is also called the Geoid Undulation)

Note that with N, that if the geoid is above the ellipsoid, N is positive. If the geoid is below the ellipsoid, N is negative.

How does mass effect the geoid and the ellipsoid?

Where a mass deficiency exists, the geoid will dip below the mean ellipsoid and where a mass surplus exists, the geoid will rise above the mean ellipsoid.

Where are the largest undulations?

Well, the largest undulations known, with the minimum in the Indian Ocean at a value of N = -100 metres and the maximum in the northern part of the Atlantic Ocean with N = +70 metres.

So how do we describe the shape and size of the Earth?

There are three surfaces to be considered:

The topography - the physical surface of the earth.

The Geoid - the level surface (also a physical reality).

The Ellipsoid - the mathematical surface for computations.

Mean Sea Level (MSL) points, an approximation to the geoid, and can be used as reference surfaces for height measurements (i.e. orthometric heights).

Ellipsoidal heights (such as those derived by GPS) have to be adjusted before they can be compared to the orthometric heights given on topographic maps.

The deviation between the geoid and an reference ellipsoid is called Geoid undulation (N). Geoid undulations can be used to adjust the ellipsoidal heights (H = h +/- N).

This is an introduction to the Geoid and the science of Geodesy. It hopefully clears up some questions about this surface.

I'll explain in more detail some of the formulations of the Geoid and how geodesists over time have tried to model it and some of the efforts being conducted presently to come up with a global gravity model to aid in height determination at a later date.

There are some very interesting projects going on in Africa, South America, and Canada.

The Robinson Projection: Not shaped like an Egg

noreply@blogger.com (Terra ETL) — Sun, 21 Oct 2007 23:53:00 +0000

A Pleasing View of the World

The Robinson Projection came into being in 1963 and was introduced by Dr. Arthur H. Robinson.

This projection can be classified as a pseudo-cylindrical projection because of its straight parallels, along each of which the meridians are spaced evenly. The central meridian is also a straight line and all other meridians are curved.

The projection is neither equal-area nor conformal, therefore abandoning both for a compromise for creating what Dr. Robinson felt produced a better overall view of the world. This was the first map projection to be developed for commercial interests. Rand McNally felt that many of the map projections in use did not present the earth as a whole very well. With Mercator the poles were distorted. Robinson, was essentially contracted to develop a map projection that did not maintain angle, direction, or limit distortion, but was sanctioned to produce a map projection that "looked good" for books and atlases.

Remember maps are designed for one of the following 4 reasons:

Conformality - the shapes of places are accurate
Distance - measured distances are accurate
Area/Equivalence - the areas represented on the map are proportional to their area on the earth
Direction - angles of direction are portrayed accurately

Dr. Robinson specified the projection to be constructed by referring to a table of cartesian coordinate values at specific intersections of latitude and longitude. The intermediate locations are to be found by interpolation. Dr. Robinson developed the projection through a series of trials, continually iterating till he settled upon the meridian shapes and parallel spacing most pleasing to the eye. In comparison with other Map Projections, they are mainly developed or are formulated as mathematical equations.

Parallels are straight parallel lines, equally spaced between latitudes 38 degrees north and south. Space decreases beyond these limits. The Equator is 0.8487 times as long as the circumference of a sphere of equal area. The central meridian is a straight line 0.5072 as long as the Equator. Other meridians are equally spaced elliptical arcs and concave toward the central meridian. The scale is true along latitudes 38 degrees north and south, constant along any given latitude, and the same for the latitude of opposite sign (Robinson 1974; Snyder and Voxland 1989).

This map projection is also based on a sphere, not an ellipsoid. This is an important point to remember, as the earth is being modelled differently.

What is the true shape of the Earth?

The Earth, in actual fact, is shaped more like an egg, hence, even an ellipsoid is not the best model. In the past, when datum's were more local (such as NAD27, SAD69, Cape Datum, etc.), various ellipsoids were tied to local points on Earth. For NAD27, this was Meades Ranch, Kansas.

But back to the Earth's shape; it is very close to an oblate spheroid — a rounded shape with a bulge around the equator — although the precise shape (the geoid) varies from this by up to 100 metres.

The rotation of the Earth creates the equatorial bulge so that the equatorial diameter is 43 km larger than the pole to pole diameter.

What about height? A whole other story.

Interesting, eh? There are so many different ways to see the Earth. When we look at height systems and describe the geoid, we are describing an equipotential surface. But height is a whole other story, as gravity is involved and it is not as simple as getting the "Zed" or "Zee" measurement from your GPS.

I'll explain height later and how we can determine MSL (and where the "mean" actually comes from).

Google Maps: Is the Earth a Sphere or Ellipsoid?

noreply@blogger.com (Terra ETL) — Thu, 18 Oct 2007 18:02:00 +0000

Google Maps sees the Earth as a Spheroid, not an Ellipsoid. This came up through a discussion on the PROJ mailing list and I thought it was interesting to point out how Open Source can even handle projected lat/long systems (such as Google Maps) using a very familiar tool called cs2cs.

Christoper Schmidt wrote about it on his blog and also on his blog he points out the EPSG code to use. The magical number for the Google Mercator Projection (of a lat/long grid based on a sphere) is: 900913

Now onto the fun, showing how we can use Open Source to have our data show within a KML project and Google Maps. Quoting from Frank's FAQ, he provides an excellent example, we see the following the use of cs2cs:

"cs2cs +proj=latlong +datum=WGS84 +to +proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +no_defs"

Notice the sphere is being used? a=b

Because we are dealing with a sphere, the Y values will be greatly different from those on an ellipsoid (30 to 100 metres or more).

Quoting Frank again:

"In this case, and many other cases using spherical projections, the desired approach is to actually treat the lat/long locations on the sphere as if they were on WGS84 without any adjustments when using them for converting to other coordinate systems. The solution is to "trick" PROJ.4 into applying no change to the lat/long values when going to (and through) WGS84. This can be accomplished by asking PROJ to use a null grid shift file for switching from your spherical lat/long coordinates to WGS84.

cs2cs +proj=latlong +datum=WGS84 +to +proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs

Note the strategic addition of +nadgrids=@null to the spherical projection definition"

As you can see the value of Open Source and the mailing list and Open Source software is that people are sharing knowledge - whether it be via blogs, lists, or some other means of communication. There is a community out there that supports each other. These are actual users facing everyday problems and looking for solutions. The answers do exist, just the question has to be asked, and the community comes together to help.

Is WGS84 really WGS84? Is it correctly defined?

noreply@blogger.com (Terra ETL) — Fri, 12 Oct 2007 03:41:00 +0000

While doing some research on things that geodesists like - questioning MSL and the geoid, questioning how exact our definitions of ellipsoids and the earth are, I stumbled upon a very interesting article, written by Muneendra Kumar, PhD (Retired, National GeoSpatial Intelligence Agency) and James P Reilly, PhD (New Mexico State University) and they were asking:

Is definition of WGS 84 correct?

The following is written by these two distinguished gentlemen and I hope it proves to be as interesting to everyone as it was to me.

In the end, though, for most mapping purposes, WGS84 is still WGS84 and your latitude and longitude, or your Northing and Easting will remain the same.

As a side note, the North Pole is moving south and Sea Level is not level!

More on that little bit of geodesy in a later blog.

Enjoy and feel free to write me if you have any questions or comments.

Is Definition of WGS84 Correct?

by

MUNEENDRA
KUMAR, PHD AND JAMES P REILLY, PHD

A Geodetic Analysis

The first and original version of the "WGS 84", defined by a special committee of the Defense Mapping Agency (DMA), was released in September 1987.

As this task of updating the WGS 72 was concurrent with development of the North
American Datum (NAD) 1983, the committee members always had many in-depth
discussions with the members of the special committee of the National Geodetic
Survey (NGS). This approach ensured the; correct geodetic definition both for
WGS 84 and NAD 83. Around 1992, it was decided by DMA that, in future update(s)
of the "WGS 84" for accuracy enhancement, the academia and other satellite
geodesy experts would be associated. However, that scientific participation was
not followed and three subsequent updates were carried out without in-depth
discussions of satellite geodetic theory and/or correct statistical evaluation.
The non-scientific procedure(s) allowed definition deficiencies to creep in.
This paper outlines the geodetic details of the three updated versions of 1994,
1996, and 2001 and brings out in "open" the definition deficiencies in the
current version WGS 84 (G1150), which otherwise will remain hidden within the
National Geospatial- Intelligence Agency (NGA).The correctly defined "WGS 84",
the coordinate system used in GPS, is a critical requirement for the geodetic
integrity and accurate GPS positioning.

1984 "Original" Definition

The WGS 84 was originally defined with BIH Conventional Terrestrial System (CTS) for Reference Epoch "RE (84.0)". The main satellite data sets used were from the
Navy Navigation satellite System (NNSS). At the time of release in 1987, the
accuracy achieved was in the order of ± 1 - 2 meter and as such the tidal
effects, as specified in the International Association of Geodesy (IAG)
Resolution 16 of 1983 were not considered.

The "Three" WGS 84 Updates

1994 "WGS 84 (G730)"

This version was updated with the International Earth Rotation Service(IERS) realized International Terrestrial Reference Frame (ITRF) 19921, RE (88.0). During this update, NGA moved the RE (88.0) of the defining
ITRF to RE (94.0), which is incorrect. For this "change", DMA geodesists did not
have the capability and expertise. And, they did not have the authority to
override IERS. Note: With a new origin and orientation of its three axes, WGS 84
(G730) is geodetically a different coordinate system than the original WGS 84.
For mapping, the two could be considered the same. 1 First six ITRF solutions,
viz., ITRF 1988, ITRF 1989, ITRF 1990, ITRF 1991, ITRF 1992, and ITRF 1993, were realized for the RE (88.0). As the ITRF 1993 was based on all the data sets
available up to the end of year 1993 and thus realized in 1994, it would not
have been possible for DMA to define the WGS 84 (G730), which was realized using
the GPS data for the week starting 2 January 1994.

1996 "WGS 84 (G873)"

At the time of this update, the ITRF 1994, RE (93.0) was used (Note: ITRF96 (93.0) was not available). But, DMA geodesists again incorrectly moved the epoch of the
defining "RF" to RE (97.0). And, for geodetic application, they created the
third WGS 84. In addition, ignoring IAG Resolution No. 16 of 1983 and bypassing
IERS Conventions (IERS, 96), which recommend the "Zero-tide" model, National
Imagery and Mapping Agency (NIMA) geodesists adopted an "arbitrary" practice to
use "Tide-free" model. Note: According to IERS, the positions in the "Tide-free"
environ are non-realistic and not observable.

2001 Current "WGS 84 (G1150)"

During the updating of this version, the ITRF00, RE (97.0) was used.
But, like the 1994 and 1996 versions, NIMA geodesists incorrectly moved the RE
of the defining RF from (97.0) to (01.0), They also kept the 1996 practice for
"Tide-free" model, even after being alerted that the world's eminent geodesists
support the "Zerotide" of the IAG' standing Resolution No. 16 of 1983.
Furthermore, during the adjustment of the GPS tracking stations network, about
65% stations were held fixed (Note: An objection by the first author was not
even discussed). This over constrained adjustment is statistically incorrect and
not acceptable. Note: For geodetic positioning, this is the fourth version of
WGS 84.

The "Version" Identifiers

The "G730", "G873", and "G1150" indicate the GPS-week, of which the data sets were used to realize the three updates. As these "identifiers" do NOT specifically identify any definite time epoch, they do NOT have any geodetic significance.

Important "Contrast" To Note In SIRGAS 2000, the "RE" of the defining ITRF has NOT been "moved".

Analytical Conclusion

The current "WGS 84 (G1150)" is incorrectly defined, does not
comply with IAG Resolution No. 16 of 1983, and its time epoch is not definitive.
Furthermore, the adjustment of the GPS tracking station network is statistically
incorrect.

Reference

IERS, 96 IERS Conventions, Tech Note 21, July 1996.

Shapefiles and PRJ - Tying them together

noreply@blogger.com (Terra ETL) — Wed, 10 Oct 2007 01:16:00 +0000

ESRI has developed a de-facto standard for Shapefiles, but sometimes we as users' of Shapefiles forget something, a Shapefile is actually a minimum of three files.

The three mandatory/required files are:

.shp - the file that stores the feature geometry.
.shx - the file that stores the index of the feature geometry.
.dbf - the dBASE file that stores the attribute information of features.

A recent thread on the GDAL Mailing List led to the first link about Shapefiles. The key to the thread was a discussion about where Projections are stored in the Shapefile definition. As it can be seen, there is no location in the required files for projections.

Therefore ESRI adopted a new file type (PRJ) with the extension .prj.

So how are projections defined in this file?

As we know coordinate systems in terms of mapping can either geographic (longitude, latitude) or projected (X, Y). In the PRJ definition, the coordinate system is composed of several objects, with every object having a keyword in uppercase. Objects can be composed of other objects. ESRI calls the string in this file a Projection Engine (PE). The Sole purpose of the Projection Engine is to store the metadata for a coordinate system in a string, or in a .prj file. This string, which ESRI also calls a PE (not for Physical Education!) string, must be continuous and not broken.

Now the scary part: You can define your own units, datums, and spheroids!

An example, taken from the ESRI website, shows how we can define in .prj file a projected or geographic definition.

As we know projected coordinate systems (of which maps are) are based upon a geographic coordinate system (latitude, longitude), so the in their sample file, a projected coordinate system first is defined.

For example, UTM zone 10N on the NAD83 datum is defined as

PROJCS["NAD_1983_UTM_Zone_10N",
,
PROJECTION["Transverse_Mercator"],
PARAMETER["False_Easting",500000.0],
PARAMETER["False_Northing",0.0],
PARAMETER["Central_Meridian",-123.0],
PARAMETER["Scale_Factor",0.9996],
PARAMETER["Latitude_of_Origin",0.0],
UNIT["Meter",1.0]]

The geographic coordinate system name is followed by the datum, the prime meridian, and the angular unit of measure.

The geographic coordinate system string for UTM zone 10N on NAD 1983 is:

GEOGCS["GCS_North_American_1983",
DATUM["D_North_American_1983",
SPHEROID["GRS_1980",6378137,298.257222101]],
PRIMEM["Greenwich",0],
UNIT["Degree",0.0174532925199433]]

The full string representation of NAD 1983 UTM zone 10N is:

PROJCS["NAD_1983_UTM_Zone_10N",
GEOGCS["GCS_North_American_1983",
DATUM["D_North_American_1983",
SPHEROID["GRS_1980",6378137,298.257222101]],
PRIMEM["Greenwich",0],
UNIT["Degree",0.0174532925199433]],
PROJECTION["Transverse_Mercator"],
PARAMETER["False_Easting",500000.0],
PARAMETER["False_Northing",0.0],
PARAMETER["Central_Meridian",-123.0],
PARAMETER["Scale_Factor",0.9996],
PARAMETER["Latitude_of_Origin",0.0],
UNIT["Meter",1.0]]

As I stated earlier, you can define your own to use the predefined names for map projection and parameter object's.

I hope the above information aids in your understanding of Shapefiles and how projections and coordinate systems are defined.

My best advice for dealing with PRJ files; copy one that you already have and use it as a base, then modify the parameters as you need.

Good luck and have fun!

GeoTunis 2007 - November 15-17, 2007 - Tunis Science City

noreply@blogger.com (Terra ETL) — Mon, 08 Oct 2007 20:06:00 +0000

GeoTunis is occurring between 15th and 17th, November 2007 at the Tunis Science City in Tunisia.

As the website states: 'The task being an equal knowledge development and a stronger control of the digital information and telecommunications technologies with the purpose to decrease the digital gap between peoples. This symposium makes real the resolutions taken during the first national conference on map production "Geotunis 2006" and takes place in the same time with the International group world day celebration on the geographic information systems.'

OSGeo is hoping to be there as well, and information on OSGeo's participation can be found at here. This group is looking at promoting OSGeo and the Open Source Philosophy as it applies to Geospatial. They are also hoping to establish a stronger Francophone/French Speaking chapter that will include many French speaking nations.

I worked in Tunisia several years back with Schlumberger, involved with the Finder Data Management software and Tunisia's State Owned Oil Company - ETAP.

It is a beautiful country with great people and great food. The "Thé à la Menthe" and the "Chorba" are incredible.

With any luck, I'll make it to GeoTunis and be able to meet some new and old friends!

cs2cs & BeTA2007 - Open Source, Germany & NTv2

noreply@blogger.com (Terra ETL) — Sun, 07 Oct 2007 22:20:00 +0000

Recently in the PROJ.4 Mailing List there was a discussion about working with the DHDN / Gauss Krueger to WGS84 conversion.

One of the posters listed a site and document that covered BeTA2007 (which means Bundeseinheitliche Transformation fur ATKIS) and allowed me to brush up on my German.

In reviewing the document, we find out that the method of Operation for the datum conversion is based on NTv2 and the coordinate reference system is ETRS89/UTM.

Within the document listed above, they even demonstrate how to do a coordinate coversion using cs2cs and how to use the +nadgrids parameter.

An example from the document shows that the +nadgrids does not have to be specific to North America, only that the data file containing the grid has the same format. This format has been specified by the Canadian Government and has been adopted by many countries (such as South Africa and Australia, to name a few).

This is the published example, using the BETA2007.gsb file and a data file containing these two points:

(1) 2490000.00 5652000.00
(2) 2504000.00 5628000.00

>> cs2cs \
+proj=tmerc +lat_0=0 +lon_0=6 +k=1.0000000 \
+x_0=2500000 +y_0=0 +ellps=bessel +units=m \
+nadgrids=./BETA2007.gsb
+to \
+proj=utm +ellps=GRS80 +zone=32

Resulting in:

2490000.00 5652000.00 <-- Input (1)
279488.01 5654871.71 0.00 <-- UTM 32/GRS80 Answer for (1)

2504000.00 5628000.00 <-- Input (2)
292503.36 5630318.18 0.00 <-- UTM 32/GRS80 Answer for (1)

So cs2cs works with other grid systems as long as they are defined the same as the NTv2 standard.

So all the above makes more sense, let me define, from Wikipedia, what ETRF is: "The European Terrestrial Reference System 1989, usually referred to as ETRS89, is a three-dimensional geodetic frame of reference - a mapping coordinate system used as the standard high accuracy system for GPS in Europe. It coincided with the World Geodetic System 1984 in 1989, hence the name, and is based on the same GRS80 ellipsoid. Unlike WGS84 or ITRS it is centred on Europe and diverges from them with the movements of the tectonic plates associated with this landmass."

Please see: http://en.wikipedia.org/wiki/ETRS89 for ETRS89 and http://en.wikipedia.org/wiki/World_Geodetic_System_1984 for WGS84

The parameters for a datum shift (dx, dy, dz, rx, ry, rz, etc.) between ETRS89 and WGS84 are not constant, due to the movement of the Eurasian geophysical plate with respect to WGS84. The differences between both datums can grow by several centimetres a year. Currently they are a couple of decimetres in difference. For many applications, these differences are not relevant. Coordinates or positions in WGS84 have usually been obtained by GPS and this results in an accuracy at the level of several metres. However, satellite positioning techniques continuously improve in accuracy, also without using differential stations. So the differences between the datum's will grow. Note that many nations are using WGS84 to define boundaries nowadays - so WGS84 is relevant.

One poster from France, to the mailing list, replied that they solved the problem of moving plates by adding a date to the location, and they stated their method as follows:

"We have solved this problem with moving plates by adding a simple date with the coordinates of WGS84. That way you can always go back to the original position of that particular date in WGS84 (or any). The WGS84 datum (with date) will stay accurate for ever, since it is always possible to trace back where the plates were at that specific date.

latitude,
longitude,
(WGS84),
date"

There are always solutions and answers to questions out there. Just feel free to ask, whether it be to a mailing list or your local expert, people are always willing to help and pass on knowledge.

MapBender 2.4.3 Released & Online Training - An Orchestra of Data & Maps

noreply@blogger.com (Terra ETL) — Sun, 07 Oct 2007 03:10:00 +0000

While at FOSS4G2007 here in Victoria, I had my first introduction to MapBender and Arnulf Christi. My introduction was through a workshop entitled "Mapbender, Orchestrating the Geodata Concert". It was indeed a concert. Through the design of the software, you can pull together an instrumental or a ballad of data and imagery with ease.

Looking for a very good description of this orchestra, led me to their website, where I quote "Mapbender is the software and portal site for geodata management of OGC OWS architectures. The software provides web technology for managing spatial data services implemented in PHP, JavaScript and XML. It provides a data model and interfaces for displaying, navigating and querying OGC compliant map services. The Mapbender framework furthermore provides authentication and authorization services, OWS proxy functionality, management interfaces for user, group and service administration in WebGIS projects."

With Arnulf's planning and a strong understanding of MapBender, the workshop was a success, as we were Guinea Pigs for their online course. I do recommend to everyone new to MapBender.

I'm still finding out what software is out there in the OSGeo world, but MapBender ranks very high on my list of tools I want to keep in my toolbox.

The Press Release found on OSGeo states that minor changes were made, bugs fixes completed, a move to Trac (to keep track of changes), and Wiki to keep the project Human(e and) Readable. This is a key point, because sometimes reading code can be very inhumane (I know from experience!).

A full fledge training course (Online, I must add again!), can be found here.

I strongly encourage everyone to look at this project and delve into the training course, then look at adding MapBender to your list of tools that will allow you to build your Orchestra of Data & Maps from Services Worldwide.

EPSG Definitions and Searches Online

noreply@blogger.com (Terra ETL) — Wed, 03 Oct 2007 04:06:00 +0000

The EPSG has been around for what seems forever and the codes and definitions have made their way into Open Source GeoSpatial and Commercial software.

One difficulty has been the way many packages have implemented the database. Schlumberger converted the tables into CSV files for lookup with their initial implementation, then they adopted Mentor Software's approach and Mentor's C++ libraries. Different people see the data outside of the traditional MS Access approach and most often turn to some database that is not owned by Bill Gates! I think Larry Ellison was happy about this. EPSG realising this, then started releasing the data and data model in many different RDBMS formats (SQL scripts to create the tables and populate the database).

They released Version 6.14 on 2 September 2007.

There are various software providers and oil service companies that have taken the SQL and have produced web-sites that allow users to query the EPSG dataset in many different ways.

Three very useful sites are:

Petrosys - EPSG Coordinate Reference Browser

Developed by Petrosys for the oil and gas industry.

SpatialReference.org

Developed by Howard Butler and Christopher Schmidt. They had a very simple aim: "hopes assist others in their understanding, recording, and usage of spatial reference systems". And they are succeeding. The site provides various formats for the codes to implement into web-mapping and software development.

EPSG Database at Ocean

This EPSG viewer was developed by Concept Systems Ltd., division of ION Geophysical Corporation. ION was previously known as Input/Output (I/O) and on September 21, 2007 they changed their name to better represent their services.

Have fun with these sites, and if you have any questions about the EPSG database, do not hesitate to contact me at the Terra ETL Website, under About Us.

Autodesk donates CS-Map to OSGeo after Mentor Purchase

noreply@blogger.com (Terra ETL) — Tue, 25 Sep 2007 21:13:00 +0000

Autodesk announced today at FOSS4G2007 here in Victoria, that CS-Map is being donated to OSGeo and the Open Source community.

The Press Release shows Autodesk's commitment to Open Source and this paradigm of development and collaboration.

For anyone who has worked with Norm Olsen's libraries (CS-Map was developed by Mentor Software), you realise his work has filtered into many industries worldwide.

Schlumberger Information Solutions has used Norm's libraries for many years in the Finder Data Management software - one of the tools I used for many years while working for them.

Frank Warmerdam, OSGeo President, states in the Press Release "The latest planned contribution supports theprojections and transformations necessary to support over 3,000 coordinatesystems worldwide and has capabilities not previously available to the opensource community. I am also very excited to have Norm Olsen joining thecommunity and I look forward to increased community collaboration andinnovation-hallmarks of the open source community."

As you know Geodesy and Map Projections have been my bread and butter and passion for many years - along with good ol'databases.

I look forward to hopefully working with Autodesk and OSGeo in implementing these libraries into the community with Frank Warmerdam and his PROJ.4 libraries.

This is indeed good news for the Open Source community and OSGeo.

MapServer 5.0 Released - Enhancements & Performance Improvements

noreply@blogger.com (Terra ETL) — Fri, 21 Sep 2007 20:17:00 +0000

When I first started using Open Source for a project in Switzerland, I went immediately to the UMN MapServer. My first introduction to this was loading and using Shape Files to create a relatively simple internet mapping site - my first!

My initial impression of Open Source, was WOW! The ease, the speed and the amount of documentation and resources available made me a believer in the Geospatial Open Source world. I then moved onto loading the data into a PostGIS database developed by Paul Ramsey at Refractions Research here in Victoria. Again, WOW! This was not difficult. I had maps and a database up and running in less than a day and I was able to spatial queries as well.

Today, the Open Source Geospatial Foundation (OSGeo) proudly announced on their web-site a major release for UMN MapServer, Version 5.0.

In the Press Release it states:

"MapServer 5.0 is the first major release release since version 4.0 in July of 2003. While there have been regular releases every 6 months or so this is the first time developers felt the new feature set warranted the "major release" label."

Some of the new features include many small bug fixes, more enhancements and performance improvements.

A detailed listing of the new features include:

style and label attribute attribute binding;
lookup table-based raster color correction;
dynamic charting (pie and bar); explicit label prioritizing;
enhanced debugging and logging;
dynamic allocation for layers, classes, styles and symbols;
improved memory management and garbage collection for MapScript;
numerous improvements for OGC service support.

This release also incorporates map rendering using the Anti-grain Geometry (AGG) graphics library. According to the Press Release this bring higher and better maps to the internet because of improved cartographic quality.

As the Press Release, and I concur, I to am "excited about the future possibilities of bringing high-end cartography to on-demand web mapping".

My hats off to Steve Lime and the UMN Developers worldwide!

Interesting Fact about Oracle Spatial

noreply@blogger.com (Terra ETL) — Tue, 18 Sep 2007 23:50:00 +0000

I was interested in doing some more work with Oracle Spatial and how it compares in speed to some of the other databases out there that work with Spatial data, such as PostGIS and MySQL.

While reading through the Wikipedia definition of Oracle Spatial I discovered that some Canadian's at the Canadian Hydrographic Service first incorporated spatial data into Oracle with Version 4.

This was even before my time. I started working with Version 6!

Looks like the CHS was ahead of their time.

During the period between Version 4 and the introduction of "SDO" (Spatial Data Option) in Version 7, the CHS and Oracle rewrote the kernel. As anyone who uses Oracle Spatial knows, the tables are prefixed with SDO_.

Just an interesting tid-bit about Oracle. It is amazing the information you can find out there on the web - with Wikipedia, one wonders "Where will Encyclopedia Britannica find a home in the digital world?"

MapGuide Open Source 1.2 Released & DM Solutions Fusion

noreply@blogger.com (Terra ETL) — Fri, 14 Sep 2007 19:26:00 +0000

MapGuide Open Source 1.2.0 has been released.

With this release of MapGuide Open Source, there have been enhancements that include:

1) Support for Unmanaged Data Sources
2) Cartographic Enhancements - Phase I

This includes the ability to use new symbols for Points and Labeling Linear Features (e.g. Highway Shields)

3) Feature Join Enhancements
4) Support coordinate system overrides on feature sources

More details can be found here.

Point 4 is interesting, as in MapGuide Open Source, there is within a feature source, an optional tag called SupplementalSpatialContextInfo. If an entry exists for a given Spatial Context, then the specified coordinate system override will be used. The coordinate system override will be given preference over the coordinate system defined in the data.

In the previous version, the SupplementalSpatialContextInfo was used only for data containing spatial contexts with undefined coordinate systems (I'll talk more about co-ordinate systems and map projections in future blogs). This SupplementalSpatialContextInfo would allow a coordinate system to be specified for a given spatial context. Essentially, the new SupplementalSpatialContextInfo can now be used to override coordinate systems for all spatial contexts in a feature source -- regardless of whether it contains a coordinate system or not.

Why did this modification come about?

Sometimes a feature source geometry may contain an incorrect coordinate system or does not have a coordinate system defined. What can be now? The above solution provides a convenient way to change the coordinate system without having to change the data of the feature source.
MapGuide Open Source handles spatial data with a specified coordinate system and with this modification it allows for the specification of a coordinate system override thereby letting the developer use data that in the past you may not have been able to use.

Symbols and Labels

Many GIS and Mapping applications are in need of very sophisticated symbolization. In the oil and gas industry, we see so many symbols and sometimes one symbol may have more than one definition (dry well, dry & abandoned, etc.).

In many parts of the world, countries published maps must adhere to high standards that are often even written into law. Amazingly though, sometimes even these maps are missing some of most standard information, such as Datum and Map Projection. The Cartographic Enhancements - Phase I introduces a high quality symbolization engine for MapGuide Open Source and allows for point symbols to be used as labels. This is especially pratical in the E&P industry.

Leaders Collaborating - MapGuide Open Source Fusion

As it can be seen Autodesk and the Open Source community are working together successfully in producing a user-friendly, practical, and powerful web-mapping tool.

DM Solutions and Autodesk have been working together on MapGuide Open Source Fusion. Dave McIlhagga and the DM Solutions group provide a very good description of Fusion here.

Take a look at how collaboration between these two companies have helped to move MapGuide ahead. Fusion is also available for UMN MapServer.

Open Source - A new software development paradigm?

Open Source development has been around for awhile, but it looks like it is gaining more momentum and with Autodesk's participation, we are seeing leaders in CAD and GIS realizing the advantages of working collaboratively with others to develop products that can be used and accepted by others.

I'll be talking more about Open Source Software development and some of the business models used and the many advantages of using Open Source in many different industries.

GDAL2Tiles - Summer of Code - EPSG:4326

noreply@blogger.com (Terra ETL) — Sat, 25 Aug 2007 03:52:00 +0000

Being involved in the Open Source world as it applies to Geospatial is interesting.

Google does almost every year, a Summer of Code, and OSGeo was involved in this as well. More information can be found on the Wiki about this years projects.

Tiling and speed have always been issues with Internet Mapping - especially with Raster Images.

Just recently Klokan Petr Pridal described his Summer of Code project as being able "to allow easy publishing of raster maps on the Internet. Your raster file (like TIFF/GeoTIFF, MrSID, ECW, JPEG2000, JPEG, PNG) is converted into a directory structure of small tiles (TMS compatible), which you can just copy to the webserver. Simple webpages with viewers based on Google Maps and OpenLayers are generated as well - so anybody can comfortably explore your maps on-line and you do not need to install or configure any special software (like mapserver) and the map displays very fast in the webbrowser. "

Through Open Source we are seeing innovation in the way software is being developed and the speed of take-up and understanding by developers worldwide. Tools, such as the one's developed by Klokan will soon be on internet maps worldwide, using libraries that have being developed by Frank Warmerdam (FWTools, GDAL) to make mapping easier.

Of course, there are always co-ordinate system problems involved and some restrictions. Currently the software (if using Google) is restricted to EPSG:4326 (Good ol'WGS84!). There is one line that scares us Surveyors - "World files and embeded georeference is used during tile generation, but you can publish a picture without proper georeference too".

No Georeference means - where are we? Be careful out there. Co-ordinates are always important in any mapping application. It is always important to know your:

1) Datum

2) Map Projection

3) Ellipsoid

as a very bare minimum. Depending on the Map Projection used, then you always have to consider the False Easting, False Northing, Central Meridians, Latitudes, etc..

As the technology increases and becomes faster and easier, we can tend to make mistakes easily and make too many assumptions about the data. Not all data is EPSG:4326. Remember, maps are only as good as the people who produce and understand them. When you have people reading and relying on your maps, especially when pulling data (for eg. via WFS), there may be different co-ordinate systems - hence roads may move, lakes may be on cities, etc., etc..

As the blog posting several days back pointed out, boundaries are important and the idea of Redrawing the Map was discussed. We'll see what happens in resolving this political dispute, but again the accuracy of the data and the maps produced will be important.

Now, for an excellent example of GDAL2Tiles - Summer of Code, take a look at the beautiful country of the Czech Republic and see some tiling and speed that can put some commercial software to shame.

An example of tiling using Open Layers is the city of Moscow.

See if you can find the Kremlin and Red Square.

Keep on watching this space and I'll bring more news about Open Source for GeoSpatial and someday soon will be producing some samples using Oil and Gas, Mining and Forestry Data.

It is a whole new world out there and geomatics is changing rapidly.

FOSS4G2007 - Victoria, BC

noreply@blogger.com (Terra ETL) — Mon, 20 Aug 2007 23:25:00 +0000

FOSS4G2007 is happening soon in the current location I've planted some roots - Victoria, B.C., Canada.

Who knows whether these roots will be permanent or not. It seems I move every two years, whether in North America or overseas. The world seems so small with Air Travel nowadays.

The opening and closing plenary speakers have all be selected and they have accepted. The keynote speakers :

Geoff Zeiss, Director of Technology, Autodesk
Tyler Mitchell, Executive Director, OSGeo
Peter Rushforth, Technology Advisor, GeoConnections

There is also a series of Lightening Talks, consisting of eight speakers, with five minutes each, that will try to being as concise an overview of open source geospatial software, community building, open geodata projects, and many demonstrations. Demonstrations always seem to grab the attention of the listeners. A picture is worth a 1000 words!

Damian Conway, will be opening the conference, with his observations of open source culture in Geek Eye for the Straight Guy.

The closing plenary will include a panel discussion featuring:

Peter Batty, Independent Consultant
Tim Bowden, Director, Mapforge Geospatial
Mark Sondheim, BC Integrated Land Management Bureau
Frank Warmerdam, President, OSGeo

And of course, I'll be presenting there, about map projections, datum's, and co-ordinate systems and how to make your way around them using Open Source Geospatial Software.

I'll be instructing this lab with Frank Warmerdam (the creator of GDAL, PROJ.4, and many other Open Source libraries).

L-03: Datums, Coordinate Systems, Map Projections & Datum Transformations

Location. Location. Location. With so many maps and datums out there, how does a person know what datum is correct? How come my GPS coordinates don't match up on my map? Why is there a shift of 100 metres? How do I transform between different datums? What is a datum? What is the EPSG? Why have GIS Vendors and Oracle adopted them? Does offshore or onshore make a difference? How come there are so many datums? This presentation looks to provide some answers to some of these questions and to point out that latitude and longitude are not absolute.

Over the decades that surveyors have been trying to map the Earth, history and politics have shaped the way we see the world. Are the borders actually there? What if one nation adopts a standard, but the other does not? Does really matter what the co-ordinate system is? Why when I draw the a UTM Projection, the lines are curved, not in a grid? Is the OGC adopting these standards? So many questions and this presentation aims to answer some of them and provide some light on a complicated and sometimes unclear topic.

Hope to see everyone there. It looks to be a great conference!