ABSTRACT

We propose a new solution for data integration and semantic enrichment in support of Situation Management (SIMA). Our solution applies to any modality (e.g. text, images, audio, signals etc.) and embraces the diversity of data sources, types, and models, placing no restrictions on processes, applications, or users. It is database centric and proceeds in stages to address the unified storage of structured data and its semantic enrichment in a way that remains viable in an Ultra-Large Scale systems environment. The result is a layered data integration architecture that can accommodate any kind of data to coherently support the multiplicity of processing required for SIMA.

full paper

The principal problem spanning the Intelligence Community today is how to integrate the great variety of disparate data into one single coherent repository of knowledge. Current practice whereby all data-models would be merged into a single “Uber-model” simply does not work. We require a solution that remains viable in a freely evolving, interdependent collective of human and computational systems, very little of which will ever be under our control. Our approach is database-centric and proceeds in stages. The first addresses the unified storage of the broad spectrum of artifacts existing within the Intelligence Enterprise today regardless of modality or representation. The second builds upon the foundation provided by the first to address the unified storage of structured data and semantic data integration. In both we embrace the diversity of data-models employed throughout the Intelligence Community. The result is a layered data architecture that can accommodate any kind of data without placing restrictions on vocabulary, structure, semantics, or constraints in a way that addresses today’s Intel needs while providing a seamless transition path toward a future of ULS systems imbued with semantic technologies.

Full Paper

Stephen G. Eick, Colin Alworth, Alex Decherd, Jesse Fugitt, Brian Horst, Andrzej Wolosewicz, Andrew Eick

Abstract—The focus of “Business Intelligence” is to enable corporations to analyze business data and gain insights that result in better business performance. One type of data that is underutilized by corporations is that involving locations. It is estimated that 80% of all data maintained by corporations has a location component, either directly involving longitude and latitude or indirectly by geo-encoding information such as addresses. To analyze geospatial business data, we have developed a thin client Web 2.0 AJAX platform called thincVIew™. Using thincVIew it is possible to develop rich mashup analytical sites. Although thincVIew is superficially similar to Google Maps and Microsoft’s Live Maps, the difference is that thincVIew’s API is designed for analysis and correlation rather than providing directions and serving up ads. In this paper we describe thincVIew and illustrate its use in a retail case study analyzing customers and store locations and shelf-level inventory and sales performance.

Index Terms—visual intelligence, geospatial analysis, mashups, Web 2.0, thinc client, GeoRSS, KML, Retail Analysis, Planogram

Introduction

Our focus is on developing mashup sites for analyzing time-stamped, geo-encoded business data. This type of data is quite common and includes, for example, streams of news articles where the events in the articles are geo-located, the sequence of observed positions of a delivery vehicle as recorded by an onboard GPS, the positions of emergency personnel with active RFID badges in a hospital, customer purchases through time geo-encoded by customer address, and even item sales encoded aisle, shelf, and position within a retail store.

Although it is possible to display this information using Google’s or Microsoft’s web-based mapping applications, these platforms do not permit rich analysis. The problem is that these web mapping applications are intended to support searching, to provide directions, and to serve up local ads. Both Google’s and Microsoft’s sites are advertising supported. Thus the APIs for these applications are optimized to serve up relevant ads. Using these platforms it is not possible to correlate geospatial information by time, by event type, by trend, etc. to discover important analytical relationships.

Furthermore, traditional GIS systems are not designed for web-based analysis of time-oriented geospatial business intelligence data. These applications are often overwhelming complex, run on high-end desktop workstations, and are optimized to analyze geospatial layers. What is needed for business users are lightweight web-based tools that enable business information to be encoded and analyzed on both traditional geospatial maps and non traditional maps such as a floor plan layout.

To address this need we have developed a Web 2.0 AJAX web mapping application called thincVIew™ aimed at the business intelligence market. Our technology shows geospatial information on a lightweight map, on a timeline, and in linked analysis charts. Presenting the information in multiple views enables analysts to correlate the dimensions and discover relationships that are hidden when the data is displayed in a single perspective. As a result they are able to perform richer and deeper analysis. This enables users to find patterns that are hidden in existing presentations.

Although “linked view” visualization systems are well known, we believe ours is unique in that it is totally browser-based. In addition to our maps, we have developed a set of thin client visualization components, e.g. timeline, time wheel, node and link graphs, and standard business charts that work with our web application to provide a rich analysis environment. What is innovative about our components is that our web application is nearly as responsive and dynamic as the incumbent thick desktop applications. Further, ad-hoc and “what-if” scenarios can be explored by the analysts using our technology. With our web application analysts are able to combine data sources to create a mashup web site and then on the fly instantiate linked analysis charts. Then using sweep selects, highlighting, or filtering, analysts may interact with any of the charts to explore different dimensions of the data. In this way, they are able to discern patterns hidden in the data, giving them different insights into the problem.

Figure 1. thincVIew geospatial web application.

Our web application implements and extends many of the ideas embodied in traditional mashups. It can ingest live or static data from several different sources and present it for rich analysis. Furthermore, by building on web technology, we can use ideas from the social networking sites. Analysts may edit and annotate the information in our web application and thereby create shared work products. The edits and annotations propagate automatically to other browsers on the site. In the remainder of this paper we describe our technology and web application in detail.

thincVIew Analytical Web Application

thincVIew includes a light weight JavaScript map, interactive timelines, and linked analysis components. In contrast to existing thin client mapping applications, thincVIew is an analytic application that provides a rich interactive environment for analyzing time-based geospatial event data. As shown in

Figure 1, thincVIew ingests RSS and GeoRSS streams and creates an interactive mashup. Each GeoRSS item is displayed on the map and in the timeline. RSS items without a geospatial element are only displayed in the timeline. The environment provides linking and several correlation tools to help analysts gain insights from the information. By combining the visual components of a map on a single web page an analyst creates an integrated analytical mashup. This approach to analysis is sometimes called Visual Discovery (Eick, Visual Discovery and Analysis, 2000). The advantage of multiple components is that each presents the mashup events using a different visual metaphor which might be more appropriate to the dataset.

Washington DC Metro System with Crime Data

Figure 1 shows the Washington DC Metro System with the locations of street crimes. The street crimes are both time and location-stamped to when and where the crime occurred and appear both on the map and in the timeline. The map imagery is provided from an OGC Web Mapping Se
rver (WMS). WMS is a standard interface for clients to request rasterized image and map tiles. The figure shows when the crimes occurred, where they were, and how they correlate to metro stops and the metro line.

The thincVIew map and timelines are interactive. The user may grab the map or timeline with the mouse and drag it around. As this occurs, the map and timeline pan and zoom smoothly. The way this works is as the map is panned the JavaScript asynchronously requests new image tiles. When the user zooms in, the map animates between zoom levels using five discrete steps. These trips make the interface feel smooth to the users. This style of user interface is sometimes called a Direct Manipulation (Shneiderman, 1981).

[more]

Ingesting GeoRSS Streams

GeoRSS is a flexible way to ingest geospatial event data. In

Figure 1 the metro system routes, subway stops, and street crimes are each pulled from separate GeoRSS feeds. The feed for the metro lines consists of a static sequence of line segments with color codes for the different subway lines. Similarly the feed for the metro stop is also a static list of geo-positions represented as items in the GeoRSS feed. However, the street crime feed is a list of items each of which represents a crime with a location to specify where the crime occurred. To specify the location within the GeoRSS specification there is a namespace extension node where arbitrary shapes may be specified using GML. For example, Figure 2 shows the GeoRSS item describing the Orange Line. The physical location of the of the Line is specified using a as a sequence of latitudes and longitudes.

Within the GeoRSS specification there is no capability to define drawing styles, e.g. colors and line thicknesses, for the metro routes. To address that need we extended GeoRSS using a namespace extension to add styling properties and certain shapes that we needed for our application. The shapes we added are ellipses, sectors, and slices of a circle. Although these shapes could be represented in GeoRSS using a GML generic polygon, it is rather cumbersome. To avoid any namespace clashes, we put all our extensions into the thincml namespace. For example, the specification

represents a sector of a circle or a pie wedge if the inside radius is zero.

Figure 2. GeoRSS item representing the Orange Line. The thincml properties set the default drawing style.

The thincVIew web application has the capability to ingest an arbitrary number of GeoRSS streams. See

Figure 3 (left). A stream may represent static information such as the locations of the Washington DC metro routes or dynamic information such as recent earthquake locations. For dynamic feeds thincVIew monitors each data stream and periodically polls it to access the latest information.

Figure 3. Left: ingesting GeoRSS feeds as layers. Right: Analysts may override the default styling of the items embedded in the feed.

thincVIew Interactive Timeline

As shown in Figure 4, the thincVIew timeline is a light weight JavaScript web component for visualizing and interacting with time stamped data. The timeline includes a number of innovative features including:

· Positions GeoRSS items that appear on the map on the timeline

· Supports interactive panning and zooming

· Provides synchronized detail timeline and overview timelines

· Provides both watch mode where new events are automatically positioned and analysis mode where the display does not automatically update

· Pans to most recent events in watch mode

· Provides multiple time scales including hour, day, week, and month

· Links to other visualizations such as the map using tooltips and selection

· Provides rich API to program against

· Includes built in toolbar

· Increase visual scalability using an over plotting indicator (Eick & Karr, Visual Scalability, 2002)

· Uses a best fit algorithms to position labels to avoid over plotting

The timeline visualization itself can be thought of as a series of rectangular tiles that each represent one unit of time and are placed horizontally to fill up a specified number of pixels on the web page. The unit of time represented depends on the scale of the timeline, which can be set to Year, Month, Day, Hour, Minute, or Second.

Figure 4. Dynamic timeline displays RSS items. The timeline is linked to the map and other visual widgets and automatically pans when new items become available.

thinvView timelines are interactive. As the user drags a timeline to the left or right using the mouse, new tiles are created and destroyed to give them the impression they are “panning” through time. This style of interaction is intuitive and easy to understand for users. In addition, simple buttons on the toolbar allow the user to zoom the timeline in or out, change the current timeline scale, pan to the most recent event, and reset the timeline scales.

Figure 5 shows synchronized overview and detail thincVIew timelines. The overview timeline is set to Month scale and the detailed timeline is also set to Month scale. Each tile is labelled with its starting date and end date. Small vertical dashes are placed intelligently on the tile to indicate familiar time intervals and subintervals. For instance, day tiles would show each hour of the day as the smaller subinterval marks and the larger tick marks are used to divide the day into quarters. The tiles are automatically generated and labelled using configurable settings for background color and date format. The advantage of using two (or more) timelines is the fact they can be synchronized together. As the user manipulates one of the timelines, the other timeline scrolls accordingly. The blue lens in the center of the bottom timeline represents the entire time period displayed in the top timeline. This capability is desirable when attempting to display a summary level and detailed view of the same data.

Fig
ure 5. Overview and detail timelines are synchronized to show multiple timescales.

The rendering and interaction code on the thincVIew timeline is sometimes tricky. For rendering, the simplest approach is to plot objects corresponding to the data items on the timeline canvas. This approach could support basic interactive operations like tooltips, selecting and highlighting individual items or toggling an item’s visibility. However, implementing more advanced features such as hiding a group of items becomes computationally expensive in JavaScript. A second disadvantage of this approach is that it does not extend gracefully to display data from several RSS feeds.

To overcome these problems, our implementation renders events from distinct RSS feeds separate transparent overlays on top of the raw timeline canvas. Each overlay is independently populated using the appropriate feed and styling properties. The advantage of this approach is that it is easy to add or remove a feed from the timeline or to configure properties of a feed, such as toggling its visibility, because the feed corresponds to a single overlay on the timeline which can easily be manipulated by setting DOM properties.

Selection and Linking between the Timeline and Map

Sometimes a user wants to know when a closely related set of geospatial events occurred or where a set of temporally related events occurred. To help answer these questions, data items on the thincVIew map and timeline are linked to enable analysts to spot temporal and spatial relationships. Data items are linked in three ways. First, when the user holds the mouse over a data point on the timeline, the corresponding data point on the map is highlighted.

Second, as is also shown in Figure 6, the user is able to sweep select groups of items. In this mode, the user sweeps out a region and thereby selects a set of items. Instantly, these items are highlighted on both the timeline and map displays. In Figure 6, the user is in the process of selecting a group of items on the timeline, which will also tell “where” these events occurred by watching the map display change as the linked items are selected. Sweep selecting on the map tells the user “when” the events occurred by observing the changes in the timeline. In the example the sequence of selected crimes that occurred over a several hour period all took place along the red line. More importantly, however, this operation causes the selected data items on the thincVIew map and timeline instantly to be highlighted as shown by the white outlines, and thereby helps the user visualize the “where” and “when” about the object’s events. Selection linking unifies the data across the visualizations and allows the analysts to construct a mental model of the events in both time and space.

Third, the user also is able to change the glyphs and icons on the display. In this scenario, showing crime patterns, the crimes are displayed as simple color-coded circles with color indicating the crime type. The user could, for example, go into the properties of the data feed, and change the styling so that “robbery” crimes display as yellow triangles, and “violent crimes” to appear as purple squares. In this way, the user is able to stratify the visualization, searching for latent patterns.

Figure 6. Sweep selecting a time range highlights the locations of these items on the map.

Minimizing Clutter for Real-time Feeds with Fadeout

In monitoring mode the thincVIew constantly polls for the new events and renders these events on the map and timeline. Eventually the map will become cluttered with too many items and too much information. For situational awareness applications, old information is often not relevant to the current state and is distracting. To help minimize visual clutter thincVIew has several options involving event fadeout. These include:

· Polling frequency which determines the frequency that the RSS provider is accessed to request new items

· Fade Out period which cause items on the map and timeline to gradually fade and disappear to avoid cluttering the display with stale information

· Pan to latest item causes either the map, timeline or both to pan to any new events as they arrive

Figure 7. Options for live data feeds to set the polling rates and item fade out period.

Ad Hoc Visual Analysis with Linked Charts

Viewing data on a thincVIew map and timeline is helpful, but is limited in that it is only possible to spot certain types of patterns and outliers in the data. What is missing is a capability to correlate spatial and non spatial dimensions. To overcome this limitation, thincVIew lets the analyst create linked sets of visual components. Each visual component shows the data items using its own visual metaphor. The items in each visual metaphor are linked so that tooltips and selections propagate among the visual metaphors.

Using thincVIew a user performs ad hoc analysis by dynamically creating linked visualizations. In the example, as shown in Figure 8 and Figure 9, the user instantiates analysis components dynamically. These analysis components are resizable and dragable, and can even be minimized into the data tray at the bottom. Being live windows, as new data points are ingested into the system, the visual components will re-render themselves as needed.

With this powerful tool, the user is able to construct a mental model for the dataset, and do fine grained analysis of portions of the data as they expand and peer into the data. Since the tools are dynamic, the user is able to filter the data, and then construct graphs from the browser. For example, in our sample dataset an analyst may use the bar chart to filter the crimes data by selecting the “Robberies” and then create a time wheel to see when the crimes were reported, to discover if there is a pattern in the time dimension. The time wheel is a 24 hour clock with the day of week shown as concentric circles starting with Sunday. By displaying the crimes on the map, the analyst is able to see if there is a geographic pattern as well. Since the displays are linked, the user is able to sweep select regions, and see them highlight on the other visualizations. With these linked components, the analysts are able to discern whether a robbery is a single crime of chance, or part of a larger pattern indicating more organization. The analyst is able to quickly stratify the data and look for patterns. While the filtering engine is not the industrial strength query analysis tool that today’s database engines provide, the lightweight and ready access to the data enable the user to arrive at conclusions quickly to take action.

Figure 8. Ad hoc analysis using linked visual metaphors runn
ing in a browser.

Figure 9. Selections and tooltips propagate among the linked visual components, map, and timeline.

Selection and linking helps users identify patterns in the data which are difficult to see without these charts. For example, the bar chart shows that the number of thefts is about 2.5 times greater than the number of criminal damage complaints which is approximately 50% larger than the number robberies. Selecting the crimes on the time wheel which occurred late at night and correlating these items with their locations on the map shows that most of these crimes occurred along the red line. Noticing these events in the time line shows that that occurred on a sequence of three days. Furthermore, there appear to be two distinct patterns of events on the time wheel. These are a sequence of crimes in the early evening and another sequence in the late evening and early morning. There is one exception. One criminal damage incident occurred in the early morning as indicated by the single green circle.

Case Study: Multi-level Analysis of Retail Purchasing Patterns

In recent years, more and more retailers are offering customer loyalty programs. These programs have many different variations but they serve two basic roles. The first is to help keep current customers coming back by offering incentives or programs that reward loyalty to the retailer. The second is to provide valuable market research data for the retailer on customer demographics and buying habits. However, as may be obviously apparent, the resulting dataset is enormous with significantly more variables than any person could ever analyze. The thincVIew application provides a set of visualizations that enable the user to interact and interrogate their data quickly and efficiently through a web browser.

Store and Customer Location Analysis

In our example, a hypothetical retailer has two stores within the Washington D.C. metro area and a handful of customers who participate regularly in the loyalty program. The customers have been grouped into three unique segments: “brand aspirationals”, “value-price shoppers”, and “price-sensitive affluents.” Instead of writing and processing database queries in a desktop application, thincVIew enables the user to identify an anomaly in the shopping habits of two customer segments, as shown in Figure 10. The analysis problem is to understand the shopping patterns of the different customer segments. There are two stores locations in the image indicated by red circles. Each customer transaction is shown on the timeline and the customer’s home address has been geo-encoded and positioned on the map.

The images in Figure 10 show geospatial and temporal shopping patterns of the customer segments. The left image shows that the “brand aspirationals” prefer to shop at the northern location whereas the “price-sensitive affluents” segment prefers the southern location.

Figure 10. Locations of customers color coded by customer segment. The two images show that the “brand aspirationals” customer segment shop at the northern location whereas the “price sensitive affluents” segment shops at the southern location.

Additionally, beyond seeing the result of the geographical analysis indicating customer preference for a store by customer type and not location, the linked functionality of the visualizations enables the user to also review what specific customers are buying and when they are shopping. Individual transactions are shown in the timeline color-coded by the customer segment.

Improved data analysis can help drive better store placement, leading to improved sales per square foot, fewer markdowns, and higher returns. Using existing data from point of sale and merchandising systems, thincVIew provides a visual capability for understanding customer buying patterns. Users can quickly see buying by location, by customer segment, by geography, by time of day, and can fuse in other business data to make more informed business decision. thincVIew’s ability to combine business data such as customer loyalty information with customer demographics provides an integrated geospatial view of customer behaviour.

With this type of information, retailers can perform site selection for new stores, identify growth opportunities, and identify opportunities for consolidation. Furthermore, retailers can identify buying patterns within or among key customer segments which could drive more targeted promotions, better merchandising, improved product mix, or even modified store hours for greater sales per square foot. For example, Figure 11 shows an analyst correlating purchases, customer location, customer segment with product category.

Figure 11. Correlating customer purchases with customer segments.

Floor Plan, Aisle and Shelf Performance

While analyzing customer data to improve store performance is a specific application, the flexibility of thincVIew enables a user to perform many different types of analysis. Starting again at the same display of customer and store locations, the user can zoom from a geospatial map down to a floor plan (Figure 12) for an individual store and even down to a shelf view (Figure 13). These second two applications are interesting because they involve non traditional maps. In Figure 12, for example, the aisles are color-coded to show sales performance.

Figure 12. Sales performance by aisle correlated with product and item category.

Shelf Performance using Planograms Analysis

While the planogram is a key tool for space planning and product layout, this visually rich tool is often not used for measurement but replaced by simple bar graphs and pie charts. As shown in Figure 13, thincVIew enables retailers to dynamically link actual product performance data to each planogram, whether it’s store specific
or down to a shelf level. By integrating with perpetual inventory systems, POS data, RFID or scanner data, etc., thincVIew provides a visually-driven measurement tools for retail analysis.

Figure 13. Shelf-level performance correlated with inventory level and product positions.

Because thincVIew is accessible via a web browser, retailers can allow key suppliers access to this data as well. They can quickly gauge compliance with programs by product type, by location, by geography, etc. Both retailer and supplier can collaborate on key performance metrics and view the results in real-time in a visually compelling tool that uses the same diagrams and visuals as the planning process. Working together, retailers and suppliers can close the loop and measure in the same manner which they plan.

System Implementation

The thincVIew is an AJAX thin client web application. It contains no Java Applets and the client is written entirely in JavaScript and HTML. Building a cross-platform enterprise AJAX application is extremely difficult. There are many subtle issues involving security, timing, JavaScript, proxies, etc. that make the programming challenging. Our approach toward overcoming these issues involves breaking the problem up into layers and creating a JavaScript class library that we call thinc™.

Implementation of Visual Metaphors using thinc™ JavaScript library

The thincVIew map, timeline, time wheel, bar chart, and other visual components are implemented using a set of JavaScript classes built using our thinc JavaScript library. Using thinc components an application developer can create advanced, lightweight browser-based visualizations and mashups. The overall architecture consists of data components, visualization components, and framework/utility components. From a data ingestion perspective, there are several classes that allow the framework to pull in RSS and GeoRSS data feeds.

By default, thincVIew uses an XmlHttpRequest wrapper classes to asynchronously retrieve XML data from the server. This is how RSS and GeoRSS events are pulled asynchronously from the server by the web clients. The advantage of this approach is that XML data is quite standard and well understood. The disadvantage is that parsing XML data on a JavaScript client is somewhat cumbersome and slow. Recently, however, it has become popular in mashup applications to use the HTML