Altova Blog

Automate Data Mapping and Transformation with FlowForce Server

2013-05-08T09:15:00.000-04:00

Altova FlowForce Server, launched on April 29, 2013, includes numerous enhancements over earlier beta releases and one of the most exciting new features is the implementation of StyleVision Server. Now a FlowForce Server job can automate a complete data transformation workflow by executing MapForce Server for data mapping and pipelining results to StyleVision Server to render a variety of output formats.

This post describes a straightforward FlowForce Server mapping and transformation job as illustrated in these messages from the FlowForce Server Log, with the most recent step at the top of the list:

We’ll start with the GPS log files created by a digital camera. We wrote about these files last January in the post titled Process Multiple Input Files in a Single Data Mapping. We’ll use the mapping output with a StyleVision SPS stylesheet adapted from the XPath Enhances XML Reports post to produce a time and elevation report for each file.

A FlowForce Server For-each job step repeats based on the result of an expression. We can use For-each to build a list of files in a folder, then repeat one or more steps for each file. Here is how it looks in the job configuration page:

The line labeled Execute function defines the mapping to be used by MapForce Server, and the input parameter {file} refers to each file in the list C:\CameraGPS\example\*.LOG.

We can add an execution step to instruct StyleVision Server to perform the transformation:

The data mapping creates output files by adding .gpx to the name of the input file, and now we can define the transformation input using the {file} variable with the new file suffix. We chose to create .html output, but we could just as easily create other formats for a multi-channel publishing implementation.

The transformation working directory is the location where StyleVision Server unpacks the contents of the .pxf file containing the stylesheet, XML Schema, and other needed components. Using a dedicated working folder will keep the workflow more organized.

We want to allow network users to drop new .LOG files into the C:\CameraGPS\example folder and we want run the FlowForce Server job on a regular schedule, but we don’t want to process the same files over and over. We can define one more job step to move the processed file to a different location:

The complete FlowForce Server job is a series of three steps that loops for each .LOG file found in the folder. We can set up a repeating trigger for the workweek or any other appropriate schedule:

Here is a portion of a .LOG file created by the camera that is an example of one input file:

We can drop this file into the C:\CameraGPS\example folder, where it will be processed based on the FlowForce Server job trigger:

When the timer triggers execution of the FlowForce Server job, the Web interface Job Log page displays this series of messages for the complete job:

The contents of the C:\CameraGPS\example folder now look like this:

We can examine the 121130.LOG.gpx file in XMLSpy:

And we can open the .html file in any Web browser:

In future blog posts we will enhance this FlowForce Server example to illustrate jobs with error handling and more complete cleanup of working files.

FlowForce Server is available for Windows, Linux, and soon for Mac OS platforms. To get started yourself, click here to download a free trial!

New Server Products for Data Transformation

2013-04-29T08:21:00.000-04:00

On the heels of our RaptorXML announcement earlier this month, we are very excited to tell you about the availability of three more new products that round out the Altova server software product family.

These high-performance server products are optimized for multi-CPU servers and are highly complementary when used together for defining and managing information pipelines. Cross-platform support is immediately available for Windows and Linux, with MacOS support expected just around the corner in May. Supported languages include English, German, Spanish, and Japanese.

Let’s take a look at each of these new products in detail.

FlowForce Server

After receiving valuable customer feedback during the beta period, we have announced general availability of FlowForce Server, a powerful new tool for managing multi-step, enterprise-level data aggregation, processing, and reporting tasks.

The straight-forward FlowForce Web browser interface makes it easy to configure and monitor jobs, whether they require a single step, or multiple cascading steps starting with data aggregation and conversion, and completing with multi-channel report generation.

A flexible and highly customizable system of job triggers lets you schedule FlowForce Server job execution based on each job's unique requirements. Jobs can be triggered by date and time, by the arrival of files in hot folders, or by http command – and you can define multiple triggers per job. To take full advantage of server resources and meet the demands of busy data transformation workflows, multiple jobs – even multiple instances of the same job – can run simultaneously on FlowForce Server.

FlowForce Server provides strong security measures through user, role, job credential, and permission definitions that enable multiple simultaneous jobs while preventing unauthorized access to restricted data repositories or system resources.

So, what kind of jobs can you set up on FlowForce Server? Most customers will want to use it in conjunction with MapForce Server and/or StyleVision Server for high-performance automation of data integration and multichannel publishing projects that doesn’t require writing or compiling any source code.

MapForce Server

You may already be familiar with MapForce, Altova’s desktop developer software for aggregating, mapping, and converting data. We describe it as an any-to-any data mapping tool because it lets you move between any combination of XML, SQL database, flat file, EDI, Excel, XBRL, and Web services data. With the introduction of MapForce Server, it’s now easy to fully automate those transformations that have been defined in MapForce.

MapForce Server performs data transformations based on preprocessed and optimized data mappings stored in MapForce Server Execution files prepared by MapForce and uploaded over a network. Preprocessing enables faster performance and reduced memory footprint for most data mappings.

When operating under the management of FlowForce Server, MapForce Server is a powerful transformation engine suitable for the high-volume, quick turn-around demands of today’s leading edge data centers. The standalone version is suitable in a department level or other small environment that does not require the extensive multi-tasking, job scheduling, and security features of FlowForce Server.

StyleVision Server

StyleVision Server provides high-performance automation of report and document generation tasks based on templates designed in Altova StyleVision. A graphical XSLT stylesheet and report design tool, StyleVision helps you create one design to render XML and XBRL data in Web and print formats including HTML, Word, RTF, and PDF, simultaneously. Using StyleVision Server, you can generate reports from XML and XBRL data on a recurring basis faster than ever before.

Like MapForce Server, StyleVision Server is an important option for document production by FlowForce Server, and the standalone version can be run from the command line and is suitable in smaller deployments that may not require all the features of FlowForce Server.

This new line of Altova server products represents a huge leap forward in effective automation of essential business processes, whether they require a single step, or multiple cascading steps starting with data aggregation and conversion via MapForce Server and completing with multi-channel report generation via StyleVision Server. These high performance workflow servers help customers design information pipelines quickly and easily, and then deploy them in a flexible way, utilizing the power and speed afforded by multi CPU servers. We’re excited to offer this new family of server products to our customers developing data integration and reporting projects using Altova MissionKit developer tools, as well as welcoming new customers who need a multi-platform, lightning fast orchestration engine.

As with all Altova tools, we invite you to try these new server products before making a purchasing decision. Download a free, 30-day trial now!

Announcing RaptorXML, the Hyper-fast XML and XBRL Server

2013-04-16T09:05:00.000-04:00

We are very excited to be at the XBRL 26 Conference in Dublin, Ireland today to announce a brand new server product in the Altova family of XML and XBRL tools!

Altova RaptorXML is a hyper-fast XML and XBRL validation and processing server. It’s Altova’s third-generation XML and XBRL engine, built completely from scratch to help organizations efficiently validate, process, transform, and query the vast and ever-increasing amounts XML and XBRL data being generated as a result of XBRL compliance regulations and myriad other big data trends.

RaptorXML is written to be highly scalable for today’s multi-CPU and multi- core computers and servers. This, along with high performance code optimizations and an extremely low memory footprint, has helped make RaptorXML a lightning-fast XML and XBRL server that can meet the demands of today’s data processing applications.

Simply put: we architected RaptorXML to combine the performance benefits afforded by modern parallel computing environments with strict compliance to the latest versions of all relevant XML and XBRL standards.

RaptorXML includes support for the very latest versions of all relevant standards and has been submitted to rigorous regression and conformance testing. The server will be available in two versions, both of which are available for Windows, Linux, and MacOS platforms.

RaptorXML Server supports validation and processing of:

XML 1.0 & 1.1
XInclude 1.0
Xlink 1.0
XML Schema 1.0 & 1.1
XPath 1.0, 2.0 & 3.0
XSLT 1.0, 2.0 & 3.0 (subset)
XQuery 1.0 & 3.0
And more

RaptorXML+XBRL Server supports all the features of RaptorXML Server, with the addition of processing and validating the XBRL family of standards:

XBRL 2.1
XBRL Dimensions
XBRL Formula 1.0
XBRL Functions
XBRL Definition Links

Developers creating solutions using Altova MissionKit XML development and XBRL development tools will be able to power their server applications with RaptorXML for hyper-performance, increased throughput, and efficient memory utilization, giving them the opportunity to validate and process large amounts of XML or XBRL data cost-effectively.

Check out the complete list of supported XML and XBRL standards and more details on this groundbreaking new server product.

RaptorXML will be available to download and purchase in May.

Exciting New Project Management Tools

2013-03-26T09:25:00.001-04:00

We’re excited to announce some powerful new features for project management debuting in MetaTeam this week. Seasoned project managers and team leaders will find new support for interactive Gantt charts both familiar and a welcomed addition to the powerful team and project collaboration tools they’ve already been working with in MetaTeam.

This release also adds important and oft-requested support for Microsoft Project integration including import/export functionality, new options for specifying completion criteria, and improved work package definition.

Read all about it over on the MetaTeam blog or give MetaTeam a try now.

Editing, Converting, and Generating JSON

2013-03-12T09:00:00.000-04:00

As the use of JSON as a data transport protocol increases, I thought it would be useful to take a look at JSON support in XMLSpy. There’s been much debate about advantages of JSON vs. XML, but when you boil it down, there are simply some cases for which JSON is the best choice, and others where XML makes more sense. This article on the XML Aficionado blog unpacks this topic quite well.
While you might need to choose between JSON and XML depending on the development task at hand, you don’t have to choose between code editors – XMLSpy supports both technologies and will even convert between the two. Let’s take a look at how that works.
Editing JSON
To make JSON editing as easy as possible, Altova extended its intelligent XML editing features to the JSON editor. If you choose to edit JSON in text view, XMLSpy helps you along with syntax coloring, bracket matching, source folding, entry helper windows and menus, and so on.

Here’s that same JSON file in grid view, which offers a graphical representation of the JSON structure with drag and drop editing. Both views provide JSON syntax checking and advanced error checking features.

Another common requirement is converting XML to/from JSON, which is a one-click option on the XMLSpy convert menu. The JSON data above has been converted to valid XML:

Having JSON editing and conversion functionality directly inside the XML editor is quite useful, especially if you’re using the free Eclipse or Visual Studio integration package.
Check it out and let us know what you think.

Web Interface Simplifies Automation of Data Transformations

2013-03-05T08:12:00.000-05:00

FlowForce Server is a new product designed to provide automation of data transformations performed by dedicated high-speed servers. FlowForce Server can start jobs based on a variety of triggers, runs multiple jobs simultaneously, and can even run multiple instances of the same job, depending on workflow.

Monitoring all these complex activities is critical to success in a busy production environment. The FlowForce Server Web interface includes customizable views into operations, simplifying management from anywhere on the network.

The screen shot above shows the Home page of the FlowForce Server Web browser interface, displaying all currently running jobs and active triggers. Six instances of the gpxElevationUSGS job are running, each identified by a unique job ID. Four job triggers are also active, three watching hot folders, and one based on a timer. Each FlowForce Server job automates a MapForce data transformation.

The orange arrows below each grid are clickable update buttons, and the job names link to the definition pages for each job.

The blue headings at the top are also clickable buttons. The Log link displays the Log View, a detailed history of all system activity, shown in a truncated version below.

Each message line describes one step in one instance of a FlowForce Server job, and may have links to more information, as we described in our earlier blog post titled Automate Data Transformation with FlowForce Server. If the status is anything other than 0, the more link opens a detailed error message.

In a busy environment, all this can be just too much information. In that case, the Log View offers several alternatives to help you find critical information quickly. The options bar above the list lets you filter items displayed by a particular job or Message severity.

If you only want to know if anything unexpected occurred, select Warning as shown above to hide all normal Info messages.

You can also sort the list by any of the column headings Date, Severity, Module, User, or Instance ID in ascending or descending order. When the FlowForce Server is running multiple simultaneous jobs and instances, it’s very likely individual steps from different jobs will be shuffled. Sorting by Instance ID can let you more easily review the job history.

Click here to learn more about FlowForce Server Beta 3 at the Altova Web site, or visit the FlowForce Server Beta 3 download page to get started automating data transformations in your data center!

Compare Folders Fast and Efficiently with DiffDog

2013-02-20T09:23:00.001-05:00

Three user preferences in DiffDog work together to accelerate directory differencing tasks into hyperdrive, especially when you compare folders with thousands of files.

DiffDog can perform specialized comparisons. For instance, XML comparison mode commands DiffDog to determine whether two files contain equivalent XML content, even if they would be different in a text comparison because of spacing, line-ending, or attribute order variations. For the fastest directory comparisons, choose Quick Comparison mode, which only looks at file sizes and modification dates.

When you compare large directories, most often you want to quickly locate differences. Choose the Hide Equal setting to avoid listing all identical files.

Show/Hide empty directories should be set to Hide, otherwise subdirectories that contain identical files will be listed, but since their contents are equal, none of the files are shown.

After you choose settings and click the Compare button, DiffDog flies through the directory comparison.

When the comparison is finished DiffDog displays a concise listing showing only the differences, with a summary message at the bottom of the window. The toolbar in the screenshot below illustrates the settings for Quick Comparison on, Show Equal files off, and Show Empty directories off.

DiffDog saves your preferences to apply to the next folder comparison, and DiffDog even includes a quick synchronization tool to merge folders in either direction.

Find out for yourself how fast DiffDog can run your folder comparisons – download a fully-functional, free 30-day trial!

MySQL to Microsoft Access Database Migration

2013-02-13T10:33:00.000-05:00

A couple weeks ago, my colleague wanted to migrate a database supporting the Project Management Body of Knowledge (PMBOK) from MySQL to Microsoft Access, in response to requests from MetaTeam Blog readers.

Create table statements already existed and were easily run in Access. DatabaseSpy performed a critical database schema comparison to verify the new table structures and relationships. Then, a DatabaseSpy database contents compare and merge moved hundreds of rows of data into the 11 tables. DatabaseSpy resolved all the differences between SQL dialects and datatypes of MySQL and Microsoft Access and completed the task in seconds.

Of course, you could just as easily go the other direction to migrate an organically grown Access database to MySQL or another enterprise-grade database.

You can read the details in A Relational PMP Study Aid, Part 2 on the MetaTeam Blog.

Part 1, titled A Relational PMP Study Aid, describes how candidates can use the PMBOK to prepare for Project Management Professional (PMP) certification.

Altova MetaTeam is the online project management and collaboration environment for teams that want to achieve high performance results by working smarter. If you haven’t seen it yet, you can check out a 2-minute MetaTeam video introduction here.

Automate Data Transformation with FlowForce Server

2013-02-05T08:20:00.000-05:00

Altova designed FlowForce Server to provide comprehensive automation, management, and control over data transformations performed by dedicated high-speed servers. FlowForce Server Beta 3 is currently available to users of MapForce Enterprise and Professional Editions at no charge during the beta test period.

FlowForce Server can provide hot folder automation of data mappings and maintains a detailed activity log users can monitor remotely in a Web browser window. The screenshot below shows the log for FlowForce Server running the MapForce data mapping CameraLogToGPX we wrote about in the blog post titled Process Multiple Input Files in a Single Data Mapping. This mapping used wildcards to specify multiple input files for processing.

It only takes a few minutes to set up, run, and review the results of jobs like this on FlowForce Server.

Wildcards or Hot Folders?

Wildcards and hot folders increase the complexity of a data transformation workflow, and using them successfully requires careful planning. Let’s take a minute to look a little deeper at the scenario we want to implement.

Assume we are the IT department in a company that publishes nature and hiking guides. We employ photographers who go out trekking and record their routes as they go, using the GPS tracking feature of their digital cameras. We want to convert the camera GPS log files to XML-based .gpx format for mapping and other processing.

We will publish a folder on our network where photographers can drop off their GPS log files. This will be the hot folder FlowForce Server watches for new files to supply as input to the CameraLogToGPX mapping.

We only need to process each input file once. So, after the data transformation is complete, we can remove the input file from the hot folder. We also want to place the output file in a separate folder.

This suggests the following FlowForce Server job steps:

Look in the hot folder to see if new a input files has arrived
Perform the data mapping on the input file and place the output file in a separate folder
Move the input file to a permanent location

The diagram below shows a folder structure we can use for the workflow, with files ready to drop into the hot folder for processing:

The hot folder is C:\CameraGPS\hotFolder and the generated .gpx files will be placed in C:\CameraGPS\outputFiles. When the data mapping is done, input files will be moved to C:\CameraGPS\completedInput.

Deploy the Mapping on a Server

MapForce Beta is a component of FlowForce Server that works just like MapForce, and adds a feature to deploy mapping files to a FlowForce Server. Existing data mappings need little or no special preparation for deployment.

The only thing we need to think about are the filenames of the input and output files. We will instruct FlowForce Server to provide the input filename as a job parameter as new files arrive in the hot folder for processing. The original mapping assigned a wildcard filename for the input component, which is no longer needed. We will also want FlowForce to specify the location of the output file.

We can open the mapping in MapForce Beta, remove the filename from the input component, and add a remove-folder filepath function to the output file so FlowForce Server can set the destination. The screenshot below shows the new filename definitions in the mapping.

In the MapForce Beta dialog to deploy the mapping, we can choose to immediately open the mapping in a FlowForce Server job definition window in a Web browser to finish defining the job operations.

Defining the Job in FlowForce Server

The screenshot below shows the complete job steps defined in a FlowForce Server job properties window:

The job trigger is defined at the bottom of the window. Every 30 seconds, FlowForce Server will check the hot folder. If the contents have changed, FlowForce Server will execute the job steps. Each Execution step could be a deployed MapForce mapping, a system step, or even another FlowForce Server job.

The name of each new file entering the hot folder becomes the parameter called {triggerfile} that we use in the mapping step as the input filename, and in the move step as the name of the file to be moved.

The Working-directory parameter in the mapping step defines where the output files will be placed.

FlowForce Server also includes features to set automatic run and stop times for jobs, user permissions and roles, and Queue settings to define minimum time between job runs and maximum parallel instances of a job.

In our scenario, we are likely to receive multiple input files in groups as they are copied from photographers’ memory cards. Multiple parallel runs can greatly improve throughput. As a rule of thumb, you might want to match the number of cores or CPUs in the machine running FlowForce Server.

Exploring the Job Log

Each time FlowForce Server runs our job, six lines are added to the Log View shown in the illustration at the top of this post. The first and last lines record the start and completion of the job, and each Execution step generated its own start and completion messages. The phrase “completed with status: 0” means the step was successful with no errors.

We can click the more links for a detailed report on each Execution step. The screenshot below shows the message for the MapForce mapping Execution step:

Each file dropped into the hot folder generates an individual FlowForce Server job instance, even if multiple files are added as a group. This makes it easy to track any individual input file that generates an error.

When we dropped four files into the hot folder, FlowForce Server ran four jobs, and the contents of the output folder looked like this:

Click here to read more about FlowForce Server Beta 3 at the Altova Web site, or visit the FlowForce Server Beta 3 download page to get started automating data transformations in your data center!

Process Multiple Input Files in a Single Data Mapping

2013-01-29T08:17:00.000-05:00

Typical data conversion tasks require processing numerous input files that arrive in batches. Altova MapForce includes features that let you handle groups of files with minimal intervention.

For instance, we recently copied a set of files from the memory card of a digital camera with GPS support. Each .LOG file is a CSV containing GPS coordinates for a single route.

We quickly designed a mapping to convert the CSV data to XML-based .gpx format and processed all three files to generate three output files in a single execution:

First, we used a wildcard character in the Input file name in the Properties dialog for the input component of the mapping. This instructs MapForce to individually process every file in the working directory that matches the wildcard.

If you are designing a complex conversion, or if the input files are very large, you can use a single unique filename to develop the mapping, then change to a wildcard when you are satisfied with the mapping output.

File Path Functions

The built-in MapForce Function Library includes file path functions we can use to manage output file names. If we define a single output file, it will be appended with new data when we process each successive input.

You can combine file path functions with other string functions for complete control of output file names and locations. We decided to leave the output in the same directory as the input files, but to create more descriptive filenames, and to use the .gpx file extension.

The portion of the mapping shown below uses the string concat function with file path functions to generate the output file 1211190converted.gpx from 1211190.LOG, and so on.

You can also use file path functions to generate strings and insert them as output. The XML Schema for .gpx files contains a metadata description element. We decided to insert the input file name into the metadata to explicitly link the output file to the original data. This strategy makes the output file self-documenting, and can help with debugging if you need to trace unexpected output back to the original source.

The portion of the mapping shown below inserts the source file name into a string and maps the string to the metadata <desc> element:

The resulting description is on line 4 of the mapping Output preview:

Filtering Input Data

The core of this data mapping required a filter on the input file. The camera GPS log files are recorded according to the National Marine Electronics Association (NMEA) specification. A portion of one of the input files is shown below:

After the first line, each recorded point is described by two NMEA sentences, where the sentence type is identified in the first field. Each GGA sentence includes the time, latitude, longitude, elevation, and additional data about the quality of the fix. Each RMC sentence contains the time, latitude, longitude, and date.

An RMC sentence contains the minimum data we need to generate a .gpx <trkpt> element, so we can use a filter to select only those lines from the input, as shown here:

If the message type in the first field of a row contains "$GPRMC" it is passed through for processing. If not, the row is ignored.

The actual data in the input file also required some manipulation. For each latitude and longitude, we had to combine multiple fields in the source that defined degrees, minutes, and seconds and convert to decimal degrees. We needed to combine the time and date fields and record the result in ISO 8601 format as required by .gpx, such as 2012-11-19T20:43:23Z. We defined each of those conversions as user functions to encapsulate their complexities and separate them from the main mapping.

This mapping also provides an opportunity to reuse the getElevationUS user function we defined in the earlier post Expect the Unexpected - Altova MissionKit Solves a Number Format Mystery. This time we rounded the elevation data to three decimal points, representing the nearest millimeter.

The core section of the CamerlogToGPX data mapping with user functions looks like this:

And here is one of the output files showing a <trk>, <trkseg>, and several <trkpt> elements.

The MapForce Output menu provides a selection that lets us validate the output files against the .gpx XML Schema:

If you would like use Altova MapForce to process input files in batches for your own data mappings, click here to download a free trial.

Generating a Schema from Multiple XML Instances

2013-01-22T09:31:00.001-05:00

I was recently talking with one of our Support Engineers about common questions he receives from customers. One that comes up a lot is schema generation. XMLSpy will generate a valid XSD from a DTD, a relational database structure, and of course, an XML instance. But what many customers don’t know is that the XML editor will also create an XML Schema based on a group of related XML files – and this is a common requirement.

Let’s take a look at how it works.

Let’s say I have inherited a Web development project, having received a group of XML files as a starting point – but no XML Schema. To generate an XSD for those files, I first need to add them to an XMLSpy project. This will be useful afterwards since the project groups all the related files in one, easy to manage group of folders where I can assign XSLT transformations, perform batch operations such as project-wide validation, and so on.

It’s worth taking a moment to talk about the XMLSpy project itself. The folders may correspond to actual directories on the file system, or I can define file-type extensions for each folder so that common files are kept together in one convenient place. The project folders are virtual folders and do not need to correspond to any hierarchical organization of the files on my server.

I can also assign different XSL transformation parameters to each folder and have the same physical file be contained in more than one project folder. This is especially useful when I need to keep data in one XML file and use different XSL stylesheets to produce different output. And, of course, XMLSpy project management integrates with most source control systems.

OK, back to generating the schema. The first step is to select New Project from the Project menu. XMLSpy creates a skeleton project with place holder folders for various file types. It’s easy to add custom folders, as needed. To add the files I received to this new project, I right click on the XML Files folder and select Add Files…

And browse to select the XML files for the Web site. This is a good time to save the project with a descriptive name, and this also saves the newly added file associations to the project.

Now, all I need to do is right click on the XML Files folder and select Generate DTD / Schema… and review or update the configurable options.

This will generate an XSD based on all the XML files included in the folder. After clicking OK, I’m prompted to save the auto-generated XML Schema, and it opens in the graphical XML Schema View. Here are the global elements generated based on the XML instance files:

Clicking the tree icon next to any element displays the detailed content model.

The graphical design paradigm of the XSD editor makes it easy to refine and edit this schema as needed.

That’s it – in a few quick steps I have a valid schema generated based on a group of instance files.

Which XMLSpy features help make your everyday work easier? Or, is there a time-saving feature you’d like to see added? We’d love to hear from you. Please leave a comment below!

Expect the Unexpected – Altova MissionKit Solves a Number Format Mystery

2013-01-10T08:27:00.000-05:00

Every time you receive data from an outside source, there is a chance it won’t arrive in the form you expect. This can require special accommodations for the rare and unlikely to make a real-world data mapping and transformation solution robust and reliable.

We processed literally dozens of .gpx files, containing hundreds of coordinates each, through the MapForce mapping we wrote about in the blog post Web Service as a Look-Up Table to Refine GPS Data. Then one day we ran a new file and encountered the error below, which caused the mapping to fail:

Reaching into the Altova MissionKit to combine features of MapForce and XMLSpy, we quickly diagnosed the issue and developed a solution we can also reuse in future mapping projects.

We first suspected a problem with the input data, so we opened the file in XMLSpy, where it passed well-formedness and XML validation tests. Fortunately, each data point has a unique time stamp, so we searched for 23:06:22, marking the last set of GPS coordinates that processed successfully. That time stamp occurred once at line 1772 of the input file.

Nothing looked obviously wrong in the source data that immediately followed. We simply commented out the next data point, and saved the file to reprocess the mapping:

This time the mapping ran successfully:

Now we were suspicious of the data returned by the Web service. Even though the U.S. Geological Survey National Geospatial Program runs the Web service, maybe the underlying database contained invalid data.

We inserted a simple .csv file into the mapping as an alternate output and mapped the elevation results for each set of source coordinates to examine the Web service output.

One line in the output file diagnostic.csv contained the same value quoted in the previous error message:

It’s scientific notation! The Web service returned a number formatted in scientific notation! The round-precision function in our data mapping that processes the Web service result requires decimal input.

Datatype Conversion

One strategy might be to write a function that recognizes the Web service result as scientific notation and explicitly calculate the numeric value. The MapForce error message “Conversion to decimal failed for ‘-1.24202767892712E-06’” suggests a simpler solution.

This is a good time to think about datatypes. The Web service component in our mapping clearly indicates it returns a text string. MapForce automatically performs type conversions from string to decimal number when a mapping connects a string as input to a mathematical formula. In most cases, this frees developers from thinking about explicit type conversions as data moves between formats. In our mapping, MapForce successfully performed type conversion from a string to decimal 178 times before encountering the entry in scientific notation.

Scientific notation is normally used for numbers that are too large or too small to be conveniently recorded in decimal form. In MapForce, the decimal datatype does not specify the size, or value, of the number. Instead, it identifies the XML decimal type, a character sequence of digits with a period as the decimal separator.

In XML -- and in MapForce -- the double datatype supports scientific notation. We can explicitly cast data in scientific notation as the double datatype, then round the result.

This solution is easy to test in a simple mapping, using text files for both the input and output. We inserted a simple variable before the round-precision function and assigned its datatype as double. For our first test, we used the data captured from the USGS Web service as input, to run the same data without redundantly performing the Web service calls. This mapping also lets us easily build more test cases with new input data.

The mapping processed successfully, generating this output:

Build a User Function

User functions in MapForce are defined in one mapping file, and can be added to the Function Library for use in other mapping files, even by multiple users. User functions also encapsulate complicated operations and help make the overall data flow of a large mapping design much more traceable.

We had already modified the simple Web service call by choosing the database for eastern or western continental US based on the longitude. Now, adding explicit datatyping to the result makes calling getElevation even more complex. We chose to define everything in a user function.

Applying the User Function

In the mapping below, we inserted the new getElevationUS function.

At this point it’s good to recall why we rounded the elevation returned by Web service in the first place. The Web service returns a value in meters, and two decimal places -- or each centimeter -- is less than a half inch.

We could have included the round operation as part of the getElevationUS function, but the function will be more useful in future data mappings if it does not round the raw elevation data.

Output of the revised mapping is shown below, using the same input .gpx file that caused the initial problem. We searched the output file for 23:06:22, the time stamp we used to find the last good coordinates before the error. The following point starting on line 902 is the one that failed.

At first we were disappointed all this effort boiled down to an elevation that rounds to 0. Then we mapped the suspicious coordinates on a Google map:

Part of the route followed a bridge over a tidal inlet. Even if we never reuse the getElevationUS function in a future data mapping, it’s very likely other .gpx files for other trips will lead across other tidal inlets, where they might generate more very small elevation values.

If you would like use the tools in the Altova MissionKit to create user-defined functions for your own data mappings, click here to download a free trial.

Are You Looking For A Relational View of the PMBOK?

2013-01-02T15:14:00.001-05:00

Altova MetaTeam is project management software for high performance team collaboration.

MetaTeam is first and foremost simple and friendly. You can see that for yourself in the examples on the MetaTeam blog.

But MetaTeam is also designed to support standards and methodologies, as well as to work alongside advanced Critical Path scheduling tools like Microsoft Project or ProjectLibre.

Because of this, we encourage our team to be certified as Project Management Professionals (PMP).

We have a tool for our own PMP exam prep that may be helpful for others. It is a SQL database mapping the project management standard into a relational model.

Read on for why we created it, using our DatabaseSpy product, and to download it for your own use.

The PMP credential is based on the Project Management Institute's standard Project Management Body of Knowledge (PMBOK). The print form is an overview called A Guide to the Project Management Body of Knowledge.

The PMBOK is the leading project management standard worldwide — and MetaTeam dovetails with it by design.

Preparing for the PMP exam is a non-trivial exercise in learning the details, processes and standard lingo of project management.

Now, why would someone want a relational database of the standard project management information?

I'm glad you asked!

The reason is that the body of knowledge contains a very large amount of information that is linked in some very relationally-oriented ways.

And yet the relationships are not easily explored in paper-form or PDF.

For a sense of the challenge, consider these numbers. PMI's PMBOK Guide has:

Roughly 500 pages
42 processes
Five process groups
Nine knowledge areas, also structuring the 42 processes
Hundreds of tools and techniques tied to specific processes
Roughly 100 deliverables and assets
Hundreds of inputs and outputs links, and
A large glossary of must-know terms

And, of course there are many items only specified by name in the Guide, or just hinted at, with the details to be discovered by the reader elsewhere.

How to deal with all that?

For us it the next step was easy: break out DatabaseSpy and create a relational database we could use to make sense of all that information, and more.

Yes, we are geeks here! :)

Creating and populating a database using DatabaseSpy is no big thing, of course.

But since I love DatabaseSpy, I’m going to give a quick overview of the process.

The link to download the schema and data is at the bottom of this post.

The first task was to connect to a local MySql server. The Connection Wizard made that a snap.

After connecting we quickly created a new database in the SQL editor by clicking the New menu and select SQL Editor, then executing a database create statement.

The create statement is in the SQL file you can download below. We called the database “pmbok”.

With the database made, we started to design by clicking the New menu and selecting Design Editor. First thing to do is to switch to the “pmbok” database by changing the root object in the properties window.

For our database we wanted to focus on the PMBOK’s processes’ inputs, outputs, and tools and techniques. The goal was to be able to quickly answer questions like:

What inputs are common to “Estimate Activity Resources” and “Estimate Costs”?
Where are “Performance Reports" created and used?
What processes rely on “Work performance information” vs “Work performance measurements”?

To do this we started with the basics.

First we needed a PROCESS table, along with PROCESS_GROUP and KNOWLEDGE_AREA tables.

You can execute the script below, but if you are following along, right click on the design and select create new table called “process”.

The first column is an ID. We made primary tables have varchar(45) IDs to hold meaningful keys based on the PMBOK names of the items. For the dependent tables and we either used that, or compound meaningful keys or auto-increments depending on the data.

We also gave most every table a NAME column. Right click on the new table to begin adding the columns.

The rest of the process table looks like this screenshot.

The “pmbok” schema is small, just 11 tables. It is definitely a simple model—which makes it more practical for studying for the PMP.

The IDs tying the tables together are the name of the process, process group, knowledge area, etc. used in the PMBOK, with “_” for spaces and very few abbreviations. Take a second to have a look at the data before continuing.

The shortcut to finishing this design is to execute the database script in a SQL window and then just drag the tables out of the Online Browser into the design.

Executing the script will also populate the tables with process, knowledge area, inputs, outputs and techniques, deliverables, and other data.

With all that in your database we can now go back to the SQL window and create queries to answer questions like the examples we gave above.

Our first query looked for the common elements to the Estimate Activity Resources and Estimate Costs processes. We can answer that with:

select *
from process_input_or_output
where process_input_or_output='input'
and process_id in
('estimate_activity_resources', 'estimate_costs')

Next we wanted to know where Performance Reports played a role:

select * from process_input_or_output
where deliverable_or_asset_id='performance_reports'

QED!

And for the third question about the difference between the two work performance items we can use:

select * from process_input_or_output a
where a.deliverable_or_asset_id='work_performance_info'
union
select * from process_input_or_output b
where b.deliverable_or_asset_id='work_performance_measurements'

As you can see, it is easy to browse, list and make connections between concepts using this database.

We find it very useful for study and in our work with MetaTeam. I hope you will as well!

Here is the schema and data download.

Please let me know if it makes your own PMP studies a bit easier.

A quick corporate word about this database script and its contents. Please feel free to use it as an overview of standard project management thinking under a share-alike license for non-commercial purposes. It is offered as-is with no warrantee or fitness for any purpose. It does not map to A Guide to the Project Management Body of Knowledge precisely and includes information and assumptions not found in PMI's summary of the standard committee's work. Undoubtedly the data contains errors and omissions that you must handle on your own. And please remember that it is just an example and not intended as a substitute for anything or for any specific purpose. If you have questions about the schema, data or usage please contact us.

Brand New Altova Web Site

2012-12-12T10:20:00.000-05:00

We are so excited to announce the launch of our new, improved, completely redesigned Web site! The new Altova.com is cleaner, simpler, and more beautiful than ever.

Our goal was to make our site easier to navigate so that you can find exactly what you’re looking for quickly, whether that’s Product information, Support resources, or technical reference materials. Of course, it’s as simple as ever to download a free trial or update your software to the latest version.

At the same time, we hope the new site will help you discover a little something you didn’t know about Altova. For instance, did you know that we have a comprehensive line of Database Tools in addition to the XML Tools and Developer Tools that we’re known for? Did you know that we offer free, online product and technology training? And have you tried our new, cloud-based project collaboration and team management tool?

You can learn all about how Altova products and supported technologies can help your team meet critical business needs in our completely revamped Technology Solutions Center.

Despite all the planning, proposals, testing, and reworking that go in to a new Web site, the true test of success is how the site works for you, our customers. Please share any feedback or suggestions you may have for the new site, either by leaving a comment here on the blog or contacting our Marketing team by email. We look forward to and greatly appreciate your input!

Version 2013 Service Pack 1 Delivers Windows 8 Support and More

2012-11-15T12:43:00.001-05:00

On November 15, 2012, we released Service Pack 1 of Altova MissionKit and the complete line of Altova XML, SQL, and UML tools. This service pack delivers the usual round of bug fixes as well as some important new updates to existing support:

Windows 8 support. Altova’s desktop applications now run under Windows 8 in addition to Windows 7 and earlier platforms.
Windows Server 2012 support. Altova tools now run on Windows 2012 Server in addition to earlier versions.
Visual Studio 2012 integration. Altova tools that support integration with Visual Studio have been updated to support the latest version of Microsoft’s IDE.
Internet Explorer 10 support. Altova desktop products with browser integration now all support Internet Explorer 10. In addition, the Authentic Browser plug-in supports IE 10 in desktop mode.
XMLSpy 2013 Chinese language version. Version 2013 of XMLSpy is now available in Chinese as well as English, German, Japanese, and Spanish.

Version 2013 Service Pack 1 is a free update for all customers of Version 2013 products, as well as any customer with valid Support and Maintenance. Download and install SP1 now.

The Altova MissionKit 2013 Introduces Seamless Integration of XMLSpy, MapForce, and StyleVision in Java Applications

2012-11-08T09:29:00.000-05:00

The Altova MissionKit 2013, empowers developers to integrate XMLSpy, MapForce, or StyleVision functionality seamlessly in custom Java applications for Windows. This frequently-requested capability adds to existing support for integrating these MissionKit tools in Visual Basic or C# applications, giving developers flexibility to add some or all XMLSpy, MapForce, or StyleVision views and functionality to their own custom apps.

Version 2013 of XMLSpy, MapForce, and StyleVision include a new API that allows each to run inside a window within a Java application developed using the Java Abstract Window Toolkit (AWT) or Java Swing. This functionality allows MapForce 2013, for instance, to be embedded in larger applications where data mapping and transformation is only one requirement.

Altova provides sample applications with XMLSpy, MapForce, and StyleVision illustrating use of the Java API. You can access the sample applications from the command line or from within Eclipse.

The screen shot above shows the MapForce version of sample application, which has launched MapForce, opened a mapping file, and displayed validation results. The buttons for each MapForce window, such as Mapping, DB Query, Output, etc. below the mapping window all operate exactly as if MapForce is running in its normal stand-alone mode. Right-click context menus in the mapping window are also supported.

The Java API also includes extensive documentation to help Java developers get started quickly. An example from the MapForce API documentation is shown below.

Please note: you must install the corresponding free XMLSpy MapForce, or StyleVision Integration Package to access the sample applications and Java integration functionality. Although Java is known as a cross-platform development tool, MapForce integration functionality is available for the Windows platform only.

Try Version 2013 Now

Version 2013 is a free update for Altova customers with an active Support and Maintenance Package. Visit the Altova Download page to update now! If you’re not already a customer, you may download a free, fully-functional 30-day trial.

Trick or Treat with Altova at DevConnections!

2012-10-30T14:51:00.001-04:00

It’s no trick - we will be in Las Vegas October 30 – November 1 at the Bellagio Hotel for the fall DevConnections event, in booth #408. If you are attending the event we would love for you to stop by our booth, where we will be showcasing MissionKit, Altova’s suite of XML, SQL and UML tools.

Altova MissionKit 2013 provides specialized functionality for Microsoft developers and IT pros, including seamless integration with Visual Studio®, C# and C++ code generation, powerful functionality for SQL Server® database management and reporting, integration with SharePoint® Server, functionality for working with Open XML (OOXML), diff/merge for Word documents, and much more.

While you are at our booth, don’t forget to enter our drawing for a chance to win a free Altova MapForce Basic Edition license. If you also mention that you saw this blog post – we might have a special treat for you too!

UModel, BPMN and MetaTeam tag-team business processes

2012-10-18T14:08:00.000-04:00

The other day I was parsing the Business Analysis Body of Knowledge (BABOK) and exploring some of the new features in UModel when I suddenly had a Reese’s moment – UModel and MetaTeam taste great together.

In fact, using BPMN to visualize the processes in a team is so natural it’s almost like UModel acts as a diagrammatic front-end to MetaTeam.

To flesh this idea out, I quickly created an example IT team and mapped a simple process, linking all the pieces into MetaTeam. Then I posted a UModel generated report, including those links, in the team’s Knowledge Base.

Here’s how it worked…

It doesn’t matter if you start in MetaTeam or UModel, but I started with MetaTeam so that I had my reference points ready before I modeled the process.

As I go through this you will see that MetaTeam is providing the who and UModel is defining the how.

First I created a team from the My Teams page. The team is called IT Team. If you are following along and want a short cut you could generate an example team.

Next I created two business roles in the Roles app. I want to show one simple interaction between developers and QA specialists, so my roles are “Developer” and “QA”.

In real life, these roles would have lots of responsibilities, but for this example I just need two: “Maintenance work” and “Black box testing”. Click on the responsibilities tab in the Roles app and click the Add link at the bottom on the left.

What I want to show is how a work item is started by a developer, completed, passed to QA for arms-length testing, and either scheduled for release or sent back to the developer for more work.

The developers are going to be picking items out of the “Improvements” to-do list. And the testers are going move the completed and validated items to a “Release list”.

Now we’ve got MetaTeam all set, let’s turn to UModel to map the process.

Start by creating a new project. In the new project right-click on the root element in the Model tree. Select New diagram, Business Process Diagrams, BPMN 2 Business Process Diagram.

UModel will want to include the BPMN 2 profile. That’s the profile we’ll be using so just click Ok. (A profile is technical definition distinguishing all the different diagramming lingua you saw in the new diagram menu).

In the Model tree double click the package name and diagram names in turn and type in the names you want to use. Your new diagram should look like this image.

Now the fun part! Click the diagram and look to the Layout toolbar for the Pool icon.

Create a pool with two lanes. Call one of them QA and the other one Development. I like a bit more size in my text so I opened the styles dialog and bumped up the font.

We are outlining a set of responsibilities for roles, so we need to identify them in the diagram. A good way to do that is with the grouping element.

Create one group called “Maintenance work”, in the Development lane, and another called “Black box testing” in the QA lane.

Next right-click the black box testing responsibility and click Hyperlinks, Insert/Edit Hyperlinks…

In the hyperlinks dialog click Add Web Link. This is where MetaTeam comes back into the picture.

Open the Black box testing responsibility in MetaTeam and copy the URL. Paste it into the hyperlink in UModel. When people click on the link arrow in the BPMN diagram their browser will immediately open to the responsibility documentation in MetaTeam. (They will have to log in, of course).

When you close the hyperlink dialog your model now has a link to the responsibility in MetaTeam.

Now its time to start adding the activity nodes to the process. Click on a start element and place it in the model. Then add a user task node.

User tasks are the perfect place to identify the role that is going to perform the task. Of course you could also link a user task to an individual account in MetaTeam; however, in most cases people identify a role because even the most stable teams change over time.

There are several more elements to this process. I’m going to skip ahead to where we link in one of the to-do lists.

Remember back a-ways I said that the QA engineers would move validated work to a release queue. That’s one of our to-do lists. Linking it up to MetaTeam works the same as for the roles and responsibilities.

Clicking the “Release list” link will take you directly to that to-do list in the Todos app where you can see the tasks, documentation and discussion. And when you assign a Release Engineer role to that todo list the person (or people) in that role will be notified when there are new release-ready features.

Finally, let’s generate this diagram into HTML that we can post in the Knowledge Base. We can certainly put the UModel (.ump) file in the repository too, but HTML gives the same visualization and linking for anyone on the team, not just the business analysts and developers using UModel.

Generating a report in UModel is easy. In brief the process is as follows. Click on the Project menu and select Generate Documentation… The dialog that opens has A LOT of options. You’ll want to play around with it to see what you want to include, but there are a couple things I would definitely suggest.

Make sure the output format is HTML. Uncheck “Split output to multiple files”. And check “Embed CSS in HTML”. When you use those settings with this simple project you get just two files: an HTML report and a clickable BPMN image.

With that done, go back to MetaTeam and open the Knowledge Base app. Click on the File tab. We could store this report within either of the roles it addresses, but since it applies to multiple roles let’s stick it at the top of the File tree.

Click Add and select the report file. Then again for the image.

Clicking on the HTML filename opens the report. And clicking on the linked elements in the BPMN opens to the MetaTeam page for that item. How cool is that?

One final thing you can try. If you click on the info icon beside the model image in the repository you can copy the image link and paste it into the image dialog of any description, comment or Knowledge Base article you write.

Here’s where what the link looks like in the info box.

And here I’m using it in a description of the responsibility. (Make sure it doesn’t have any trailing white space in the URL).

And that’s how easy it is to link up two powerful tools for business analysis — UModel and MetaTeam.

Go ahead and give it a try. And if you think up other ways to use UModel and MetaTeam together please let me know!

Do You Believe in (XML) Magic?

2012-10-16T09:00:00.000-04:00

One of the most exciting features in our recent Version 2013 release is Smart Fix XML validation in XMLSpy. Smart Fix is so magical it can revolutionize XML validation as we know it. While the XMLSpy validator has always been powerful, with new features added often, Smart Fix really delivers the “wow” factor: automatic error correction.

Let’s take a look at how it works.

One-click Validation Error Correction

The XMLSpy XML validator supports document- or project-wide validation with a nine-tabbed message window that provides detailed information about each validation error. This includes the reason for the error, a hotlink to the error location, links to the corresponding definition in the associated schema, and links to any relevant W3C specifications. Now, XMLSpy 2013 also offers a Smart Fix, shown in the right-hand window above. In addition to information about the error, Smart Fix offers a valid solution for fixing it, which you can review and then implement with one click.

In the screenshot above, I mistakenly listed the <price> element before the <quantity> element, but the schema specifies that <quantity> should come first. I can simply click the Fix + Validate button to make this valid change and move on to the next error (if there is one). It’s that easy. Smart Fix takes the guess work, retyping, and hair pulling out of XML validation.

Here’s another example where an element is accidentally duplicated. Smart Fix zeros in and lets me get rid of the duplicate with one click using the Smart Fix tool bar, or with a double-click on the fix itself.

Let’s look at a third example that’s a bit different. I’ve accidentally included two @ signs in an email address, which violates the defined pattern. (I’m also working in Grid View now – Smart Fix works equally well in XMLSpy’s graphical editing mode.) Smart Fix offers two suggestions, both of which would create a valid document. I double click the first option to simply remove the second @ sign, and XMLSpy automatically makes the correction.

Smart Fix reduces the time and energy spent troubleshooting and fixing frustrating XML validation errors considerably, freeing you to focus on more productive aspects of development.

More New Features in XMLSpy 2013

Smart Fix is just one of numerous new features added in the most recent release.

Current customers can update to the latest version here. If you’re new to XMLSpy, download a free, 30-day trial to check out Version 2013 in action.

Using Table Row and Column Conditions in StyleVision 2013

2012-10-09T10:00:00.000-04:00

The ability to conditionally hide table rows or columns has been a popular feature request from StyleVision users. StyleVision 2013 adds the capability in the form of conditional expressions on table rows and columns. Conditional processing can be set on individual columns and rows of static and dynamic tables, as well as on column and row headers, to display or hide the column, row, or header depending on the truth of the condition. The conditions are specified using StyleVision’s XPath Expression editor. If a condition evaluates to true, the column, row, or header is displayed; otherwise it is not (i.e., is hidden).

Let’s take a look at some scenarios for which this new feature is exceptionally useful.

Here are some scenarios in which this type of condition could be used:

On a column, row, or table, enter the XPath expression false() to hide the column, true() to display it
A column is output only if the sum of all the values in that column exceeds a certain integer value
A column or row is output only if no cell in that column or row, respectively, is empty
A column or row is output only if a certain cell-value exists in that column or row, respectively

To specify a row or column condition, simply select the target row or column and, from the right-click menu, select “Edit Conditional Processing.”

As a quick introduction to another valuable and time-saving new feature in StyleVision 2013, note the new “Evaluator” button in the Edit XPath Expression dialog. Clicking on it presents StyleVision’s new XPath Evaluator extension, which makes it possible to easily and contextually check the results of XPath expressions while you’re editing.

The XPath expression in the XPath 2 Expression text box can be edited, and the expression can be immediately evaluated, with the results displayed in the Results pane. Evaluation options include:

Evaluate on typing: If this check box is selected, the XPath expression is evaluated: (i) with every keystroke used to edit the expression, and (ii) when the mode is switched from Builder mode to Evaluator mode.
Evaluate now: This button is enabled when the Evaluate on Typing option is not checked. Click it to evaluate the expression.

For information about other productivity-enhancing new features in StyleVision 2013 and other MissionKit 2013 tools, please see What’s New in the Altova Product Line?. To further explore the many new StyleVision 2013 features, you can also download a free 30-day trial version today!

Watermarks in StyleVision 2013

2012-10-03T13:40:00.000-04:00

Watermark support has been added in StyleVision 2013, in response to customer requests. A watermark is text or an image that is displayed on the background of each page of a document section. Watermarks are often used in draft and confidential documents in order to emphasize a document’s status or security constraints.

In StyleVision 2013, watermarks may be used for print-oriented output formats, including PDF, RTF, and Word (Open XML). In keeping with StyleVision’s overall design model, with a single layout design used for multiple output types, you only need to define watermarks once, and they are applied to all print-output views.

Here’s an example of a StyleVision design containing a watermark:

Let’s take a look at the steps involved in using watermarks in StyleVision 2013 by modifying the QuickStart example (quickstart.sps) included in the Tutorials folder (installed with StyleVision), in order to produce the design shown in the image above.

In the Design view, if you select “Edit Properties” in the Initial Document Section, you’ll see a new “Watermark…” button. Clicking the button presents the Edit Watermark dialog:

StyleVision supports both image and text watermarks. For both types of watermarks, the general idea is to specify:
· A condition used to determine when the watermark should be included (which can simply be “true()” in order to include the watermark on all pages)
· The watermark content (image or text, with related attribute setting preferences)
· Watermark size and position settings

For our example scenario, let’s assume we want to place a red “CONFIDENTIAL” text watermark across printed pages. To do so, we click the Text tab in the Edit Watermark dialog and:
· Enter “true()” for the condition
· Enter “CONFIDENTIAL” for the watermark text
· Select red for the watermark color

StyleVision’s support for watermarks is powerful and flexible. You can include both text and image watermarks, for example, with different XPath conditional expressions used to specify what should appear when. You can also specify different watermarks (or pairs of image and text watermarks) for each document section in a StyleVision design.

Check out watermark options and the other new features in StyleVision 2013 by downloading a free 30-day trial version today!

Visit us at Oracle OpenWorld next week!

2012-09-28T09:00:00.000-04:00

Fall is here, which must mean we are off to San Francisco, CA for the 2012 Oracle OpenWorld show. If you are planning on attending come by and see Altova at the Moscone center from Sept. 30^th – October 4^th in booth #3408 for a demo of Altova’s tools for Oracle Users and see the amazing new features in the newly released Altova MissionKit 2013!

While you are visiting our booth tell us about your latest projects and challenges, collaborate on best practices or let us show you some of the latest and greatest features available in MissionKit 2013, like Smart Fix XML Validation in XMLSpy, mapping SQL stored procedures in MapForce, watermark support in StyleVision, or our seamless integration options in Java applications. Don’t forget to enter our drawing for a chance to win a free Altova MapForce Basic Edition license!

Stored Procedures in Database Mappings

2012-09-25T11:27:00.000-04:00

In response to user requests, the new release of MapForce 2013 includes robust support for stored procedures in database mappings as:

input components (for stored procedures that provide results)
output components (for stored procedures that insert or update data in a database)
or, stored procedures may be inserted as a function-like call (allowing users to provide input data, execute the stored procedure, and read/map the output data to other components)

The screenshot below shows the mapping of a stored procedure in SQL Server to create an XML file.

Stored Procedures are programs that are hosted and run on a database server, callable from a client application. These programs are normally written in SQL but some databases can also handle calls to routines in languages like Java, Visual Basic , or other code. Stored procedures can perform simple operations like retrieving data, inserting new data, or updating database tables, or stored procedures can perform complex database administration tasks. Support for stored procedures varies among database vendors.

Database architects often use stored procedures to implement business logic in the database, which can simplify data management, especially when requirements change over time.

Stored procedures often require input parameters. MapForce allows you to provide values for all stored procedure input parameters that are required when the mapping is executed.

The procedure at the center of the database mapping shown above returns a table of data showing all the managers in the chain of command above the specified employee ID, which is supplied as an input parameter. The employee ID in this example is defined in the mapping as a constant with the value 67. You can also supply the parameter in another form, such as a calculated value or a data element retrieved from elsewhere in the database.

One issue with mapping stored procedures is the required data structure is not visible in the mapping until the first time the procedure is executed. MapForce provides a context menu that lets you run the stored procedure to reveal the data structure and individual data types for mapping:

The Output component of the mapping shown above is an XML document. Executing the mapping generates this result:

The MapForce integrated Help and example files demonstrate various applications of stored procedures in database mappings.

Find out for yourself how easy it is to include stored procedures in your own database mapping projects! Download a free 30-day trial of MapForce 2013.

Altova MissionKit 2013 with Smart Fix

2012-09-19T07:00:00.000-04:00

Altova Software Version 2013 was released today, and, as always, numerous features in this year’s autumn launch have been developed in response to customer requests. You asked for the ability to use SQL stored procedures in MapForce data mapping projects, to embed XSDs in WSDL documents in XMLSpy, to add watermarks in StyleVision, and to integrate MissionKit functionality flexibly in Java applications. All those features and more are there in Version 2013. We’ve also added some features that can only be described as magical.

Let’s take a look at the highlights…

Version 2013 Highlights

Smart Fix – it’s XML alchemy

We are especially excited to introduce Smart Fix validation, which takes intelligent XML editing in XMLSpy to a whole new level.

With Smart Fix, in addition to reporting comprehensive information about validation errors, XMLSpy enumerates the possible corrections for fixing them and will make the required changes automatically, based on your selection! This feature is truly magical. It reduces the time and frustration spent hunting down and fixing validation errors considerably. There is simply no other XML editor that even comes close to this.

Mapping SQL Stored Procedures in MapForce

Support for these very important tools for designing, querying, and maintaining database architectures has been a frequent user request, especially among more DBA-oriented users.

MapForce, Altova’s any-to-any data mapping tool, has long provided powerful functionality for mapping, processing, and transforming SQL database data, and now it also includes robust support for stored procedures. You can utilize stored procedures in a data mapping project as:

· Input components (for stored procedures that provide results)

· Output components (for stored procedures that insert or update data in a database)

· Data processing function calls (allowing users to provide input data, execute the stored procedure, and read/map the output data to other components)

Watermark Support in StyleVision

One of the challenges of multi-channel publishing is being able to flexibly add output-specific features to a single layout design. For instance, when creating reports, media that is typically printed or presented in document format may require the addition of a watermark (such as diagonally page-spanning “confidential” or “draft” in the background), which is something not usually found on an HTML Web page.

Designers and developers now have the ability in StyleVision 2013 to incorporate image and/or free text watermark content in PDF, RTF, and Word output, while leaving the HTML output unchanged. This adds even more power and flexibility to multi-channel publishing of XML, SQL database, and XBRL content in StyleVision.

Seamless Integration Options in Java Applications

Another MissionKit 2013 feature added with flexibility in mind is a new API that allows developers to integrate XMLSpy, MapForce, StyleVision, and Authentic functionality seamlessly in custom Java applications for Windows.

This frequently-requested capability adds to support for integrating MissionKit tools in Visual Basic or C# applications, giving developers even more options for adding powerful MissionKit views and functionality to their custom apps. Example applications, such as the one for XMLSpy shown below, are also provided to give you a head start implementing this functionality.

These are just a few of the numerous new features added to the Altova XML, SQL, and UML product line in Version 2013. You can read more details here.

Try Version 2013 Now

Version 2013 is a free update for Altova customers with an active Support and Maintenance Package. Visit the Altova Download page to update now!

If you’re not already a customer, you may download a free, fully-functional 30-day trial.

How Does MetaTeam Generate Responsibility Matrixes?

2012-09-11T10:14:00.000-04:00

Creating a responsibilities matrix (often referred to as a RACI matrix) is a very basic team best practice

The concept of aligning responsibilities with roles, and roles with team members, makes sense for small teams, to departments, to major efforts following Prince2, the PMBOK, ITIL or a similar approach.

In fact, if you didn’t create a matrix, you and your teammates would still have a foggy one in your mind’s eye. The problem with not creating an actual matrix is that everyone’s mental matrix differs slightly, allowing forgetfulness, avoidance and conflict to rule the day.

For example, if your work on an ERP system involves exchange rates it is intuitive to assume that the accountants have that covered. The accountants, on the other hand, really just need a developer to hook up an exchange rate service, so to them the work is basically IT’s responsibility. A responsibility matrix clears up this kind of confusion.

MetaTeam makes creating matrixes quick and easy. More importantly, MetaTeam links your matrixes with day-to-day work so they remain accurate and useful.

Let’s make some examples in a quick tutorial to show how easy it is…

First, we are going to create a basic responsibilities matrix. Then we will expand on it, and in the process also create a roles matrix.

Let’s meet the team. Our example team is implementing the accounting module of a new ERP system for Boulder Machinery Inc, a mid-sized manufacturing company with operations on three continents.

There are 15 people working on this module with an estimated duration of 16 months. The team leader is John Carfe, a professional Project Manager. It is sponsored by Diane Webber, CIO, and her colleague Daniel Johnson, the company Controller. The team is roughly split between technologists and accountants.

If you are working along in your own MetaTeam account, create a team called “Accounting Module Implementation”. On the My Teams page select your new team as the current team.

You may not have 15 users to assign to your example team if you are just getting started with MetaTeam. But you can follow along with this example just using a personal email address to create a test user that can be added to your new team – don’t worry you can always delete the account later.

Next assign your new team’s members. Go to the Directory app’s People tab. Find each team member and click the add icon to the left of their name. You will see the Current Team box on the right of the window fill up as you do this.

Now you are ready to create the roles. Click on the Roles app icon in the top nav bar.

We’ll add the team roles as a list in the Roles tab. Then we will switch to the Members tab to make the assignments and print the matrix.

Every team automatically includes three roles: Admin, Sponsor and Leader. The Admin role is special. It gives assignees additional MetaTeam privileges allowing them to add, edit and delete content and make assignments. Being in the Admin role is essentially equivalent to being the creator of the team.

Sponsor and Leader are roles that are common to essentially all teams, so MetaTeam creates them for you. Let’s leave Sponsor as-is. But change Leader to Project Manager.

Next add the other roles for this team. I will enter a typical set of roles, but again, feel free to just enter a couple in your own example team, if you like.

Now the responsibilities. I came up with a number of responsibilities for this example team. In the real world you might have many more than this – that is the nature of teamwork.

And here we get to the fun part. To make your responsibilities matrix simply click the Responsibilities tab and start checking boxes.

When you have checked off the responsibilities you want to assign to at least a couple of roles, take a look back at the Roles tab. Now you see the responsibilities listed with the roles you assigned them to.

Click back to the Responsibilities tab. Look in the lower left corner for the report icon.

Click that icon to open a report-form responsibilities matrix. This report gives you both responsibilities mapped to roles and to team members.

However, we haven’t made role assignments yet, so let’s continue.

Our team members are all set and we already entered the roles. All we need to do is make the assignments of members to roles. This is easy!

Open the Members tab. You see a grid of members against roles. Just check the boxes for the roles each member should perform.

To get a printable roles matrix just look in the lower left corner of the grid for the print icon.

Click the icon to open the roles matrix report.

There you have it: handsome printable roles and responsibilities matrixes.

But wait, you may be thinking, what about RACI?

Again, RACI stands for Responsible, Accountable, Consulted and Informed. It is a great way of clearly stating how a shared responsibility will be apportioned between the roles involved.

Setting RACI values in the responsibility matrix is straightforward.

For each responsibility where you want to set a RACI value click on its name. You can see the responsibilities names in the Roles tab below the assigned roles, or on the Responsibilities tab along the top of the grid.

In the open responsibility you see a section listing the assigned roles below the name and description of the responsibility. Beside each role are three selects.

The first select lets you apportion the responsibility across the roles by percent of effort. The second is our RACI values. The third select lets you estimate a level of effort in hours per week.

It is simple to set the RACI values as needed. With that done, click over to the Roles tab. Your responsibility is now listed with the RACI value for each role. Likewise, back on the Responsibilities tab when you click the report icon the print view now shows one or more R, A, C or I values according to your settings.

And there you have it. A complete responsibility assignment matrix and a role matrix. The perfect way to make the areas of responsibility simple and clear to all team members!

To see the full print versions of my example team you can click here for the roles matrix example and here for the responsibilities matrix.

My example team is fairly large. But all it takes for you to try creating your own matrixes is a test team with couple of members and one or two roles and responsibilities.

Give it a try. A few minutes after you signup you will have your own matrixes, and you will see how easy it is to get your team organized in MetaTeam!