Investigating source data using Investigate Stage

2010-02-27T00:00:00.002-05:00

Investigating source data using Investigate Stage

Learn how to set up and process an Investigate job to provide data from which you can create reports in the IBM Information Server Web console.

Follow these steps to set up and link an Investigate job

Create a QualityStage job by adding Data Quality stages and DataStage sequential files and stages to the Designer canvas. Connect the stages together for the flow of data from source file to each stage.

The job should contain a Sequential file(Source File)--->Copy Stage--->2 Investigate Stages --->Sequential Files(3 Files.Name,AreaToken,Area Pattern)

Check this diagram for the job canvas

Introduction to WebSphere QualityStage

2010-01-16T21:34:00.003-05:00

WebSphere QualityStage comprises a set of stages, a Match Designer, and related capabilities that provide a development environment for building data-cleansing tasks called jobs.

Using the stages and design components, you can quickly and easily process large stores of data, selectively transforming the data as needed.

WebSphere QualityStage provides a set of integrated modules for accomplishing data re-engineering tasks:

Investigating
Conditioning (standardizing)
Designing and running matches
Determining which data records survive

The probabilistic matching capability and dynamic weighting strategies of WebSphere QualityStage help you create high-quality, accurate data and consistently identify core business information such as customer, location, and product throughout the enterprise. WebSphere QualityStage standardizes and matches any type of information. By ensuring data quality, WebSphere QualityStage reduces the time and cost to implement CRM, business intelligence, ERP, and other strategic customer-related IT initiatives.

Scenarios for data cleansing

Organizations need to understand the complex relationships that they have with their customers, suppliers and distribution channels. They need to base decisions on accurate counts of parts and products to compete effectively, provide exceptional service, and meet increasing regulatory requirements. Consider the following scenarios:

Banking: One view of households

To facilitate marketing and mail campaigns, a large retail bank needed a single dynamic view of its customers’ households from 60 million records in 50 source systems. The bank uses WebSphere QualityStage to automate the process. Consolidated views are matched for all 50 sources, yielding information for all marketing campaigns. The result is reduced costs and improved return on the bank's marketing investments. Householding is now a standard process at the bank, which has a better understanding of its customers and more effective customer relationship management. 2.Pharmaceutical: Operations information A large pharmaceutical company needed a data warehouse for marketing and sales information. The company had diverse legacy data with different standards and formats, information that was buried in free-form fields, incorrect data values, discrepancies between field metadata and actual data in the field, and duplicates. It was impossible to get a complete, consolidated view of an entity such as total quarterly sales from the prescriptions of one doctor. Reports were difficult and time-consuming to compile, and their accuracy was suspect. Most vendor tools lack the flexibility to find all the legacy data variants, different formats for business entities, and other data problems. The company chose WebSphere QualityStage because it goes beyond traditional data-cleansing techniques to investigate fragmented legacy data at the level of each data value. Analysts can now access complete and accurate online views of doctors, the prescriptions that they write, and their managed-care affiliations for better decision support, trend analysis, and targeted marketing. 3.Insurance: One real-time view of the customer A leading insurance company lacked a unique ID for each subscriber, many of whom participated in multiple health, dental, or benefit plans. Subscribers who visited customer portals could not get complete information on their account status, eligible services, and other details. Using WebSphere QualityStage, the company implemented a real-time, in-flight data quality check of all portal inquiries. WebSphere QualityStage and WebSphere MQ transactions were combined to retrieve customer data from multiple sources and return integrated customer views. The new process provides more than 25 million subscribers with a real-time, 360-degree view of their insurance services. A unique customer ID for each subscriber is also helping the insurer move toward a single customer database for improved customer service and marketing.

Data Stage Job Status Values

2009-03-03T14:01:00.009-05:00

The Data Stage Job Status values can be found in
$DSHOME/include/dsapi.h file

These are the values defined in dsapi.h file for JOB STATUS:

#define DSJS_RUNNING 0 /* Job running */
#define DSJS_RUNOK 1 /* Job finished a normal run with no warnings */
#define DSJS_RUNWARN 2 /* Job finished a normal run with warnings */
#define DSJS_RUNFAILED 3 /* Job finished a normal run with a fatal error */
#define DSJS_VALOK 11 /* Job finished a validation run with no warnings */
#define DSJS_VALWARN 12 /* Job finished a validation run with warnings */
#define DSJS_VALFAILED 13 /* Job failed a validation run */
#define DSJS_RESET 21 /* Job finished a reset run */
#define DSJS_CRASHED 96 /* Job was stopped by some indeterminate action */
#define DSJS_STOPPED 97 /* Job was stopped by operator intervention (can't tell run type) */
#define DSJS_NOTRUNNABLE 98 /* Job has not been compiled */
#define DSJS_NOTRUNNING 99 /* Any other status */

JobStatus

ETL Data warehouse Concepts....Ascential Data Stage,Informatica ,SQL,PL/SQL,Unix

Investigating source data using Investigate Stage

Investigating source data using Investigate Stage

Learn how to set up and process an Investigate job to provide data from which you can create reports in the IBM Information Server Web console.

Follow these steps to set up and link an Investigate job

Introduction to WebSphere QualityStage

Data Stage Job Status Values

The Data Stage Job Status values can be found in $DSHOME/include/dsapi.h file

These are the values defined in dsapi.h file for JOB STATUS:

The Data Stage Job Status values can be found in
$DSHOME/include/dsapi.h file