TechInfoBI

DWH

2014-03-03T04:53:00.001-08:00

Src-Kimball Articals

Data Latency

2014-03-03T04:50:00.003-08:00

The data latency requirement describes how quickly the data must be delivered

to end users. Data latency obviously has a huge effect on the architecture

and the system implementation.

Create Stored Procedure in SQL Server

2013-01-23T21:56:00.001-08:00

Create Stored Procedure in SQL Server

A stored procedure groups one or more Transact-SQL statements into a logical unit, stored as an object in a SQL Server database.

Benefits of using stored procedures:

Stored procedures help centralize your Transact-SQL code in the data tier. Websites or applications that embed ad hoc SQL are notoriously difficult to modify in a production environment. When ad hoc SQL is embedded in an application, you may spend too much time trying to find and debug the embedded SQL. Once you’ve found the bug, chances are you’ll need to recompile the page or program executable, causing unnecessary application outages or application distribution nightmares. If you centralize your Transact-SQL code in stored procedures, you’ll have only one place to look for SQL code or SQL batches. If you document the code properly, you’ll also be able to capture the areas that need fixing.

Stored procedures can help reduce network traffic for larger ad hoc queries. Programming your application call to execute a stored procedure, rather then push across a 500 line SQL call, can have a positive impact on your network and application performance, particularly if the call is repeated thousands of times a minute.

Stored procedures encourage code reusability. For example, if your web application uses a drop-down menu containing a list of cities, and this drop-down is used in multiple web pages, you can call the stored procedure from each web page rather than embed the same SQL in multiple places.

Stored procedures are Secure with the data Security.

Example Create Simple Procedure

CREATE PROCEDURE [dbo].[Proc_GETCOUNTRY]
AS
BEGIN
SET NOCOUNT ON;
select Country_id, Country from tbl_Country order by Country
END

Now if you want to see the result from this procedure then execute it with this given command.

EXEC Proc_GETCOUNTRY

Result is

Country_id	Country
1	America
2	Canada
3	Brazil
4	Japan
5	India
6	China

Alter Stored Procedures

If You want to make some Changes in your Existing procedure then you will only get that procedure and with ALTER keyword you can alter your procedure .The Example is here.

ALTER PROCEDURE [dbo].[Proc_GETCOUNTRY]
AS
BEGIN
SET NOCOUNT ON;
select top 2 Country from tbl_Country
END

Now Execute it with F5 key or you can Write EXEC Proc_GETCOUNTRY

The Result is:

Country

America

Canada

Dropping Stored Procedures

You can drop a stored procedure from the database using the DROP PROCEDURE command. The syntax for dropping a stored procedure is:

DROP PROCEDURE { [ schema_name. ] procedure } [ ,...n ]

This command takes one argument; the name of the procedure or procedures to drop. For example:

DROP PROCEDURE dbo.Proc_GETCOUNTRY

How It Works

Once a stored procedure is dropped, its definition information is removed from the database’s system tables. Any cached query execution plans are also removed for that stored procedure. Code references to the stored procedure by other stored procedures or triggers will fail upon execution once the stored procedure has been dropped.

Creating a Parameterized Stored Procedure in Sql Server

In the previous recipe, I demonstrated a non-parameterized stored procedure, meaning that no external parameters were passed to it. The ability to pass parameters to them is part of why stored procedures are one of the most important database objects in SQL Server. Using parameters, you can pass information into the body of the procedure in order to return customized search information, or use parameters to influence or execute INSERT, UPDATE, or DELETE statements against tables. A procedure can have up to 2100 parameters (although it’s unlikely you’ll want to use nearly that many).

Example Parameterized Stored Procedure

CREATE PROCEDURE [dbo].[Proc_GETCOUNTRY]
(
@Country_id int
)
AS
BEGIN
SET NOCOUNT ON;
select Country from tbl_Country Where Country_id=@Country_id
END

To get the result pass the value to procedure and get the result here how to pass the value in procedure.

EXEC Proc_GETCOUNTRY 3

Result is

Country

Brazil

Performing Data Manipulation Add and Update Rows

CREATE PROC [dbo]. Proc_SaveCountryName
(
@CountryName varchar(50)
)
AS
BEGIN
INSERT into dbo.tbl_Country( Country) VALUES ( @CountryName)
END

EXEC Proc_SaveCountryName 'England'

Result is

Country_id	Country
1	America
2	Canada
3	Brazil
4	Japan
5	India
6	China
7	England

Update rows using parameter values

CREATE PROC [dbo]. Proc_UpdateCountryName
(
@ID int,
@CountryName varchar(50)
)
AS
BEGIN
update dbo.tbl_Country SET country=@CountryName where Country_id=@ID
END

EXEC Proc_UpdateCountryName 3 ,'NewZealand'

Result is

Country_id	Country
1	America
2	Canada
3	NewZealand
4	Japan
5	India
6	China
7	England

How it works

In this Proc_SaveCountryName we have pass one parameter value to the procedure and it has saved the record in tbl_country table.
EXEC Proc_SaveCountryName 'England'
So In the next example we have updated the record with passing two paremeter value to the procedure like
EXEC Proc_UpdateCountryName 3 ,'NewZealand'
so we have updated the 3rd row in table with 'NewZealand' Country name in place of'Brazil'.

Stored Procedure with Output Parameter in Sql Server

Output parameters enable a stored procedure to return data to the calling client procedure. The keyword output is required both when the procedure is created and when it is called. Within the stored procedure, the output parameter appears as a local variable. In the calling procedure or batch, a variable must have been created to receive the output parameter. When the stored procedure concludes, its current value is passed to the calling procedure’s local variable.

Output Parameter Example

ALTER PROCEDURE [dbo].[Proc_SaveCountry]
(
@Country nvarchar(30),
@Status tinyint output
)
AS BEGIN
SET NOCOUNT ON;
if exists (select Country_Id from tbl_Country where Country=@Country)
Begin
Set @Status=0
end
ELSE
Begin
insert into tbl_Country (Country) values (@Country)
set @Status=1
End

END

In the Above Example this line means if exists (select Country_Id from tbl_Country where Country=@Country)
Begin
Set @Status=0 end
It will check the country_id is already exist in database or not if not exists set @Status =0 means in front end coding by getting this value we can give the message that record already exists.
In the else part if country id is exists it will succesfully insert the record and set status =1 means record successfully inserted.

Using Output parameter Getting last Inserted Country_id value.

CREATE PROCEDURE [dbo].[Proc_SaveCountry]
(
@Country nvarchar(30),
@Country_id tinyint output
)
AS BEGIN
SET NOCOUNT ON;
insert into tbl_Country (Country) values (@Country)
Select @Country_id =@@IDENTITY
END

In this example the last inserted value in tbl_country is retreived by @@IDENTITY keyword and assign to output parameter @country_id

Fetch Stored Procedure

The fetch stored procedure retrieves the data. A sophisticated fetch procedure can accept various parameters and respond with a single row, filtered rows, or all rows, depending on the requirement and the parameters, so multiple fetch procedures are not necessary for various scopes of data. The null default is used in the where clause to effectively nullify the criterion if the parameter is not supplied.
This stored procedure also handles lock-timeout and deadlock issues using the techniques covered in the next chapter (we will disscuss it later), “Managing Transactions, Locking, and Blocking.” The following sample fetch stored procedure retrieves product information for the OBXKites database:

CREATE PROCEDURE pProduct_Fetch
(
@ProductCode CHAR(15) = NULL,
@ProductCategory CHAR(15) = NULL
) AS
SET NoCount ON
SELECT Code, ProductName, ProductDescription, ActiveDate,
DiscontinueDate, ProductCategoryName, [RowVersion] --,
-- Product.Created, Product.Modified
FROM dbo.Product
JOIN dbo.ProductCategory
ON Product.ProductCategoryID
= ProductCategory.ProductCategoryID
WHERE ( Product.Code = @ProductCode
OR @ProductCode IS NULL )
AND ( ProductCategory.ProductCategoryName
= @ProductCategory
OR @ProductCategory IS NULL
) IF @@Error <> 0 RETURN -100
RETURN

The following command executes the pProduct_Fetch stored procedure and retrieves data for all the products when called without any parameters:
EXEC pProduct_Fetch

Result (columns and rows abridged):

Code	Name	Modified
1001	Basic Box Kite 21 inch	2002-02-18 09:48:31.700
1002	Dragon Flight	2002-02-18 15:19:34.350

1003	Sky Dancer	2002-02-18 09:48:31.700

With a @ProductCode parameter, the fetch stored procedure returns only the selected product:
EXEC pProduct_Fetch
@ProductCode = ‘1005’

Result (columns abridged):

Code	Name	Modified
1005	Eagle Wings	2002-02-18 09:48:31.700

The second parameter causes the stored procedure to return all the products within a single product category:
EXEC pProduct_Fetch
@ProductCategory = ‘Book’

Result (rows and columns abridged):

Code	Name	Modified
1036	Adventures in the OuterBanks	2002-02-25 17:13:15.430
1037	Wright Brothers Kite Designs	2002-02-25 17:13:15.430

1038	The Lighthouses of the OBX	2002-02-25 17:13:15.430
1039	Outer Banks Map	2002-02-25 17:13:15.430
1040	Kiters Guide to the Outer Banks	2002-02-25 17:13:15.430

Update Stored Procedure in Sql Server

The update stored procedure accepts the primary method of identifying the row (in this case the product code) and the new data. Based on the new data, it performs a SQL DML update statement.

The previous example procedure handles lost updates by checking the rowversion timestamp column. Each time the row is updated, SQL Server automatically updates the rowversion value. If the rowversion is different, the row must have been updated by another transaction, and the rowversion condition in the where clause prevents the update.

Update with RowVersion

This version of the update procedure updates all the columns of the row, so all the parameters must be supplied even if that column is not being updated. The procedure assumes that the rowversion column was selected when the data was originally retrieved. If the rowversion value differs from the one retrieved during the select, the update fails to take place. The procedure determines that using the @@rowcount global variable and reports the error to the calling object. As a sample update procedure, here’s the code for the pProduct_Update_RowVersion stored procedure from the OBXKites database:

CREATE PROCEDURE pProduct_Update_RowVersion
(
@Code CHAR(15),
@RowVersion Rowversion,
@Name VARCHAR(50),
@ProductDescription VARCHAR(50),
@ActiveDate DateTime,
@DiscontinueDate DateTime )
AS
SET NoCount ON
UPDATE dbo.Product
SET ProductName = @Name,
ProductDescription = @ProductDescription,
ActiveDate = @ActiveDate,DiscontinueDate = @DiscontinueDate
WHERE Code = @Code AND [RowVersion] = @RowVersion
IF @@ROWCOUNT = 0
BEGIN
IF EXISTS ( SELECT * FROM Product WHERE Code = @Code)
BEGIN
RAISERROR (‘Product failed to update because another transaction updated the row since your
last read.’, 16,1)
RETURN -100
END
ELSE
BEGIN
RAISERROR (‘Product failed to update because the row has been deleted’, 16,1)
RETURN -100
END
END
RETURN

To test the timestamp version of the update stored procedure, the pProduct_Fetch procedure will return the current timestamp for product code “1001”:
EXEC pProduct_Fetch 1001

Code	Name	RowVersion
1001	Basic Box Kite 21 inch	0x0000000000000077

The pProduct_Update_Rowversion stored procedure must be called with the exact same rowversion value to perform the update:
EXEC pProduct_Update_Rowversion
1001,0x0000000000000077,
‘updatetest’,‘new description’,‘1/1/2002’,NULL
The procedure updates all the columns in the row, and in the process, the rowversion column is reset to a new value.

Delete Stored Procedure in Sql Server

The delete stored procedure executes the delete DML command. This procedure can be the most complex stored procedure, depending upon the level of data-archival and logical deletion. This sample delete procedure taken from the OBXKites database transforms the @ProductCode into a @ProductID, verifies that the product does in fact exist, and then deletes it:

CREATE PROCEDURE pProduct_Delete
(
@ProductCode INT
)
AS
SET NOCOUNT ON
DECLARE @ProductID UniqueIdentifier
SELECT @ProductID = ProductID
FROM Product
WHERE Code = @ProductCode
If @@RowCount = 0
BEGIN
RAISERROR(‘Unable to delete Product Code %i - does not exist.’, 16,1, @ProductCode)
RETURN
END
ELSE
DELETE dbo.Product
WHERE ProductID = @ProductID
RETURN

To test the pProduct_Delete stored procedure, the following stored procedure attempts to call a product. Because there is no product code “99,” the error trapping raises the error: EXEC pProduct_Delete 99
Result:
Unable to delete Product Code 99 - does not exist.

Data Warehouse Concepts

2012-11-05T20:21:00.001-08:00

Data Warehouse Concepts

What is a Data Warehouse?

According to Inmon, famous author for several data warehouse books, "A data warehouse is a subject oriented, integrated, time variant, non volatile collection of data in support of management's decision making process".

Example: In order to store data, over the years, many application designers in each branch have made their individual decisions as to how an application and database should be built. So source systems will be different in naming conventions, variable measurements, encoding structures, and physical attributes of data. Consider a bank that has got several branches in several countries, has millions of customers and the lines of business of the enterprise are savings, and loans. The following example explains how the data is integrated from source systems to target systems.

Example of Source Data

System Name	Attribute Name	Column Name	Datatype	Values
Source System 1	Customer Application Date	CUSTOMER_APPLICATION_DATE	NUMERIC(8,0)	11012005
Source System 2	Customer Application Date	CUST_APPLICATION_DATE	DATE	11012005
Source System 3	Application Date	APPLICATION_DATE	DATE	01NOV2005

In the aforementioned example, attribute name, column name, datatype and values are entirely different from one source system to another. This inconsistency in data can be avoided by integrating the data into a data warehouse with good standards.

Example of Target Data(Data Warehouse)

Target System	Attribute Name	Column Name	Datatype	Values
Record #1	Customer Application Date	CUSTOMER_APPLICATION_DATE	DATE	01112005
Record #2	Customer Application Date	CUSTOMER_APPLICATION_DATE	DATE	01112005
Record #3	Customer Application Date	CUSTOMER_APPLICATION_DATE	DATE	01112005

In the above example of target data, attribute names, column names, and datatypes are consistent throughout the target system. This is how data from various source systems is integrated and accurately stored into the data warehouse.

See Figure below for Data Warehouse Architecture Diagram.

Data Warehouse & Data Mart
A data warehouse is a relational/multidimensional database that is designed for query and analysis rather than transaction processing. A data warehouse usually contains historical data that is derived from transaction data. It separates analysis workload from transaction workload and enables a business to consolidate data from several sources.

In addition to a relational/multidimensional database, a data warehouse environment often consists of an ETL solution, an OLAP engine, client analysis tools, and other applications that manage the process of gathering data and delivering it to business users.

There are three types of data warehouses:
1. Enterprise Data Warehouse - An enterprise data warehouse provides a central database for decision support throughout the enterprise.
2. ODS(Operational Data Store) - This has a broad enterprise wide scope, but unlike the real entertprise data warehouse, data is refreshed in near real time and used for routine business activity.
3. Data Mart - Datamart is a subset of data warehouse and it supports a particular region, business unit or business function.

Data warehouses and data marts are built on dimensional data modeling where fact tables are connected with dimension tables. This is most useful for users to access data since a database can be visualized as a cube of several dimensions. A data warehouse provides an opportunity for slicing and dicing that cube along each of its dimensions.

Data Mart: A data mart is a subset of data warehouse that is designed for a particular line of business, such as sales, marketing, or finance. In a dependent data mart, data can be derived from an enterprise-wide data warehouse. In an independent data mart, data can be collected directly from sources.

What is Star Schema?
Star Schema is a relational database schema for representing multidimensional data. It is the simplest form of data warehouse schema that contains one or more dimensions and fact tables. It is called a star schema because the entity-relationship diagram between dimensions and fact tables resembles a star where one fact table is connected to multiple dimensions. The center of the star schema consists of a large fact table and it points towards the dimension tables. The advantage of star schema is slicing down, performance increase and easy understanding of data.

Steps in designing Star Schema

· Identify a business process for analysis (like sales).

· Identify measures or facts (sales dollar).

· Identify dimensions for facts (product dimension, location dimension, time dimension, organization dimension).

· List the columns that describe each dimension.(region name, branch name, region name).

· Determine the lowest level of summary in a fact table(sales dollar).

Important aspects of Star Schema & Snow Flake Schema

· In a star schema every dimension will have a primary key.

· In a star schema, a dimension table will not have any parent table.

· Whereas in a snow flake schema, a dimension table will have one or more parent tables.

· Hierarchies for the dimensions are stored in the dimensional table itself in star schema.

· Whereas hierachies are broken into separate tables in snow flake schema. These hierachies helps to drill down the data from topmost hierachies to the lowermost hierarchies.

Glossary:

Hierarchy
A logical structure that uses ordered levels as a means of organizing data. A hierarchy can be used to define data aggregation; for example, in a time dimension, a hierarchy might be used to aggregate data from the Month level to the Quarter level, from the Quarter level to the Year level. A hierarchy can also be used to define a navigational drill path, regardless of whether the levels in the hierarchy represent aggregated totals or not.

Level
A position in a hierarchy. For example, a time dimension might have a hierarchy that represents data at the Month, Quarter, and Year levels.

Fact Table
A table in a star schema that contains facts and connected to dimensions. A fact table typically has two types of columns: those that contain facts and those that are foreign keys to dimension tables. The primary key of a fact table is usually a composite key that is made up of all of its foreign keys.

A fact table might contain either detail level facts or facts that have been aggregated (fact tables that contain aggregated facts are often instead called summary tables). A fact table usually contains facts with the same level of aggregation.

Example of Star Schema:

In the example figure sales fact table is connected to dimensions location, product, time and organization. It shows that data can be sliced across all dimensions and again it is possible for the data to be aggregated across multiple dimensions. "Sales Dollar" in sales fact table can be calculated across all dimensions independently or in a combined manner which is explained below.

· Sales Dollar value for a particular product

· Sales Dollar value for a product in a location

· Sales Dollar value for a product in a year within a location

· Sales Dollar value for a product in a year within a location sold or serviced by an employee

Snow Flake Schema

A snowflake schema is a term that describes a star schema structure normalized through the use of outrigger tables. i.e dimension table hierachies are broken into simpler tables. In star schema example we had 4 dimensions like location, product, time, organization and a fact table(sales).

In snow flake schema, the example diagram shown below has 4 dimension tables, 4 lookup tables and 1 fact table. The reason is that hierarchies(category, branch, state, and month) are being broken out of the dimension tables(PRODUCT, ORGANIZATION, LOCATION, and TIME) respectively and shown separately. In OLAP, this snow flake schema approach increases the number of joins and poor performance in retrieval of data. In few organizations, they try to normalize the dimension tables to save space. Since dimension tables hold less space, snow flake schema approach may be avoided.

Example of Snow Flake Schema:

Fact Table
The centralized table in a star schema is called as FACT table. A fact table typically has two types of columns: those that contain facts and those that are foreign keys to dimension tables. The primary key of a fact table is usually a composite key that is made up of all of its foreign keys.

In the example fig 1.6 "Sales Dollar" is a fact(measure) and it can be added across several dimensions. Fact tables store different types of measures like additive, non additive and semi additive measures.

Measure Types

· Additive - Measures that can be added across all dimensions.

· Non Additive - Measures that cannot be added across all dimensions.

· Semi Additive - Measures that can be added across few dimensions and not with others.

A fact table might contain either detail level facts or facts that have been aggregated (fact tables that contain aggregated facts are often instead called summary tables).

In the real world, it is possible to have a fact table that contains no measures or facts. These tables are called as Factless Fact tables.

Steps in designing Fact Table

· Identify a business process for analysis(like sales).

· Identify measures or facts (sales dollar).

· Identify dimensions for facts(product dimension, location dimension, time dimension, organization dimension).

· List the columns that describe each dimension.(region name, branch name, region name).

· Determine the lowest level of summary in a fact table(sales dollar).

Example of a Fact Table with an Additive Measure in Star Schema: Figure 1.6

In the example figure 1.6, sales fact table is connected to dimensions location, product, time and organization. Measure "Sales Dollar" in sales fact table can be added across all dimensions independently or in a combined manner which is explained below.

· Sales Dollar value for a particular product

· Sales Dollar value for a product in a location

· Sales Dollar value for a product in a year within a location

· Sales Dollar value for a product in a year within a location sold or serviced by an employee

Dimension Table
Dimension table is one that describe the business entities of an enterprise, represented as hierarchical, categorical information such as time, departments, locations, and products. Dimension tables are sometimes called lookup or reference tables.

Location Dimension
In a relational data modeling, for normalization purposes, country lookup, state lookup, county lookup, and city lookups are not merged as a single table. In a dimensional data modeling (star schema), these tables would be merged as a single table called LOCATION DIMENSION for performance and slicing data requirements. This location dimension helps to compare the sales in one region with another region. We may see good sales profit in one region and loss in another region. If it is a loss, the reasons for that may be a new competitor in that area, or failure of our marketing strategy etc.

Example of Location Dimension: Figure

Country Lookup

Country Code	Country Name	DateTimeStamp
USA	United States Of America	1/1/2005 11:23:31 AM

State Lookup

State Code	State Name	DateTimeStamp
NY	New York	1/1/2005 11:23:31 AM
FL	Florida	1/1/2005 11:23:31 AM
CA	California	1/1/2005 11:23:31 AM
NJ	New Jersey	1/1/2005 11:23:31 AM

County Lookup

County Code	County Name	DateTimeStamp
NYSH	Shelby	1/1/2005 11:23:31 AM
FLJE	Jefferson	1/1/2005 11:23:31 AM
CAMO	Montgomery	1/1/2005 11:23:31 AM
NJHU	Hudson	1/1/2005 11:23:31 AM

City Lookup

City Code	City Name	DateTimeStamp
NYSHMA	Manhattan	1/1/2005 11:23:31 AM
FLJEPC	Panama City	1/1/2005 11:23:31 AM
CAMOSH	San Hose	1/1/2005 11:23:31 AM
NJHUJC	Jersey City	1/1/2005 11:23:31 AM

Location Dimension

Location Dimension Id	Country Name	State Name	County Name	City Name	DateTime Stamp
1	USA	New York	Shelby	Manhattan	1/1/2005 11:23:31 AM
2	USA	Florida	Jefferson	Panama City	1/1/2005 11:23:31 AM
3	USA	California	Montgomery	San Hose	1/1/2005 11:23:31 AM
4	USA	New Jersey	Hudson	Jersey City	1/1/2005 11:23:31 AM

Product Dimension
In a relational data model, for normalization purposes, product category lookup, product sub-category lookup, product lookup, and and product feature lookups are are not merged as a single table. In a dimensional data modeling(star schema), these tables would be merged as a single table called PRODUCT DIMENSION for performance and slicing data requirements.

Example of Product Dimension: Figure

Product Category Lookup

Product Category Code	Product Category Name	DateTimeStamp
1	Apparel	1/1/2005 11:23:31 AM
2	Shoe	1/1/2005 11:23:31 AM

Product Sub-Category Lookup

Product Sub-Category Code	Product Sub-Category Name	DateTime Stamp
11	Shirt	1/1/2005 11:23:31 AM
12	Trouser	1/1/2005 11:23:31 AM
13	Casual	1/1/2005 11:23:31 AM
14	Formal	1/1/2005 11:23:31 AM

Product Lookup

Product Code	Product Name	DateTimeStamp
1001	Van Heusen	1/1/2005 11:23:31 AM
1002	Arrow	1/1/2005 11:23:31 AM
1003	Nike	1/1/2005 11:23:31 AM
1004	Adidas	1/1/2005 11:23:31 AM

Product Feature Lookup

Product Feature Code	Product Feature Description	DateTimeStamp
10001	Van-M	1/1/2005 11:23:31 AM
10002	Van-L	1/1/2005 11:23:31 AM
10003	Arr-XL	1/1/2005 11:23:31 AM
10004	Arr-XXL	1/1/2005 11:23:31 AM
10005	Nike-8	1/1/2005 11:23:31 AM
10006	Nike-9	1/1/2005 11:23:31 AM
10007	Adidas-10	1/1/2005 11:23:31 AM
10008	Adidas-11	1/1/2005 11:23:31 AM

Product Dimension

Product Dimension Id	Product Category Name	Product Sub-Category Name	Product Name	Product Feature Desc	DateTime Stamp
100001	Apparel	Shirt	Van Heusen	Van-M	1/1/2005 11:23:31 AM
100002	Apparel	Shirt	Van Heusen	Van-L	1/1/2005 11:23:31 AM
100003	Apparel	Shirt	Arrow	Arr-XL	1/1/2005 11:23:31 AM
100004	Apparel	Shirt	Arrow	Arr-XXL	1/1/2005 11:23:31 AM
100005	Shoe	Casual	Nike	Nike-8	1/1/2005 11:23:31 AM
100006	Shoe	Casual	Nike	Nike-9	1/1/2005 11:23:31 AM
100007	Shoe	Casual	Adidas	Adidas-10	1/1/2005 11:23:31 AM
100008	Shoe	Casual	Adidas	Adidas-11	1/1/2005 11:23:31 AM

Organization Dimension
In a relational data model, for normalization purposes, corporate office lookup, region lookup, branch lookup, and employee lookups are not merged as a single table. In a dimensional data modeling(star schema), these tables would be merged as a single table called ORGANIZATION DIMENSION for performance and slicing data.

This dimension helps us to find the products sold or serviced within the organization by the employees. In any industry, we can calculate the sales on region basis, branch basis and employee basis. Based on the performance, an organization can provide incentives to employees and subsidies to the branches to increase further sales.

Example of Organization Dimension: Figure

Corporate Lookup

Corporate Code	Corporate Name	DateTimeStamp
CO	American Bank	1/1/2005 11:23:31 AM

Region Lookup

Region Code	Region Name	DateTimeStamp
SE	South East	1/1/2005 11:23:31 AM
MW	Mid West	1/1/2005 11:23:31 AM

Branch Lookup

Branch Code	Branch Name	DateTimeStamp
FLTM	Florida-Tampa	1/1/2005 11:23:31 AM
ILCH	Illinois-Chicago	1/1/2005 11:23:31 AM

Employee Lookup

Employee Code	Employee Name	DateTimeStamp
E1	Paul Young	1/1/2005 11:23:31 AM
E2	Chris Davis	1/1/2005 11:23:31 AM

Organization Dimension

Organization Dimension Id	Corporate Name	Region Name	Branch Name	Employee Name	DateTime Stamp
1	American Bank	South East	Florida-Tampa	Paul Young	1/1/2005 11:23:31 AM
2	American Bank	Mid West	Illinois-Chicago	Chris Davis	1/1/2005 11:23:31 AM

Time Dimension
In a relational data model, for normalization purposes, year lookup, quarter lookup, month lookup, and week lookups are not merged as a single table. In a dimensional data modeling(star schema), these tables would be merged as a single table called TIME DIMENSION for performance and slicing data.

This dimensions helps to find the sales done on date, weekly, monthly and yearly basis. We can have a trend analysis by comparing this year sales with the previous year or this week sales with the previous week.

Example of Time Dimension: Figure

Year Lookup

Year Id	Year Number	DateTimeStamp
1	2004	1/1/2005 11:23:31 AM
2	2005	1/1/2005 11:23:31 AM

Quarter Lookup

Quarter Number	Quarter Name	DateTimeStamp
1	Q1	1/1/2005 11:23:31 AM
2	Q2	1/1/2005 11:23:31 AM
3	Q3	1/1/2005 11:23:31 AM
4	Q4	1/1/2005 11:23:31 AM

Month Lookup

Month Number	Month Name	DateTimeStamp
1	January	1/1/2005 11:23:31 AM
2	February	1/1/2005 11:23:31 AM
3	March	1/1/2005 11:23:31 AM
4	April	1/1/2005 11:23:31 AM
5	May	1/1/2005 11:23:31 AM
6	June	1/1/2005 11:23:31 AM
7	July	1/1/2005 11:23:31 AM
8	August	1/1/2005 11:23:31 AM
9	September	1/1/2005 11:23:31 AM
10	October	1/1/2005 11:23:31 AM
11	November	1/1/2005 11:23:31 AM
12	December	1/1/2005 11:23:31 AM

Week Lookup

Week Number	Day of Week	DateTimeStamp
1	Sunday	1/1/2005 11:23:31 AM
1	Monday	1/1/2005 11:23:31 AM
1	Tuesday	1/1/2005 11:23:31 AM
1	Wednesday	1/1/2005 11:23:31 AM
1	Thursday	1/1/2005 11:23:31 AM
1	Friday	1/1/2005 11:23:31 AM
1	Saturday	1/1/2005 11:23:31 AM
2	Sunday	1/1/2005 11:23:31 AM
2	Monday	1/1/2005 11:23:31 AM
2	Tuesday	1/1/2005 11:23:31 AM
2	Wednesday	1/1/2005 11:23:31 AM
2	Thursday	1/1/2005 11:23:31 AM
2	Friday	1/1/2005 11:23:31 AM
2	Saturday	1/1/2005 11:23:31 AM

Time Dimension

Time Dim Id	Year No	Day Of Year	Quarter No	Month No	Month Name	Month Day No	Week No	Day of Week	Cal Date	DateTime Stamp
1	2004	1	Q1	1	January	1	1	5	1/1/2004	1/1/2005 11:23:31 AM
2	2004	32	Q1	2	February	1	5	1	2/1/2004	1/1/2005 11:23:31 AM
3	2005	1	Q1	1	January	1	1	7	1/1/2005	1/1/2005 11:23:31 AM
4	2005	32	Q1	2	February	1	5	3	2/1/2005	1/1/2005 11:23:31 AM

Slowly Changing Dimensions
Dimensions that change over time are called Slowly Changing Dimensions. For instance, a product price changes over time; People change their names for some reason; Country and State names may change over time. These are a few examples of Slowly Changing Dimensions since some changes are happening to them over a period of time.

Slowly Changing Dimensions are often categorized into three types namely Type1, Type2 and Type3. The following section deals with how to capture and handling these changes over time.

The "Product" table mentioned below contains a product named, Product1 with Product ID being the primary key. In the year 2004, the price of Product1 was $150 and over the time, Product1's price changes from $150 to $350. With this information, let us explain the three types of Slowly Changing Dimensions.

Product Price in 2004:

Product ID(PK)	Year	Product Name	Product Price
1	2004	Product1	$150

Type 1: Overwriting the old values.
In the year 2005, if the price of the product changes to $250, then the old values of the columns "Year" and "Product Price" have to be updated and replaced with the new values. In this Type 1, there is no way to find out the old value of the product "Product1" in year 2004 since the table now contains only the new price and year information.

Product

Product ID(PK)	Year	Product Name	Product Price
1	2005	Product1	$250

Type 2: Creating an another additional record.
In this Type 2, the old values will not be replaced but a new row containing the new values will be added to the product table. So at any point of time, the difference between the old values and new values can be retrieved and easily be compared. This would be very useful for reporting purposes.

Product

Product ID(PK)	Year	Product Name	Product Price
1	2004	Product1	$150
1	2005	Product1	$250

The problem with the above mentioned data structure is "Product ID" cannot store duplicate values of "Product1" since "Product ID" is the primary key. Also, the current data structure doesn't clearly specify the effective date and expiry date of Product1 like when the change to its price happened. So, it would be better to change the current data structure to overcome the above primary key violation.

Product

Product ID(PK)	Effective DateTime(PK)	Year	Product Name	Product Price	Expiry DateTime
1	01-01-2004 12.00PM	2004	Product1	$150	12-31-2004 11.59PM
1	01-01-2005 12.00PM	2005	Product1	$250

In the changed Product table's Data structure, "Product ID" and "Effective DateTime" are composite primary keys. So there would be no violation of primary key constraint. Addition of new columns, "Effective DateTime" and "Expiry DateTime" provides the information about the product's effective date and expiry date which adds more clarity and enhances the scope of this table. Type2 approach may need additional space in the data base, since for every changed record, an additional row has to be stored. Since dimensions are not that big in the real world, additional space is negligible.

Type 3: Creating new fields.
In this Type 3, the latest update to the changed values can be seen. Example mentioned below illustrates how to add new columns and keep track of the changes. From that, we are able to see the current price and the previous price of the product, Product1.

Product

Product ID(PK)	Current Year	Product Name	Current Product Price	Old Product Price	Old Year
1	2005	Product1	$250	$150	$2004

The problem with the Type 3 approach, is over years, if the product price continuously changes, then the complete history may not be stored, only the latest change will be stored. For example, in year 2006, if the product1's price changes to $350, then we would not be able to see the complete history of 2004 prices, since the old values would have been updated with 2005 product information.

Product

Product ID(PK)	Year	Product Name	Product Price	Old Product Price	Old Year
1	2006	Product1	$350	$250	$2005

What are ETL Tools?
ETL Tools are meant to extract, transform and load the data into Data Warehouse for decision making. Before the evolution of ETL Tools, the above mentioned ETL process was done manually by using SQL code created by programmers. This task was tedious and cumbersome in many cases since it involved many resources, complex coding and more work hours. On top of it, maintaining the code placed a great challenge among the programmers.

These difficulties are eliminated by ETL Tools since they are very powerful and they offer many advantages in all stages of ETL process starting from extraction, data cleansing, data profiling, transformation, debuggging and loading into data warehouse when compared to the old method.

There are a number of ETL tools available in the market to do ETL process the data according to business/technical requirements. Following are some those.

Popular ETL Tools

Tool Name	Company Name
Informatica	Informatica Corporation
DT/Studio	Embarcadero Technologies
DataStage	IBM
Ab Initio	Ab Initio Software Corporation
Data Junction	Pervasive Software
Oracle Warehouse Builder	Oracle Corporation
Microsoft SQL Server Integration	Microsoft

ETL Concepts
Extraction, transformation, and loading. ETL refers to the methods involved in accessing and manipulating source data and loading it into target database.

The first step in ETL process is mapping the data between source systems and target database(data warehouse or data mart). The second step is cleansing of source data in staging area. The third step is transforming cleansed source data and then loading into the target system.

Note that ETT (extraction, transformation, transportation) and ETM (extraction, transformation, move) are sometimes used instead of ETL.

Glossary of ETL (Reference:www.Oracle.com)

Source System
A database, application, file, or other storage facility from which the data in a data warehouse is derived.

Mapping
The definition of the relationship and data flow between source and target objects.

Metadata
Data that describes data and other structures, such as objects, business rules, and processes. For example, the schema design of a data warehouse is typically stored in a repository as metadata, which is used to generate scripts used to build and populate the data warehouse. A repository contains metadata.

Staging Area
A place where data is processed before entering the warehouse.

Cleansing
The process of resolving inconsistencies and fixing the anomalies in source data, typically as part of the ETL process.

Transformation
The process of manipulating data. Any manipulation beyond copying is a transformation. Examples include cleansing, aggregating, and integrating data from multiple sources.

Transportation
The process of moving copied or transformed data from a source to a data warehouse.

Target System
A database, application, file, or other storage facility to which the "transformed source data" is loaded in a data warehouse.

ETL Tools - Informatica

Informatica is a powerful ETL tool from Informatica Corporation, a leading provider of enterprise data integration software and ETL softwares.

The important Informatica Components are:

· Power Exchange

· Power Center

· Power Center Connect

· Power Exchange

· Power Channel

· Metadata Exchange

· Power Analyzer

· Super Glue

In Informatica, all the Metadata information about source systems, target systems and transformations are stored in the Informatica repository. Informatica's Power Center Client and Repository Server access this repository to store and retrieve metadata.

Source and Target:
Consider a Bank that has got many branches throughout the world. In each branch data may be stored in different source systems like oracle, sql server, terradata, etc. When the Bank decides to integrate its data from several sources for its management decisions, it may choose one or more systems like oracle, sql server, terradata, etc. as its data warehouse target. Many organisations prefer Informatica to do that ETL process, because Informatica is more powerful in designing and building data warehouses. It can connect to several sources and targets to extract meta data from sources and targets, transform and load the data into target systems.

Guidelines to work with Informatica Power Center

· Repository: This is where all the metadata information is stored in the Informatica suite. The Power Center Client and the Repository Server would access this repository to retrieve, store and manage metadata.

· Power Center Client: Informatica client is used for managing users, identifiying source and target systems definitions, creating mapping and mapplets, creating sessions and run workflows etc.

· Repository Server: This repository server takes care of all the connections between the repository and the Power Center Client.

· Power Center Server: Power Center server does the extraction from source and then loading data into targets.

· Designer: Source Analyzer, Mapping Designer and Warehouse Designer are tools reside within the Designer wizard. Source Analyzer is used for extracting metadata from source systems.
Mapping Designer is used to create mapping between sources and targets. Mapping is a pictorial representation about the flow of data from source to target.
Warehouse Designer is used for extracting metadata from target systems or metadata can be created in the Designer itself.

· Data Cleansing: The PowerCenter's data cleansing technology improves data quality by validating, correctly naming and standardization of address data. A person's address may not be same in all source systems because of typos and postal code, city name may not match with address. These errors can be corrected by using data cleansing process and standardized data can be loaded in target systems (data warehouse).

· Transformation: Transformations help to transform the source data according to the requirements of target system. Sorting, Filtering, Aggregation, Joining are some of the examples of transformation. Transformations ensure the quality of the data being loaded into target and this is done during the mapping process from source to target.

· Workflow Manager: Workflow helps to load the data from source to target in a sequential manner. For example, if the fact tables are loaded before the lookup tables, then the target system will pop up an error message since the fact table is violating the foreign key validation. To avoid this, workflows can be created to ensure the correct flow of data from source to target.

· Workflow Monitor: This monitor is helpful in monitoring and tracking the workflows created in each Power Center Server.

· Power Center Connect: This component helps to extract data and metadata from ERP systems like IBM's MQSeries, Peoplesoft, SAP, Siebel etc. and other third party applications.

· Power Center Exchange: This component helps to extract data and metadata from ERP systems like IBM's MQSeries, Peoplesoft, SAP, Siebel etc. and other third party applications.

ETL Tools - Informatica

Power Exchange:
Informatica Power Exchange as a stand alone service or along with Power Center, helps organizations leverage data by avoiding manual coding of data extraction programs. Power Exchange supports batch, real time and changed data capture options in main frame(DB2, VSAM, IMS etc.,), mid range (AS400 DB2 etc.,), and for relational databases (oracle, sql server, db2 etc) and flat files in unix, linux and windows systems.

Power Channel:
This helps to transfer large amount of encrypted and compressed data over LAN, WAN, through Firewalls, tranfer files over FTP, etc.

Meta Data Exchange:
Metadata Exchange enables organizations to take advantage of the time and effort already invested in defining data structures within their IT environment when used with Power Center. For example, an organization may be using data modeling tools, such as Erwin, Embarcadero, Oracle designer, Sybase Power Designer etc for developing data models. Functional and technical team should have spent much time and effort in creating the data model's data structures(tables, columns, data types, procedures, functions, triggers etc). By using meta deta exchange, these data structures can be imported into power center to identifiy source and target mappings which leverages time and effort. There is no need for informatica developer to create these data structures once again.

Power Analyzer:
Power Analyzer provides organizations with reporting facilities. PowerAnalyzer makes accessing, analyzing, and sharing enterprise data simple and easily available to decision makers. PowerAnalyzer enables to gain insight into business processes and develop business intelligence.

With PowerAnalyzer, an organization can extract, filter, format, and analyze corporate information from data stored in a data warehouse, data mart, operational data store, or otherdata storage models. PowerAnalyzer is best with a dimensional data warehouse in a relational database. It can also run reports on data in any table in a relational database that do not conform to the dimensional model.

Super Glue:
Superglue is used for loading metadata in a centralized place from several sources. Reports can be run against this superglue to analyze meta data.

Power Mart:
Power Mart is a departmental version of Informatica for building, deploying, and managing data warehouses and data marts. Power center is used for corporate enterprise data warehouse and power mart is used for departmental data warehouses like data marts. Power Center supports global repositories and networked repositories and it can be connected to several sources. Power Mart supports single repository and it can be connected to fewer sources when compared to Power Center. Power Mart can extensibily grow to an enterprise implementation and it is easy for developer productivity through a codeless environment.

Note:This is not a complete tutorial on Informatica. We will add more Tips and Guidelines on Informatica in near future. Please visit us soon to check back. To know more about Informatica, contact its official website www.informatica.com.

Informatica - Transformations
In Informatica, Transformations help to transform the source data according to the requirements of target system and it ensures the quality of the data being loaded into target.

Transformations are of two types: Active and Passive.

Active Transformation
An active transformation can change the number of rows that pass through it from source to target i.e it eliminates rows that do not meet the condition in transformation.

Passive Transformation
A passive transformation does not change the number of rows that pass through it i.e it passes all rows through the transformation.

Transformations can be Connected or UnConnected.

Connected Transformation
Connected transformation is connected to other transformations or directly to target table in the mapping.

UnConnected Transformation
An unconnected transformation is not connected to other transformations in the mapping. It is called within another transformation, and returns a value to that transformation.

Following are the list of Transformations available in Informatica:

· Aggregator Transformation

· Expression Transformation

· Filter Transformation

· Joiner Transformation

· Lookup Transformation

· Normalizer Transformation

· Rank Transformation

· Router Transformation

· Sequence Generator Transformation

· Stored Procedure Transformation

· Sorter Transformation

· Update Strategy Transformation

· XML Source Qualifier Transformation

· Advanced External Procedure Transformation

· External Transformation

In the following pages, we will explain all the above Informatica Transformations and their significances in the ETL process in detail.

Aggregator Transformation
Aggregator transformation is an Active and Connected transformation. This transformation is useful to perform calculations such as averages and sums (mainly to perform calculations on multiple rows or groups). For example, to calculate total of daily sales or to calculate average of monthly or yearly sales. Aggregate functions such as AVG, FIRST, COUNT, PERCENTILE, MAX, SUM etc. can be used in aggregate transformation.

Expression Transformation
Expression transformation is a Passive and Connected transformation. This can be used to calculate values in a single row before writing to the target. For example, to calculate discount of each product or to concatenate first and last names or to convert date to a string field.

Filter Transformation
Filter transformation is an Active and Connected transformation. This can be used to filter rows in a mapping that do not meet the condition. For example, to know all the employees who are working in Department 10 or to find out the products that falls between the rate category $500 and $1000.

Joiner Transformation
Joiner Transformation is an Active and Connected transformation. This can be used to join two sources coming from two different locations or from same location. For example, to join a flat file and a relational source or to join two flat files or to join a relational source and a XML source.

In order to join two sources, there must be atleast one matching port. at least one matching port. While joining two sources it is a must to specify one source as master and the other as detail.

The Joiner transformation supports the following types of joins:

· Normal

· Master Outer

· Detail Outer

· Full Outer

Normal join discards all the rows of data from the master and detail source that do not match, based on the condition.
Master outer join discards all the unmatched rows from the master source and keeps all the rows from the detail source and the matching rows from the master source.
Detail outer join keeps all rows of data from the master source and the matching rows from the detail source. It discards the unmatched rows from the detail source.
Full outer join keeps all rows of data from both the master and detail sources.

Lookup Transformation
Lookup transformation is Passive and it can be both Connected and UnConnected as well. It is used to look up data in a relational table, view, or synonym. Lookup definition can be imported either from source or from target tables.

For example, if we want to retrieve all the sales of a product with an ID 10 and assume that the sales data resides in another table. Here instead of using the sales table as one more source, use Lookup transformation to lookup the data for the product, with ID 10 in sales table.

Difference between Connected and UnConnected Lookup Transformation:
Connected lookup receives input values directly from mapping pipeline whereas UnConnected lookup receives values from: LKP expression from another transformation.

Connected lookup returns multiple columns from the same row whereas UnConnected lookup has one return port and returns one column from each row.

Connected lookup supports user-defined default values whereas UnConnected lookup does not support user defined values.

Normalizer Transformation
Normalizer Transformation is an Active and Connected transformation. It is used mainly with COBOL sources where most of the time data is stored in de-normalized format. Also, Normalizer transformation can be used to create multiple rows from a single row of data.

Rank Transformation
Rank transformation is an Active and Connected transformation. It is used to select the top or bottom rank of data. For example, to select top 10 Regions where the sales volume was very high or to select 10 lowest priced products.

Router Transformation
Router is an Active and Connected transformation. It is similar to filter transformation. The only difference is, filter transformation drops the data that do not meet the condition whereas router has an option to capture the data that do not meet the condition. It is useful to test multiple conditions. It has input, output and default groups. For example, if we want to filter data like where State=Michigan, State=California, State=New York and all other States. It’s easy to route data to different tables.

Sequence Generator Transformation
Sequence Generator transformation is a Passive and Connected transformation. It is used to create unique primary key values or cycle through a sequential range of numbers or to replace missing keys.

It has two output ports to connect transformations. By default it has two fields CURRVAL and NEXTVAL(You cannot add ports to this transformation). NEXTVAL port generates a sequence of numbers by connecting it to a transformation or target. CURRVAL is the NEXTVAL value plus one or NEXTVAL plus the Increment By value.

Stored Procedure Transformation
Stored Procedure transformation is a Passive and Connected & UnConnected transformation. It is useful to automate time-consuming tasks and it is also used in error handling, to drop and recreate indexes and to determine the space in database, a specialized calculation etc.

The stored procedure must exist in the database before creating a Stored Procedure transformation, and the stored procedure can exist in a source, target, or any database with a valid connection to the Informatica Server. Stored Procedure is an executable script with SQL statements and control statements, user-defined variables and conditional statements.

Sorter Transformation
Sorter transformation is a Connected and an Active transformation. It allows to sort data either in ascending or descending order according to a specified field. Also used to configure for case-sensitive sorting, and specify whether the output rows should be distinct.

Source Qualifier Transformation
Source Qualifier transformation is an Active and Connected transformation. When adding a relational or a flat file source definition to a mapping, it is must to connect it to a Source Qualifier transformation. The Source Qualifier performs the various tasks such as overriding default SQL query, filtering records; join data from two or more tables etc.

Update Strategy Transformation
Update strategy transformation is an Active and Connected transformation. It is used to update data in target table, either to maintain history of data or recent changes. You can specify how to treat source rows in table, insert, update, delete or data driven.

XML Source Qualifier Transformation
XML Source Qualifier is a Passive and Connected transformation. XML Source Qualifier is used only with an XML source definition. It represents the data elements that the Informatica Server reads when it executes a session with XML sources.

Advanced External Procedure Transformation
Advanced External Procedure transformation is an Active and Connected transformation. It operates in conjunction with procedures, which are created outside of the Designer interface to extend PowerCenter/PowerMart functionality. It is useful in creating external transformation applications, such as sorting and aggregation, which require all input rows to be processed before emitting any output rows.

External Procedure Transformation
External Procedure transformation is an Active and Connected/UnConnected transformations. Sometimes, the standard transformations such as Expression transformation may not provide the functionality that you want. In such cases External procedure is useful to develop complex functions within a dynamic link library (DLL) or UNIX shared library, instead of creating the necessary Expression transformations in a mapping.

Differences between Advanced External Procedure and External Procedure Transformations:
External Procedure returns single value, where as Advanced External Procedure returns multiple values.
External Procedure supports COM and Informatica procedures where as AEP supports only Informatica Procedures.

Note:This is not a complete tutorial on Informatica. To know more about Informatica, contact its official website www.informatica.com.

Database - RDBMS
There are a number of relational databases to store data. A relational database contains normalized data stored in tables. Tables contain records and columns. RDBMS makes it easy to work with individual records. Each row contains a unique instance of data for the categories defined by the columns.

RDBMS are used in OLTP applications(e.g. ATM cards) very frequently and sometimes datawarehouse may also use relational databases. Please refer to Relational data modeling for details to know how data from a source system is normalized and stored in RDBMS databases.

Popular RDBMS Databases

RDBMS Name	Company Name
Oracle	Oracle Corporation
IBM DB2 UDB	IBM Corporation
IBM Informix	IBM Corporation
Microsoft SQL Server	Microsoft
Sybase	Sybase Corporation
Terradata	NCR

Business Intelligence:
Business Intelligence is a technology based on customer and profit oriented models that reduces operating costs and provide increased profitability by improving productivity, sales, service and helps to make decision making capabilities at no time. Business Intelligence Models are based on multi dimensional analysis and key performance indicators (KPI) of an enterprise.

Business Intelligence applications that are based on Business Intelligence Models are created by Business Intelligence software which provides the aggregated details about suppliers, customers, internal activities, business to business transactions to the managers or whoever needs it to take better corporate decisions.

Many business questions or situations need to be analyzed in order to achieve the target of an enterprise with the help of several managers or executives in each cadre. Below are some of the samples of these questions.

Business Intelligence: Finance:
What is the net income, expenses, gross profit, and net profit for this quarter, year?

Business Intelligence: Accounts:
What is the sales amount this month and what is the outstanding pending payment?

Business Intelligence: Purchase:
Who is the vendor to be contacted to purchase products?

Business Intelligence: Production:
How many products are manufactured in each production unit today, weekly, monthly?

Business Intelligence: Sales:
How many products have been sold in each area today, weekly, monthly?

Business Intelligence: Quality:
How many products have been defective today, weekly, monthly, quarterly, yearly?

Business Intelligence: Service:
Are the customers satisfied with the quality?

These business intelligence questions are related with why, what, how, when, and business intelligence reports(olap reports) are useful in providing solutions to the above questions by means of reporting, score cards, balance score cards that are helpful in managerial decisions.

Business Intelligence Tools

Business Intelligence Tools help to gather, store, access and analyze corporate data to aid in decision-making. Generally these systems will illustrate business intelligence in the areas of customer profiling, customer support, market research, market segmentation, product profitability, statistical analysis, inventory and distribution analysis.

With Business Intelligence Tools, various data like customer related, product related, sales related, time related, location related, employee related etc. are gathered and analysed based on which important strategies or rules are formed and goals to achieve their target are set. These decisions are very efficient and effective in promoting an Organisation's growth.

Since the collected data can be sliced across almost all the dimensions like time,location, product, promotion etc., valuable statistics like sales profit in one region for the current year can be calculated and compared with the previous year statistics.

Popular Business Intelligence Tools

Tool Name	Company Name
Business Objects	Business Objects
Cognos	Cognos
Hyperion	Hyperion
Microstrategy	Microstrategy
Microsoft Reporting Services	Microsoft
Crystal	Business Objects

OLAP & its Hybrids

OLAP, an acronym for Online Analytical Processing is an approach that helps organization to take advantages of DATA. Popular OLAP tools are Cognos, Business Objects, Micro Strategy etc. OLAP cubes provide the insight into data and helps the topmost executives of an organization to take decisions in an efficient manner.

Technically, OLAP cube allows one to analyze data across multiple dimensions by providing multidimensional view of aggregated, grouped data. With OLAP reports, the major categories like fiscal periods, sales region, products, employee, promotion related to the product can be ANALYZED very efficiently, effectively and responsively. OLAP applications include sales and customer analysis, budgeting, marketing analysis, production analysis, profitability analysis and forecasting etc.

ROLAP
ROLAP stands for Relational Online Analytical Process that provides multidimensional analysis of data, stored in a Relational database(RDBMS).

MOLAP
MOLAP(Multidimensional OLAP), provides the analysis of data stored in a multi-dimensional data cube.

HOLAP
HOLAP(Hybrid OLAP) a combination of both ROLAP and MOLAP can provide multidimensional analysis simultaneously of data stored in a multidimensional database and in a relational database(RDBMS).

DOLAP
DOLAP(Desktop OLAP or Database OLAP)provide multidimensional analysis locally in the client machine on the data collected from relational or multidimensional database servers.

OLAP Analysis

Imagine an organization that manufactures and sells goods in several States of USA which employs hundreds of employees in its manufacturing, sales and marketing division etc. In order to manufacture and sell this product in profitable manner, the executives need to analyse(OLAP analysis) the data on the product and think about various possibilities and causes for a particular event like loss in sales, less productivity or increase in sales over a particular period of the year.

During the OLAP analysis, the top executives may seek answers for the following:
1. Number of products manufactured.
2. Number of products manufactured in a location.
3. Number of products manufactured on time basis within a location.
4. Number of products manufactured in the current year when compared to the previous year.
5. Sales Dollar value for a particular product.
6. Sales Dollar value for a product in a location.
7. Sales Dollar value for a product in a year within a location.
8. Sales Dollar value for a product in a year within a location sold or serviced by an employee.
OLAP tools help executives in finding out the answers, not only to the above mentioned measures, even for the very complex queries by allowing them to slice and dice, drill down from higher level to lower level summarized data, rank, sort, etc.

Example of OLAP Analysis Report

Time Dimension Id	Location Dimension Id	Product Dimension Id	Organization Dimension Id	Sales Dollar	DateTimeStamp
1	1	100001	1	1000	1/1/2005 11:23:31 AM
3	1	100001	1	750	1/1/2005 11:23:31 AM
1	1	100001	2	1000	1/1/2005 11:23:31 AM
3	1	100001	2	750	1/1/2005 11:23:31 AM

In the above example of OLAP analysis, data can be sliced and diced, drilled up and drilled down for various hierarchies like time dimension, location dimension, product dimension, and organization dimension . This would provide the topmost executives to take a decision about the product performance in a location/time/organization. In OLAP reports, Trend analysis can be also made by comparing the sales value of a particular product over several years or quarters.

OLAP Database – Multidimensional

This is a type of database that is optimized for data warehouse, data mart and online analytical processing (OLAP) applications. The main advantage of this database is query performance.

Relational databases make it easy to work with individual records, whereas multidimensional databases are designed for analyzing large groups of records. Relational database is typically accessed using a Structured Query Language (SQL) query. A multidimensional database allows a user to ask questions like "How many mortgages have been sold in New Jersey city" and "How many credit cards have been purchased in a particular county?".

Popular Multidimensional Databases

Database Name	Company Name
Crystal Holos	Business Objects
Hyperion Essbase	Hyperion
Oracle Express	Oracle Corporation
Oracle OLAP Option	Oracle Corporation
AWMicrosoft Analysis Services	Microsoft
PowerPlay Enterprise	Cognos

SQL SERVER 2008 Questions & Answers

2012-11-05T01:27:00.000-08:00

SQL SERVER 2008 Questions & Answers

What are the basic functions for master, msdb, model, tempdb and resource databases?

The master database holds information for all databases located on the SQL Server instance and is theglue that holds the engine together. Because SQL Server cannot start without a functioning masterdatabase, you must administer this database with care.

The msdb database stores information regarding database backups, SQL Agent information, DTS packages, SQL Server jobs, and some replication information such as for log shipping.

The tempdb holds temporary objects such as global and local temporary tables and stored procedures.

The model is essentially a template database used in the creation of any new user database created in the instance.

The resoure Database is a read‐only database that contains all the system objects that are included with SQL Server. SQL Server system objects, such as sys.objects, are physically persisted in the Resource database, but they logically appear in the sys schema of every database. The Resource database does not contain user data or user metadata.

What is Service Broker?

Service Broker is a message‐queuing technology in SQL Server that allows developers to integrate SQL Server fully into distributed applications. Service Broker is feature which provides facility to SQL Server to send an asynchronous, transactional message. it allows a database to send a message to another database without waiting for the response, so the application will continue to function if the remote database is temporarily unavailable. (Read More Here)

Where SQL server user names and passwords are stored in SQL server?

They get stored in System Catalog Views sys.server_principals and sys.sql_logins.

What is Policy Management?

Policy Management in SQL SERVER 2008 allows you to define and enforce policies for configuring and managing SQL Server across the enterprise. Policy‐Based Management is configured in SQL Server Management Studio (SSMS). Navigate to the Object Explorer and expand the Management node and the Policy Management node; you will see the Policies, Conditions, and Facets nodes.

What is Replication and Database Mirroring?

Database mirroring can be used with replication to provide availability for the publication database. Database mirroring involves two copies of a single database that typically reside on different computers. At any given time, only one copy of the database is currently available to clients which are known as the principal database. Updates made by clients to the principal database are applied on the other copy of the database, known as the mirror database. Mirroring involves applying the transaction log from every insertion, update, or deletion made on the principal database onto the mirror database.

What are Sparse Columns?

A sparse column is another tool used to reduce the amount of physical storage used in a database. They are the ordinary columns that have an optimized storage for null values. Sparse columns reduce the space requirements for null values at the cost of more overhead to retrieve nonnull values. (Read More Here)

What does TOP Operator Do?

The TOP operator is used to specify the number of rows to be returned by a query. The TOP operator has new addition in SQL SERVER 2008 that it accepts variables as well as literal values and can be used with INSERT, UPDATE, and DELETES statements.

What is CTE?

CTE is an abbreviation Common Table Expression. A Common Table Expression (CTE) is an expression that can be thought of as a temporary result set which is defined within the execution of a single SQL statement. A CTE is similar to a derived table in that it is not stored as an object and lasts only for the duration of the query.

What is MERGE Statement?

MERGE is a new feature that provides an efficient way to perform multiple DML operations. In previous versions of SQL Server, we had to write separate statements to INSERT, UPDATE, or DELETE data based on certain conditions, but now, using MERGE statement we can include the logic of such data modifications in one statement that even checks when the data is matched then just update it and when unmatched then insert it. One of the most important advantages of MERGE statement is all the data is read and processed only once.

What is Filtered Index?

Filtered Index is used to index a portion of rows in a table that means it applies filter on INDEX which improves query performance, reduce index maintenance costs, and reduce index storage costs compared with full‐table indexes. When we see an Index created with some where clause then that is actually a FILTERED INDEX.

Which are new data types introduced in SQL SERVER 2008?

The GEOMETRY Type: The GEOMETRY data type is a system .NET common language runtime (CLR) data type in SQL Server. This type represents data in a two‐dimensional Euclidean coordinate system.

The GEOGRAPHY Type: The GEOGRAPHY datatype’s functions are the same as with GEOMETRY. The difference between the two is that when you specify GEOGRAPHY, you are usually specifying points in terms of latitude and longitude.

New Date and Time Datatypes: SQL Server 2008 introduces four new datatypes related to date and time: DATE, TIME, DATETIMEOFFSET, and DATETIME2.

DATE: The new DATE type just stores the date itself. It is based on the Gregorian calendar and handles years from 1 to 9999.

•TIME: The new TIME (n) type stores time with a range of 00:00:00.0000000 through 23:59:59.9999999. The precision is allowed with this type. TIME supports seconds down to 100 nanoseconds. The n in TIME (n) defines this level of fractional second precision, from 0 to 7 digits of precision.

•The DATETIMEOFFSET Type: DATETIMEOFFSET (n) is the time‐zone‐aware version of a datetime datatype. The name will appear less odd when you consider what it really is: a date + a time + a time‐zone offset. The offset is based on how far behind or ahead you are from Coordinated Universal Time (UTC) time.

•The DATETIME2 Type: It is an extension of the datetime type in earlier versions of SQL Server. This new datatype has a date range covering dates from January 1 of year 1 through December 31 of year 9999. This is a definite improvement over the 1753 lower boundary of the datetime datatype. DATETIME2 not only includes the larger date range, but also has a timestamp and the same fractional precision that TIME type provides

What are the Advantages of using CTE?

•Using CTE improves the readability and makes maintenance of complex queries easy.

•The query can be divided into separate, simple, logical building blocks which can be then used to build more complex CTEs until final result set is generated.

•CTE can be defined in functions, stored procedures, triggers or even views.

•After a CTE is defined, it can be used as a Table or a View and can SELECT, INSERT, UPDATE or DELETE Data.

How can we rewrite sub‐queries into simple select statements or with joins?

Yes we can write using Common Table Expression (CTE). A Common Table Expression (CTE) is an expression that can be thought of as a temporary result set which is defined within the execution of a single SQL statement. A CTE is similar to a derived table in that it is not stored as an object and lasts only for the duration of the query.

E.g.

USE AdventureWorks GO WITH EmployeeDepartment_CTE AS ( SELECT EmployeeID,DepartmentID,ShiftID FROM HumanResources.EmployeeDepartmentHistory ) SELECT ecte.EmployeeId,ed.DepartmentID, ed.Name,ecte.ShiftID FROM HumanResources.Department ed INNER JOIN EmployeeDepartment_CTE ecte ON ecte.DepartmentID = ed.DepartmentID GO

What is CLR?

In SQL Server 2008, SQL Server objects such as user‐defined functions can be created using such CLR languages. This CLR language support extends not only to user‐defined functions, but also to stored procedures and triggers. You can develop such CLR add‐ons to SQL Server using Visual Studio 2008.

What are synonyms?

Synonyms give you the ability to provide alternate names for database objects. You can alias object names; for example, using the Employee table as Emp. You can also shorten names. This is especially useful when dealing with three and four part names; for example, shortening server.database.owner.object to object.

What is LINQ?

Language Integrated Query (LINQ) adds the ability to query objects using .NET languages. The LINQ to SQL object/relational mapping (O/RM) framework provides the following basic features:

•Tools to create classes (usually called entities) mapped to database tables

•Compatibility with LINQ’s standard query operations

•The DataContext class, with features such as entity record monitoring, automatic SQL statement generation, record concurrency detection, and much more

What is Isolation Levels?

Transactions specify an isolation level that defines the degree to which one transaction must be isolated from resource or data modifications made by other transactions. Isolation levels are described in terms of which concurrency side‐effects, such as dirty reads or phantom reads, are allowed.

Transaction isolation levels control:

•Whether locks are taken when data is read, and what type of locks are reque• How long th

•Whether a read operation referencing

•Blocks until the exclusive lock on the row is freed.

•Retrieves the committed version of the row that

statement or transaction started. • Reads the uncommitted data mod

EXCEPT clause is similar to MINU

returns all rows in the first query that are not returned in the second query. Each SQL statement within the EXCEPT query and MINUS query must have the same number of fieldsin the result sets with similar data types.

WXPath uses a set

expression that you’ll use is the location path expression, which returns back a set ofcalled a node set. XPath can use both an unabbreviated and an abbreviated syntax. The following is the unabbreviated syntax for a location path:

Using the NOLOCK

improve concurrency on a busy system. When the NOLOCK hint is included in a SELECT statement, no locks are taken when data is read. The result is a Dirty Read, which meansthat another process could be updating the data at the exact time you are reading it. Therare no guarantees that your query will retrieve the most recent data. The advantage to performance is that your reading of data will not block updates from taking place, and updates will not block your reading of data. SELECT statements take Shared (Read) lockThis means that multiple SELECT statements are allowed simultaneous access, but other processes are blocked from modifying the data. The updates will queue until all the readshave completed, and reads requested after the update will wait for the updates to complete. The result to your system is delay (blocking).

SQL Server now supports the use of TRY...CATCH con

handling. TRY...CATCH lets us build error handling at the level we need, in the way wto, by setting a region where if any error occurs, it will break out of the region and head to an error handler. The basic structure is as follows: BEGIN TRY

<code>

END TRY

BEGIN CA

<code>

END CAT

error can be dealt.

RaiseError generates an

RAISERROR can either reference a user‐defined message stored in the sys.messages catalogview or build a message dynamically. The message is returned as a server error message to the calling application or to an associated CATCH block of a TRY…CATCH construct. (Read More Here)

Master database is system databas

configuration. When SQL Server 2005 is installed it usually creates master, model, msdb,tempdb resource and distribution system database by default. Only Master database is thone which is absolutely must have database. Without Master database SQL Server cannot be started. This is the reason it is extremely important to backup Master database.

rebuild the system databases. This procedure is most often used to rebuild the master database for a corrupted installation of SQL Server.

WThe xml data type lets yo

An XML fragment is an XML instance that is missing a single top‐level element. You can create columns and variables of the xml type and store XML instances in them. The xml data type and associated methods help integrate XML into the relational framework of SServer.

What is Data Compression?

pression comes in two flavors:

Row Compression

Row compression changes the format of physical storage of data. It minimize the metadata

In SQL SERVE 2008 Data Com

•Page Compression

(column information, length, offsets etc) associated with each record. Numeric data types and fixed length strings are stored in variable‐length storage format, just like Varchar.

uses the following techniques to compress data:

•Prefix Compression.

These prefixes are saved in compression information headers (CI) which resides after page header. A reference number is assigned to these prefixes and that reference number is replaced where ever those prefixes are being used.

in CI. The main difference between prefix and dictionary compression is that prefix is only restricted to one column while dictionary is applicable to the complete page.

The Transact‐SQL programming la

Console Commands for SQL Server. DBCC commands are used to perform following tasks.

•Tasks that gather and display various types of inform

•Validation operations on a database, table, index, catalog, fi

allocation of database pages. • Miscellaneous tasks such as en memory.

Run following query in Query Editor.

USE <database_name>;

GO SELE ,name AS table_name FROM sys.tables WHERE OBJECTPROORDER BY schema_name, table_name; GO

There are multiple ways to do this.

“Detach Database” from one ser

Manually script all the objects using SSMS and run the script on new server.

Use Wizard of SSMS. (Read More Here)

There are multiple ways to do this.

1This method is used when

be inserted into this table from another table. If columns listed in insert clause and select clause are same, they are not required to list them. ELECT INTO

This method

data from one table is to be inserted into newly created table from another table. New table is created with same data types as selected columns.

Catalog views return info

Views are the most general interface to the catalog metadata and provide the most efficient way to obtain, transform, and present customized forms of this information. All user‐available catalog metadata is exposed through catalog views.

A Pivot Table can automatica

spreadsheet and create a second table displaying the summarized data. The PIVOT operator turns the values of a specified column into column names, effectively rotating a table.

Filestream allows you integrated within the database. It enables SQL Server based applications to store unstructured data such as documents, images, audios, videos etc. in the file system. FILESTREAMbasically integrates the SQL Srver Database Engine with New Technology File System (NTFS); it basically stores the data in varbinary (max) data type. Using this data type, the unstructured data is stored in the NTFS file system and the SQL Server Database Engine manages the link between the Filestream column and the actual file located in the NTFS. Using Transact SQL statements users can insert, update, delete and select the data stored in FILESTREAM enabled tables.

A dirty read occurs w

incorrect or unedited data. Suppose, A has changed a row, but has not committed the changes. B reads the uncommitted data but his view of the data may be wrong so that is Dirty Read.

sqlcmd is enhanced

other two options. In other words sqlcmd is better replacement of isql (which will be deprecated eventually) and osql (not included in SQL Server 2005 RTM). sqlcmd can work two modes ‐ i) BATCH and ii) interactive modes.

Aggregate functions perform a

Aggregate functions ignore NULL values except COUNT function. HAVING clause is used, along with GROUP BY, for filtering query using aggregate values. Following functions are aggregate functions.

AVG, M CHECKSUM_AGG, SUM, COUNT, STDEV, COU

VAR, MAX, VARP

TABLESAMPLE allows you to extract a sThe rows retrieved are random and they are not in any order. This sampling can be basedon a percentage of number of rows. You can use TABLESAMPLE when only a sampling of rows is necessary for the application instead of a full result set. (Read More Here)

ROW_NUMBER() returns athe result set. This is only a number used in the context of the result set, if the result changes, the ROW_NUMBER() will change.

Ranking functions return a rank

functions are non‐deterministic. Different Ranking functions are:

Returns the sequential number of a row within a partition of a result setthe first row in each partition.

Returns the rank of each row within the partition of a result se

Returns the rank of rows within the partition of a result set, without a(Read More Here )

The UNIJOIN command. However, when using the UNION command all selected columns need to be of the same data type. With UNION, only distinct values are selected.

The UNION ALLall values.

rows, instead it just pulls all rows from all tables fitting your query specifics and combines them into a table.has following types of index pages or nodes:

only one.
• branch no nodes which can be two or more.

leaf nodes A leaf node contains in nodes which can be many.

General Questions of SQL SERVER

What is RDBMS?
Relational Data Base Management Systems (RDBMS) are database management systems that maintain data records and indices in tables. Relationships may be created and maintained across and among the data and tables. In a relational database, relationships between data items are expressed by means of tables. Interdependencies among these tables are expressed by data values rather than by pointers. This allows a high degree of data independence. An RDBMS has the capability to recombine the data items from different files, providing powerful tools for data usage.
What are the properties of the Relational tables?
Relational tables have six properties:
•Values are atomic.
•Column values are of the same kind.
•Each row is unique.
•The sequence of columns is insignificant.
•The sequence of rows is insignificant.
•Each column must have a unique name.
What is Normalization?
Database normalization is a data design and organization process applied to data structures based on rules that help building relational databases. In relational database design, the process of organizing data to minimize redundancy is called normalization. Normalization usually involves dividing a database into two or more tables and defining relationships between the tables. The objective is to isolate data so that additions, deletions, and modifications of a field can be made in just one table and then propagated through the rest of the database via the defined relationships.
What is De‐normalization?
De‐normalization is the process of attempting to optimize the performance of a database by adding redundant data. It is sometimes necessary because current DBMSs implement the relational model poorly. A true relational DBMS would allow for a fully normalized database at the logical level, while providing physical storage of data that is tuned for high performance. De‐normalization is a technique to move from higher to lower normal forms of database modeling in order to speed up database access.
What are different normalization forms?
1NF: Eliminate Repeating Groups
Make a separate table for each set of related attributes, and give each table a primary key. Each field contains at most one value from its attribute domain.
2NF: Eliminate Redundant Data
If an attribute depends on only part of a multi‐valued key, remove it to a separate table.
3NF: Eliminate Columns Not Dependent On Key
If attributes do not contribute to a description of the key, remove them to a separate table. All attributes must be directly dependent on the primary key. (Read More Here)
BCNF: Boyce‐Codd Normal Form
If there are non‐trivial dependencies between candidate key attributes, separate them out into distinct tables.
4NF: Isolate Independent Multiple Relationships
No table may contain two or more 1:n or n:m relationships that are not directly related.
5NF: Isolate Semantically Related Multiple Relationships
There may be practical constrains on information that justify separating logically related many‐to‐many relationships.
ONF: Optimal Normal Form
A model limited to only simple (elemental) facts, as expressed in Object Role Model notation.
DKNF: Domain‐Key Normal Form
A model free from all modification anomalies is said to be in DKNF.
Remember, these normalization guidelines are cumulative. For a database to be in 3NF, it must first fulfill all the criteria of a 2NF and 1NF database.
What is Stored Procedure?
A stored procedure is a named group of SQL statements that have been previously created and stored in the server database. Stored procedures accept input parameters so that a single procedure can be used over the network by several clients using different input data. And when the procedure is modified, all clients automatically get the new version. Stored procedures reduce network traffic and improve performance. Stored procedures can be used to help ensure the integrity of the database.
e.g. sp_helpdb, sp_renamedb, sp_depends etc.
What is Trigger?
A trigger is a SQL procedure that initiates an action when an event (INSERT, DELETE or UPDATE) occurs. Triggers are stored in and managed by the DBMS. Triggers are used to maintain the referential integrity of data by changing the data in a systematic fashion. A trigger cannot be called or executed; DBMS automatically fires the trigger as a result of a data modification to the associated table. Triggers can be viewed as similar to stored procedures in that both consist of procedural logic that is stored at the database level. Stored procedures, however, are not event‐drive and are not attached to a specific table as triggers are. Stored procedures are explicitly executed by invoking a CALL to the procedure while triggers are implicitly executed. In addition, triggers can also execute stored procedures.
Nested Trigger: A trigger can also contain INSERT, UPDATE and DELETE logic within itself, so when the trigger is fired because of data modification it can also cause another data modification, thereby firing another trigger. A trigger that contains data modification logic within itself is called a nested trigger.
What is View?
A simple view can be thought of as a subset of a table. It can be used for retrieving data, as well as updating or deleting rows. Rows updated or deleted in the view are updated or deleted in the table the view was created with. It should also be noted that as data in the original table changes, so does data in the view, as views are the way to look at part of the original table. The results of using a view are not permanently stored in the database. The data accessed through a view is actually constructed using standard T‐SQL select command and can come from one to many different base tables or even other views.
What is Index?
An index is a physical structure containing pointers to the data. Indices are created in an existing table to locate rows more quickly and efficiently. It is possible to create an index on one or more columns of a table, and each index is given a name. The users cannot see the indexes; they are just used to speed up queries. Effective indexes are one of the best ways to improve performance in a database application. A table scan happens when there is no index available to help a query. In a table scan SQL Server examines every row in the table to satisfy the query results. Table scans are sometimes unavoidable, but on large tables, scans have a terrific impact on performance.
What is a Linked Server?
Linked Servers is a concept in SQL Server by which we can add other SQL Server to a Group and query both the SQL Server dbs using T‐SQL Statements. With a linked server, you can create very clean, easy to follow, SQL statements that allow remote data to be retrieved, joined and combined with local data. Stored Procedure sp_addlinkedserver, sp_addlinkedsrvlogin will be used add new Linked Server.
What is Cursor?
Cursor is a database object used by applications to manipulate data in a set on a row‐by‐row basis, instead of the typical SQL commands that operate on all the rows in the set at one time.
In order to work with a cursor we need to perform some steps in the following order:
•Declare cursor
•Open cursor
•Fetch row from the cursor
•Process fetched row
•Close cursor
• Deallocate cursor (Read More Here)
What is Collation?
Collation refers to a set of rules that determine how data is sorted and compared. Character data is sorted using rules that define the correct character sequence, with options for specifying case sensitivity, accent marks, kana character types and character width. (Read More Here)
What is Difference between Function and Stored Procedure?
UDF can be used in the SQL statements anywhere in the WHERE/HAVING/SELECT section where as Stored procedures cannot be. UDFs that return tables can be treated as another rowset. This can be used in JOINs with other tables. Inline UDF's can be thought of as views that take parameters and can be used in JOINs and other Rowset operations.
What is sub‐query? Explain properties of sub‐query?
Sub‐queries are often referred to as sub‐selects, as they allow a SELECT statement to be executed arbitrarily within the body of another SQL statement. A sub‐query is executed by enclosing it in a set of parentheses. Sub‐queries are generally used to return a single row as an atomic value, though they may be used to compare values against multiple rows with the IN keyword.
A subquery is a SELECT statement that is nested within another T‐SQL statement. A subquery SELECT statement if executed independently of the T‐SQL statement, in which it is nested, will return a resultset. Meaning a subquery SELECT statement can standalone and is not depended on the statement in which it is nested. A subquery SELECT statement can return any number of values, and can be found in, the column list of a SELECT statement, a FROM, GROUP BY, HAVING, and/or ORDER BY clauses of a T‐SQL statement. A Subquery can also be used as a parameter to a function call. Basically a subquery can be used anywhere an expression can be used.
What are different Types of Join?
Cross Join
A cross join that does not have a WHERE clause produces the Cartesian product of the tables involved in the join. The size of a Cartesian product result set is the number of rows in the first table multiplied by the number of rows in the second table. The common example is when company wants to combine each product with a pricing table to analyze each product at each price.
Inner Join
A join that displays only the rows that have a match in both joined tables is known as inner Join. This is the default type of join in the Query and View Designer.
Outer Join
A join that includes rows even if they do not have related rows in the joined table is an Outer Join. You can create three different outer join to specify the unmatched rows to be included:
•Left Outer Join: In Left Outer Join all rows in the first‐named table i.e. "left" table, which appears leftmost in the JOIN clause are included. Unmatched rows in the right table do not appear.
•Right Outer Join: In Right Outer Join all rows in the second‐named table i.e. "right" table, which appears rightmost in the JOIN clause are included. Unmatched rows in the left table are not included.
•Full Outer Join: In Full Outer Join all rows in all joined tables are included, whether they are matched or not.
Self Join
This is a particular case when one table joins to itself, with one or two aliases to avoid confusion. A self join can be of any type, as long as the joined tables are the same. A self join is rather unique in that it involves a relationship with only one table. The common example is when company has a hierarchal reporting structure whereby one member of staff reports to another. Self Join can be Outer Join or Inner Join.
What are primary keys and foreign keys?
Primary keys are the unique identifiers for each row. They must contain unique values and cannot be null. Due to their importance in relational databases, Primary keys are the most fundamental of all keys and constraints. A table can have only one Primary key.
Foreign keys are both a method of ensuring data integrity and a manifestation of the relationship between tables.
What is User Defined Functions? What kind of User‐Defined Functions can be created?
User‐Defined Functions allow defining its own T‐SQL functions that can accept 0 or more parameters and return a single scalar data value or a table data type.
Different Kinds of User‐Defined Functions created are:
Scalar User‐Defined Function
A Scalar user‐defined function returns one of the scalar data types. Text, ntext, image and timestamp data types are not supported. These are the type of user‐defined functions that most developers are used to in other programming languages. You pass in 0 to many parameters and you get a return value.
Inline Table‐Value User‐Defined Function
An Inline Table‐Value user‐defined function returns a table data type and is an exceptional alternative to a view as the user‐defined function can pass parameters into a T‐SQL select command and in essence provide us with a parameterized, non‐updateable view of the underlying tables.
Multi‐statement Table‐Value User‐Defined Function
A Multi‐Statement Table‐Value user‐defined function returns a table and is also an exceptional alternative to a view as the function can support multiple T‐SQL statements to build the final result where the view is limited to a single SELECT statement. Also, the ability to pass parameters into a TSQL select command or a group of them gives us the capability to in essence create a parameterized, non‐updateable view of the data in the underlying tables. Within the create function command you must define the table structure that is being returned. After creating this type of user‐defined function, It can be used in the FROM clause of a T‐SQL command unlike the behavior found when using a stored procedure which can also return record sets. (Read Here For Example)
What is Identity?
Identity (or AutoNumber) is a column that automatically generates numeric values. A start and increment value can be set, but most DBA leave these at 1. A GUID column also generates numbers; the value of this cannot be controlled. Identity/GUID columns do not need to be indexed.
What is DataWarehousing?
•Subject‐oriented, meaning that the data in the database is organized so that all the data elements relating to the same real‐world event or object are linked together;
•Time‐variant, meaning that the changes to the data in the database are tracked and recorded so that reports can be produced showing changes over time;
•Non‐volatile, meaning that data in the database is never over‐written or deleted, once committed, the data is static, read‐only, but retained for future reporting.
•Integrated, meaning that the database contains data from most or all of an organization's operational applications, and that this data is made consistent.

Which TCP/IP port does SQL Server run on? How can it be changed?
SQL Server runs on port 1433. It can be changed from the Network Utility TCP/IP properties –> Port number, both on client and the server.
What are the difference between clustered and a non‐clustered index? (Read More Here)
A clustered index is a special type of index that reorders the way records in the table are physically stored. Therefore table can have only one clustered index. The leaf nodes of a clustered index contain the data pages.
A non clustered index is a special type of index in which the logical order of the index does not match the physical stored order of the rows on disk. The leaf node of a non clustered index does not consist of the data pages. Instead, the leaf nodes contain index rows.
What are the different index configurations a table can have?
A table can have one of the following index configurations:

No indexe

A clustered index
A clustered index and many nonclustered indexes
A nonclustered index
Many nonclustered indexes
What are different types of Collation Sensitivity?
Case sensitivity ‐ A and a, B and b, etc.
Accent sensitivity ‐ a and á, o and ó, etc.
Kana Sensitivity ‐ When Japanese kana characters Hiragana and Katakana are treated differently, it is called Kana sensitive.
Width sensitivity ‐ A single‐byte character (half‐width) and the same character represented as a double‐byte character (full‐width) are treated differently than it is width sensitive.
What is OLTP (Online Transaction Processing)?
In OLTP ‐ online transaction processing systems relational database design use the discipline of data modeling and generally follow the Codd rules of data normalization in order to ensure absolute data integrity. Using these rules complex information is broken down into its most simple structures (a table) where all of the individual atomic level elements relate to each other and satisfy the normalization rules.
What's the difference between a primary key and a unique key?
Both primary key and unique key enforces uniqueness of the column on which they are defined. But by default primary key creates a clustered index on the column, where are unique creates a nonclustered index by default. Another major difference is that, primary key doesn't allow NULLs, but unique key allows one NULL only.
What is difference between DELETE & TRUNCATE commands?
Delete command removes the rows from a table based on the condition that we provide with a WHERE clause. Truncate will actually remove all the rows from a table and there will be no data in the table after we run the truncate command.
TRUNCATE
•TRUNCATE is faster and uses fewer system and transaction log resources than DELETE.
•TRUNCATE removes the data by deallocating the data pages used to store the table’s data, and only the page deallocations are recorded in the transaction log.
•TRUNCATE removes all rows from a table, but the table structure, its columns, constraints, indexes and so on, remains. The counter used by an identity for new rows is reset to the seed for the column.
•You cannot use TRUNCATE TABLE on a table referenced by a FOREIGN KEY constraint. Because TRUNCATE TABLE is not logged, it cannot activate a trigger.
•TRUNCATE cannot be rolled back.
•TRUNCATE is DDL Command.
•TRUNCATE Resets identity of the table
DELETE
•DELETE removes rows one at a time and records an entry in the transaction log for each deleted row.
•If you want to retain the identity counter, use DELETE instead. If you want to remove table definition and its data, use the DROP TABLE statement.
•DELETE Can be used with or without a WHERE clause
•DELETE Activates Triggers.
•DELETE can be rolled back.
•DELETE is DML Command.
•DELETE does not reset identity of the table.
When is the use of UPDATE_STATISTICS command?
This command is basically used when a large processing of data has occurred. If a large amount of deletions any modification or Bulk Copy into the tables has occurred, it has to update the indexes to take these changes into account. UPDATE_STATISTICS updates the indexes on these tables accordingly.
What is the difference between a HAVING CLAUSE and a WHERE CLAUSE?
They specify a search condition for a group or an aggregate. But the difference is that HAVING can be used only with the SELECT statement. HAVING is typically used in a GROUP BY clause. When GROUP BY is not used, HAVING behaves like a WHERE clause. Having Clause is basically used only with the GROUP BY function in a query whereas WHERE Clause is applied to each row before they are part of the GROUP BY function in a query.
What are the properties and different Types of Sub‐Queries?
Properties of Sub‐Query
•A sub‐query must be enclosed in the parenthesis.
•A sub‐query must be put in the right hand of the comparison operator, and
•A sub‐query cannot contain an ORDER‐BY clause.
•A query can contain more than one sub‐query.
Types of Sub‐query
•Single‐row sub‐query, where the sub‐query returns only one row.
•Multiple‐row sub‐query, where the sub‐query returns multiple rows,. and
•Multiple column sub‐query, where the sub‐query returns multiple columns
What is SQL Profiler?
SQL Profiler is a graphical tool that allows system administrators to monitor events in an instance of Microsoft SQL Server. You can capture and save data about each event to a file or SQL Server table to analyze later. For example, you can monitor a production environment to see which stored procedures are hampering performances by executing too slowly.
Use SQL Profiler to monitor only the events in which you are interested. If traces are becoming too large, you can filter them based on the information you want, so that only a subset of the event data is collected. Monitoring too many events adds overhead to the server and the monitoring process and can cause the trace file or trace table to grow very large, especially when the monitoring process takes place over a long period of time.
What are the authentication modes in SQL Server? How can it be changed?
Windows mode and Mixed Mode ‐ SQL & Windows.
To change authentication mode in SQL Server click Start, Programs, Microsoft SQL Server and click SQL Enterprise Manager to run SQL Enterprise Manager from the Microsoft SQL Server program group. Select the server then from the Tools menu select SQL Server Configuration Properties, and choose the Security page.
Which command using Query Analyzer will give you the version of SQL server and operating system?
SELECT SERVER PROPERTY ('product version'), SERVER PROPERTY ('product level'), SERVER PROPERTY ('edition').
What is SQL Server Agent?
SQL Server agent plays an important role in the day‐to‐day tasks of a database administrator (DBA). It is often overlooked as one of the main tools for SQL Server management. Its purpose is to ease the implementation of tasks for the DBA, with its full‐function scheduling engine, which allows you to schedule your own jobs and scripts. (Read More Here)
Can a stored procedure call itself or recursive stored procedure? How much level SP nesting is possible?
Yes. Because Transact‐SQL supports recursion, you can write stored procedures that call themselves. Recursion can be defined as a method of problem solving wherein the solution is arrived at by repetitively applying it to subsets of the problem. A common application of recursive logic is to perform numeric computations that lend themselves to repetitive evaluation by the same processing steps. Stored procedures are nested when one stored procedure calls another or executes managed code by referencing a CLR routine, type, or aggregate. You can nest stored procedures and managed code references up to 32 levels.
What is Log Shipping?
Log shipping is the process of automating the backup of database and transaction log files on a production SQL server, and then restoring them onto a standby server. Enterprise Editions only supports log shipping. In log shipping the transactional log file from one server is automatically updated into the backup database on the other server. If one server fails, the other server will have the same db and can be used this as the Disaster Recovery plan. The key feature of log shipping is that it will automatically backup transaction logs throughout the day and automatically restore them on the standby server at defined interval.
Name 3 ways to get an accurate count of the number of records in a table?
SELECT * FROM table1
SELECT COUNT(*) FROM table1
SELECT rows FROM sysindexes WHERE id = OBJECT_ID(table1) AND indid < 2
What does it mean to have QUOTED_IDENTIFIER ON? What are the implications of having it OFF?
When SET QUOTED_IDENTIFIER is ON, identifiers can be delimited by double quotation marks, and literals must be delimited by single quotation marks. When SET QUOTED_IDENTIFIER is OFF, identifiers cannot be quoted and must follow all Transact‐SQL rules for identifiers.
What is the difference between a Local and a Global temporary table?
A local temporary table exists only for the duration of a connection or, if defined inside a compound statement, for the duration of the compound statement.
A global temporary table remains in the database permanently, but the rows exist only within a given connection. When connection is closed, the data in the global temporary table disappears. However, the table definition remains with the database for access when database is opened next time.
What is the STUFF function and how does it differ from the REPLACE function?
STUFF function is used to overwrite existing characters. Using this syntax, STUFF (string_expression, start, length, replacement_characters), string_expression is the string that will have characters substituted, start is the starting position, length is the number of characters in the string that are substituted, and replacement_characters are the new characters interjected into the string. REPLACE function to replace existing characters of all occurrences. Using the syntax REPLACE (string_expression, search_string, replacement_string), where every incidence of search_string found in the string_expression will be replaced with replacement_string.
What is PRIMARY KEY?
A PRIMARY KEY constraint is a unique identifier for a row within a database table. Every table should have a primary key constraint to uniquely identify each row and only one primary key constraint can be created for each table. The primary key constraints are used to enforce entity integrity.
What is UNIQUE KEY constraint?
A UNIQUE constraint enforces the uniqueness of the values in a set of columns, so no duplicate values are entered. The unique key constraints are used to enforce entity integrity as the primary key constraints.
What is FOREIGN KEY?
A FOREIGN KEY constraint prevents any actions that would destroy links between tables with the corresponding data values. A foreign key in one table points to a primary key in another table. Foreign keys prevent actions that would leave rows with foreign key values when there are no primary keys with that value. The foreign key constraints are used to enforce referential integrity.
What is CHECK Constraint?
A CHECK constraint is used to limit the values that can be placed in a column. The check constraints are used to enforce domain integrity. (Read More Here)
What is NOT NULL Constraint?
A NOT NULL constraint enforces that the column will not accept null values. The not null constraints are used to enforce domain integrity, as the check constraints.

How to get @@ERROR and @@ROWCOUNT at the same time?
If @@Rowcount is checked after Error checking statement then it will have 0 as the value of @@Recordcount as it would have been reset. And if @@Recordcount is checked before the error‐checking statement then @@Error would get reset. To get @@error and @@rowcount at the same time do both in same statement and store them in local variable. SELECT @RC = @@ROWCOUNT, @ER = @@ERROR
What is a Scheduled Jobs or What is a Scheduled Tasks?
Scheduled tasks let user automate processes that run on regular or predictable cycles. User can schedule administrative tasks, such as cube processing, to run during times of slow business activity. User can also determine the order in which tasks run by creating job steps within a SQL Server Agent job. E.g. back up database, Update Stats of Tables. Job steps give user control over flow of execution. If one job fails, user can configure SQL Server Agent to continue to run the remaining tasks or to stop execution.
What are the advantages of using Stored Procedures?
•Stored procedure can reduced network traffic and latency, boosting application performance.
•Stored procedure execution plans can be reused, staying cached in SQL Server's memory, reducing server overhead.
•Stored procedures help promote code reuse.
•Stored procedures can encapsulate logic. You can change stored procedure code without affecting clients.
•Stored procedures provide better security to your data.
What is a table called, if it has neither Cluster nor Non‐cluster Index? What is it used for?
Unindexed table or Heap. Microsoft Press Books and Book on Line (BOL) refers it as Heap. A heap is a table that does not have a clustered index and, therefore, the pages are not linked by pointers. The IAM pages are the only structures that link the pages in a table together. Unindexed tables are good for fast storing of data. Many times it is better to drop all indexes from table and then do bulk of inserts and to restore those indexes after that.
Can SQL Servers linked to other servers like Oracle?
SQL Server can be linked to any server provided it has OLE‐DB provider from Microsoft to allow a link. E.g. Oracle has an OLE‐DB provider for oracle that Microsoft provides to add it as linked server to SQL Server group
What is BCP? When does it used?
BulkCopy is a tool used to copy huge amount of data from tables and views. BCP does not copy the structures same as source to destination. BULK INSERT command helps to import a data file into a database table or view in a user‐specified format.
What command do we use to rename a db, a table and a column?
To rename db
sp_renamedb ‘oldname’ , ‘newname’
If someone is using db it will not accept sp_renmaedb. In that case first bring db to single user using sp_dboptions. Use sp_renamedb to rename database. Use sp_dboptions to bring database to multi user mode.
E.g.
USE master;
GO
EXEC sp_dboption AdventureWorks, 'Single User', True
GO
EXEC sp_renamedb 'AdventureWorks', 'AdventureWorks_New'
GO
EXEC sp_dboption AdventureWorks, 'Single User', False
GO
To rename Table
We can change the table name using sp_rename as follows,
sp_rename ‘oldTableName’ ‘newTableName’
E.g.
SP_RENAME ‘Table_First’, ‘Table_Last’ GO
To rename Column
The script for renaming any column :
sp_rename ‘TableName.[OldcolumnName]’, ‘NewColumnName’, ‘Column’
E.g.
sp_RENAME ‘Table_First.Name’, ‘NameChange’ , ‘COLUMN’ GO
15 © Copyright 2000-2008 Pinal Dave. All Rights Reserved. SQLAuthority.com
What are sp_configure commands and set commands?
Use sp_configure to display or change server‐level settings. To change database‐level settings, use ALTER DATABASE. To change settings that affect only the current user session, use the SET statement.
E.g.
sp_CONFIGURE ’show advanced’, 0 GO RECONFIGURE GO sp_CONFIGURE GO
You can run following command and check advance global configuration settings.
sp_CONFIGURE ’show advanced’, 1 GO RECONFIGURE GO sp_CONFIGURE GO
How to implement one‐to‐one, one‐to‐many and many‐to‐many relationships while designing tables?
One‐to‐One relationship can be implemented as a single table and rarely as two tables with primary and foreign key relationships. One‐to‐Many relationships are implemented by splitting the data into two tables with primary key and foreign key relationships.
Many‐to‐Many relationships are implemented using a junction table with the keys from both the tables forming the composite primary key of the junction table.
What is an execution plan? When would you use it? How would you view the execution plan?
An execution plan is basically a road map that graphically or textually shows the data retrieval methods chosen by the SQL Server query optimizer for a stored procedure or ad‐hoc query and is a very useful tool for a developer to understand the performance characteristics of a query or stored procedure since the plan is the one that SQL Server will place in its cache and use to execute the stored procedure or query. From within Query Analyzer is an option called "Show Execution Plan" (located on the Query drop‐down menu). If this option is turned on it will display query execution plan in separate window when query is ran again.

INNER JOIN

2012-11-04T22:33:00.002-08:00

select * from Employee

select * from Department1

Select Employee.empid,Employee.empnumber,Employee.empfirstname,Employee.emplastname,
Employee.empemail,Employee.dateofjoin,Department1.deptname,Department1.Departmentid from Employee
INNER JOIN department1 on employee.departmentid=department1.departmentid

Select Employee.empid,Employee.empnumber,Employee.empfirstname,Employee.emplastname,
Employee.empemail,Employee.dateofjoin,Department1.deptname,Department1.Departmentid from Employee
INNER JOIN department1 on employee.departmentid=department1.departmentid Where Department1.Departmentid=10

How to Remove Duplicate Rows but Log the Duplicates in SSIS

2012-11-03T00:50:00.001-07:00

Create table D
( A int,B int)

Insert D values (1,23)
Select * from D

A    B
1    23
1    23
16    59
12    12
13    45
12    12
45    56

SQL Server Scnerio

2012-11-02T04:54:00.001-07:00

Create Table A1
( Row1 int,
Row2 int,
Row3 int,
Row4 int)

Insert into A1 values (0,0,0,18)
Select * from A1

Row1	Row2	Row3	Row4
0	6	0	0
0	0	30	0
4	0	0	0
0	0	0	18

Select Row1=MAX(Row1),Row2=MAX(row2),Row3=MAX(Row3),Row4=MAX(Row4) From A1

Row1	Row2	Row3	Row4
4	6	30	18

Lookup,Incremental load(Latest Records),removedup

2012-10-31T23:13:00.001-07:00

How TO EXECUTE dowloded pacakage

2012-10-31T02:11:00.000-07:00

Write data from multiple tables to single flat file.

2012-10-31T00:06:00.000-07:00

Take 3 DataFlow Tasks in Control Flow

and One Flat File Destination
(click on the image for larger view)

Handling extra/missing columns in Delimited Flat file

2012-10-30T22:39:00.003-07:00

Delimited Flat files (CSV) are one of the important non-relational source for SSIS ETL but if we get robust file all the time then we can use inbuild flat file source in SSIS. Surprisingly we are not that much lucky and get the data in which columns are missing (or we can say extra columns) like as follow...

Flat File INPUT

we Want Flatfile OUTPUT

Flat file source which reads all data into one column and then splits them based on index of the comma in a derived column.

Here I'm giving you the demonstration to handle such file with the help of derived column.We have to write expression (sometime complex) to get the proper data in column format. In the above example I am getting all the data as String and after splitting we are saving it as string only (If you want to change the data type then you can use Data Conversion task). If number of columns are more then this approach is not recommended, In that case you should go with the first 2 approach.

Step1: Inside the Data Flow Task (DFT), Select a Flat File Source and Derived column.

Step2: Configure the FlatFile Source in such a way that you will get all input data in single column.

Note: We are keeping the Column name as "C" in the example.

Step3: Double click the derived column and use the following logic.

For separating the column "C1" from the column "C", we need to fetch the data from position 1 to first occurrence of delimiter " , " so finding the first delimiter we have to use FINDSTRING SSIS function (Returns the location of the specified occurrence of a string within a character expression).

SUBSTRING(C,1,FINDSTRING(C,",",1) - 1)

Here we have to do "-1" because we have to exclude delimiter from the column value.

For separating the column "C2" from the column "C", we have to fetch the data after the first delimiter to 2nd delimiter but before that we have to check whether we have 2nd delimiter in place

FINDSTRING(C,",",2) != 0 ? SUBSTRING(C,FINDSTRING(C,",",1) + 1,FINDSTRING(C,",",2) - FINDSTRING(C,",",1) - 1) : SUBSTRING(C,FINDSTRING(C,",",1) + 1,LEN(C) - FINDSTRING(C,",",2))

For separating the column "C3" from the column "C"
FINDSTRING(C,",",2) == 0 ? "NULL" : (FINDSTRING(C,",",3) != 0 ? SUBSTRING(C,FINDSTRING(C,",",2) + 1,FINDSTRING(C,",",3) - FINDSTRING(C,",",2) - 1) : "NULL")

For separating the Last column "C4" from the column "C",we have to fetch the data after the third delimiter to end of the row. and for calculating the last position of the row, we have to use LEN SSIS function
FINDSTRING(C,",",3) == 0 ? "NULL" : SUBSTRING(C,FINDSTRING(C,",",3) + 1,LEN(C) - FINDSTRING(C,",",2))

Step4: And result would be as expected..

SCD in SSIS

2012-10-30T05:42:00.000-07:00

Create table Scdemp
(EmpBusinessId int,EmpName Varchar(20),age int,MaritalStatus varchar(10),ManagerId int)
go
Insert Into scdemp values (1010,'M',45,'M',1008)

Create table DestScdemp
(EmpBusinessId int,EmpName Varchar(20),age int,MaritalStatus varchar(10),ManagerId int,EmpSurrogateId Int,StartDate Date,Enddate DATE )

Insert multiple rows into a table

2012-10-30T05:26:00.004-07:00

CREATE TABLE Sample1(UserId INT IDENTITY,UserName VARCHAR(8),Sales Decimal(6,2))
GO
INSERT INTO Sample1 (UserName, Sales)
SELECT 'Ram', 100.00
UNION ALL
SELECT 'Raheem ', 700.00
UNION ALL
SELECT 'Robert', 400.00
UNION ALL
SELECT 'Sachin', 800
UNION ALL
SELECT 'Pawan', 1000.00
UNION ALL
SELECT 'Mahesh',50.00

Select * from sample1

Merge Join

2012-10-29T02:48:00.005-07:00

OLEDB SOURCE1:

Create Table OrderMaster
(
      OrderID Varchar(5),
      ClientCode Varchar(10),
      ProjectName varchar(25),
      OrderDate DateTime
)
INSERT INTO OrderMaster
SELECT 'A1001','CHEM02','NY WaterField',GETDATE() UNION ALL
SELECT 'A1002','ACCU01','Plainfield Soil',GETDATE()
GO

Flat File Source:

OrderID,SampleNumber,Matrix
A1001,A1001-01,Water
A1001,A1001-02,Soil
A1002,A1002-01,Water
A1002,A1002-02,Water

Oledb Destination:

Create Table OrderMasterAndDetail
(
OrderID Varchar(5),
ClientCode Varchar(10),
SampleNumber Varchar(10),
Matrix Varchar(10)
)
GO

Alter & Update Table with Null

2012-10-28T23:50:00.002-07:00

select * from Prod

update Prod set Quantity=NULL where rowguid='D4544D7D-CAF5-46B3-AB22-5718DCC26B5E'

Alter table Prod alter column quantity smallint null

DELETE DUP REC"S

2012-10-25T22:31:00.004-07:00

DELETE DUP REC"S

create table A (I int, J int, K int)

insert A select 1,1,1
insert A select 1,1,1
insert A select 1,1,1
insert A select 1,1,1
insert A select 2,1,1
insert A select 2,1,1
insert A select 2,2,1
insert A select 2,2,2
insert A select 2,2,3
insert A select 2,2,4
insert A select 3,3,3
insert A select 3,3,3
insert A select 3,3,3

select * from A

1. simple table recreate

select * into B from A where 1 = 0
insert B select distinct * from A
begin tran
delete A
insert A select * from B
commit tran
drop table B

2. delete and replace duplicates

select * into B from A where 1 = 0
insert B select i,j,k from A group by i,j,k having count(*) > 1
begin tran
delete A from B where A.i = B.i and A.j = B.j and A.k = B.k
insert A select * from B
commit tran
drop table B

3. delete duplicates one by one leaving single row
set rowcount 1
select 1
while @@rowcount > 0
delete A
where 1 < (select count(*) from A a2 whereA.i = a2.i and A.j = a2.j and A.k = a2.k)
set rowcount 0

4. delete all duplicates for one row value at a time
select *, cnt = 0 into B from A where 1 = 0
declare @rowcount int
select 1
while @@rowcount <> 0
begin
insert B (i,j,k,cnt) select top 1 i,j,k, count(*) - 1 from A group by i,j,k having count(*) > 1
select @rowcount = cnt from B
set rowcount @rowcount
delete

deleting duplicates.

2012-10-25T22:14:00.000-07:00

CREATE TABLE DUPTest (A int not null,
B int not null,
C int not null,
ID int not null identity) on [Primary]
GO
INSERT INTO DUPTest (A,B,C) VALUES (1,1,1)
INSERT INTO DUPTest (A,B,C) VALUES (1,1,1)
INSERT INTO DUPTest (A,B,C) VALUES (1,1,1)

INSERT INTO DUPTest (A,B,C) VALUES (1,2,3)
INSERT INTO DUPTest (A,B,C) VALUES (1,2,3)
INSERT INTO DUPTest (A,B,C) VALUES (1,2,3)

INSERT INTO DUPTest (A,B,C) VALUES (4,5,6)
GO
Select * from DUPTest
GO

Delete from DUPTest where ID <
(Select Max(id) from DUPTest t where DUPTest.A = t.A and
DUPTest.B = t.B and
DUPTest.C = t.C)
GO
Select * from DUPTest
GO

Removing duplicate records by using Oracle's ROWID

2012-10-25T21:55:00.001-07:00

Select col1,col2,col3,col4 from TableName where ROWID in (select max(ROWID) from TableName group by col1,col2)

Copy a table from one database to another database

2012-10-24T22:28:00.002-07:00

Copy a table from one database to another database in another ms sql server using ..

Copy a table from one database to another database in another ms sql server using sql server management studio :
If your table was in the same ms sql server but in another database, you would copy it using t-sql like this :

SELECT * INTO NewTable FROM existingdb.dbo.existingtable

SELECT * INTO NewTable FROM Sarma.dbo.DimAccount

However, if your database is in another ms sql server, you can do this :
Connect to your database using management studio. Right click on your table and select Script Table As > Create To > New Query Editor Window
Connect to the other database you want the table to be copied, create a new query and paste the sql previously generated. Run the query and you are done!

SELECT *
INTO TargetDatabase.dbo.MyTable
FROM SourceDatabase.dbo.MyTable

SQL QUERIES

2012-10-23T02:59:00.001-07:00

CREATE TABLE Person

(

LastName varchar(20),

FirstName varchar(20),

Address varchar(50))

sp_help Person

ALTER TABLE Person ADD City varchar(30)

SElect * from Person

ALTER table Person ALTER COLUMN firstname varchar(30)

INSERT INTO Person Values('Hetland','Camilla','Hegabakka 24','Sandness')

INSERT INTO Person Values('Pettersen','Kari','Storgt 20','Stavanger')

INSERT INTO Person (LastName,Address) values ('Rasmussen','Storgt 67')

Select * from Person

Update Person SET FirstName='Nina' WHERE LastName='Rasmussen'

Renaming a TABLE:

sp_rename 'Person', 'Persons'

alter table persons add year date

insert into persons (year) values ('1951')

select * from persons

delete from persons where YEAR ='1951-01-01'

update persons set year='1976-08-21' where firstname='Nina'

Functions available to create EXPRESSIONS

2012-10-23T00:12:00.000-07:00

Functions available to create EXPRESSIONS

Mathematical Functions
FUNCTION	Result	DataType
ROUND( 2.23 , 2 )	2	Integer
ROUND( 2.53 , 2 )	3	Integer
CEILING(2.23 )	3	Numeric
FLOOR(2.9)	2	Numeric
ABS(3.2)	3	Numeric
ABS(3.9)	4	Numeric
ABS(-3.2)	3	Numeric
ABS(-3.9)	4	Numeric

String Functions
FUNCTION	Result	DataType
LOWER( "ABD" )	abc	String
UPPER( "abd" )	ABD	String
LTRIM( " abc " )	"abc "	String
RTRIM( " abc " )	" abc"	String
TRIM( " abc " )	"abc"	String
REVERSE( "abc" )	"cba"	String
RIGHT( abc, 2 )	"bc"	String
SUBSTRING( "abcde", 2 , 3 )	"bcd"	String
REPLICATE("a",3)	"aaa"	String
FINDSTRING("abcd","ab",1)	"ab"	String
REPLACE("Apple", "pp", "n app")	"An apple"	String

Date/Time Functions
FUNCTION	Result	DataType
DATEADD( "mm", 1, (DT_DATE)"12/24/2009" )	1/24/2010 0:00	DateTime
DATEADD( "dd", -1, (DT_DATE)"12/24/2009" )	12/23/2009 0:00	DateTime
DATEADD( "yy", 1, (DT_DATE)"12/24/2009" )	12/24/2010 0:00	DateTime
DATEDIFF( "dd", (DT_DATE)"12/24/2009", DT_DATE)"01/26/2010" )	33	Integer
DATEDIFF( "mm", (DT_DATE)"12/24/2009", (DT_DATE)"01/26/2010" )	1	Integer
DATEDIFF( "yy", (DT_DATE)"12/24/2009", (DT_DATE)"01/26/2010" )	1	Integer
DATEPART( "dd", (DT_DATE)"12/24/2009" )	24	Integer
DATEPART( "mm", (DT_DATE)"12/24/2009" )	12	Integer
DATEPART( "yy", (DT_DATE)"12/24/2009" )	2009	Integer
DAY( (DT_DATE)"12/24/2009" )	24	Integer
MONTH( (DT_DATE)"12/24/2009" )	12	Integer
YEAR( (DT_DATE)"12/24/2009" )	2009	Integer

NULL Functions
FUNCTION	Result	DataType
ISNULL( «expression» )	TRUE/FALSE	Boolean
NULL(DT_DATE)	NULL	DateTime

Type Casts

FUNCTION

(DT_STR, «length», «code_page»)

(DT_WSTR, «length»)

(DT_NUMERIC, «precision», «scale»)

(DT_DECIMAL, «scale»)

Operators
FUNCTION	Comments
?:	If then Else -> "Condition"?"True Action" : "False Action"
&&	Logical AND
\|\|	Logical OR

The Sort Transformation-----SSIS

2012-10-21T23:14:00.000-07:00

The Sort Transformation-----SSIS

How to copy a SQL Server table?

2012-10-19T05:10:00.002-07:00

select * into <destination table> from <source table>

Example:
Select * into employee_backup from employee

OR

select only a few columns into the destination table like below

select col1, col2, col3 into <destination table>
from <source table>

Example:
Select empId, empFirstName, empLastName, emgAge into employee_backup
from employee

OR

 copy only the structure of the source table.

select * into <destination table> from <source table> where 1 = 2

Example:
select * into employee_backup from employee where 1=2

OR

 copy a table across two database in the same SQL Server instance.

select * into <destination database.dbo.destination table>
from <source database.dbo.source table>

Example:
select * into Mydatabase2.dbo.employee_backup
from mydatabase1.dbo.employee

Concatenate Function using Derived Column T/r SCreenShot

2012-10-19T05:05:00.001-07:00

TechInfoBI

DWH

Data Latency

Create Stored Procedure in SQL Server

Creating a Parameterized Stored Procedure in Sql Server

Example Parameterized Stored Procedure

CREATE PROCEDURE [dbo].[Proc_GETCOUNTRY] ( @Country_id int ) AS BEGIN SET NOCOUNT ON; select Country from tbl_Country Where Country_id=@Country_id END

To get the result pass the value to procedure and get the result here how to pass the value in procedure.

EXEC Proc_GETCOUNTRY 3

Result is

Performing Data Manipulation Add and Update Rows

CREATE PROC [dbo]. Proc_SaveCountryName ( @CountryName varchar(50) ) AS BEGIN INSERT into dbo.tbl_Country( Country) VALUES ( @CountryName) END

EXEC Proc_SaveCountryName 'England'

Result is

Update rows using parameter values

CREATE PROC [dbo]. Proc_UpdateCountryName ( @ID int, @CountryName varchar(50) ) AS BEGIN update dbo.tbl_Country SET country=@CountryName where Country_id=@ID END

EXEC Proc_UpdateCountryName 3 ,'NewZealand'

Result is

How it works

Stored Procedure with Output Parameter in Sql Server

Output Parameter Example

ALTER PROCEDURE [dbo].[Proc_SaveCountry] ( @Country nvarchar(30), @Status tinyint output ) AS BEGIN SET NOCOUNT ON; if exists (select Country_Id from tbl_Country where Country=@Country) Begin Set @Status=0 end ELSE Begin insert into tbl_Country (Country) values (@Country) set @Status=1 End END

Using Output parameter Getting last Inserted Country_id value.

CREATE PROCEDURE [dbo].[Proc_SaveCountry] ( @Country nvarchar(30), @Country_id tinyint output ) AS BEGIN SET NOCOUNT ON; insert into tbl_Country (Country) values (@Country) Select @Country_id =@@IDENTITY END

In this example the last inserted value in tbl_country is retreived by @@IDENTITY keyword and assign to output parameter @country_id

Fetch Stored Procedure

The following command executes the pProduct_Fetch stored procedure and retrieves data for all the products when called without any parameters: EXEC pProduct_Fetch

Result (columns and rows abridged):

With a @ProductCode parameter, the fetch stored procedure returns only the selected product: EXEC pProduct_Fetch @ProductCode = ‘1005’

Result (columns abridged):

The second parameter causes the stored procedure to return all the products within a single product category: EXEC pProduct_Fetch @ProductCategory = ‘Book’

Result (rows and columns abridged):

Update Stored Procedure in Sql Server

The update stored procedure accepts the primary method of identifying the row (in this case the product code) and the new data. Based on the new data, it performs a SQL DML update statement.

Update with RowVersion

To test the timestamp version of the update stored procedure, the pProduct_Fetch procedure will return the current timestamp for product code “1001”: EXEC pProduct_Fetch 1001

Delete Stored Procedure in Sql Server

To test the pProduct_Delete stored procedure, the following stored procedure attempts to call a product. Because there is no product code “99,” the error trapping raises the error: EXEC pProduct_Delete 99 Result: Unable to delete Product Code 99 - does not exist.

Data Warehouse Concepts

SQL SERVER 2008 Questions & Answers

INNER JOIN

select * from Employee

select * from Department1

Select Employee.empid,Employee.empnumber,Employee.empfirstname,Employee.emplastname,Employee.empemail,Employee.dateofjoin,Department1.deptname,Department1.Departmentid from EmployeeINNER JOIN department1 on employee.departmentid=department1.departmentid

Select Employee.empid,Employee.empnumber,Employee.empfirstname,Employee.emplastname,Employee.empemail,Employee.dateofjoin,Department1.deptname,Department1.Departmentid from EmployeeINNER JOIN department1 on employee.departmentid=department1.departmentid Where Department1.Departmentid=10

How to Remove Duplicate Rows but Log the Duplicates in SSIS

SQL Server Scnerio

Select Row1=MAX(Row1),Row2=MAX(row2),Row3=MAX(Row3),Row4=MAX(Row4) From A1

Lookup,Incremental load(Latest Records),removedup

How TO EXECUTE dowloded pacakage

Write data from multiple tables to single flat file.

Handling extra/missing columns in Delimited Flat file

SCD in SSIS

Insert multiple rows into a table

Merge Join

Alter & Update Table with Null

update Prod set Quantity=NULL where rowguid='D4544D7D-CAF5-46B3-AB22-5718DCC26B5E'

Alter table Prod alter column quantity smallint null

DELETE DUP REC"S

deleting duplicates.

Removing duplicate records by using Oracle's ROWID

Select col1,col2,col3,col4 from TableName where ROWID in (select max(ROWID) from TableName group by col1,col2)

Copy a table from one database to another database

Copy a table from one database to another database in another ms sql server using ..

SELECT * INTO NewTable FROM existingdb.dbo.existingtable

SELECT * INTO NewTable FROM Sarma.dbo.DimAccount

SELECT *INTO TargetDatabase.dbo.MyTableFROM SourceDatabase.dbo.MyTable

SQL QUERIES

Functions available to create EXPRESSIONS

The Sort Transformation-----SSIS

How to copy a SQL Server table?

Concatenate Function using Derived Column T/r SCreenShot

CREATE PROCEDURE [dbo].[Proc_GETCOUNTRY]
(
@Country_id int
)
AS
BEGIN
SET NOCOUNT ON;
select Country from tbl_Country Where Country_id=@Country_id
END

CREATE PROC [dbo]. Proc_SaveCountryName
(
@CountryName varchar(50)
)
AS
BEGIN
INSERT into dbo.tbl_Country( Country) VALUES ( @CountryName)
END

CREATE PROC [dbo]. Proc_UpdateCountryName
(
@ID int,
@CountryName varchar(50)
)
AS
BEGIN
update dbo.tbl_Country SET country=@CountryName where Country_id=@ID
END

ALTER PROCEDURE [dbo].[Proc_SaveCountry]
(
@Country nvarchar(30),
@Status tinyint output
)
AS BEGIN
SET NOCOUNT ON;
if exists (select Country_Id from tbl_Country where Country=@Country)
Begin
Set @Status=0
end
ELSE
Begin
insert into tbl_Country (Country) values (@Country)
set @Status=1
End

END

CREATE PROCEDURE [dbo].[Proc_SaveCountry]
(
@Country nvarchar(30),
@Country_id tinyint output
)
AS BEGIN
SET NOCOUNT ON;
insert into tbl_Country (Country) values (@Country)
Select @Country_id =@@IDENTITY
END

The following command executes the pProduct_Fetch stored procedure and retrieves data for all the products when called without any parameters:
EXEC pProduct_Fetch

With a @ProductCode parameter, the fetch stored procedure returns only the selected product:
EXEC pProduct_Fetch
@ProductCode = ‘1005’

The second parameter causes the stored procedure to return all the products within a single product category:
EXEC pProduct_Fetch
@ProductCategory = ‘Book’

To test the timestamp version of the update stored procedure, the pProduct_Fetch procedure will return the current timestamp for product code “1001”:
EXEC pProduct_Fetch 1001

To test the pProduct_Delete stored procedure, the following stored procedure attempts to call a product. Because there is no product code “99,” the error trapping raises the error: EXEC pProduct_Delete 99
Result:
Unable to delete Product Code 99 - does not exist.

Select Employee.empid,Employee.empnumber,Employee.empfirstname,Employee.emplastname,
Employee.empemail,Employee.dateofjoin,Department1.deptname,Department1.Departmentid from Employee
INNER JOIN department1 on employee.departmentid=department1.departmentid

Select Employee.empid,Employee.empnumber,Employee.empfirstname,Employee.emplastname,
Employee.empemail,Employee.dateofjoin,Department1.deptname,Department1.Departmentid from Employee
INNER JOIN department1 on employee.departmentid=department1.departmentid Where Department1.Departmentid=10

SELECT *
INTO TargetDatabase.dbo.MyTable
FROM SourceDatabase.dbo.MyTable