Team blog of MCS @ Middle East and Africa

TCP CICS transactions integration and automated rollback

Mon, 12 Jan 2015 09:27:25 +0000

Introduction

Integrating BizTalk applications with Mainframe programs can be done using many methods. One of the most convenient methods is to use the TCP CICS integration model using the standard listener TRM or enhanced listener ELM program link. Both these methods requires installation of a concurrent server (listener) on the Mainframe to handle TCP and sockets communication. Host Integration Server comes packed with an example concurrent server. One problem that you might face when using this model is the case of a processing of a transaction on the Mainframe and while this processing is happening on the Mainframe the socket communication fails. In this case you will face a condition where the transaction is already done on the Mainframe but the application on BizTalk is not aware that the transaction is performed so if there is no mechanism against this then the same transaction can be executed twice.

Discussion & Resolution

A resolution of this problem is to edit the concurrent server program on the Mainframe to issue a CICS ABEND in the case of a socket failure. This insures that if the response is not delivered to the application then the transaction would be rolled back on the Mainframe, this though requires that all Mainframe programs to be used should not implement any SYNCPOINT.

To change the HIS provided sample for the concurrent listener to implement this approach the following should be added to the program MSCOMTI.cbl in the “STANDARD-LISTENER” procedure and after processing the “WRITE-BUF-TO-SOCKET” procedure.

PERFORM WRITE-BUF-TO-SOCKET THRU

WRITE-BUF-TO-SOCKET-EXIT

IF CHILD-SERVER-ERROR < 0 THEN

MOVE SS-CLOSE-SOCKET TO NEXT-STATE

MOVE 'Y' TO ABEND-STAT

ELSE

Then within the same procedure after the loop ends add the following code to issue the ABEND.

IF ABEND-IS-REQUIRED THEN

MOVE TP-ABEND-1 TO TRACE-ID

PERFORM TRACE-POINTS THRU TRACE-POINTS-EXIT

EXEC CICS ABEND ABCODE('ABC') END-EXEC

END-IF.

This will make sure that the transaction on the Mainframe will rollback if no response can be delivered to the application due to any connectivity problems.

Full source of the customized listener is here.

Writing a custom HostApps ESB adapter provider to integrate ESB toolkit with a Mainframe

Sat, 10 Jan 2015 14:53:35 +0000

Adapter provider framework is how the ESB toolkit configures the off-ramps to send requests using dynamic send ports. An adapter provider is an implementation of the base abstract class BaseAdapterProvider. The adapter provider is simply a component that is executed as part of the send pipeline to add the required context properties on the message before it is processed by the adapter on the dynamic send port to send the message.

HostApps adapter is BizTalk way to communicate with Mainframe programs using CICS integration. To use this adapter as part of the BizTalk ESB toolkit then you will need to allow dynamic ports to be able to send requests to this adapter. The ESB way to do this is by implementing a custom adapter provider.

The steps to implement a new custom adapter provider are simple. The following are the steps required to implement a HostApps adapter provider:

1- Create a new C# class library DLL and strongly sign the DLL.
2- Add references to the following DLLs

a. Microsoft.BizTalk.Messaging.dll

b. Microsoft.BizTalk.Pipeline.dll

c. Microsoft.Practices.ESB.Adapter.dll

d. Microsoft.Practices.ESB.Itinerary.Pipelines.dll

e. Microsoft.XLANGs.BaseTypes.dll

3- Add a new class with any name and inherit from BaseAdapterProvider
4- Override the public properties AdapterName and AdapterContextPropertyNamespace and return any name for your adapter provider. The following is an example for the HostApps adapter provider.

public override string AdapterName

{

Using Host Integration Server COBOL Import Wizard

Sat, 07 Jun 2014 10:37:04 +0000

Introduction

Host Integration Server allows application integration with the Mainframe system using the Transaction Integrator (TI) component. The method to implement this integration involves building what is called a TI assembly. TI Assembly is actually built using the definition of the interface between the Mainframe program and the definition of the input and output COM areas. This involves in many cases reading large COBOL files and manually adding them to the TI library. The HIS development tools have added the import utility to simplify importing from COBOL copy book files but as we will see in this post this still have many challenges and I will provide in this post a systematic approach to facilitate importing a COBOL copy book file.

Discussion

COBOL copy books are the way Mainframe programs define the inputs and outputs. You can think of these as the signature of C# methods. These are mainly text files with specific structure. You can find more details about the definition of the structure of the COBOL copy book files here. A COBOL copy book file would look something like the below:

Now usually you would receive two COM area definitions like the above, one for the input COM area and another for the output (if they are different) from the Mainframe team. So the challenge here is how to use the HIS import utility to import these into one method in the TI assembly. Usually when you try to import the import wizard would crash and would not give you enough description of where was the problem so you would end up adding these things manually and that would be ok for smaller programs (like the above) but for more complex programs where you have more than 20-50 fields in input and outputs that would be a very tedious task. The section below would provide a simple approach and walkthrough on how to import these files and use the import wizard successfully every time.

Solution & Walkthrough

The HIS import wizard is very sensitive to the spaces and structure of the COBOL copy book file. So for example a following misalignment would fail the import.

So the first thing you need to do when dealing with COBOL copy books is to align them perfectly. To do this I recommend using a tool such as Notepad ++ as it understands this type of file and it actually shows you very helpful vertical guidelines.

So this simplify spotting misalignments.

The second thing you need to do is to merge the inputs and outputs COM area definitions in one COBOL copy book file while aligning the outputs as if it is part of the input and I will show you later how these would be separated. Just give it a name with the same program name while adding any suffix such as “-OUT” to it. Please also note that every line has to be terminated with a dot.

Now that you have created and saved the COBOL copy book file open Visual Studio and create a new client TI library in a TI project. Now import it as follows:

Host Integration Server 2013 TI Assembly GAC deployment Walkthrough

Tue, 27 May 2014 09:42:13 +0000

Introduction

Host Integration Server (HIS) is used to integrate with Legacy Host technology. One of the Host integration scenarios is to call Mainframe programs or transactions using a component in HIS called Transaction Integrator (TI). The way to use TI is to create what is called a TI assembly and then use this to call the Host systems from the Windows System. The TI assembly can be either used in a folder or deployed to the Global Assembly Cache (GAC). I wanted to deploy the assembly to the GAC to make sure it is available from anywhere in the integration solution and because I had strong named assemblies in the solution that required to reference this assembly. This blog post will provide the challenge and problems faced while trying to do this and how these were resolved to finally deploy the TI assembly to the GAC and use it from there in a BizTalk solution.

Discussion

Host Integration Server (HIS) is used to integrate with Legacy Host technology. One of the Host integration scenarios is to call Mainframe programs or transactions using a component in HIS called Transaction Integrator (TI). TI technology has been around for a while (it was used to be called COMTI) but it went through many changes and updates. In HIS 2013 the TI component undergone major changes. There are two scenarios to use the TI, either Host Initiated Processing (HIP) where the call is initiated from the Host server to the Windows server, in this scenario the Windows server acts as the backend and the Host is acting as the channel or initiating system. The other scenario is Windows Initiated Processing (WIP) where the call is initiated from the Windows platform and it is routed to the Host system. The diagram below illustrates this scenario.

Figure 1: Windows Initiated Processing Scenario

To be able to implement a WIP scenario you need to implement what is called a TI assembly. This assembly actually governs the interface and mapping of parameters between the Windows system and the Host system. The way you create this assembly (as this is a special assembly type) is using the TI project template that ships with the HIS 2013. I am not going to describe how to create the TI assembly as you can find many references on how to do this in the HIS 2013 SDK here and on MSDN here.

Once you have created the TI assembly you will have the following view in VS 2012.

Figure 2: TI Assembly in VS 2012

Now to be able to deploy an assembly in the GAC you need to have that assembly signed with a key and then would have a strong name. If you click on the Library node (root node in the above figure) and check the properties you will see that it has the following properties:

Figure 3: TI Library Properties

The property I am interested in is the KeyFile property which is according to the documentation if you provide a Key file in this property then the generated assembly would be signed with this key and would have a strong name (sweet). Unfortunately for some reason this does not work as expected and I could not get it to work and the assembly generated is always not signed.

I have investigated many other options to sign the assembly after being compiled and this is doable but I wanted something more streamed and easier to be done if I needed to re-generate the assembly.

Once I signed the assembly (as I will show you in the next section) and GACed it. When I create a BizTalk port using the BAHA adapter and try to configure it with the GACed assembly I found that the dialog to select the assembly from the GAC is always empty! As per the below figure.

SharePoint Provider Hosted App Walkthrough: Part 2 Dig deep

Mon, 31 Mar 2014 19:22:25 +0000

Introduction

In the previous post here I showed how to write a new provider hosted App using a step by step process. In this post I will build up on that knowledge and show you the following:

How to customize your provider hosted app so that you get information from the SharePoint site. (Such as user email).
How to customize the look and feel of your provider hosted App so that it reflects the SharePoint theme so that the user does not feel a completely disconnected experience. Also how that can be selective so that if the same web site was viewed outside of SharePoint it is displayed as a normal web site.
How to link back to SharePoint and publish information back to the site including posting to the personal feeds.
How to host the provider hosted App on Azure web site and how to register this on your private tenant.

So let’s get started, shall we.

Getting Information from SharePoint

To allow the application developers to connect to the source SharePoint site, the request coming from the SharePoint site has a query parameter called “SPHostURl” this contains the source URL of the SharePoint site. So to retrieve information you need to use the client object model to connect to the SharePoint site. Now another problem is how to establish authorization and get the current user identity to be able to perform actions using the currently logged on user identity. To establish this you will already find in any new SharePoint provider hosted App a class called “TokenHelper” this will specifically help you to create the client context using the token. Now once you got the token you do not want to re-connect with the SharePoint site to retrieve the token with every request and hence the need of a token cache. I have included with this article a “TokenDictionary” class that would allow you specifically to cache your tokens.

Note: One thing extra to note while working with provider hosted apps is that you need to make sure you pass the “Request.QueryString” from every page to the next so that you do not lose your SharePoint context.

Now let’s see the code to get the user email address:

// Retrieve Email address from SharePoint

// Set up SharePoint Context

Uri

Small tip: Where is the XSD Definition Tab in Visual Studio 2012 with Host Integration Server 2013

Tue, 07 Jan 2014 21:09:01 +0000

Today I faced an issue that took me some time to figure out with Host Integration Server 2013. Usually when you develop transaction Integration (or application integration) with BizTalk server you require XSD schemas to be able to perform mappings and send messages using a send port. With Host Integration Server and up to 2010 version we did that by writing an application definition DLL within Visual Studio 2010 and then while you have the DLL description open in the designer you would click on the “XSD Definition” tab within your Visual Studio window as below:

Then you copy all contents of this tab into a newly created XSD schema in a BizTalk project.

Now the problem I faced was that this tab is missing in HIS 2013 :( So how can we do this. At first I thought that maybe this is an installation issue as I had in that environment HIS 2010 and VS 2010. But it was not. It turned out that now you simply save the DLL and VS 2012 automatically generates for you the XSD (along with many other things such as WCF and ASMX services) inside a bin folder. So Vola use that XSD.

Happy HISing ;)

Using an IIS hosted MEX endpoint with an in-process WCF receive location in BizTalk Server

Sat, 28 Dec 2013 11:52:29 +0000

Introduction

Receiving messages and requests using WCF endpoints and receive locations is a very well-known practice that is used in many BizTalk integration projects. These WCF services can be either a published orchestration or using specific input and output schemas. In any of these there are two known messaging patterns that can be used either in-process or isolated. In-process means that the BizTalk server host instance will be responsible for hosting the WCF host and any requests will be directly received by the BizTalk host. Isolated model means that the WCF service host will be hosted inside IIS and a request received by this service will then be processed inside IIS W3WP worker process and once it requires persistence it will be submitted to the BizTalk Message Box database for normal publish and subscribe processing to occur. In this case this request can be picked up by any of the BizTalk processing host instances and hence will traverse the processes boundaries. In this post I will not go any further in describing these two models but I want to concentrate on a specific problem I faced.

So imagine the following scenario, you want to implement an in-process WCF receive service but you want the MEX endpoint to be published on IIS so you want the consumer of this service to be able to submit specific requests and expect specific responses rather than submitting and receiving Message objects. This can be done easily if you want to host the processing or receiving endpoint inside IIS, so working in isolated mode. But unfortunately this is not supported for in-process. Usually we do this by using the WCF publishing wizard and publish a metadata exchange endpoint only to IIS and point to the receive location required to process the request which is usually hosted inside IIS (isolated). Now if you select an in-process receive location it will publish the endpoint as usual and you will be able to create the MEX WCF service on IIS but when you try to use it you will receive the following error.

This is normal and expected as the in-process receive location have to be specified using an absolute URL rather than a relative one as you usually would configure an isolated one. Since the URL is absolute the MEX host will try to add it as a new base address. Now the problem is that this address is not supported by IIS. Now if you try to configure the service using the suggested configuration in this error this will not work in any way.

Solution

Usually this is not a supported configuration and hence there is no out of the box method to get this to work. Now I wanted this configuration to work so I started reflecting the BizTalk assemblies to figure out the problem.

Now if you open the MEX endpoint service SVC file you will find the following in it.

<%@ ServiceHost Language="c#" Factory="Microsoft.BizTalk.Adapter.Wcf.Metadata.MexServiceHostFactory, Microsoft.BizTalk.Adapter.Wcf.Runtime, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" %>

So as you can see here it is using a custom host factory to create the WCF service host. This custom host factory is inside an assembly called “Microsoft.BizTalk.Adapter.Wcf.Runtime” that is hosted in the GAC. So I reflected this assembly to see what is happening there and here is what I found inside the CreateServiceHost.

So as expected, it adds the URL to the list of base addresses if the address of the receive location is in absolute URL format. Now this is correct and needed if the processing endpoint is hosted inside the IIS but since this is hosted inside BizTalk worker process this is not needed and hence we would not need such a line.

So what I ended up doing is to create new service host factory. I started by thinking of inheriting from the already existing host factory, well you guessed it, this class is sealedL. That leaved me with the only other option which is to create a new class with exactly the same code (reflected) but with these lines removed. So the CreateServiceHost function is now as follows:

removed. So the CreateServiceHost function is now as follows:

Integrating Apache ActiveMQ with BizTalk Server 2013 using WCF-WebHTTP adapter and REST API

Wed, 11 Dec 2013 13:22:44 +0000

Introduction

In this blog post I will present a very simple solution to integrate with Apache ActiveMQ platform. Actually I got this question from a friend I could not believe how easy it is to integrate with this platform using the new BizTalk 2013 WCF-WebHTTP adapter. The adapter provides many features out of the box that really makes the integration engineer life much easier and better.

In this scenario I have installed Apache ActiveMQ on a private VM, created a test queue and made sure that I am able actually to send messages to that test queue. The target then was to create a way to receive messages from this queue to BizTalk server.

Discussion

The following procedure is what I did to perform this POC.

1- First I started by installing ActiveMQ and to my surprise that was super easy. So I downloaded the binaries from here and I also downloaded the JDK from here. I installed the JDK then extracted the ActiveMQ to any folder of my choice.

2- I then had to set the JAVA_HOME environment variable (I do not know why the JDK setup did not do this) but I did that manually to point to the JDK folder as C:\Program Files\Java\jdk1.7.0_45

3- I then started the ActiveMQ platform by browsing to the folder where I extracted it and then typing the command:

bin\activemq

4- Then I opened the new administration portal using the URL (http://localhost:8161/hawtio) then I logged in using the username admin and password admin.

5- Clicked on “ActiveMQ”

6- Then I created a new Queue called “Testing” by clicking “Create” then “Create Queue”

7- Click on the queue on the left then click “Send” then make sure you change the format to XML type in a valid XML message and finally send the message.

Creating an Active/Active SQL Cluster using Hyper-V: Part4 Configuration Revisited

Wed, 04 Dec 2013 09:06:00 +0000

In part 1of this series I showed you how to configure the virtual storage required for the cluster. In part 2 of this series I showed you how to configure two SQL instances on the created windows cluster. In part 3 I showed you how to configure these two SQL instances into an Active/Active configuration. After a while and as I was playing around with the configuration I have noticed that I cannot really move the SQL server services from one node to another automatically! I had to move the DTC first then I can move the SQL server service. Then it hit me! Why am I complicating things and doing everything by hand although the Windows Failover cluster can do much of the plumping in the correct way. So in this follow-up post I will show you how you can configure things the right way (the windows failover cluster way ).

Introduction

In this series of posts I will walk you through the processes of creating an Active/Active SQL server cluster using Hyper-V and Microsoft iSCSI target software for virtualized SAN. The target is to create first a storage server hosted on a normal Windows 2008 R2 server. Then connect to this server using two other machines as iSCSI initiators. Then I will create the windows cluster along with the DTC clustered service. A clustered SQL server instance will then be created. Finally another clustered SQL server instance will be created and Active/Active configuration of both instances will be applied.

Solution Architecture

The solution is fairly simple as per the below configuration.

Revisiting the Cluster configuration and Dependencies

The old dependency was not set correctly since the SQL server service and the associated DTC service are not the same clustered service but are managed differently and I established the link between them using an external service dependency. This rendered the SQL service not movable since the cluster really does not know that it is required to move the DTC service to be able to move the SQL service but instead it treats it as an external service that must exist for the SQL service to start.

The old dependency diagram was like this:

The problem is actually in the dependency on an external clustered service rather than a cluster service resource.

Then I remembered that the Failover Cluster can add the DTC as a generic service resource to the same clustered service, or even better it actually knows of DTC and has a special type of resource as per the below image:

So to configure everything correctly I performed the following:

1- Brought all services offline including all two SQL server services and even disks.

2- I removed all DTC related configuration by deleting the two clustered services for DTC including all dependencies from the SQL server services.

Solved: Nested and conditional parent and child Table Looping Functoid in BizTalk mapping

Sat, 31 Aug 2013 10:37:13 +0000

Introduction

BizTalk mapping tool is a very powerful tool once you get the hang of it. Sometimes you would need to produce some structure in the output schema that did not exist in the source schema, the Table Looping functoid is built for this specific task.

In some other cases you would need to generate a parent child kind of structure. In this blog post I will present the solution to use out of the box functoids to be able to generate this structure.

Problem

Suppose you have the following as the input schema.

And the following output schema.

The requirement is to generate two Message elements in the ProcessRoot output schema. It is also required to generate some fields in the fields array inside each message. These fields are specific for each message and depend on the message ID field. So to be more specific let’s say that we have the following XML input:

<ns0:ProcessRequest xmlns:ns0="http://BTSTest.InputSchema">

<Parameter>Parameter_0</Parameter>

</ns0:ProcessRequest>

And we want to generate the following XML output from it:

<ProcessRoot ID

BizTalk Cop installation and configuration for VS 2012

Sat, 17 Aug 2013 18:13:51 +0000

I faced a requirement today that I need to enable some BizTalk best practices and create a custom check-in policy for these practices while the developers are checking in code. While searching the internet I found the BizTalk Cop (which is a great tool) that does exactly that, but unfortunately this tool supports only Visual Studio 2010 and BizTalk 2010 and does not support BizTalk 2013 and visual studio 2012. So the challenge was how to enable this to work with Visual Studio 2012. Another challenge that I had was to create a default custom check-in policy and a rules set that is enabled by default on TFS that would only allow the developers to check-in their code if they enable this rule set.

The solution that I performed involved the following changes (updates)

I copied the DLL “BizTalkCop.Rules.dll” to the folder “C:\Program Files (x86)\Microsoft Visual Studio 11.0\Team Tools\Static Analysis Tools\FxCop\Rules”
I created a file called “BizTalkRules.ruleset” (attached at the end of this article) containing the set of rules that are enabled by default for the projects.
I copied the file “BizTalkRules.ruleset” to the folder “C:\Program Files (x86)\Microsoft Visual Studio 11.0\Team Tools\Static Analysis Tools\Rule Sets”
Changed the file “devenv.exe.config” in the folder “C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE” so that it redirects all FxCop dependencies from the 10.0 and 9.0 versions to the 11.0 version in the assembly binding section as follows:

Changed the file “FxCopCmd.exe.config” in the folder “C:\Program Files (x86)\Microsoft Visual Studio 11.0\Team Tools\Static Analysis Tools\FxCop” so that it redirects all FxCop dependencies from the 10.0 and 9.0 versions to the 11.0 version in the assembly binding section as follows:

Restart VS 2012
I enabled a default check-in policy using the source control settings as per the below screen shoots

SQL 2012 Unattended installation & Configuration file creation

Tue, 30 Jul 2013 12:18:22 +0000

Introduction

In many cases you might be faced with a requirement to perform an unattended SQL Server 2012 installation. To perform SQL Server 2012 installation you need to perform many choices and steps to configure your installation so the question is how to provide all these details during an unattended installation?

Solution

There are many methods to perform SQL 2012 unattended installation, one of these methods is to perform image preparation which is greatly enhanced in SQL 2012. This might be a topic for a different post but for this post I wanted to concentrate on another method which is to use a configuration file to configure the installation.

To perform an unattended installation using a configuration file it is simple you just need to write the following command line at a new command prompt.

Setup.exe /ConfigurationFile=SQLConfigurationFile.ini

Simple, right? But the question now is how can I generate this configuration file?

Actually the answer is also very simple.

1. You simply start setup normally by double clicking on Setup.exe

2. Click on installation on the left.

3. Click on “New SQL server stand-alone installation or add features to an existing installation”

4. Go through the normal setup and choose all components you wish to be installed during the unattended installation.

5. On the “Ready to Install” screen you will find the path to the created configuration file as per the screen below

6. Copy the created configuration file and then cancel this installation.

Fixed: BizTalk 2013 BAM Portal configuration problem

Sun, 28 Jul 2013 10:23:40 +0000

The Problem

When configuring BizTalk Server 2013 RTM on a multi-computer environment you might face multiple issues that are new and did not exist in previous versions of BizTalk server. One of these issues that I faced was when you try to configure the BAM portal after you have configured the BAM Alerts service. Of course you already know that the BAM alerts in BizTalk 2013 is using a new model of Database mail instead of SQL server 2005 notification services (finally) but it seems this caused some problems. In this scenario the configuration failed and when you open the log you see this error.

Error encountered: Could not install BAM Portal, error with regard to “BAM Management Web Service User”

Error thrown: “Attempted to read or write protected memory. This is often an indication that other memory is corrupt.”

The Solution

When I started looking at this error I started thinking that the BizTalk configuration application is corrupted so I restarted it, with no luck. I then restarted the entire machine with no luck. I then started to look into the log file more carefully and found that there is another error being reported before this one complaining about a problem with one database role called “NSSubscriberAdmin” in the BAM Alert Application database.

I opened this database and I could not find this role defined in it. I started to search the internet for someone who faced this problem with BizTalk 2013 and I found one here. He is suggesting to just add the role “NSSubscriberAdmin” to the BAM Alert Application database. So I did this and it worked.

BUT when I started thinking of what I did, I thought this is wrong since the configuration application did something using this role and this role is not used anywhere in the application so this must hunt me back sometime later. So I opened the SQL management studio and looked at this role and what the configuration application did to it. I found that it actually added the BAM Management Web Service user to this role! So this will be used by this service; what I found also is that there is another role in this database called “BTS_SubscriberAdmin” role (rings any bells ;) ) so I am thinking this is the true role that the configuration application should have used. So what I did is that I added the “BAM Management Web Service User” to this role “BTS_SubscriberAdmin”. I hope that this is enough to make the solution work with no problems. J

Happy BizTalking ;)

Fixed: BizTalk 2013 BAM tools configuration problem

Fri, 26 Jul 2013 12:48:57 +0000

The Problem

When configuring BizTalk Server 2013 RTM on a multi-computer environment you might face multiple issues that are new and did not exist in previous versions of BizTalk server. One of these issues that I faced was when the BizTalk Server and the database server are two separate servers. I was trying to configure only BAM tools and without the configuration of the BAM alerts. Of course you already know that the BAM alerts in BizTalk 2013 is using a new model of Database mail instead of SQL server 2005 notification services (finally) but it seems this caused some problems. In this scenario the configuration failed and when you open the log you see this error.

[Info] BAMTools Determining version of Microsoft SQL Server installed on server.

[Error] BAMTools Error configuring

Error determining version of Microsoft SQL Server installed on server. (Microsoft.BizTalk.Bam.CfgExtHelper.Utility)

A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server) (.Net SqlClient Data Provider)

The system cannot find the file specified ()

The Solution

This is a known and reported issue in BizTalk server 2013 and there are two workarounds for this issue:

1- Install SQL Server 2005 Notification services on the BizTalk server machine. (again L) I personally do not like this option as you are installing something you are not using.

2- Configure the SQL server 2012 database mail and configure both the BAM tools and BAM alerts at the same time. This is a good option if you are going to use BAM alerts but if you are not going to use it then you are configuring a feature that you will not be using.

The official and recommended solution for this is to install the already available Hot Fix for this issue. This hot fix would be automatically installed on the BizTalk server if it has internet access, but like my case where the servers does not have internet access I had to manually download and install it. Please find the download for this hot fix here.

http://support.microsoft.com/kb/2838133

Securing Dynamic Data ASP.NET SQL Azure Published Web Site with ACS and Facebook as an Identity Provider

Sat, 27 Apr 2013 16:06:11 +0000

The Scenario

I wanted to implement an Azure web site that is using the Azure Access Control Service and integrates with an external identity provider to authenticate and authorize users. At first I thought of using Windows Live ID but it has a problem that the only claim offered by WLID is the unique identified which is simply a number and represents nothing from the user. Then I thought why not make things more interesting and use Facebook. I think things got more interesting than I thought. J

I wanted to implement a Dynamic database access web site so that it generates the views on top of an existing SQL Azure database and lets the end user manipulate the database tables and filter them. This is using Linq-to-SQL classes.

I am using Visual Studio 2012 latest version.

The Steps highlights

The steps at a glance of how to get this up and going are as below:

1- Create your project:

a. Create your database.

b. Create a new Dynamic ASP.NET project.

c. Add a Linq-To-SQL model to your database.

d. Change the Framework version of the project.

e. Set the “ScaffoldAllTables” to true.

2- Download the latest “Identity and access tool”

3- Create a new Azure web site and download the publishing settings.

4- Setup your identify provider:

a. Create a new Azure ACS namespace

SharePoint Group Calendar, Adding default users

Wed, 10 Apr 2013 06:22:23 +0000

The Problem

One feature of SharePoint is to create a calendar and make it a group calendar. This allows the user to see the schedule of list of people at the same time on the same view as per the image below.

To configure the Calendar as a group Calendar you need to edit the List settings and then click on “List name, description and navigation” and finally select the radio button as per the image below.

So all this is SharePoint out of the box functionality. So where is the problem?

The problem is that to get the view of all calendar of your work group every time you open the calendar view you need to add each one of them individually! And I could not find any way to persist this information so that it would open the calendar view automatically with the people added as per the list you see in the first image. So lets get to work.

The Solution

The way I fixed this is using Java script.

To be able to do this I used the following script.

var sel = SP.UI.ApplicationPages.CalendarSelector.instance().getSelector(1, 'WPQ12');
sel.selectEntities(ret, true);

You need two things to be able to perform the above script, the ID of the calendar web part (WPQ12 above) and the XML script to be able to add the required individuals. The way to do this is to follow the following steps:

Step 1: Getting the XML to add the required users

Install and start Fiddler
Click on the ribbon on the small arrow below “people” then “Add Person or Group”
This will give you the dialog to select the users, add all required users to be added by default to the Calendar view and then click “Ok” as image below
See Fiddler log to find an entry to add the users as per the below.
Then click on the response as text view you will find the Entities string as per the image below.
Copy this text to be used later in your script.

Step 2: Getting the web part ID

Open the page with the calendar web part
Open the IE developer tools (F12)
Click on the selector arrow and then on the calendar web part (most top level)
In the developer tools view you should see Id of the web part as below

Step 3: Create the required script file

Use the information gathered to construct a JS file as below

function _firstTime() {

//need to select the calendar tab so we can override the onlick method on some of the buttons.
SelectRibbonTab('Ribbon.Calendar.Calendar', true);

//give the ribbon time to load
setTimeout('_doWireUp();',2000);

}

function _doWireUp()
{
//change the onclick event for the group buttons to make sure it reloads our default group
var weekElem = document.getElementById('Ribbon.Calendar.Calendar.Scope.WeekGroup-Large');
if(weekElem)
weekElem.onclick = function() {setTimeout('_setDefaultResources();',1000);return false;};

var dayElem = document.getElementById('Ribbon.Calendar.Calendar.Scope.DayGroup-Large');

if(dayElem)
dayElem.onclick = function() {setTimeout('_setDefaultResources();',1000);return false;};

_setDefaultResources();
}

function _setDefaultResources() {

// This is the entities XML from step1
var ret ='\u003cEntities c…………………………………………………………………………………………………………..\u002fEntities\u003e';

// Put here the web part ID from step 2

var sel = SP.UI.ApplicationPages.CalendarSelector.instance().getSelector(1, 'WPQ12');
sel.selectEntities(ret, true);

A Small trick: How to extract a BizTalk BAM definition from an Excel sheet without installing Excel client

Tue, 26 Feb 2013 06:36:51 +0000

This is a small trick that I have been doing for sometime now and wanted to share. The scenario is that you have prepared the BizTalk BAM definition and you ready to deploy it to the production environment, you run the bm.exe command and you see the situation; the BAM definition file is an Excel sheet and since you do not have Excel installed on the production server the installation has failed.

The options you have are either to your back to your development environment and export the XML from the Excel sheet or to install Excel on the production environment (not really an option).

The trick is actually to open the Excel sheet in notepad as a normal text file and you would see something like this:

WOW wait a minute this XML seems familiar, right?

Well you are correct this is the XML definition for the BAM view and activity. So simply copy this text and paste it in a new XML document and there you have it the XML definition of your BAM activity

Happy BizTalking

How to configure SQL Database mail so send emails using Office 365 (Exchange Online): A walkthrough

Mon, 25 Feb 2013 13:17:55 +0000

Introduction

SQL Server has a feature called database mail. This feature allows the database server to send emails to any external entity using SMTP server. The problem happens if you have installed an on-premise SQL server and an online (Office 365) Exchange server in the cloud. How can you use this Exchange server in the cloud to send database emails?

This blog post provides a complete walkthrough on how to configure this. This is based on the description provided in the KB article http://support.microsoft.com/kb/2600912.

The Walkthrough

The steps start with the following.

Step 1: Get the SMTP settings for your Exchange online server

1- Go to the address http://dev.office.com to sign up for a trial account for office 365.

2- After the exchange service is provisioned go to the tenant administration page and click on Outlook

3- Click on the “” icon and then click options

4- Click on “Settings for POP or IMAP access…”

SharePoint Provider Hosted App Walkthrough: Part 1 Preparation

Sun, 24 Feb 2013 16:24:47 +0000

SharePoint and office development model is changing to a new App model. This is the new development model for SharePoint. In this blog series I will walk you through creating a provider hosted SharePoint App that is hosted in an App part. This will show the following key features:

· How to prepare your environment for your App development?

· How to develop a provider hosted App that is hosted in Azure websites and linked to SharePoint Online?

· How to use the Client Object Model (CSOM) to read and write data back to SharePoint using the credentials of the logged in user?

· How to handle multi-tenancy in your App?

· How to package and sell your App on the market place?

In this blog post (which is first of the series) I will provide some introduction and then will walk you through how to prepare and test your development environment with your first hello world provider hosted App.

Introduction

So the question here is, why the new development model? Looking back at SharePoint 2010, the common development approaches had a few challenges associated with them.

With Farm Solutions, AKA Fully Trusted Solutions, custom code was deployed to the server. These types of solutions generally meant there were a lot of things to deploy to the server, even if they were all deployed in a SharePoint solution package. This also meant that most deployments required a server touch as an IT Pro with console access would need to upload the solution to the server’s solution store for deployment. In addition server outages and most SharePoint errors could usually be traced to custom code deployed to the server. These types of solutions are only available with on-prem deployments and not with hosted deployments that only allow sandbox solutions.

Sandbox solutions have their own challenges in that they are limited in the number of things you can do. Developers can work within a subset of the full server-side API and are blocked from making external database or web service calls.

Both of these options, farm & sandbox solutions, require the developer to have a solid understanding and be familiar with the SharePoint server-side API.

To address many of the challenges developers and site owners had in previous versions of SharePoint, Microsoft has introduced a new development option for SharePoint 2013: The SharePoint App Model.

In this new model apps do not necessary live within SharePoint. Instead the app’s business logic executes within the context of the client (browser) or externally from SharePoint. This external option could be another non-SharePoint Web server or a cloud server. Apps are also more secure in that when they need to access SharePoint resources such as lists and libraries they must be explicitly granted permissions to do so. This is implemented using OAuth. When an app is created, the developer specifies which permission the app needs in order to function. When the app is installed, the user installing the app is prompted to accept the permission requests the app needs (if they deny the permissions, the app is not installed). Once granted permissions, the apps can then talk to SharePoint using the Client Side Object Model (CSOM) or using some of the new OData services in SharePoint.

Create a password dialog for Visual Studio deployment project

Wed, 19 Sep 2012 13:58:10 +0000

Introduction

Most of us have worked with Visual Studio deployment projects to create setup projects for components we developed. In many cases we used the custom user interface dialogs that come with Visual Studio to take extra parameters from the user. The problem happens when you require extra parameters than Visual Studio provides or custom user interface dialogs. In my case what I needed is a dialog to take username and password fields from the user and hence I wanted to apply a style to one of the edit box controls to be a password field. In the following I will show you how to create a new custom dialog and how to mark an edit box as a password field.

How to create a custom Visual Studio Setup dialog?

Visual Studio setup project has the functionality to add extra dialogs from a library of custom dialogs as shown in the figure below.

So what I wanted to do is to customize one of the three textboxes dialog and create new one where I change one of the fields to be a password field. To do this you will need ORCA MSI editor from the Windows SDK Components for Windows Installer Developers here. The steps are as follows.

1- Close the Visual Studio.

2- Browse to the folder “C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\Tools\Deployment\VsdDialogs\0”and make a copy of the file “VsdCustomText3Dlg.wid” and rename it to “VsdUsernamePasswordDlg.wid”

3- Open the file “VsdUsernamePasswordDlg.wid”in ORCA.

4- Change all occurrences of “CustomTextC”to “CustomTextD” in all tables and all values.

5- In the module signature table make sure you edit the GUID and replace it with a new GUID

Hyper-V Physical to virtual: Moving a natively booted VHD to be a real virtual machine

Tue, 07 Aug 2012 21:28:47 +0000

Introduction

I had a requirement to migrate an existing native boot Windows Server 2008 R2 machine that was booting from an attached VHD on a physical machine to be a virtual machine in Hyper-V. The problem is that this machine had multiple boots and so the booting record was on the physical drive and the VHD image really did not have any booting information. Also because of the fact that it is booting in a physical machine it was using SATA AHCI of that machine and had the drivers to use that which is completely different form the Hyper-V implementation of the feature so I had to switch to Native IDE storage controller instead.

Resolution

What I did was first I created a differencing VHD disk from that virtual machine as I wanted to make sure I am not changing anything in the source VHD in case I decided to switch back to native boot. Then I created the virtual machine and used IDE controller for the HDD pointing to the differencing VHD file. Now because the VHD did not have any booting information I had to build it and I had to add the boot manager to that machine. So here are the steps.

1- Mounted the VHD using the host storage disk management.

2- Copied the files “bootmgr” and all contents of the folder “Boot” from a similar OS machine (Windows 2008 R2) to the root of the VHD.

3- Made sure that the copied files still have the system attribute set correctly.

4- Un-mounted the VHD from the host.

5- Attached a Windows 2008 R2 setup DVD to the DVD drive of the virtual machine.

6- Started the Virtual machine and booted from the setup DVD.

7- Clicked next on the setup screen with default options.

8- Clicked repair my computer and opened command prompt.

ADFS 2.0 Client Certificate Authentication using test certificates

Fri, 03 Aug 2012 23:03:50 +0000

Introduction

I was developing a WCF based solution that required services to be authenticated using ADFS 2.0. The requirement was that the clients will be using client based certificates to authenticate to services. If you have a complete AD CS solution implemented that wouldn’t have been that difficult since the platform already takes care of most of the stuff already mentioned here but I wanted to test this scenario with my own testing certificates that are generated using command line tool “makecert.exe” and not a CA.

The Solution

The steps that you need to follow are:

· Step 1: Create a claims aware WCF service using WIF.

· Step 2: Add ADFS as an STS to the WCF service.

· Step 3: Add this as a relying party in ADFS using the generated metadata.

· Step 4: Build a client application and add a service reference.

· Step 5: Change the issuer address to make sure it is using the ADFS certificate endpoint.

· Step 6: Create a Certificate to Act as Your Client Root Certificate Authority.

· Step 7: Create a Certificate Revocation List File from the Root Certificate.

· Step 8: Install Your Client Root Certificate Authority on the Client and Server Machines

HIS hidden gem: Disable BiDi reversal

Sat, 26 May 2012 11:23:34 +0000

Recently I was working on a very complex integration project using BizTalk Server 2010 stack. This project involved integrating with legacy IBM mainframe system. The integration involved working with BizTalk Adapter for Host Applications to call Mainframe hosted applications as part of specific processes. The application mainly involved passing parameters to this COBOL application and getting the results back. Some of the parameters and results returned contained Arabic latters and characters. Arabic is a considered a complex language which requires complex language processor which is called a Transcriber. If you are familiar with the Arabic language it needs to actions to be rendered correctly:

1- The shape of the Arabic letters changes according to their context; meaning that the way we write the latter differs depending upon which letter is before and after the letter we are writing. This phase we call the shaping phase.

2- The second phase is the reversal; Arabic language is written from right to left and hence the characters need to be reversed for it to be rendered correctly. This phase we call the reversal phase.

Usually when you are dealing with Arabic programs and languages you need to execute both these phases for correct display of the sentence. Now HIS 2010 contains a special engine that deals with complex languages such as Arabic this is the TransBIDI.dll and by default this is working as normal and performs both phases as expected.

The problem we faced while integrating with the COBOL program is that it required only the execution of the shaping phase but wanted the letters not to be reversed! Although this (for me) is a strange request and initially I started to think about some sort of customizations and complex scenarios to my surprise I have found that HIS server (starting of HIS 2004) recognized this need and provided a solution to it. Although the KB article I have found talks about the DB2 adapter I found that it applies to all HIS host adapters including the Host Application adapter. Here is an excerpt of the KB article on how to apply this hot fix.

This hotfix enables the CP420PhysicalStorage registry value. You can use this registry value to define how the OLE DB Provider for DB2 treats the string data according to the physical storage. By default, after you apply this hotfix, Host Integration Server 2004 reverses the string data that is sent to the mainframe or received from the mainframe. However, for some mainframe programs, you may not want Host Integration Server 2004 to reverse the string data.
To force Host Integration Server 2004 to process the string data without reversing the data, add the CP420PhysicalStorage registry value to the Host Integration Server 2004-based computer. To create the registry value, follow these steps:
Warning Serious problems might occur if you modify the registry incorrectly by using Registry Editor or by using another method. These problems might require that you reinstall your operating system. Microsoft cannot guarantee that these problems can be solved. Modify the registry at your own risk.

1. Click Start, click Run, type regedit, and then click OK.

2. Locate the following registry key:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Host Integration Server

3. Right-click Host Integration Server, point to New, and then click DWORD Value.

4. Type CP420PhysicalStorage as the name of the registry value, and then press ENTER.

5. Double-click CP420PhysicalStorage, type 1 in the Value data box, and then click OK.

To return Host Integration Server to the default value and reverse the string data, set the CP420PhysicalStorage registry value to 0.

The complete KB article can be found here: http://support.microsoft.com/default.aspx?scid=KB;EN-US;890353

I hope this saves someone else’s day as it saved mine

Solved: Moving SharePoint 2010 Designer Workflows between Sites

Sun, 22 Apr 2012 09:31:09 +0000

In man occasions you would be faced by the need to move workflows you developed using SharePoint designer from one site to another. This might be the case if for example you developed and tested the workflow on a testing environment and now wants to move to the production environment without the need to re-develop the workflow.

The steps to perform this is rather simple:

Export the workflow to Visio.
Edit the VWI archive and delete the configuration file.
Import the workflow back to your destination site.

So open the SharePoint designer and open the source site. click on the workflows link on the left and then the workflow you want to move. Click on the ribbon on export to Visio button.

Now rename the file exported (VWI file) to be a ZIP file and open it with Windows Explorer. You will find a file named “workflow.xoml.wfconfig.xml”

Just delete this file. Now rename the archive back to be a VWI file.

Now open the destination site and click the import from Visio and browse to the edited VWI file. This will allow you to re-associate your workflow as if it was created or exported from Visio rather than the designer while preserving any development made in the workflow it self.

Creating an Active/Active SQL Cluster using Hyper-V: Part3 the Active/Active Configuration

Wed, 18 Apr 2012 09:05:00 +0000

Introduction

Solution Architecture

The solution is fairly simple as per the below configuration.

Setting Active-Active Configuration

Since we need to configure an Active/Active configuration for this cluster and we do not want any instance to depend on other components from the other instance we will have to add another DTC clustered service to the windows cluster. This is to allow the separation of the DTC service between both instances. I will also show you how to configure the SQL service to depend on its relating DTC service instance so that it moves it along with the SQL instance.

Prepare the second DTC cluster

1- Go to the iSCSI target and create or add another shared disk to be used by the second SQL cluster

2- Go to one of the nodes and then open the iSCSI initiator a