Kent Weare's BizTalk Blog

BizTalk Light and Easy Viewing Series - SAP Adapter

noreply@blogger.com (Kent Weare) — Mon, 06 Jul 2009 13:09:00 +0000

A couple months back Mick Badran, a BizTalk MVP from Australia, got a few MVPs and other BizTalk experts together to put on a series of webcasts that cover various topics in BizTalk Server 2009.One of the topics that I decided to tackle was the new SAP Adapter ( as found in BizTalk Adapter Pack). In this webcast I cover the following topics:

SAP Background Information
Why do companies use SAP
Challenges with integrating with SAP
Previous SAP Adapters
BizTalk Adapter Pack 2.0
Demos

Alan Smith has launched a site that will host the Light and Easy series in addition to other Microsoft related webcasts. If you are interested in seeing this webcast here is the link: http://cloudtv.cloudapp.net/ViewWebcast.aspx?webcastid=2521554266599869567

If you have any questions or comments about the webcast, feel free to leave a comment on the blog.

BizTalk Light and Easy Viewing Webcast Series - High Availability with BizTalk Server 2009

noreply@blogger.com (Kent Weare) — Mon, 06 Jul 2009 12:49:00 +0000

A couple months back Mick Badran, a BizTalk MVP from Australia, got a few MVPs and other BizTalk experts together to put on a series of webcasts that cover various topics in BizTalk Server 2009.

One of the topics that I decided to tackle was High Availability in BizTalk Server 2009. In this webcast I cover the following topics:

High Availability - What is it?
How do I achieve High Availability with BizTalk Server?
Impact on Adapters and Hosts
Impact on Databases
Role of Enterprise Single Sign On
Deployment scenarios
Changes for BizTalk 2009

Alan Smith has launched a site that will host the Light and Easy series in addition to other Microsoft related webcasts. If you are interested in seeing this webcast here is the link. http://cloudtv.cloudapp.net/ViewWebcast.aspx?webcastid=2521554265716170751

If you have any questions or comments about the webcast, feel free to leave a comment on the blog.

Microsoft Tech Days Canada is back for 2009

noreply@blogger.com (Kent Weare) — Mon, 29 Jun 2009 12:25:00 +0000

Microsoft Tech Days 2009 is coming back to Canada this fall to the following cities:

Vancouver (September 14-15)
Toronto (September 29-30)
Halifax (November 2-3)
Calgary (November 17-18)
Montreal (December 2-3)
Ottawa (December 9-10)
Winnipeg (December 15-16)

Looks like Riderville (Regina) is off the list this year if you are from Saskatchewan you will want to head to Calgary or Winnipeg.

There is currently an early bird special special of $299 which is a savings of $300. I went last year and you can't go wrong with a Microsoft conference at this price point. The session agendas include a wide variety of topics so whether you are a hardcore developer or IT Pro you are bound to learn something new at this conference.

Follow this link for more details.

Clustering MQ Series and BizTalk Send/Receive

noreply@blogger.com (Kent Weare) — Sat, 20 Jun 2009 16:35:00 +0000

We have a 3rd party application that uses MQ Series as an integration bridge. BizTalk is used to manage a business process and perform message transformation between this system and our ERP.

We have been using this configuration for the past couple years and it came time to upgrade this 3rd party application. This application now supports Websphere MQ 6.0 where as we were previously running Websphere MQ 5.3. Since it was time for an upgrade and we do have a pretty large BizTalk farm with a team that is responsible for supporting both BizTalk and MQ we decided to install MQ 6.0 on this same cluster(as BizTalk)...for better or worse(it is supported).

In our previous configuration we had MQ 5.3 running on an Active/Passive Cluster that did not include BizTalk. When running BizTalk 2006 and wanting to use the MQ Series Adapter, there is a component that you must install on the MQ Series servers. The component that is installed is called MQSAgent*. Now if you are running BizTalk 2004 then the component is called MQSAgent where as if you are running BizTalk 2006 it is called MQSAgent2. According to this doc, the component is still called MQSAgent2 for BizTalk 2009. In our previous configuration, I must admit it was rock solid. We never had any fail over issues whatsoever.

We recently ran into some issues in the Test environment with the new configuration. We found that you needed to ensure that MS Distributed Transaction Coordinator (MS DTC) and MQ Series on the same node in order for it to function correctly. MSDTC is leveraged to support Guaranteed Reliable message delivery.

The specific problem that we ran into was that when we failed the MQ Resource group over to a new node, BizTalk would essentially just hang(with respect to MQ message processing). We couldn't send or receive messages with MQ. I then ran into this kb: The BizTalk Server Adapter for MQSeries version 2.0 no longer retrieves messages from a clustered MQSeries queue manager when the queue manager fails over to a different cluster node. This paragraph accurately describes our scenario:

You may configure the Microsoft BizTalk Server Adapter for MQSeries version 2.0 to receive messages from a clustered MQSeries queue manager. If the queue manager fails over to a different cluster node, the BizTalk Server Adapter for MQSeries no longer retrieves messages from the clustered MQSeries queue. When this behavior occurs, the following event is logged in the Application event log:

Event Type: Warning
Event Source: BizTalk Server 2006
Event Category: BizTalk Server 2006
Event ID: 5740
Date: 6/20/2009Time: 10:16:40 AM
User: N/A
Computer:
Description:The adapter "MQSeries" raised an error message. Details "Error encountered on opening Queue Manager name = ISVC.MSG4.QM.T Reason code = 2059.".
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Later on in the kb it describes a "Workaround" *cough* HACK */cough* that is suppose to terminate the MQSAgent from the node that previously was hosting the MQ/MSDTC Resource group.


Option Explicit
On Error Resume Next

Dim sComputerName, oWMIService, colRunningServices, oService, colProcessList, objProcess

If Wscript.Arguments.Count = 0 Then

 sComputerName = "."
 Call ServStat
 Wscript.Quit
End If

Sub ServStat
Set oWMIService  = GetObject("winmgmts:" _
 & "{impersonationLevel=impersonate}!\\" & sComputerName& "\root\cimv2")
Set colRunningServices = oWMIService.ExecQuery _
 ("Select * from Win32_Service where DisplayName='Distributed Transaction Coordinator'")

For Each oService in colRunningServices
 'Wscript.Echo oService.DisplayName  & VbTab & oService.State
 if (oService.State="Stopped") Then
  'Wscript.Echo "Stopped"
  ' find the dllhost
  Set colProcessList = oWMIService.ExecQuery ("SELECT * FROM Win32_Process WHERE Name = 'DLLHOST.EXE'")

  For Each objProcess in colProcessList
   if inStr(objProcess.CommandLine, "6D06157A-730B-4CB3-BD11-D48AC6B8A4BB")>0 then

    'Wscript.Echo objProcess.ProcessId
    Dim objShell
    Set objShell = CreateObject("WScript.Shell")
    objShell.Run "cmd /k kill -f " & objProcess.ProcessId & "& exit"
    WScript.Quit
   end if
  Next
 end if
Next

End Sub

I have highlighted two issues with this script in red:

The ID 6D06157A-730B-4CB3-BD11-D48AC6B8A4BB that they are referring to in the script is for the MQSAgent (BizTalk 2004) and not BizTalk 2006. The MQSAgent2 ID (BizTalk 2006) is C691D827-19A0-42E2-B5E8-2892401481F5.

The other issue is that the script will issue the following command to kill the MQSAgent# process on the server that use to host the DTC/MQ Resource group. objShell.Run "cmd /k kill -f " & objProcess.ProcessId & "& exit The problem with this is that the kill command is not available on Windows 2003 and I suspect Windows 2008 as well. This command has been replaced with Taskkill. I opted to go a different route since the script they have provided is based upon WMI and VBscript I have replaced this line with objProcess.Terminate()

My testing to this point have been very successful. I have not had to manually intevene whatsoever and have not lost any messages when failing resources over or rebooting servers. The only delay that you will see is if you have messages inflight while you bounce resources is that if the Queue is still down that BizTalk will use the Send Port's configuration to issue retries. This is very standard BizTalk behaviour and is performing as expected.

So what does this script do? It essentially looks to see if the DTC service is running on the node that this script is running on. If the DTC service is running, the script will exit since we still need the MQSAgent to run on this server with DTC. If DTC is not running then it will look for a process called 'DLLHOST.EXE' that has a Process ID (PID) of the MQSAgent. If it finds an instance of this process it will terminate it so that this process does not lock up BizTalk from sending or receiving messages with MQ Series.

This sounds like an opportunity to cluster the MQSAgent application. However, the guidance from Microsoft is to not cluster it:

There is no requirement to cluster the MQSAgent (MQSAgent2) COM+ application that is used with the BizTalk Server MQSeries adapter. To provide high availability for this component, install the component on each cluster node. If the COM+ application stops, the next call from the client will start it.

I don't know...seems pretty clunky to me. I will have to follow up further with support.

Adventures with the HTTP Adapter and Yahoo Finance API

noreply@blogger.com (Kent Weare) — Sun, 07 Jun 2009 16:39:00 +0000

I was asked to investigate what would be involved in connecting BizTalk to a Yahoo Finance "api" in order to retrieve stock quotes. This is not a mission critical application but they wanted to be able to consume this information. The client "COTS" application can consume a WSDL, but not a HTTP response that includes comma delmitted data. So we figured that we could expose this data via a web service. There are certainly many ways to expose this data and this is not the point of this post.

The point of the post is to discuss some of the pitfalls that I ran into when trying to connect to this Yahoo API using the BizTalk HTTP Adapter. At first, I thought the problem was rather trivial, I opened IE, pointed it at the Yahoo URL, included the Stock Ticker and the format that I was interested in. The browser returned a string of data that included the stock quote and the other relevant data.

http://download.finance.yahoo.com/d/quotes.csv?s=MSFT&d=t&f=sl1d1t1c1ohgvj1pp2wern

I then saved a copy of this data into a text file, ran it through the BizTalk Flat file schema wizard, created a Receive Pipeline based upon this schema and now had "typed" data for use inside of BizTalk.

At this point, I was a little unsure of what Yahoo was expecting when I made this Http Request. I created a "dummy" schema which only had a root node and figured that I would submit the request to Yahoo to see what was going to happen. Initially, I had a static send port where I hard coded the URL. The URL was very important since it contains the HTTP Request query parameters. I figured that once I get this working, then I will focus on making it dynamic so that the client application can drive which stock quote is returned.

However, this is when I started to run into issues. When I tried to run my application using this configuration I was prompted with the following response from YAHOO: Missing Symbols List.

Based upon this error, I figured that something was up with the query parameters. Yahoo is expecting something along the lines of ?s=MSFT&d=t&f=sl1d1t1c1ohgvj1pp2wern to be past as part of the HTTP Request. After performing some BING searches someone suggested using a dynamic send port to pass these query parameters in. That didn't help either.

I then decided to open up Fiddler to see what was being passed as a successful request. Fiddler is a tool that can be used to inspect HTTP Requests and Responses.
When you use the Request Builder feature in Fiddler, it will default the HTTP request mode to "GET". It makes sense, but I then thought what if I switch this to POST? Look for the red line towards the bottom of the next image: "Missing Symbols List".

At this point, I am starting to understand the problem a little better. After another Bing search, I found the following document that indicates: "The HTTP send adapter gets messages from BizTalk Server and sends them to a destination URL on an HTTP POST request". Using Fiddler, I was able to determine that using a GET request worked without issue. Now knowing that the BizTalk HTTP Adapter is going to use a POST request, I figured that I needed to be able to get Fiddler to work with a POST request and then get BizTalk to use this same approach when posting data to Yahoo.

I am not going to get into the differences between POST and GET here as it has been done so many times before, but here is a good summary of the differences.

Since the query string is essentially being ignored anyways, I removed it from the URL Address text box. I then copied the query parameters into the "Request Body" text box without the '?'

Success! The next challenge is to get BizTalk to pass this data through the HTTP adapter as a POST request.

I found an old forum post by CranCran77 that discussed sending a message of type RawString to a different website that was also looking for HTTP Get. I have used the RawString class before when sending emails via BizTalk, so I was able to add this class to my project quickly.

In the image above, I have highlighted the "Construct Yahoo Request" Message Assignment shape. Below, I have the details of what is inside this message construct shape. Here I am assigning values to my message body that is of type "RawString". This RawString class has been added to a .Net Helper Assembly.

After the message is sent, I can look in the tracking database and can see that this data was transmitted as part of the message body.

Since the parameters that Yahoo requires are being sent as part of the message body, we may use a static Send Port and do not have to provide a query string.

With the emergence of SOAP and now WCF, the use of the HTTP adapter is limited. But as you can see there are still some "services" that exist on the web that rely upon HTTP. Unfortunately there is not a lot of good documentation on the HTTP adapter so hopefully this post fills in some of the gaps.

Another BizTalk Online Resource - http://www.biztalk247.com/

noreply@blogger.com (Kent Weare) — Fri, 22 May 2009 02:32:00 +0000

I just wanted to plug another good BizTalk resource: http://www.biztalk247.com/. Saravana Kumar has given the site an extensive facelift and it looks great!

On the site, you will find links to Blogs, installation guides, BizTalk posters, Hot News, Web Casts, tutorials and links to lastest in BizTalk books.

While I am in the plugging mood, a book that I have started to read is Richard Seroter's SOA Patterns with BizTalk Server 2009. So far I am a couple chapters in and I am impressed with it. I will post a more comprehensive review once I have completed it.

TechEd 2009 - Day 5

noreply@blogger.com (Kent Weare) — Sat, 16 May 2009 04:56:00 +0000

Friday was the final day for TechEd 2009 North America. There were fewer people around and it had that feeling of writing a final exam on the last day of the semester. I started out the day with BizTalk and ended with a BizTalk RFID session. In between those two sessions I spent some time learning about DUET and WCF. Read the rest of the blog for more details.

Stephen Kaufman had a good session on BizTalk 2009 Application Lifecycle Management (ALM) using the new features of BizTalk 2009 and its integration with Team Foundation Server(TFS). BizTalk is now a first class citizen in the Visual Studio 2008 family and therefore has added benefits when you deploy TFS as well. This includes bug tracking, task management and automated builds to name a few.

Next Stephen ran us through some of the new unit testing capabilities. He went through writing unit tests for both Maps and Schemas. Having the ability to write these tests using Microsoft tools and then having the ability to publish the results to TFS is extremely powerful. Writing and executing unit tests is a good practice to follow. Like most developers I don't mind writing and executing the tests, but I hate documenting the results. If this can be published to a tracking tool automatically then I am all for this type of functionality.

Automating your builds was next on the agenda. If you are using automated builds in BizTalk 2006, then chances are you have a Build server with Visual Studio installed on it. This is no longer required as there is an option in the BizTalk 2009 installation that allows you to specify only the build components. This allows you to execute build scripts using MSBuild without the need of visual studio. I strongly suggest automating your builds and deployments. The benefits are consistency, repeatability and times savings. For those of you who were around in the BizTalk 2004 days know how much of a pain deployment was back then. Having a scripted deployment is truly a breath of fresh air compared to those days.

The DUET session was good, however I walked away a little discouraged with the complexity of the DUET architecture. Being a Microsoft developer at an organization that uses SAP as a system of record forces you to find different ways to integrate with SAP. From a middleware perspective, this path is very clear: BizTalk. However, there are situations when BizTalk may not be the clear cut choice. I would like to think of these types of scenarios to involve User to System interactions and potentially human workflow scenarios. BizTalk is very good with asynchronous scenarios. While it certainly can function in synchronous scenarios, you need to be careful in order to prevent client side timeouts.

The demos were impressive: the speaker showed a few scenarios where a user would be working in Outlook and had the ability to book time against project codes, submit a leave request and generate reports. In scenarios where you need approval, those requests will get forwarded to the appropriate person(Manager) as configured in SAP. The interactions were very smooth and responsive.

Now for the discouraging part…the amount of infrastructure and pre-requisites to get DUET functioning is significant. It would only be fair at this point to disclose that I am far from a DUET expert, but this is the way I interpreted the requirements:

Client Computers:
DUET Client
Local SQL Server Express (yes – must have)
Hidden Mail Folder

Exchange Server
DUET Service Mail Box

SAP Server
Enterprise Services need to be enabled
DUET Add on
Additional ABAP modules

DUET Server
IIS – Microsoft DUET Server Components
J2EE – SAP DUET Server Components
SQL Server

So as you can see there is a tremendous amount of infrastructure required in order to get DUET up and running. What also concerns me is the skill set in order to implement these functions. In my experience, I have not met too many people who understand both of the SAP and Microsoft technology stacks. The other thing that tends to happen on these types of projects is the finger pointing that occurs between the two technology groups. Within DUET, at least from my perspective, the line between roles and responsibilities becomes very blurry. The only way that I can see these types of implementations succeeding is to have a “DUET team” that is made up of both SAP and Microsoft resources and their mission is to ensure of the success of the technology. If you leave these two teams segregated I think you are in for a very long ride on a bumpy road.

Perhaps I am jumping to conclusions here, but I would love to hear any real world experiences if anyone is willing to share.

The next session I attended was a WCF session put on by Jon Flanders. In case you haven’t heard of Jon, he is a Connected Systems Guru having worked a lot with BizTalk, WF, WCF and Restful Services. He is also a Microsoft MVP, trainer at Pluralsight and an accomplished author.

He took a bit of a gamble in his session, but I think it paid off. He essentially wiped the slate clean and prompted the audience on the topics that we would like more info on. He did ensure that the topics listed in the abstract were covered just to ensure that no one felt slighted.
If you follow Jon’s work, you will quickly find out that he is very big supporter of Restful Services. It was really interesting to gain more insight on why he is such a proponent of the technology.

When you think about the various bindings that are available in WCF, you tend to think that the basicHttpBinding is usually the “safe” bet. Meaning that it allows for the most interoperability between your service and a variety of clients. These clients may be running on JAVA or even older versions of .Net (think asmx). Jon quickly changed my way of thinking with regards to interoperability. The webHttpBinding is truly the most interoperable binding. There was a bit of a sidebar jabbering between Jon and Clemens Vasters regarding this statement, but I will leave that one alone for now. The rational that Jon used was that http and XML are extremely pervasive across nearly every modern platform. He gave an example from his consulting experience in that a customer had a version of Linux and they were trying to connect to an ASMX web service via SOAP. In order to get this scenario working, they had to start including some hacks so that the client and service could communicate. When they brought Jon in, he convinced them to change the service to a RESTful service and once they had done that there were no more interoperability challenges. It was a very good scenario that he described and it certainly opened my eyes to REST.

I do consider myself to be more of a Contract First type of guy. Especially when communicating with external parties. Having recently communicated with a really big internet company over REST, I was frustrated at times because it wasn’t explicitly clear what my responsibility as a client was when submitting messages to their service. Sure there was a high level design document that described the various fields and a sample xml message, but what did I have to ensure that I was constructing my message according to their specification? It also wasn’t clear what the response message looked like. At first I wasn’t even sure if it was two way service? So yes some of this isn’t the fault of REST itself, but it does highlight that there can be a lot of ambiguity. Something that SOAP provides via WSDL is that the responsibilities are very explicit. When working with external parties, and especially when tight timelines are involved, I do like the explicit nature of SOAP. Now certainly with making contracts explicit you have challenges with iterations as a change to the service payload may slow down the process. The service developer needs to update his WSDL and then provide it to the client developer.

But all in all it was a very good session, and I am happy to say that I did learn to appreciate REST more so than I previously had.

The final session of the day was a BizTalk RFID session put on by BizTalk MVP Winson Woo. I had a little exposure to RFID previously, but he was able to fill in the gaps for me and I learned a lot from him in that 1h15.

BizTalk RFID is an application that comes with BizTalk, but it is not tightly coupled with BizTalk Server whatsoever. It does not hook into the message box or anything like that. As Winson put it, BizTalk Server is where the real value of RFID comes into play. I am not trying to downplay the role of BizTalk RFID, but it is essentially just reading tags. RFID readers have been out for years so there is nothing earth shattering about this. However, once you have read the data, having the ability to wrap this functionality around business process management and rules engine execution is where the value is really extracted.

Having the ability to read a tag, update your ERP system and send a message to a downstream provider is where this technology is impressive. A sample scenario could be some product being sent from a value add supplier to a retailer. The retailer wants to know when they can expect this product because the quicker they can get their hands on it, the quicker they can sell it. So as a pallet of widgets leaves the warehouse, an RFID reader detects this and pushes the data to BizTalk RFID via TCPIP. These readers essentially are network resolvable devices. On the BizTalk RFID server you are able to create what I will call a “mini-workflow” via event handlers. Within this “mini-workflow” you may write an event handler that will compose a message and sent it to BizTalk using WCF. You may also write this data to the RFID SQL Server Database. If you didn’t want to use WCF to when getting these RFID Reads, BizTalk could always poll the BizTalk RFID database as well. Once BizTalk has this data, it is business as usual. If you need to update your ERP, you would compose a message and send it to the ERP using the appropriate adapter. If you need to construct an EDI message and submit that message to your retailer via ASN you able to do that as well.

In scenarios where you have a hand held reader, you have a couple communication options. These readers will be network resolvable using TCP/IP as well, but in the event that you cannot communicate with the BizTalk RFID system, a store and forward persistence mechanism will maintain a record of all of your reads so that when you place the handheld reader into its cradle these records will be synchronized in the order that they were read.

As these reader devices evolve, so does the monitoring and tooling capabilities. SCOM and SCCM are able to determine the health of readers and push out updates to them. This is a great story as you no longer need to be running around a warehouse trying to determine whether a reader is functioning or not.

So that was TechEd 2009 in a nutshell. I hope that you have enjoyed following this series. You could tell that the tough economic climate had an effect on the attendance and atmosphere this year. There was no night at Universal studio which was a little unfortunate. I had a good time when we did this at TechEd 2007 and PDC 2008. All in all, I was happy with my TechEd 2009 experience. It was also a good opportunity to catch up with some of my MVP buddies and the BizTalk product team.

TechEd 2009 - Day 4

noreply@blogger.com (Kent Weare) — Thu, 14 May 2009 22:29:00 +0000

Thursday ended up being a great day for sessions. The two top sessions for me were "Enhancing the SAP User Experience: Building Rich Composite Applications in Microsoft Office SharePoint Server 2007 Using the BizTalk Adapter Pack" and "SOA319 Interconnect and Orchestrate Services and Applications with Microsoft .NET Services"

Enhancing the SAP User Experience: Building Rich Composite Applications in Microsoft Office SharePoint Server 2007 Using the BizTalk Adapter Pack
In this session Chris Kabat and Naresh Koka demonstrated the various ways of exchanging data between SAP and other Microsoft technologies.

Why would you want to extract data from SAP - can't you do everything in SAP?
SAP systems tend to be mission critical, sources of truth or systems of records. Bottom line they tend to be very important. However, it is not practical to expect that all information in the enterprise is contained in SAP. You may have acquired a company that used different software, you may have an industry specific application where a SAP module doesn't exist or you may have decided that building an application on a different platform was more cost effective. Microsoft is widely deployed across many enterprises making it an ideal candidate to interoperate with SAP. Microsoft's technologies tend to be easy to use, quick to build and deploy and generally have a lower TCO. (Total Cost of Ownership). Both Microsoft and SAP have recognized this and have formed a partnership to ensure of interoperability.

How can Microsoft connect with SAP?
The 4 ways that they discussed included:

RFC/BAPI calls from .Net
RFC/BAPI calls hosted in IIS
RFC/BAPI calls from BizTalk Server
.Net Data Providers for SAP

Most of their discussion involved using the BizTalk Adapter Pack 2.0 when communicating with SAP. In case you were not aware, this Adapter Pack can be used in and outside of BizTalk. They demonstrated both of these scenarios.

Best Practice
A best practice that they described was using a canonical contract(or schema) when exposing SAP data through a Service. I completely agree with this technique as you are abstracting some of the complexity away from downstream clients. You are also limiting the coupling between SAP and a consumer of your service. SAP segments/nodes/field names are not very user friendly. If you wanted a Sharepoint app or .Net app to consume your service, you shouldn't have to delegate that pain of figuring out what AUFNR(for example) means to them. Instead you should use a business friendly term like OrderNumber.

.Net or BizTalk, how do I choose?
Since the BizTalk adapter pack can be used inside or outside of BizTalk, how do you decide where to use it? This decision can be very subjective. If you already own BizTalk, have the team to develop and support the interface then it makes sense to leverage those skills and infrastructure that you have in house to build the application with BizTalk. You can also build out a service catalogue, using this approach, that allows other applications to leverage these services as well. The scale out story in BizTalk is very good so you do not have to be too concerned with a service that will be sparingly used initially and then mutates into a mission critical service that is used by many other consuming applications. Next thing you know this service can't scale and your client apps have now broke because they cannot connect. Another benefit of using BizTalk is the canonical example that I previously described. Mapping your canonical schema values to your SAP values is every easy. All you have to do is drag a line from your source schema to your destination schema.

If you do not have BizTalk, or resources to support this scenario then leveraging the Adapter pack outside of BizTalk is definitely an acceptable practice. In many ways this type of a decision comes down to your organizations' appetite to build vs buy.

From a development perspective the meta data generation is very similar. Navigating through the SAP catalogue is the same no matter whether you are connecting with BizTalk or .Net. The end result is that you are going to get schemas generated for the BizTalk solution vs code for the .Net solution.

SOA319 Interconnect and Orchestrate Services and Applications with Microsoft .NET Services
Clemens Vasters' session on .Net Services was well done. I saw him speak at PDC and he didn't disappoint again. He gave an introduction demo and explanation about the relay service and direct connect service. Even though these demos are console applications, if you sit back and think about what he is demonstrating it blows your mind. Another demo he gave involved a blog site that he was hosting on his laptop. The blog was publicly accessible because he registered his application in the cloud. This allowed the audience to hit his cloud address over the internet, but it was really his laptop that serviced the web page request. As he put it, "he didn't talk to anyone in order to make any network configuration arrangements". This was all made possible by him having an application that established a connection with the cloud and listened for any requests being made.

As mentioned in my Day 1 post, the .Net Services team has been working hard on some enhancements to the bus. The changes mainly address issues that "sparsely connected receivers" may have. What this means is that if you have a receiver that has problems maintaining a reliable connection to the cloud, that you may need to add some durability.

So how do you add durability to the cloud? Two ways (currently):

Routers
Queues

Routers have a built in buffer that will continue to retry and connect to the downstream end point. Where as a Queue will persist the message, but the message needs to be pulled from the endpoint. So Routers push and Queues need to be pulled from.

Another interesting feature of the Router is dealing with bandwidth distribution scenarios. Lets say you are exchanging a lot of information between a client who has a lot of bandwidth (say a T1) and someone with little bandwidth (say a 128 kbps). The system that has a lot of bandwidth will overwhelm the system with the little bandwidth. Another way to look at this is someone who "drinks from a fire hose". So by using buffers, the Router is able to effectively deal with this unfair distribution of bandwidth by only providing as much data to the downstream application as it can handle. At some point the downstream endpoint should be able to catch up with the upstream system once the message generation process starts to slow down.

Routers also have a multi-cast feature. You can think of this much like a UDP Multicast scenario. Best efforts are made to distribute the message to all subscribers, but there is no durability built in. However, I just mentioned that a Router that is configured to have one subscriber, has the ability to take advantage of a buffer. There is nothing from stopping you from multi-casting to a set of routers and therefore you are able to achieve durability.

A feature of Queues that I found interesting was the two modes that they operate in. The first is a destructive receive where the client pulls the message and it is deleted...no turning back. The second mode has the receiver connecting, locking the message so that it can cannot be pulled from another source, once the message is retrieved, the client then issues the delete command when its ready.

Pricing
I am sure it happens in every Azure presentation, and this one was no different, that pricing comes up. We didn't get any hard facts, but were told that Microsoft's pricing should be competitive with other competing offerings. Both bandwidth and cost per message will be part of the equation. So when you are streaming large messages you are best off looking at direct connections. Direct connections are established initially as a relay, but while the relay is taking place, the .Net Service bus performs some NAT probing. Approximately 80% of the time the .Net Service bus is able to determine the correct settings that allow for a direct connection to be established. This improves the speed of the data exchange as Microsoft no longer becomes an intermediary. It also reduces the amount of money that the data exchange costs since you are connecting directly with the other party.

Workflow
At this point, it looks like Workflow in the cloud will remain in CTP mode. The feedback that the Product Team received, strongly encouraged them to deliver .Net 4.0 workflow in the cloud instead of releasing Cloud WF based upon the 3.5 version. The .Net Services team is trying to do the right thing once; so they are going to wait for .Net 4.o workflow to finish cooking before the make it a core offering in the .Net Service stack.

TechEd 2009 - Day 3

noreply@blogger.com (Kent Weare) — Thu, 14 May 2009 16:29:00 +0000

Day 3 was a bit of a lighter day for me. I spent most of my time "in the clouds" attending sessions and having some good conversations at the .Net Services booth...thanks Clemens.

In the Architecting Solutions for the Cloud session, Clemens provided an introduction to the .Net Service bus. For those not familiar with the .Net Service Bus it is essentially an Internet Service bus that allows you to exchange information with other parties using the Cloud as an intermediary. Another way of thinking about it is building a bridge between two islands, only the islands in this case are applications. The real power of the .Net Service bus are the WCF based relay bindings. These bindings allow endpoints to make outbound connections to the Bus and then listen for messages. This makes punching holes in your Firewall a task of the past. Very Cool!!! As mentioned on my Day 1 blog there are a couple new features as part of the March CTP that allow for more capabilities in the bus including Routers and Queues. Look for more information on this in my upcoming Day 4 blog.

When prompted for a comment about private clouds, the response was a very clear in that it is not going to happen anytime soon. The reason for this is that to build up your own "private" cloud would be too cost prohibitive. People shouldn't underestimate the complexity involved in building such an elastic, dynamic platform like Azure.

The second half of the session dealt more with deploying your Web Applications to the cloud. People have been hosting their web applications with Application Server Providers for years, what is the difference with Azure? Having the ability to efficiently scale would be an answer that I would have. If you had a viral marketing project underway and you were not sure just how much bandwidth, or processing power, that you are going to need then having an environment like Azure that can scale your app in minutes is a great option. The other thing to consider is that you can scale you application tiers independently. By establishing Web and Worker roles you can allocate resources to serving pages versus doing the back ground work. So if you were doing a lot of number crunching in the back end and the Web Requests were lightweight you can configure your application to suite your needs. I doubt that there are very many ASPs that can provide you this type of granularity.

In the demos they showed how easy it was to work with the local Azure Dev Fabric and then how easy it was to deploy to the cloud. I would expect to see some more tooling around this experience that allows for scripted deployments plus some delegated administration in the cloud. Currently, the developer working on the cloud application is the only one who can deploy the project to the cloud. Obviously this methodology would not fly in many corporate environments, but this is something that they are aware of and have on their task list to work on.

When working with ASP.Net web apps, be sure to use ASP.Net Web projects instead of ASP.Net Web Sites if you have plans of deploying them to the cloud. The Cloud does not support the "Web Site" flavour.

Here are some upcoming dates to look for albeit they are not "solid" at this point:

Pricing - August 2009
Reliability/SLA - shooting for August 2009, but may slide
Launch - targeting PDC time frame release (November 17th - 20th 2009)

TechEd 2009 - Day 2

noreply@blogger.com (Kent Weare) — Wed, 13 May 2009 18:13:00 +0000

On Day 2 I spent some more time looking into BizTalk 2009 and System Center(SCOM).

The ESB Guidance 2.0 presentation from Brian Loesgen provided some excellent insight into the latest version of the ESB Guidance 2.0 CTP. I am going highlight some of the interesting bits of info that I picked up. For more details regarding ESB Guidance check out the codeplex site. You may not want to bookmark that site because it won't be living there very long. You will have to read the rest of the blog to get the punchline.

What is ESB Guidance 2.0?
It is an initiative of the Microsoft Patterns and Practices team that provides architectural guidance, patterns and practices. The building blocks that are in the package are reusable blocks of code that compliment BizTalk Server. They do not replace BizTalk servers but they allow you to use BizTalk in a "BUS" mode instead of the more traditional Hub and Spoke Model.

Why 2.0?
There was a previous version called 1.0 that left some people wanting more. The initial stab of the kit was known for a tedious installation, pushed some of the itinerary decisions onto the client and lacked some of the tooling that developers were asking for. I am happy to say that these issues have been resolved. Brian indicated that his install only took around 15 minutes instead of hours or even days with the old version. They have done a great job on this version.

Features
The itinerary designer was pretty slick. When you install ESB Guidance, an additional toolbox will be included in Visual Studio which allows you to drag these ESB related shapes onto your orchestration designer. You then are able to configure these shapes within Visual studio. I tend to look at this as if you are configuring a workflow for a message. You have a particular series of events and you want to configure it to encounter. The sum of all of these events are essentially your itinerary. So for example you may receive a message and as part of this message's interaction, you need it to be transformed to a new message format, have it passed to an orchestration for additional processing only to be transformed to an additional format on its way out the door. Instead of tightly coupling this solution within a series of maps and orchestration(s) you essentially are configuring the itinerary which will instruct BizTalk what to do with the file. BizTalk will use .Net components, that are included in the Guidance kit, to perform all of the map and orchestration resolutions at run time.

Additional info:

Only available for BizTalk Server 2009
Provides extensibility points so you can customize to meet your needs
More prescriptive guidance is available in this version
Samples of popular scenarios in SDK
Itineraries can now be published to an XML file or SQL Server repository. This worked very well. From within Visual Studio you can push the itinerary to SQL Server with a click of a mouse. You can use deploy multiple versions of the itinerary to the repository and by default the newest version will get executed.

Big News
It was announced at TechEd 2009 that the ESB Guidance will be making its way into the product offering. Starting in June, it will be known as the BizTalk Server ESB Toolkit. It will be signed code from Microsoft and available via MSDN Download centre. Private fixes will be available via MS Connect site and support will be available via Microsoft Premiere Services. There will be no additional charges for the toolkit when you have a BizTalk license.

SCOM (System Center Operations Manager)
I decided to attend a session that was a little outside my comfort zone. I have had exposure to both MOM (Microsoft Operations Manager) and its successor SCOM through my involvement with BizTalk. We have used both of these technologies to inform our team of any issues that are occurring on the BizTalk Servers. I can't imagine running BizTalk without them.

The session itself was dedicated to Cross Platform Management packs. More specifically, using SCOM to monitor your Unix/Linux operating systems and the applications that run on these platforms.

I was impressed with the experience inside of SCOM. The experience of managing and monitoring these platforms is the same as it is for Windows platform. Under the hood SCOM is issuing commands through SSH that is able to retrieve information from the platform or is able to execute a command on those systems.

In the Windows world an agent is pushed to the server that SCOM is going to be monitoring. The process is very similar for Unix/Linux however the terminology is a little different. In Unix the equivalent of a Windows service is a daemon. So you will find that daemon bits are deployed to these servers much like windows.

Out of the box you will find that SCOM has the addressed the core set of functionality that you would expect. This includes:

File Systems (both physical and logical)
Memory usage
Processor Usage
Network interfaces
Daemon availability

I wasn't able to catch the entire list of supported flavours of Unix/Linux so the following list is not comprehensive:

IBM AIX 5.3, 6.1
HP - 11.2, 11.3
Red Hat ?,?
Solaris 8, 9, 10
SUSE

If you need more visibility than this you can look to some 3rd party packages like Novell's SUSE Linux management pack. For more application specific management packs that run on Unix/Linux you can look to Bridgeways' Management Pack. With the Bridgeways' management pack you can find support for:

Apache Web Servers
PostGres Database
Oracle Database
DB2 Database
MySQL
Apache Application Server
JBOSS Application Server
WebSphere Application Server
Oracle Application Server
BlackBerry Enterprise Server
VM Ware ESX
and more

It was an interesting session. What I found was that Microsoft is very serious in this area. Customers have demanded a composite monitoring solution that allows them to watch their entire enterprise, not just their Windows Servers. Microsoft has stepped up by providing the functionality themselves or by leveraging a 3rd party management pack. Microsoft has also stepped up their game in the support area. They have increased their support capabilities so that when you do have an issue with their Unix/Linux management packs that they will have someone that can speak intelligently about the issue from a Unix/Linux perspective.

Expect SCOM 2007 R2 and these third party management packs to ship June 2009.

TechEd 2009 - Day 1

noreply@blogger.com (Kent Weare) — Tue, 12 May 2009 21:45:00 +0000

Key Note:
Besides re-iterating the theme of "Do more with less" several times throughout the presentation, the big news was that Windows 7 and Windows 2008 R2 will be available for Christmas, at least that is what the current intentions are.

We saw some very interesting demos that displayed some of the synergies between these two products:

MEDV (aka Microsoft Enterprise Desktop Virtualization) provides you the ability to run applications on a new platform that otherwise would not be able to run. For example lets take an application that was built for Windows XP that cannot run on Windows 7 (or Windows Vista for that matter). This type of scenario may kill, or delay, your desktop refresh project until you can either figure out how to run it on Win 7 or rebuild the application. Enter MEDV. Using MEDV allows you to run two Operating systems on one device simultaneously. In the demonstration, they had an application that would not natively run on Win7 but it would run on Windows XP. With a double click of the mouse the application launches before your eyes. I was expecting some significant lag in this application loading due to the fact that it is really running on Windows XP. However it was extremely snappy. Within a second the application was launched and there was no real indication that XP was even running. The differentiator was that the application had a Red outline around it. Pretty cool stuff...I was blown away. For more info on MEDV check out the following link.
APPV (aka Microsoft Application Virtualization ) provides you the ability to stream portions of applications on demand to the end client. In demo they simulated a user logging on to a brand new machine that they had never logged into and opening an Excel spreadsheet. So what's the big deal? Excel was not installed. Once the presenter double clicked on the Excel spreadsheet, Excel bits were brought down to the laptop and the spread sheet opened. This all occurred within a couple minutes. Last time I checked, the installation of Excel took quite a bit longer than that and no reboot was required. For more info on APPV check out the following link.
Branch Cache - Do you have any local branches that may be in remote areas where the bandwidth just isn't there? If so then this may a feature of Windows 7 + Windows 2008 R2 that is for you. When a user downloads a file, or Web page, from say your corporate intranet, this artifact will be cached locally within the branch. So when the next person comes looking for that same resource, it can be downloaded from the branch cache instead of the Intranet. This seems like a good way to increase productivity, reduce user frustrations and save on data communication costs. Check out the following video for more details.

Introduction to BizTalk Server 2009
I only caught the tail end of this presentation but wished that I had seen it all. There were two demos that included connecting BizTalk to the cloud. In Ofer Ashkenazi and Danny Garber's session they demonstrated two cloud scenarios: the first one connected to the Microsoft Live Mesh service and the second connected to the .Net Service Bus by participating in a relay . These adapters are currently not publicly available but I have been told that at some point they should surface.

So why is this important? I can envision several scenarios where BizTalk can be used in conjunction with the cloud. Today if you want to expose BizTalk hosted services outside your organization, you need to get your hands dirty in the DMZ. For many organizations the the risks are significant and can slow down or even stop a project. For some organizations, they will rely upon ISA servers to forward the traffic onto the BizTalk servers, others may install BizTalk App Servers in the DMZ and then poke a hole in the firewall so that BizTalk can speak with SQL Server. Others may implement their own custom proxy( like a Web Service) that will just act as a router. As you can see none of these solutions are that great. By using the .Net Service bus, BizTalk can establish an outbound connection to the bus and subscribe to messages moving through the bus. This way you do not need to open firewalls or introduce new infrastructure components into your DMZ. The other benefit is that you can continue to use the all of the tooling that BizTalk provides out of the box. Yes I could create a WCF endpoint to listen to the .Net Service bus, but then I lose out on many of the benefits that BizTalk already provides.

Programming Microsoft .Net Services
The first part of this session was primarily a review, but the second half more than made up for it. I have read Aaron Skonnard's blog several times but never had the opportunity to hear him speak. He is an excellent presenter and I highly recommend seeing any of his sessions or taking training from him.

The 2 new features in the .Net Service bus are Routers and Queues. Routers provide you the ability to multi cast messages onto multiple subscribers or it provides you the ability to send a message to 1, of many, subscribers. The Queues have been added to provide some durability around messages moving through the bus. If you have a consumer who is not always connected, pushing a message to a queue until they can make a connection provides the sender some additional assurances that their message is safe until a connection can be made. Here is a link to the recently published white papers. I know what I will be doing on the plane ride home.

Ignite Your Career: IT Architecture Career Webcast Series

noreply@blogger.com (Kent Weare) — Sat, 09 May 2009 14:41:00 +0000

UPDATE:
You can now download this discussion from here.

I will be participating in a panel discussion for the IT Architecture Career Webcast Series - Honing Your Experience And Skills For Uncertain Times (SESSION #2 OF 4) on Tuesday, May 12th.

IGNITE YOUR CAREER: IT Architecture Career Webcast Series Overview
With the economy experiencing challenges it is more important than ever for IT architects to manage their career growth and skills – and one excellent way of doing this is to learn and discuss with our peers as well as mentors. The purpose of the Architect Webcast series is to share with architects and other technical decision makers experiences and insights from a panel of leading architects in the IT industry across Canada.

If you are interested in listening on the discussion, you can register here:
https://msevents.microsoft.com/CUI/WebCastEventDetails.aspx?EventID=1032403865&EventCategory=4&culture=en-CA&CountryCode=CA

WebSphere Best on Windows

noreply@blogger.com (Kent Weare) — Sat, 02 May 2009 15:21:00 +0000

I don't think that Steve Martin is bored these days. In March he was caught up in the "Cloud Manifesto" fiasco and now he is right in the middle of a new battle with IBM. No I am not referring to Steve Martin the actor, but rather the senior director of developer platform marketing at Microsoft.

His latest battle involves WebSphere running better on Windows Server 2008 than a high priced IBM AIX system. Martin's claim is that you can get more transactions per second (11000 vs 8000) at about a third of the price. ($87161 vs $260128).

I have no idea whether the claims are true or false, but I do know that Microsoft generally puts a lot of effort into backing their claims. Obviously Steve is pretty comfortable with the claims that have been made, otherwise I am sure he wouldn't be willing to fund a third party bake off as he has mentioned in his blog.

Check out http://www.websphereloveswindows.com/ for more details on how you can effectively run WebSphere on Windows!

Should be interesting to see where this ends up.

New BizTalk HotRod: Issue 6 Q2 2009

noreply@blogger.com (Kent Weare) — Tue, 21 Apr 2009 13:18:00 +0000

The latest edition of the BizTalk HotRod can be downloaded here. In this edition you can find some good info on BRE, AS2, XSLT Mapping, Static code analysis using BizTalkCop and a preview into what looks to be an interesting Admin tool called "Terminator".

BizTalk Server 2009 Hyper-V Guide

noreply@blogger.com (Kent Weare) — Sun, 19 Apr 2009 16:08:00 +0000

Microsoft has recently posted the BizTalk Server 2009 Hyper-V Guide. You can find it here.

Here is a short description of Guide:

"The purpose of this guide is to provide practical guidance for using Microsoft BizTalk Server 2009 with Microsoft Windows Server 2008 Hyper-V. The emphasis is on BizTalk Server, but the performance evaluation methods and performance testing scenarios are useful for analyzing the performance of virtualized server applications in general. This guidance will be of interest to both the IT Pro and Developer communities."

Clustering BizTalk Hosts

noreply@blogger.com (Kent Weare) — Sat, 18 Apr 2009 14:15:00 +0000

I went through this configuration a few years ago before I started blogging. Since I had to build a parallel environment recently, I have decided to post this information.

So why would I want to Cluster a BizTalk Host?
If your BizTalk group needs to be Highly Available, but you want to limit a Host to only have 1 Host instance. The reason that you may want to do this is to prevent message duplication. Message duplication may occur when you have *some* adapters running in multiple Host Instances. Adapters such as POP3, FTP, MSMQ/MSMQT and database adapters in polling scenarios.

For example, if you have two BizTalk servers and they both have a Host Instance on them called "FTP", both Host Instances will trigger a connection to this FTP server. If a file exists on the FTP server that matches your Receive Location's file mask, both of these host instances will attempt to retrieve a copy of this file. The reason for this is that the FTP protocol does not support file locking.

The solution to this problem is to cluster the "FTP" Host so that only one Host instance is capable of running at one time. Since the Host is clustered, it is capable of running on the servers that are configured in your cluster administration configuration. In this "Active/Passive" scenario, the Host instance does exist on the "Passive" node, it is just stopped. The core Windows 2003 Cluster functionality takes care of determining whether the Host Instance is online and if it is not, it will try to start it on another node.

So there is a little background on why you need to do this, I will now go through the process of setting this up inside of BizTalk.

The first thing that you need to do is create a BizTalk Host. You do this the same way you would create a Non-Clustered Host.
You then need to create a Host Instance on every node in your cluster. Otherwise you will get presented with the following error:
Once you have created all of the host instances, go into the "Hosts" menu, find the Host that you want to cluster, right mouse click and select "Cluster".

You will then be prompted to provide the Clustered Resource Group that you want this Host to belong to. A resource group basically allows you to group services, or resources. You can then spread these Resource groups across multiple servers.

Once this is complete, you will see a new "Generic Service" added to the Clustered Resource group that was previously selected.

So what does this look like inside of the BizTalk Admin console? I have included a non-clustered host instance "ServiceProcess" just to illustrate that the icons are different. The next clue is that one "EnterpriseClusteredSQLReceive" Host instance has the caption of (Active) beside it. What this means is that this Host Instance is actively running on node "01A". It is not possible to try and start this host instance on multiple servers at the same time.

So in this post I have over simplified the Windows 2003/2008 clustering process. I have attacked it from the perspective that the OS level clustering has already been put into place. Don't under estimate the effort required to do this. To learn more about this process, check out the following link: Improving Fault Tolerance in BizTalk Server 2006 by Using a Windows Server Cluster.

Also note that BizTalk Host Instance clustering is only available in BizTalk 2006 onward.

Another BizTalk Resource: Microsoft BizTalk Server Developer Center

noreply@blogger.com (Kent Weare) — Wed, 08 Apr 2009 04:20:00 +0000

Just wanted to point out another good BizTalk resource: Microsoft BizTalk Server Developer Center. Here you will find an aggregation of popular BizTalk blogs, Web Casts, Quick Starts, Tutorials and much more. The product team is making a conscious effort to keep this site up to date with content, so you know that you will be able to find something of interest there.

Azure Roadmap

noreply@blogger.com (Kent Weare) — Sun, 05 Apr 2009 20:16:00 +0000

I didn't have the opportunity to attend MIX, but have had a chance to catch some of the Azure Session re-broadcasts.

One session of particular interest was A Lap around Microsoft .NET Services by John Shewchuk. Part of his presentation included the updated Azure Road map. I felt that this bit of info was worth sharing.

As you can see the Microsoft team has come a long way in a short period of time. I find the next milestone to be very interesting: "Pricing & SLA Confirmation". Up until this time, every time that I have heard a question asked that involved either Pricing or SLA, it has been deflected. This is understandable as it is no doubt a complex situation.

Up until this point, it is very safe to say that people are impressed with the Cloud technology that has been presented. Pricing and SLA could very well make or break Azure. If the Pricing is too cost prohibitive then adoption will suffer, if the SLAs are not strong enough...adoption will suffer.

I am also interested in knowing whether the initial "Production" applications running on the platform will have to be "big" enough (read: Profitable) for Microsoft's initial adoption? Will smaller applications, that may not be as profitable for Microsoft, be included in the initial Production release of Azure? I guess time will tell.

Private Azure Clouds?

noreply@blogger.com (Kent Weare) — Sun, 05 Apr 2009 18:17:00 +0000

Lately there has been a lot of discussion on Microsoft's recent announcement to not provide the ability to host your own cloud in your data centre. For some additional background, please check out the following links:

During the PDC time frame(Fall 2008), I started to pay attention to the Azure platform and its offerings. Initially, I thought that this stuff was pretty cool and that it would be something that I would want to host in my enterprise. It wasn't until I started to better understand just what is involved in running a platform like this that my mind has since changed. At PDC, I attended a session called ".NET Services: Messaging Services - Protocols, Protection, and How We Scale" by Clemens Vasters. I believe that it was around this time that my opinion about hosting an on premise cloud started to change.

So I have put together a few reasons why I think it is a good idea to leave it to Microsoft to host the Azure platform.

Complexity - Part of the reason why I would be interested in a cloud offering to begin with is to let someone else take care of the hard part. Windows patching, upgrades and up time are all challenges of operating a highly available solution. If someone can provide me that, then I can focus on solving my core business problem.

Scale and Elasticity - The "pay as you go" model has many benefits, especially during the initial launch of your Cloud applications and services. If you were to build this on your own, how big, or small, do you make your cloud and how quickly can you allocate more resources to an application? Adding new processing capabilities on the Azure platform is as simple as "turning a dial". I suspect that this would be very problematic for many organizations to implement.

Cost - Perhaps if you had a stock pile of cash to build a larger than required data centre, you could. However, in this climate there are not a whole lot of companies that can do this. One company, with the capital to do so would be Microsoft. Generally, environments are planned based on expected requirements + x% for growth. How many companies would have the capital to invest in a platform that gives them the ability to scale out in the fashion that Azure allows you do.

Microsoft also has a Geo-Scale initiative where your applications/data may be stored in several different data centres around the world. The benefit of this is to take advantage of lower latency where you have customers accessing your application from all over the world. Also, avoiding natural disasters and widespread power outages are additional reasons to look for Geo-scale. It was only a few years ago where we had most of Eastern Canada and Eastern United States shut down do to a widespread power outages. Yes, UPS and generators will help, but many cannot withstand multi-day outages. Having Geo-Scale ensures of availability since regional events can be offset by data centres outside the affected areas.

Infancy - The Azure platform is still relatively immature. If Microsoft was about to "productize" this offering, I believe that it would slow down innovation. Microsoft would then have to worry about what other versions of the cloud a customer is running to ensure of backwards compatibility and interoperability. Note, I am looking at this from a platform perspective. I am not letting Microsoft off the hook for ensuring that their APIs/services still function for people who have written applications on top of them.

To look at the other side of the coin, I do feel that there are some valid use cases where someone would want to be able to host an Azure cloud platform:

ISV/ASP - Microsoft has generally worked well with ISV and ASPs in the past. I can see some ASP (Application Service Providers) looking to get on board the next wave of Microsoft Technology and start hosting these applications. For many ASPs, they have been hosting Web Applications for years. Evolving to the Azure platform is a natural progression. While I am sure there are many top notch providers out there, at the present time I am just not sure how many of them would be able to host a platform as big and complex as Azure.

Government - Many government departments, have privacy regulations, or laws, that prevent them from allowing their data to exist in 3rd party data centres. Other constraints may include hosting data in foreign countries. For instance data stored in the United States may be subject to the Patriot act. If your company is outside the United States, you may not want your data subject to the Patriot act. These types of constraints, do make hosting the Azure cloud on premise to be very appealing. You get to leverage the "building blocks" that Microsoft has provided in Azure, yet have total control within your environment.

Conclusion - For me, in my current situation, the Microsoft hosted Azure Platform is more advantageous. If I have requirements to expose, or exchange, data with people outside of my organization, I would prefer to let Microsoft handle the infrastructure related challenges that allow me to focus on solving my business requirements.

BizTalk 2009 - Failed to configure EDI/AS2 Status Reporting functionalities.

noreply@blogger.com (Kent Weare) — Sun, 05 Apr 2009 14:44:00 +0000

In a previous post, I mentioned that BizTalk 2009 has gone RTM. On one of my dev boxes, I have been running the 2009 Beta. I wanted to run the new bits on this same machine. Trying to do an in place upgrade on Beta versions is often discouraged as it can yield unpredicatable results. With this in mind, I decided to un-install the Beta. The un-installation went well and then I knew that I would have to blow away the related BizTalk Database MDF and LDF files. I went ahead and did this prior to configuring the new RTM bits.

Everything went smoothly with configuration except for the configuration of the EDI/AS2 runtime. I ran into this error:

Failed to configure EDI/AS2 Status Reporting functionalities.

Failed to deploy BAM activity definitions. Please make sure that all BAM related Data Transformation Services (DTS) packages are removed along with the BAM databases.

Having not had a lot of exposure to the EDI/AS2 world this error message was rather foreign to me.

Like many solutions, I found what I was looking for with a google search. This seems to be a problem that many people have encountered with BizTalk 2006 R2 as there is a KB article.

I decided to take the "Method 2" route(from MSDN):

Method 2: Use the Bm.exe utility to remove the DTS packages

To resolve this problem, use the Bm.exe utility to remove the DTS packages. To do this, follow these steps:

1. Click Start, click Run, type cmd, and then click OK.

2. Use the CD command to locate the folder where the Bm.exe file is located.Note By default, the Bm.exe file is located in the following folder:
Microsoft BizTalk Server 2006\Tracking

3. At the command prompt, type the following command, and then press ENTER:
Bm.exe remove-all -DefinitionFile:folder\AS2ReportingActivityDefs.xmlNote The folder placeholder is the folder where BizTalk Server 2006 R2 is installed.If you receive an error message that an activity could not be removed, go to step 4.

4. At the command prompt, type the following command, and then press ENTER:
Bm.exe remove-all -DefinitionFile:folder\EdiReportingActivityDefs.xmlNote The folder placeholder represents the folder where BizTalk Server 2006 R2 is installed.If you receive an error message that an activity could not be removed, go to step 5.

5. At the command prompt, type the following command, and then press ENTER:
Bm.exe remove-all -DefinitionFile:folder\EdiReportingActivityIndexes.xmlNote The folder placeholder represents the folder where BizTalk Server 2006 R2 is installed.If you receive an error message that an activity could not be removed, go to step 6.

6. Try to configure the BizTalk Server EDI/AS2 Runtime feature again.

Once I had completed these steps, I was able to finish the EDI/AS2 configuration.

BizTalk 2009 RTM!

noreply@blogger.com (Kent Weare) — Sat, 04 Apr 2009 15:00:00 +0000

While browsing the Microsoft Volume Licensing catalogue I noticed the following:

The links work and I was able to download software for my organization. So how does a person access the links? Well if you have purchased BizTalk licenses in the past and have also bought Software Assurance (SA) then you should be entitled to download and use this software. I would check with your Microsoft Sales rep if you are unsure.

I also decided to check MSDN and it looks like the RTM bits have been posted there as well.

VANCOUVER, BC - DevTeach/SQLTeach June 9th - 11th 2009

noreply@blogger.com (Kent Weare) — Thu, 26 Mar 2009 23:26:00 +0000

DevTeach/SQLTeach Vancouver boasts an impressive conference schedule occurring from June 9th - June 11th.

The conference has tracks geared towards Agile Development, Web Development, .Net, Silverlight, Architecture, SQL Dev and SQL IT.

What is particularly intriguing about this conference is the Education Stimulus package promotion that is currently available. The promotion allows 1 attendee free admission when registering 2 other attendees. Perfect for those companies that want to send multiple people to this conference.

Check the following link for further details.

BizTalk vNext Wish list

noreply@blogger.com (Kent Weare) — Sat, 21 Feb 2009 16:21:00 +0000

So you are probably thinking, BizTalk 2009 isn't even out and you are already posting a wish list? Well yes, I was recently asked the question, so I figured I would share my thoughts with you.

Adapters

An Email Adapter(POP3/Exchange) that supports regular expressions. For instance if you are only interested in retrieving certain messages from a mailbox, you should be able to provide a regular expression against the subject field of the message. If the words in the subject matches you reg ex, then the message would be retrieved.

A native Exchange adapter. I have personally run into this and have also seen several instances of this on the MSDN forms. For many organizations, leaving their servers open to POP3 connections posses a security risk. The way that we solved this problem was limiting POP3 access to certain IP Addresses. This is "ok" for servers, but when I put in a request to have my IP Address added, that was declined as individual users should not have POP3 access. So yes this is more of "my problem" than Microsoft's, but if many others are in the same problem then perhaps adding an Exchange adapter would be beneficial.

FILE Adapter that supports regular expressions. The current adapter does support file masks using '*', '?' etc, but it would be even better if regular expressions were supported. I work in an industry where the names of files are built into the File Exchange specification. Often times this will include ranges of numbers that are considered valid. It is pretty tough to properly constrain without regular expressions. Yes, I know that a Custom Adapter could be built that supports this scenario, however it would be nice if this was included in the core product.

FTP Adapter that supports Temporary folder in ASCII mode. In order for the FTP adapter to support "once only guaranteed delivery", you are required to use a Temporary folder when moving files with the FTP Adapter as described here. The problem is that temporary files are only supported when the Adapter is used in binary mode. The issue is that when are exchanging data between heterogeneous environments like Windows and Unix that these environments use different symbols for carriage return line feeds. Using ASCII mode takes care of the conversions of these symbols. So as you can see we are in a bit of a Catch 22 situation here.

Deployment

Deployment has improved considerably from BizTalk 2004, but I still think that some improvements could be made. The organization that I work for has automated our build and deploy process including deploying a BizTalk project to a multi-node BizTalk group. It would be nice if this type of functionality was part of the core product. For instance when importing an MSI in the BizTalk Admin console if you could "Deploy to Group" that would be a welcomed feature where it does the import to the BizTalkMgmt database once and gac's the dlls on the remaining BizTalk nodes.

Binding files - To be honest I am not exactly sure what I am looking for here but think there just has to be a better way. Obviously the ability to separate code/assemblies from configuration is required. However managing that configuration could be improved. If you have a project with only a few receive locations then binding files is not too big of a deal. However once you start reaching the 10+ receive locations managing this data and the passwords becomes tedious. I personally 'love' having to export the bindings after a deployment only to play the "Find and Replace" game with the password '*'s before checking the bindings into source control. I do this so that the next time I deploy I don't have to manually set passwords during the next deployment. I do recognize that having passwords, in binding files, in clear text is not a great option but if we could somehow improve this scenario it would relieve a lot of headaches.

Design Time

The ability to update a generated schema automagically. For instance if I have generated a schema for an SAP IDoc and the IDoc has now changed, I need to run through the "Add Generated Items" wizard again. If I have renamed or modified the target namespace of the original schema I then need to update this new version of the schema. As an added bonus, a blank orchestration is added to my solution even though I already have a working solution.

The ability to debug an orchestration from Visual Studio. When I first started using BizTalk, I really missed this feature. Prior to working with BizTalk I was a ASP.Net developer and really enjoyed ASP.Net debugging. Especially having worked with classic ASP. I am not sure how they could support this feature, but it would definitely be welcomed.

Management/Operations

The ability to go back and historically look at a message. So yes this functionality does exist today however there is another catch 22 situation. If you have message body tracking enabled, then a copy of the message body will placed in the BizTalkDTADb database. However, in order to keep your BizTalk environment performing well, you need to archive and purge data from this database. The job that takes care of this is the DTA Purge and Archive job. If you neglect to enable this job, or keep your live window open for too long, you are bound to have performance problems. At one point, neglecting this job was one of the top BizTalk tickets to Microsoft support. One option is to take the extracts that this job will output and aggregate them together to create your own Long Term Archive solution. This would alleviate you from any run time issues, however it leaves you with a management problem as you need to manage this yourself. The requirement itself comes from a request to find out "What happened last month to order # 1234567"?

If you have a complex business scenario and need to find all messages related to that instance of the business process, it would be great if you could link all of the related messages to the orchestration that managed this business process. So yes, there are Ids that do link all of these interactions together, but it would be nice if you could easily view all of this information from a tool. This would allow you to quickly view what happened to that particular business process instance. Consider the following interaction, if you needed to view all of the messages for this particular business process, how could you easily achieve this?

ETC

So here are a few items that tend to show up frequently elsewhere

Support for Low Latency scenarios. BizTalk's design includes built in persistence. This is a feature that is used to support guaranteed delivery. While in many scenarios this feature is required, but there is a cost associated. For some scenarios this cost is just too expensive. Having the ability to by pass all of the persistence may satisfy some requirements that are not currently addressed.
An expandable, or larger, expression editor. The argument has always been that if you need a larger expression editor then chances are that you are doing something that you shouldn't be doing. While I agree in principle with this statement, here is something else to consider. If you follow a popular namespace convention that includes using OrganizationName.BusinessUnit.FunctionalName as a namespace in a .Net assembly it does not take too long before you are starting to scroll to the right when calling a static method.
Constructing a new message in an orchestration. We have all been there! You use some sort of questionable approach to create an instance of a message. This may include using a map that does not actually map any data, loading an XML string that matches your schema's format and assigning it to a message. It would be nice to just create a new instance of a message without having to jump through these hacks.

If you have any Wish list items of your own, please post them using the comments feature.

WindowsITPro Magazine - Get a Handle on Windows Performance Analysis

noreply@blogger.com (Kent Weare) — Mon, 16 Feb 2009 19:41:00 +0000

I recently had an opportunity to work with Michael Morales from Microsoft on an article for WindowsITPro magazine. The article focused on Performance Monitoring and my contribution was primarily focused on PAL (Performance Analysis of Logs). If you are unfamiliar with PAL you can read more about it here.

If you are interested in reading the article it is in the February 2009 edition of WindowsIT Pro magazine or you can view the article online here.

BizTalk HotRod - Issue 5, Q1 2009 is now available

noreply@blogger.com (Kent Weare) — Sun, 15 Feb 2009 17:33:00 +0000

The latest version of BizTalk HotRod is available at http://biztalkhotrod.com/default.aspx. The issue is definitely worth reading and covers the following topics:

BizTalk vs Dublin
Unit Testing in BizTalk 2009
BizTalk - Sharepoint integration
WCF + BAM
Operations Management for BizTalk using SCOM
and much more

For those of you who are still unsure where BizTalk and Dublin fit in, the BizTalk vs Dublin is a much read. BizTalk and Dublin definitely have some similar features and capabilities. The author, Stephen Kaufman , provides good scenarios as to which technology stack should be used and when it should be used.

The Unit Testing article provides a great walk through of the new unit testing capabilities that are available in BizTalk 2009. Step by step examples are provided for Testing Schemas, Maps and pipelines.