Ben Cops

BizTalk DR – marked backups and log shipping process explained

2009-10-19T16:10:00.001+01:00

For some reason the MSDN docs around this are a bit grim – Nick Heppleston has a great explanation of the process here;

Configuring BizTalk Backup for Disaster Recovery – Part 1

64bit XLANG host memory usage, virtualisation overhead & optimising filegroups for the BizTalk databases

2009-09-23T09:59:00.002+01:00

Have a look at Ewan Fairweather’s very interesting webcast on BizTalk 2009 performance. He says they saw a decrease in performance moving from 32bit to 64bit because the default settings for memory allowance for the XLANG hosts (unchanged from 32bit to 64bit) cause excessive dehydration (~70 orchestrations a second, when none should be dehydrated) at memory usage levels appropriate for 64 bit processes. By optimising these he saw a 33% increase in performance, so if you’re running BizTalk on 64bit this is definitely something to look at during the test and tune cycle.

	Default	Optimised
VirtualMemory ThrottlingCritera
OptimalUsage	900	6000
MaximalUsage	1300	6300
PrivateMemory ThrottlingCriteria
OptimalUsage	50	700
MaximalUsage	350	1000
PhysicalMemory ThrottlingCriteria	disabled

Descriptions of the settings here.
Another interesting point is that they showed scaling from 2 to 3, 4 BizTalk servers all doing the same work with one messagebox server caused a performance decrease due to excessive polling on the database. Performance increased for 3+ BizTalk servers once 3 messagebox SQL servers were used.
They also tested physical vs virtual server performance and saw that virtualisation carries the following overhead:

Throughput is down by 13%
Latency is up by 15%

One of the other items pointed out is a SQL script that segregates out various objects in the messagebox schema to different file groups. This caused significant performance improvements.
Optimising Filegroups for the Databases
Also some further links of interest:
BizTalk Server 2009 Performance Optimization Guide
Microsoft BizTalk Server 2009 Operations Guide
BizTalk Server 2009 Hyper-V Guide
Richard Seroter - Interview with Ewan Fairweather

Timezone error in ESB toolkit 2.0 - Input string was not in a correct format

2009-09-08T17:41:00.003+01:00

Well done to Payal for working this one out.
When you look at the portal webpage on a computer set to the Indian timezone offset (5.5) rather than the UK (currently 1) you get the following error

Input string was not in a correct format. 
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. 

Exception Details: System.FormatException: Input string was not in a correct format.

Source Error: 


Line 19:             if (!string.IsNullOrEmpty(tzoqry))
Line 20:                 
Line 21:                 Session["tzo"] =int.Parse ( tzoqry);
Line 22:             if (Session["tzo"] == null) 
Line 23:                 //need this to make it work in production

This is because the timezone is passed in the querystring you're redirected to:

http://localhost/ESB.Portal/Home/Homepage.aspx?tzo=5.5

And tzo=5.5 does not parse to an int. If you look at it on a UK machine you're redirected to

http://localhost/ESB.Portal/Home/Homepage.aspx?tzo=1

And it works...

BUG.

ESB toolkit on 64bit

2009-09-08T10:05:00.001+01:00

I started with this and gave up - better to go with the flow at the time.
However, here's someone who persisted:

Raj on installing ESB Toolkit 2.0 on Win 2003 64 bit OS

BizTalk ESB Toolkit 2.0 install notes

2009-07-13T15:01:00.002+01:00

There’s a few of these about but here’s the list of steps I had to go through to get the thing installed. This was on a from-scratch win2k3 32 bit virtual PC image.

Install win2k3 32bit, win2k3 sp2, sql 2k8, vstudio 2008, run windows update
Install BizTalk 2009, uddi
Install pre-reqs (.net 3.5 sp1, MSChart, vstudio 2008 sp1, visual studio sdk)
Install SQL XML 4.0 SP1 from http://www.microsoft.com/downloads/details.aspx?familyid=228DE03F-3B5A-428A-923F-58A033D316E1&displaylang=en
Install Enterprise Library 4.1 (this appears to have dropped off the list of items to install in the pre-reqs in the documentation but its still required)
Run ESB toolkit 2 installer
Import C:\Program Files\Microsoft BizTalk ESB Toolkit 2.0\Microsoft.Practices.ESB.CORE.msi into the BizTalk admin console then run to install the msi itself
Do the same with Microsoft.Practices.ESB.ExceptionHandling.msi, but set "overwrite resources" on when importing the msi file into the admin console
Do the same with Microsoft.Practices.ESB.JMS.msi, but set "overwrite resources" on when importing the msi file into the admin console - I didn't run this via the command line
Run

set-executionpolicy Unrestricted
From powershell to allow the management tool install script to run

Extract ESBSource.zip to c:\program files\microsoft BizTalk esb toolkit 2.0\ESBSource
Run sn -k Microsoft.Practices.ESB.snk in c:\program files\Microsoft BizTalk esb toolkit 2.0\keys
Run c:\program files\microsoft BizTalk esb toolkit 2.0\ESBSource\Samples\Management Portal\Install\Scripts\Management_Install.cmd
(I had to reinstall MSChart here as it was complaining about not being able to reference the datavisualization controls in system.web.ui - ran the uninstall script, reinstalled MSChart, then ran the install script)
Run c:\program files\microsoft BizTalk esb toolkit 2.0\bin\esbconfigurationtool.exe and configure, applying settings between each page. Set up the SSO configuration store
Deploy the bam definitions using

"c:\program files\microsoft BizTalk Server 2009\tracking\Bm.exe " deploy-all -DefinitionFile:"C:\Program Files\Microsoft BizTalk ESB Toolkit 2.0\Bam\Microsoft.BizTalk.ESB.BAM.Itinerary.xml"

"c:\program files\microsoft BizTalk Server 2009\tracking\Bm.exe " deploy-all -DefinitionFile:"C:\Program Files\Microsoft BizTalk ESB Toolkit 2.0\Bam\Microsoft.BizTalk.ESB.BAM.Exceptions.xml"

C:\Windows\Microsoft.NET\Framework\v3.0\Windows Communication Foundation>ServiceModelReg.exe -r -y

Because the app pool is running as a local account this applies http://support.microsoft.com/default.aspx?scid=kb;en-us;871179
I set the above to Negotiate,NTLM - this cures the 403 - unauthorized error that was being generated when the portal called into the WCF service in the exceptionmanagement service

Also - check this blog out for a load more fixes for the endless problems with the portal website

Storage Bin for BizTalk 2004/2006/R2/2009 - July 2009 archive

SAML

2009-07-13T14:58:00.001+01:00

A presentation giving a great overview on SAML 2.0

http://www.oasis-open.org/committees/download.php/20520/SAMLV2.0-basics-Oct2006.pdf

IIS Authentication and Access Control Diagnostics Tool

2009-06-29T09:31:00.001+01:00

This came in handy debugging the shonky install for the ESB Toolkit 2

Authentication and Access Control Diagnostics 1.0 (x86)

I was getting an error in the event log;

“The remote server returned an error: (401) Unauthorized”

The tool pointed me at the ESB.Exceptions.Service virtual directory. Loading the .svc file in here directly led me to change the authentication provider to “Negotiate,NTLM”

C:\Inetpub\AdminScripts>cscript adsutil.vbs set w3svc/NTAuthenticationProviders
"Negotiate,NTLM"

SSIS Package fails on execute "Integration Services evaluation period has expired"

2009-02-13T11:59:00.002Z

This was a nasty gotcha, requiring a MS support call and some mucking about.
We had a number of SQL servers, half of which would not allow execution of SSIS packages via the .net API (so it seemed), although they would directly via DTExec - it looked like we'd had the eval copy of SSIS only (odd as SQL wasn't eval) installed on half the servers.
Turns out the user executing the package didn't have access to the registry key which tells SSIS which version the software is licensed as.

The registry node in question is under
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\90\DTS\Setup

The nodes Edition and EditionType should say "Enterprise Edition (64-bit)" for the licensed copy (not sure what it would say for the eval copy). But SSIS couldn't read this key and so assumed it was a trial version - permissions to the Users group had been removed as part of the server hardening. Adding read access back for the account in question solved the problem.

BizTalk Administration group hub page hangs

2009-02-05T14:45:00.003Z

Just had the group hub page repeatedly hanging while refreshing the following;
"loading configuration of adapter handlers"

From past experience, enumerating the adapter handlers requires access to the master secret service. Restarting this on our cluster sorted this out.

Error on installing SSO on a clustered server

2008-12-18T18:52:00.002Z

The SSO service account needs Full Control permissions to the cluster as per the following instructions http://msdn.microsoft.com/en-us/library/aa559783.aspx (search for "Full Control")

Otherwise configuring SSO will fail (can't start the service) with the following in the application event log;

The application could not connect to MSDTC because of insufficient
permissions. Please make sure that the identity under which the application
is
running has permission to access the cluster.

Failed to read the needed name objects from the registry. Error Specifics:
d:\nt\com\complus\dtc\dtc\msdtcprx\src\dtcinit.cpp:435, Pid: 2136
No
Callstack,
CmdLine: "C:\Program Files\Common Files\Enterprise Single
Sign-On\ENTSSO.exe"

BizTalk 2009 - Announcement

2008-09-05T10:15:00.002+01:00

This just in from Microsoft;

BizTalk 2009 - Announcement

On Friday Sept. 5, Microsoft will announce features for BizTalk Server 2009 and future releases plans to provide customers and partners with further clarification about what to expect. The two main highlights of this announcement are: BizTalk Server vNext (renaming, timing & features) and future plans (ship rhythm & high-level themes). We know that our enterprise customers need to make longer-term plans about their infrastructure investments, often 5+ years into the future. In response to this we are providing customers with greater visibility into the BizTalk Server release cycle

The goal is to provide a BizTalk Server release approximately every two years, plus additional interim releases of service packs as appropriate. Each full BizTalk Server release will integrate the previous major release with the latest service pack(s) and new functionality. Service packs will incorporate all of the current critical, non-critical, and customer-requested updates into one convenient package that has been extensively regression-tested by Microsoft and by customers during a beta test program. To the greatest extent possible, Microsoft strives to maintain BizTalk Server application compatibility for both full and service pack releases and performs extensive application compatibility tests with each release.

1. BizTalk Server vNext:
o Naming Change: We have also updated the name of the next release from BizTalk Server 2006 R3 to “BizTalk Server 2009”. By calling the product BizTalk Server 2009, we can clearly communicate this is a full product release with new and enhanced capabilities and updated platform support for customers to take full advantage of the latest technology wave (Windows Server 2008, Visual Studio 2008, SQL Server 2008, .NET Framework 3.5).
o BizTalk Server 2009 Timing: We will deliver a public CTP by the end of this calendar year. Additionally, BizTalk Server 2009 is on track for availability in the first half of calendar year 2009.
o Features: We bucket the 2009 release into 5 core feature areas, which are detailed further on the BizTalk roadmap page. These include platform support, SOA & web services, business to business integration, device connectivity and developer and team productivity.

2. Future Plans:
o Ship Rhythm: Microsoft’s commitment to maintain a rhythm of releases roughly every 2 years.
o High-Level Themes: We outlined priorities for the next couple of releases. Including:
§ Developer productivity enhancements (e.g. complex mapping);
§ Enhanced B2B support (e.g. complex trading partner management, expanded industry standards and schemas);
§ Low-latency messaging enhancements and ESB Guidance;
§ Enhanced device support for cross-enterprise asset tracking, enterprise manageability of devices, and key industry standards;
§ Real-time business event visibility through BI / BAM Enhancements; and
§ Integration with the latest new platform capabilities (to take advantage of the latest advances in the .NET Framework, Visual Studio, and Windows Server).

Key Talking Points

BizTalk Roadmap:
• With a clear focus on our customers key enterprise connectivity challenges, Microsoft will continue to invest in BizTalk Server enhancements and maintain a rhythm of releases roughly every 2 years.
• BizTalk Server is the enterprise connectivity solution for the Microsoft application platform, used by customers to connect and interoperate with systems - e.g. LOB systems, legacy systems, smart devices (RFID), B2B (SWIFT, EDI, etc.).
• By providing high-level themes on the next releases of BizTalk Server, Microsoft hopes to give clarity to customer’s technology purchasing decisions and planning for adoption of platform technologies.

BizTalk Server 2009:
· BizTalk Server 2009 will align with the newest Microsoft platform technologies including SQL Server 2008, Windows Server 2008, Visual Studio 2008 and .NET Framework 3.5. Customers can take full advantage of the latest platform capabilities, in particular greater scalability and reliability, new Hyper-V virtualization support, and many advances in the latest developer tools.
· BizTalk Server 2009 also delivers many of the top features that have been requested by our customers, including a new UDDI v3 compliant services registry, new and enhanced LOB adapters (Oracle EBS, SQL), enhanced host systems integration (updates to MQ, CICS, IMS, CICS), a new Mobile RFID platform and management tools, enhanced B2B capabilities (updates to EDI, AS2, SWIFT), enhanced developer and team productivity through ALM integration with TFS and Visual Studio, and a new release of ESB Guidance 2.0 patterns and practices

Q: What new functionally will be delivered in BizTalk Server 2009?
A: One of the key features of BizTalk Server 2009 will be to deliver support for the latest versions of Windows Server, the .NET Framework, Visual Studio, and SQL Server. This will mean that BizTalk Server customers will continue to take full advantage of the platform’s latest improvements including: scalability for mission-critical workloads, improved support for next-generation web and service oriented applications, improved virtualization support and better business insight through Office.
Also, this BizTalk Server release will deliver additional customer-requested capabilities around enterprise connectivity. We will give more updates on specific features in the coming months, but at a high level we are planning some new investments in the release that includes:
New web service registry capabilities with support for UDDI (Universal Description Discovery and Integration) version 3.0
Enhanced service enablement of applications (through new and enhanced adapters for LOB applications, databases, and legacy/host systems)
Enhanced service enablement of “edge” devices through BizTalk RFID Mobile
Enhanced interoperability and connectivity support for industry protocols (like SWIFT, EDI, etc)
SOA patterns and best practices guidance to assist our customer’s implementations

You can find more details about BizTalk Server 2009 at http://www.microsoft.com/biztalk/en/us/roadmap.aspx

Q&A from the Sessions:

Q: Why have you changed the name from R3 to 2009?
A: BizTalk Server 2009 will be a full release of the product. Naming it “BizTalk Server 2009” clearly communicates that it delivers a full upgrade to enable customers to take advantage of the latest platform wave (Windows Server 2008, Visual Studio 2008, SQL Server 2008, .NET Framework 3.5), and also adds a significant number of new customer requested functionality and enhancements.

Q: Does this name change alter the release timeframe for BizTalk Server 2009?
A: No. We are on track to deliver BizTalk Server 2009 in the H1 CY09 timeframe as previously announced. We are very pleased with the progress we’ve been making during our development cycle so far; we’ve already released early CTPs to TAP customers and have been getting good feedback. We will be releasing another CTP during Q4 CY09 and will use this feedback to help validate the features and readiness of the software.

Q: You said the next version of BizTalk Server was “6”, and would be the “Oslo” release for BizTalk Server? Should we assume this is “6”?
A: Yes, this is the sixth version of BizTalk Server; however it is being delivered ahead of Oslo technologies. Based upon the large number of customer requests for compatibility with the 2008 technology wave, we decided that it was more important to be responsive to customer input and ship as soon as the new BizTalk Server 2009 release was ready.

Q: What can customers expect from BizTalk Server in the future?
A: The charter of BizTalk Server remains consistent – it allows the Microsoft application platform to connect and interoperate with other kinds of systems - LOB systems, legacy systems, smart devices (RFID), and B2B integration (SWIFT, EDI, etc.). This has been the focus of BizTalk Server since it was initially released back in 2000 and continues to be its charter going forward.

Q: How does BizTalk Server relate to Oslo? How will BizTalk Server begin to adopt Oslo technologies?
A: ”Oslo” is the codename for Microsoft’s forthcoming modeling platform. Modeling is used across a wide range of domains and allows more people to participate in application design and allows developers to write applications at a much higher level of abstraction. “Oslo” consists of a new modeling tool (which helps people visually interact with models in rich and intuitive manner), a new modeling language (which allows developers to efficiently define domain models in a form that is natural to the author) and a new repository (which provides a shared store for linking together all of the various model artifacts that describe an application across both design and runtime).

BizTalk Server will follow a disciplined, evolutionary path to ease customer adoption of newer platform technologies, while continuing to invest in and enhance the current mature BizTalk architecture.

In fact, you don’t need to upgrade BizTalk Server to take advantage of “Oslo” – current BizTalk Server 2006 R2 or BizTalk Server 2009 customers can benefit from “Oslo” by being able to leverage and compose their services into new composite applications. Because BizTalk Server today already provides the ability to service enable LOB systems or trading partners as web services (using WCF supported protocols), this allows the “Oslo” modeling technologies to compose the services you already have.

Q: Isn’t UDDI part of Windows Server 2008 today? How does the BizTalk UDDI support differ?
A: As part of this next BizTalk Server release, we will be transitioning the packaging/distribution of Microsoft’s UDDI capabilities from Windows Server over to BizTalk Server. Given the close relationship of web service registries with metadata repository technologies, it makes sense to more closely align the distribution and evolution of Microsoft’s registry/repository functionality. As a result, we will be offering an enhanced registry (UDDI v3 compatible) with BizTalk Server’s next release. (Current UDDI v2 customers will receive guidance on how to move to UDDI v3 capabilities in a straight-forward manner.) This enables a core foundational element of your SOA infrastructure, and helps prepare for some of the forthcoming modeling and repository investments planned in our “Oslo” technologies.

Additional Resources

For more information go to:
· PressPass Q & A with Oliver Sharp, GM of BizTalk Server: http://www.microsoft.com/presspass
· BizTalk Website: http://www.microsoft.com/biztalk/
· BizTalk Server Team Blog: http://blogs.msdn.com/biztalk_server_team_blog/
· Steven Martin, director in the Connected Systems Division Blog: http://blogs.msdn.com/stevemar/default.aspx

Immediate deadlock notifications without changing existing code

2008-08-26T08:44:00.000+01:00

Immediate deadlock notifications without changing existing code

The user has not been granted the requested logon type at this machine

2008-07-25T12:56:00.000+01:00

Yes but which type is that?

Check the logon type enum in the error message, conveniently mentioned as an int rather than a description:

Logon Type: 4

and cross reference with the table here: Win32_LogonSession Class (Windows)

Creation of Adapter FILE Configuration Store entries failed

2008-07-14T16:19:00.000+01:00

Thanks Matt: Matt Hall's Blog : BizTalk 2004: ENTSSO Error 0xC0002A18 - although I'm seeing this on 2006 R2.

Why are all configframework error messages so maliciously misleading?


WMI Failed in pAdmInst->Create() in CWMIInstProv::PutInstance(). HR=c0002a18
Creation of Adapter FILE Configuration Store entries failed

The format of the account name is not valid. Domain accounts must include the domain name. Local accounts must not include a domain or computer name.

I just ran into this and was momentarily deflected by talk of MSDTC configurations from google. Matt's blog pointed me at the (simple) answer - I'd put the wrong group as the biztalk application users group and the service account I'd specified was not in the group.

Basic Troubleshooting For Enterprise Single-Sign-On

2008-07-01T09:23:00.000+01:00

One Blog for Biztalk Engineers : Basic Troubleshooting For Enterprise Single-Sign-On (SSO)

Data Scripter Add-in for Management Studio

2008-06-19T09:09:00.001+01:00

A fantastic add-in and one that should have been part of the tool itself in the first place.

Data Scripter Add-in for Management Studio - seanprice

In place upgrading a Windows 2003 domain controller to Windows Server 2008

2008-06-10T07:55:00.000+01:00

Described in detail but not recommended:

The things that are better left unspoken : Upgrading your Active Directory to Windows Server 2008

BizTalk 2006 R2: Troubleshooting Problems with MSDTC

2008-05-20T08:22:00.001+01:00

Troubleshooting Problems with MSDTC

Finally, an authoritative treatment of the DTC configuration required to install and operate BizTalk in a multi-server environment.

Note the provided table of recommended values for various DTC security settings in different scenarios - e.g. Incoming caller authentication, not mutual when MSDTC is running on a cluster.

BAM Setup and Configuration : Troubleshooting in BAM Portal Configuration

2008-05-19T11:22:00.000+01:00

BAM Setup and Configuration : Troubleshooting in BAM Portal Configuration: "IIS is not 32-bit enabled"

IIS 6.0+ must be enabled to run 32-bit Web applications on 64-bit machines in order for the BAM Portal to function. You can enable by running the command:

cscript.exe %windir%\Inetpub\AdminScripts\adsutil.vbs set W3SVC/AppPools/Enable32BitAppOnWin64 1

mattberther.com » Drop all stored procedures

2008-05-01T11:14:00.001+01:00

mattberther.com » Drop all stored procedures:

A useful script to drop all sprocs (prior to updating them all, for example)

USE myDatabase
GO

declare @procName sysname

declare someCursor cursor FOR
SELECT name FROM sysobjects WHERE type = 'P' AND objectproperty(id, 'IsMSShipped') = 0

open someCursor
fetch next FROM someCursor INTO @procName
while @@FETCH_STATUS = 0
begin
exec('drop proc ' + @procName)
fetch next FROM someCursor INTO @procName
end

close someCursor
deallocate someCursor
go

DateTime format string for inserting into SQL strings directly

2008-04-17T16:05:00.003+01:00

To build an insert for a SQL string using datetime literals, the following format string can be used: "yyyy-MM-dd hh:mm:ss"

As in


DateTime.Now.ToString("yyyy-MM-dd hh:mm:ss")

FIX: Error message when you try to enlist two orchestrations at the same time in BizTalk Server 2006 or in BizTalk Server 2006 R2: "Value does not fall within the expected range"

2008-04-15T08:10:00.000+01:00

When using Scott Colestock's excellent deployment framework for BizTalk, you start orchestrations by the assembly load. If these are bound to more than one host (fairly common), the first time you start them you'll run into the following error:

Value does not fall within the expected range

I read about a MS fix for this on Scott's site, duly downloaded it only to find that it wouldn't install - the fix was only for R1 of 2006 and hadn't been included in R2. A trip to MS support yielded a developed, tested and released hotfix in less than a month.

FIX: Error message when you try to enlist two orchestrations at the same time in BizTalk Server 2006 or in BizTalk Server 2006 R2: "Value does not fall within the expected range"

Also see Scott's blog on this.

BizTalk 2006 SQL jobs

2008-04-14T12:06:00.000+01:00

This link: Database Structure and Jobs contains a list of all SQL jobs used by BTS2006 and a description.

You might have another few jobs in there;
bam_FRRView_ViewFRRMessagesIn_DelAlertHistJob
bam_FRRView_ViewFRRMessagesOut_DelAlertHistJob
bam_RepairView_ViewMessage_DelAlertHistJob

These are from the SWIFT accelerator - FRR stands for FIN Response Reconciliation.

I like this part best:

In the BizTalk Management (BizTalkMgmtDb) database, there's a stored procedure named dbo.adm_cleanupmgmtdb. DO NOT RUN THIS STORED PROCEDURE! If you do run this stored procedure, all the entries in the database will be deleted.

Brilliant! Don't install the procedure then!

From streams to strings...

2008-04-02T09:18:00.001+01:00

The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) - Joel on Software

An entertaining read on encodings that every pipeline component developer should read too!

Minimum Security User Rights

2008-03-31T10:53:00.001+01:00

Minimum security rights required on a task by task basis in BizTalk 2006 R2.

Minimum Security User Rights

Guidelines for install into a multi-server environment