Team Individualism (old blog)

New blog home

2006-08-23T11:44:00.000+10:00

Please redirect to:

http://blogs.msdn.com/davidmcg

The new RSS 2.0 feed can be found here:

http://blogs.msdn.com/davidmcg/rss.xml

BizTalk Out of Memory Exceptions

2006-07-17T08:30:00.000+10:00

Interesting articles to read concerning the standard out of memory exception and how to diagnose and reduce the effects of the problem:

The BizTalk service restarts unexpectedly when you map large messages in BizTalk Server 2004

Memory Growth in BizTalk Messaging

How to Capture a Memory Dump of a BizTalk Process

Other related
Format of BizTalk 2004 assemblies: How do artifacts get compiled to .NET assemblies

Format of BizTalk 2004 assemblies: Maps

Sometims putting things in the GAC isn't such a good idea as is suggested here

When discussing with some colleagues I collated the following information.

When you start the application, initially the BizTalk process starts, and consumes some memory. Then, when you start an orchestration, each of the schema and map DLLs, and the orchestration DLLs will be JIT’d and loaded. Memory may not be reclaimed until you cycle the BTS process. Next, instance data is loaded. For an orchestration with a couple of schemas and a map to convert, this is 2x DOM 1x XSLT, plus additional data kept about the XSLT as noted below.

If your memory is fragmented, or the BizTalk memory watermarks are not appropriate, you could well run out of memory even when you have adequate memory available.

Running out of memory and running out of memory address space are two completely different issues. It doesn’t matter how much memory is present in the box, 4Gb, 32Gb or even a terabyte, 32bit processes (processes, not processors) each only have 2Gb of memory addressed to them for usage. In managed code, the .Net runtime may up a proportion of that 2Gb. So in reality, the amount of memory available is more likely to be about 1 to 1.2 Gb. This is one of the reasons way for example ASP.Net app can be run in separate apppools and one of the reasons why BizTalk allows multiple hosts.

So in summary be aware of

1) High and Low WaterMarks (see BizTalk Performance white paper)
2) Available memory (is it hitting a memory limit)
3) Which orchestrations are using the most memory and consider using isolated hosts

Some user groups result in this recommendation on shutdowns:

Not sure why the BTS NT service shuts down; but the best way to monitor
this is using MOM. You can set alerts in MOM to monitor the service status
and trigger various actions depending on the situation.

http://www.microsoft.com/mom/default.mspx

Mac mini Vista Media Center Beta 2 Update

2006-07-11T15:52:00.000+10:00

Hello.

Roughly a month ago I began a project to build my first media center. It began because I finally thought I could purchase an affordable, small and quiet pc - an unlikely choice for me - the mac mini.

After a few obstacles I got Vista on and was quite impressed. I liked it so much I purchased some more ram and now have a pleasent 1Gb and jumped my performance rating from a lowly one to a meagre 2 :)

The big picture was to demonstrate at home to the family that I could pull together all our content onto one machine. Its worked to an extent - the photos are great, videos good and the music is pretty good. My recorded TV and live TV experiences have been a bit poor. Heres my progress to date:

I first purchased a Compro VideoMate U3 which worked great in XP but was slow to initialise and became jerky and unusable after only a few uses. I even had the signal checked by the local antenna specialists (signal perfect).

I then used a colleagues DVico Fusion HDTV Usb and it worked "ok" in Vista but would blue screen when swapping channels - notably between HD and SD. I opted to use the Dvico provided Vista beta drivers.

I thought I could live with the Dvico if is was a bit smaller and didnt have the blue light so I opted to use a Dvico Fusion HDTV Nano. When it works it works well but I am finding each time I switch on I can only use specific channels. Each time a different set of channels do not work. Sometimes no channels work. Again I used the Dvico Vista Beta drivers. Rescaning has the same effect - random channels work fine and others do not.

On every TV tuner:

-setup I have found that resuming from sleep typically stops the Tuner from providing any channels
-scans usually take 10minutes+
-HD signals have always been of poor quality, jerky and mostly unwatchable
nb. other than MCE no other applications are running.

Other observations:

-On occassion audio stops after a change (use of tv tuner. use of video) and I have to disable the audio device and reenable for it to work.
-Usually swapping between video, recorded Tv and the tv tuner can cause instability in MCE for example blank screen or the screen freezing but MCE center still operating.
-When going to online spotlight more often than not the first time I enter I get a white screen. The second time everything works as normal

I am set to try the digitalnow tiny usb2.0 so will post with my findings with this device....

Mac mini Vista Media Center - DVB-T Tuner

2006-06-21T12:29:00.000+10:00

At the weekend I picked up a TV tuner and a Microsoft media center remote.

I selected the tuner based on the following criteria

1) cost (sub $200)
2) support of DVB-T for Australia
3) connector - must have a coax connection (I negated the Ulead tuner for this reason)
4) driver support - the more the better
5) recommendations from forums - reliability/setup/customer support

The shortlist I got down to was:

1) DigitalNow tinyUSB 2 DVB-T Receiver - Highly recommended but out of stock. Looked like an Australia only solution. Customer service questions came back 3 days later
2) Dvico FusionHDTV DVB-T nano - Great Looking, not bad recommendations, good looking web site
3) Compro Video Mate U3 Greta looking site, made in USA and some great reviews (1/2)

I opted for the ComproUsb stick. It looks like a KitKat but black. Does not come with a remote but this suited me fine as I wanted to use the standard media centre control. I bought the two from i-tech who were very helpful and had a great price (<150)

Setup was fine, though the extension cable for USB was certainly handy as the "kitkat" otherwise protrudes out a bit. Vista picked up the device but I had to install the drivers from the CD. After several times playing around I found it made media center crash a few times. This made swapping back and forward from remote to remote dekstop via my laptop frustrating.

Once I installed newer drivers (?) from the Compro site I found it no longer crashed. My current experience is that after sleep mode (the mac mini pulsates the wee light on the front) the tuner won't run unless a recording is played first. One side efefct of installing the drivers was that the mac mini speaker statred to kick in again... so I had to apply the register hack again from my previous post.

I haven't yet been adventurous enough to try setting a program for recording then sleeping media center (to see if it wakes up) but I have managed to tape the Australia vs Brazil game and Bob the Builder in SD. This was very pleasing as the unit had been running for 5-6 hours before automatically recording the program.

I have also ordered some memory from here. I am finding that Vista with media center and the tuner are taking up around 900mb. On a 512Mb setup this could be a reason why media center sometimes stalls before operating.

Friends came round at the weekend and used the media center with ease and this was a result in terms of all the effort.

Be skeptical about (software development) estimates

2006-06-15T16:12:00.000+10:00

Really like this article, which concludes...

"If someone gives you an estimate for project duration, ask them the following questions:

1. Was this estimate made by the team which will actually be doing the work?
2. How long has this team worked together?
3. Has this team measured their own actual capacity and used that measurement for this estimate?
4. How well does the team know the domain and tools to be used?

If the answers to these questions are unsatisfactory, then the estimate is pure fantasy, it is a lie."

At which point should we use BizTalk

2006-06-15T11:12:00.000+10:00

From a business perspective a decision needs to be made when selecting BizTalk as a solution...

What is missing from the whole discussion is this:

At what point should we start using BizTalk?

When deciding on future BizTalk applicability we should perhaps think about the following criteria:

- Where a business process is to be managed/ transparent (ie. not hidden in code)*
- Where business activities are to be monitored and tracked
- Where more than one target system is involved (the n x n-1 argument)
- Where the types of business process may involve business transactions and/or long running processes
- Where a target systems interface or related business process/component may be reused at a later date
- Where integration scalability is considered to be managed separate from the application
- Where integration security is required to be handled in a central place
- Where existing internal and third party customization investment has already been made
- When the delivery time, relative to short and long term goals, is significantly enhanced by using the product

In terms of the longevity of BizTalk as a organisational choice for integration we should consider the following business drivers

- Customer and user benefit drivers
- Industry Drivers
- Cost Drivers

(add to this list legal/legislative reasons)

A great article related to this topic can be found here.
8 Business Drivers for Technology & Architecture decision making
by Mark Carroll, Architect Advisor, Microsoft ANZ

Some steps when you have decided to use BizTalk starter for integration

1) Decide on an appropriate integration topology
2) Use small, discrete request and response messages. Define your schemas first.
3) Use of BizTalk rules engine to decide which data is required
4) Don't neglect the applicability of of asynchronous design early

Feature complete

2006-06-15T11:05:00.000+10:00

I printed a small list of items to check when I believed the development effort was completed (ie. the MSF execution phase)

- test acceptance
- review pass
- demonstration is successful
- documentation is sufficient
- policing/governance is in place
- package/deployment is operational

If every (significant) development effort had this criteria in place then, theortically, quality would be higher, rework would be lower and information would be better shared.

To do this the review process should be well defined (see earlier posts). A basic rewvie process (separate from the development process) could be:

1) determine that the functional specification can be transpoed into development effort
2) determine if the first review has been adhered to, whether standards and best practices have been applied, and what discrepencies are there
3) confirms that the source control contains all lodged work and is packaged according to end product guidelines

BizTalk Mapping functoids - conditional output

2006-06-15T08:34:00.000+10:00

When trying to conditionally output an element (but not any sub elements) there are a few options in BizTalk 2004.

1) use a logical existance functoid

It only "fires" when the source input exists and it spits out a "true" value. When the true value is input into the target element it tells the mapper whether or not to
create the element

warning: logical existance functoid when looping can cause a polarized loop. It may only loop for the first element - check my previous posts

2) use a record count functoid

it fires whether or not the source input element exists and spits out an integer.
Used in combination with a greater than or equals logical functoid you can control the creation of an element

warning: If you are looping and you are looking at a sub element this may return the record count of all sub elements, which may not be what you are looking for

3) use a string size functoid

it fires whether or not the source input element exists and spits out an integer.
Used in combination with a greater than or equals logical functoid you can control the creation of an element.

Unlike option 2 it only fires for the specific sub element. Unlike option 1 it can provide a "does it exist or not" functionality. This is particularly useful when you want to set values of other elements based on non-existance.

Mac mini to Vista Cookbook

2006-06-14T00:48:00.000+10:00

Here is my cookbook for getting Vista onto a Mini mac. I loved the mac mini design and really like Media Center (or at least the concept) so this was an opportunity to learn and convince my wife it was bringing together all the "stuff" we have.

Note: watching movies and pictures in the lounge while listening to your favourite
tunes, whilst cheesy, was in fact very rewarding. The next step in my journery is DVB-T, which I will document later.

Here is my braindump of steps and links I used

- get windows Vista
This proved a challenge over the long weekend. I ended up going to work to pickup a copy. Fortunately working for Microsoft there were plenty to access, but there was plenty of warnings around its
use. Armed with the DVD (single sided) and product key I went forth into mac land.

- update apple firmware AND SMC firmware
bit confusing here but I did get it to work in the end
http://www.123macmini.com/forums/viewtopic.php?p=32442&highlight=

- do the bootcamp dance
Download bootcamp, sign your life away to beta, and create a CD driver disk (requires blank CDR)

- connect to monitor ideally with a dvi or HDMI cable
I found that a composite connection, whilst terrible to read had the side effect of not displaying anything at all.
See here for worrying examples (I thought at this point my project was dead)
I learned that the MAC option key was important in getting my disk to eject. The mac mini has no eject button.
option F12 did the trick.

- delete the 200mb EFI partition
This apparently lets the Vista install actually happen (apparently the EFI partition gets in the way)

- install vista
This tends not to work first time. I rebooted into safe mode with network support (pressing F8). Do not reinstall, just choose the reboot option and alls well.

- getting sound to work
I read about this before installing and I reckoned it would be easy. Nothing ever is. YOu have to install the XP drivers
from the disk and then copy them before the install fails. Yes it fails. I then pointed to the directory based on the
advice and ran the sigmatel exe. Sound now worked. I later read about an easier way of dealing with this:

http://blogs.msdn.com/hans_vb/archive/2006/06/06/619262.aspx

Note: using the command below results in more files and folders being created. Suprisingly this works
"D:\Install Macintosh Drivers for Windows XP.exe” /V /a

- Switch off the internal speaker

As I use the output to goto the monitor/sound system it was freaky getting the tinny output from the mac mini.
Actually, I'm being a bit harsh, the speaker did ok for the size. It boggles the mind that people hack the registry
until they find a solution. Brave heart or foolish - who knows - I'm thankful he did look

http://forum.onmac.net/showthread.php?t=1239
nb. mine was 0000

- Setting up bluetooth
http://blogs.msdn.com/hans_vb/archive/2006/06/06/619262.aspx

- seting up wireless

This was a pain in the butt. And I still working on it. WPA it didnt like, WEP it didn't like, even 802.1 authentication it didn't like. SO unsecured for a day or so. Not good. In the end it was to do with my wireless or ethernet drivers. Note: I am using RDP to get to the machine so it boots me off. I recommend fulling installing the mac supplied drivers.

My (current) two pet dislikes about Vista in beta2:

1) USer Account Control - Understand the rationale, dislike it because when administrator I generally trust myself
2) Start - Program Files - Accessories - Run , I like run, its now hard to get hold of

Links

Mac Mini – Is It The Best Media Center….PC?
Here is where I was convinced it was possible in Australia (the DVB-T mac mini vista pipe dream)
Love the developer evangelists who try it all out. Hans does great commentary

Scrum in management

2006-06-08T12:31:00.000+10:00

Recently I have been using scrum for BizTalk (approximately 6 months). I put together some material for the project manager to understand what I've been doing and what those funny diagrams and terminology I have been using.

Wikipedia definition

A nice basic intro can be found here from the author Ken Schwaber:

After I started using it in our project in January here this article came out (note I was pre Microsoft then)

And these plugins are available for Visual Studio

To be certified (!) you might like to join me at this event

Remove from Windows Assembly Download Cache?

2006-05-11T14:35:00.000+10:00

Q: How do you remove assemblies that have been added to the
C:\Windows\assembly\download cache?

A:

Start>Run

gacutil.exe /cdl

Correlation links

2006-05-03T10:01:00.000+10:00

Been doing some correlation work again and here are some nice links:

Creating an "invisible" promoted property with MessageContextPropertyBase
Promoting BizTalk schema properties into the message context

Property Promotion and Demotion in BizTalk 2004

Nice overview of BizTalk parts (and some nice pictures) from Microsoft MSDN site link

Use of Asynchronous Threading in ASP.Net

2006-04-05T10:32:00.000+10:00

Jasper Lee and I did some work a while back on threading. It proved to be funf and some important lessons were learnt on appropriate use of aysnchronous threading. Specifically how to avoid HTTP 503 errors ("The service is temporarily overloaded")

Improving web services performance by Microsoft Patterns & Practices group

Do Not Use Asynchronous Web Methods When You Depend on Worker Threads

Processing Long Running Tasks With Asynchronous Handlers and XMLHTTP

Use Threads and Build Asynchronous Handlers in Your Server-Side Web Code by Fritz Onion.

.NET Application Performance

Avoid polling for asynchronous invocation results. Polling is inefficient and uses precious processor cycles which can be used by other server threads.

On the other hand

A common approach to handling long-running calls is to have the client poll for results

How To: Submit and Poll for Long-Running Tasks

Of course if I was a bit more with it I would be using Ajax or even Atlas

Review process goals

2006-03-16T14:45:00.000+11:00

In my opinion, a good review process should:
· identify potential problems earlier in the development cycle
· Help team members learn from their more experienced colleagues
· improve quality through adherance to selected standards
production of content that is easier to understand and maintain

These goals exist primarily for two main business reasons:

· to increase productivity in developing new solutions
· maintain a good return on investment for existing solutions

Reviews could be arranegd in order

Lead / Architects review
A formal review to:
· confirm the scope of work

Peer review
An informal review where:
· Document Authors leverage the input / review of their project colleagues and customer counter-parts to product input, feedback and comments.

Paired review

An informal review where the primary developer should enlist the help of a secondary developer to:

· Logic is complex and requires explaining
· To confirm an area of speciality that another developer possesses (thus encouraging the sharing of information
· To highlight a technique or best practices to his/her peers
· A unit of work has been completed (typically a method passes its unit tests)

Group review
A formal review that should be:
· an independently peformed review to provide input, recommendations and experience for a given deliverable, concept or task package

The review process can be much more efficient with the use of regularly applied automated processes.

How do I reuse BizTalk 2004 maps outside of BizTalk?

2006-03-14T17:50:00.000+11:00

Gilles post's the answer here

CMMI vs other standards

2006-03-13T11:56:00.000+11:00

This Web page is especially useful for those comparing CMMI to other standards, approaches, or improvement models.

I particularly like the comparison with agile by Richard Turner from which I have borrowed this excellent summary

BizTalk 2004 Performance

2006-03-13T11:46:00.000+11:00

This Whitepaper on performance, talks about message box configuration.

Apparently 2 message boxes are not liked

An agile approach

2006-02-13T08:47:00.000+11:00

A reminder of agile in a nutshell. The Agile Manifesto 2001 seeks:

- Individuals and interactions over tools and processes
- Working software over comprehensive documentation
- Customer collaboration over contract negotation
- Responding to change over following a plan

Planning is more important than the plan...

The team (ideally small) should:

- Work as one
- Work in short iterations (phases)
- Deliver something each iteration
- Focus on business priorities
- Inspect and adapt

Scrum & Agile practices

2006-02-03T09:32:00.000+11:00

So, I pulled together some agile practices literature from which I have made some suggestions for well defined packagable solution. Agile tends to support short feature driven cycles; it seems appropriate as we try and complete the integration work in the same manner.

I use the word cycle here, as iteration can have the side effect of giving the reader the impression that the same bit of work is continually being repeated. Wheras I appreciate solutions should be continously improved during the execution\development phase, I do also think that each cycle should result in a finite deliverable.

Getting agile… (Martin Fowler)

Xp to Scrum

Advanced Development methods Scrum methodology to aid project management

One key phrase, and universally accepted formula:

business value = f(cost, time, functionality, quality)

Scrum encourages “Value driven projects”. Value driven projects leave the determination of the four variables in the customers' hands throughout the project.
A key feature in a scrum team is self adaptivity. The ability of the individual to adapt to the requirements that the team collectively agrees to deliver.
The first part of self-adaptivity is regular reviews of the process. Usually you do these with every iteration.

At the end of each cycle, have a short meeting and ask yourself the following questions (culled from Norm Kerth)

- What did we do well?
- What have we learned?
- What can we do better?
- What puzzles us?

Things to do

- Create a product backlog
product backlog that has been identified as required for the next release of the product (and therefore can be identified by the output of each cycle)

- Encourage self organisation in meetings (weekly meeting)
The team identifies how requirements can be turned into a product increment that delivers that value during the next iteration

- Produce weekly burn-down report
Where we are and where we have to get to – graphed!

-Set dates for demonstrations (monthly meeting)
Demonstrations increase project sponsor/manager/owner's visibility of where they are by visual inspection. It should also enhance the teams ability to focus an actual outcome rather than a purely technical or theoretical one.

- Set subsequent monthly date for review/planning meeting
Each cycle, if monthly, should be reviewed and the next one planned.

Simple BizTalk 2004 Exception Handling

2005-12-13T16:32:00.000+11:00

Some nice pretty picture in this article on Exception Handling in BizTalk 2004

List of best buys

2005-12-09T15:44:00.000+11:00

Ive always found PC PRO a good source of whats the best "now":

PC PRO A List

Top tips for 2006 - Vista?

2005-12-09T12:10:00.000+11:00

Thanks to Darrell Ryman for the following links

top tips for 2006

Gartner 'clarifies' its Vista upgrade advice

Microsoft to test feature-complete Vista by early 2006

Microsoft's Vista appears to be off schedule

Six IT trends to rock your world
Sandra Rossi

05/12/2005 07:23:16

Gartner last week unveiled six technology trends that will shape the IT industry in 2006 and beyond. The research is part of the Gartner Predicts 2006 Special Report series which will be available at gartner.com in early December.

1. By 2008, 10 percent of companies will require employee-purchased notebooks.

Gartner predicts that notebooks will begin to move from company ownership to personal ownership as prices continue to decline dramatically. Like a car allowance, staff will receive a notebook allowance. This will remove PC assets from the company books.

2. By 2010, wireless will dominate

In 2004, nearly 90 percent of the world's new telecomms connections were mobile. Growth in traditional wired voice connections continue to slow as VoIP grows. Wireless links will represent 99 percent of the world's new voice connections in 2009.

3. The job market for IT specialists will shrink 40 percent by 2010.

The coming decade will see the emergence of IT "versatilists", people whose multidisciplinary assignments, roles and experiences create a valuable blend of synthesized knowledge, competencies and context to fuel business value.

4. Business process outsourcing (BPO) service providers will capture $11 billion of insurance revenue by 2008.

Insurers are turning to external BPO providers to expedite their legacy transformation process.

5. A 50 percent growth in healthcare software investment could enable clinicians to cut the level of preventable deaths in half by 2013. Healthcare has historically under invested in IT; however, this is changing.

6. Throughout 2008, investigation of new technologies will slow as discretionary budgets divert to regulatory compliance.

Regulatory compliance spending is growing at a rate twice that of IT spending, and in many cases, discretionary IT budgets are entirely consumed by compliance efforts, stifling initiatives that are important to business growth.

Microsoft Live

2005-12-09T11:11:00.000+11:00

Since using www.start.com for the past couple of months I hear from Anthony Croaker about Microsoft Live.

"Windows Live and Microsoft Office Live are designed to deliver rich and seamless experiences to individuals and small businesses"

. The replacement of start is here.

The interesting parts so far are the gadgets, hotmail integration and the general improved UI.

Microsoft and UDDI - the future

2005-12-09T11:06:00.000+11:00

Doug Purdy writes about WCF AND UDDI

Business Rules - When and Where

2005-12-08T16:51:00.000+11:00

I will update this section soon with more information and my practical experience with rules over the past 2 years.

A far more useful place to leverage BRE is where you have a series of rule conditions that require evaluation, some of which may impact the resolution of other rules within a policy. Further more if these rules are subject to frequent changes including additions of whole new conditions then the BRE becomes a far more effective place to manage and run the execution of such rules.

Jim Bowyer, Microsoft, Canada

Links:

Risk Scoring with BizTalk Server 2004 and the Business Rules Framework.

http://www.microsoft.com/biztalk/techinfo/whitepapers/2004/business_rules.asp

http://msdn.microsoft.com/library/en-us/sdk/htm/ebiz_sdk_busrules_prog_qpsx.asp?frame=true