Development Diaries

Multiple monitors & windows 7

noreply@blogger.com (Ross Dargan) — Tue, 13 Oct 2009 08:56:00 PDT

Windows 7 has a great new feature called "aero snap" that lets you drag a window to the left or right of the screen and have it resize to that position (see here for a video of this http://lifehacker.com/5077728/snap-any-window-to-half-the-screen-size-in-windows-7).

This is great feature except if you have multiple monitors. Areo Snap won't snap correctly to the part of the screen that goes across your monitors.

You can work round this by holding down the windows key and pressing the arrow key for the direction you would like to snap in (e.g. if your window is on monitor 1 then press the start key and right and will will snap to the right part of monitor 1)

Perfect!

P.S. Also don't forget you can press the start key up to maximize, down to minimize and you can use the number keys to select items from the start bar to maximize them (or bring them to the foreground)

jQuery and asp.net

noreply@blogger.com (Ross Dargan) — Thu, 01 Oct 2009 07:53:16 PDT

There are many ways to get asp.net control values from javascript. Here are a couple of ways with pro’s/cons

Normal Javascript:

document.getElementById('ctl00_ContentPlaceHolder1_TextBox1').value

This is massivly hard coded and bad. If you change the content place holder, or refactor this into a user control this will no longer work. But this is quick.

document.getElementById('<%=TextBox1.ClientID %>').value

This is less hard coded which is good, but the code can be a bit tricky to read.

With jQuery:

$('#ctl00_ContentPlaceHolder1_TextBox1')[0].value

Again, just as bad as the first normal Javascript example

$('#<%=TextBox1.ClientID %>')[0].value

This is ok compared to the javascript version

$('input[id$=TextBox1]')[0].value

This is probably my favorite method. This tells jquery to search for an input where the id ends with TextBox1 – be careful you have no other TextBox1 ids on the page!

Note the array indexer. This is because jquery returns an array of all the items it finds. This means if you have a grid with 12 rows you can access all of the textbox1’s using

$('input[id$=TextBox1]')[0].value -> $('input[id$=TextBox1]')[12].value

Now you are probably asking why would you just use the first javascript example. This is reasonable for a text box but say you are using a drop down list – which looks better:

var CustomerInputbox = document.getElementById('<%=ddlCustomer.ClientID %>');

var CustomerID = CustomerInputbox.options[CustomerInputbox.selectedIndex].value;

var CustomerID = $('select[id$=ddlCustomer] option:selected')[0].value

That is all.

Ross

P.S. Learn to love jQuery, resistance is futile.

What don't you know?

noreply@blogger.com (Ross Dargan) — Wed, 20 May 2009 01:32:18 PDT

As part of the review process at Waterstons we have to set objectives to try and achieve throughout the coming year. I want to orientate mine around stuff I didn't know but wanted to know - here is my list:-

Large scale architecture & design: Probably some kind of training course

Cloud Computing: Microsoft have a beta product called Asure which could offer us the ability to produce some very innovative and scalable solutions.

Microsoft enterprise library application blocks: these include code to make the following easier (and best practice): Caching, Cryptography, data access, logging and security.

Code generation: Learn more about the code generation techniques we are using

Sql: My sql skills are purely self taught and there are loads of advanced techniques I just don’t know (not really sure how I would self learn this, I do know I have very little knowledge in SSIS).

Visual Studio 2010 & .net 4.0: This will be with us soon and I couldn’t really tell you what we will be getting (the beta is out now)

ADO.Net data services: a colleague sent an email around about these – I think they could be quite a different way for us to consume sql information.

BI tools: I know the theory behind BI but not the tools

Test Driven Development: The tools could be useful, but I’m really not sold on this

ASP.Net Routing: This technology allows us to create .net web sites which are search engine optimised (SEO)

ASP.NET MVC: This is a different way of developing web applications that is gaining a lot of momentum – could easily become industry standard and Microsoft are putting a lot of money and resources into it.

ASP.Net data driven websites: Creates the scaffolding for data driven sites – should do things like code generation but this is from Microsoft

FX Cop/StyleCop: These tools perform analysis on source code to identify potential coding issues

It's really interesting to write a list like this - it defiantly helps organise your (lack of!) free time a bit better - what would be on your list?

Securing Anonymous web services with Certificates

noreply@blogger.com (Ross Dargan) — Thu, 26 Mar 2009 10:57:46 PDT

We are currently working with a client who requires two BizTalk web services to talk to each other, however because they are in different locations there is no common account we can use to secure the communication. Below shows an example of the setup:

BizTalk Server 1:

This orchestration has an input port (which accepts an xml file) – this message is then passed to a request reply port (which will call a webservice hosted on BizTalk server 2). The response is then passed to the output port (which is saved to an xml file).

BizTalk server 2:

This orchestration is a simple webservice which accepts a message, does something with the contents and returns the new message.

This posting assume you have two orchestrations already talking to each other using wcf without any authentication

Ok that’s the scenario defined – lets get onto the more interesting part!

Configuring BizTalk server 1 (The WCF consumer):

Since we are going to be securing the webservice with certificates lets start by generating the client certificate. Run the following code on the server which will be connecting to the webservice (BizTalk server 1 in my case) in a command prompt:

makecert -r -pe -n "CN=ClientCertName" -b 01/01/2005 -e 01/01/2100 -sky exchange -ss my

To view the certificate type mmc into command prompt, then go to file add/remove snap in, and select Certificates. You should be given an option to select select who you will be managing the certificates for – select “My user account”

Now if you look in the Personal folder you should see your certificate. We will need to install this same certificate in a few other places so right click on it and select export. Ensure you include the private key (you actually don't need the private key on the BizTalk server hosting the webservice, but for this demo I have included it).

Now we need to install this certificate in two different locations. On BizTalk server 1 we need to install it under the user account running the host instance. To do this open up a command prompt and type:

runas /user:DOMAINNAME\USERNAME mmc

then type the password for the user. In the management console again select the certificate add in. You can then import the certificate by right clicking on personal and selecting import.

Now we can configure this server to use the certificate you have added. Go into the send port you have generated and ensure you used the type wcf-wshttp – then click configure. For the address modify the url to use Https, then go to the security tab. Here you want to change the security mode to Transport with transport client type: certificate. Then in the client certificate panel click browse. Here you should be able to select the certificate you have just generated.

That’s the first server configured.

Configuring Server 2 (The WCF Provider):

On the second server go to the receive location and select your WCF port. Click configure and go to Security. Set the security mode to Transport, and the client credential type to Certificate.

Now we need to configure IIS to host the ssl web service.

Firstly we will need to get another certificate to allow ssl to occur – so right click on the web site and go to properties, then Directory Security and click server certificate. Click the “Create a new certificate” and next followed by “Prepare the request now, but send it later” (You may be able to click send the request immediately to an online certification authority depending on your setup) Fill in all the details ensuring the field named “Common Name” is the dns name used to connect to this machine from Server 1. This will generate a text file which you can then give to a certificate authority to get a valid ssl certificate (this will have to be a certificate authority trusted by Server one – so possible a public authority like VeriSign).

Once you have your certificate you should be able to add it by re-running the above wizard.

Now what we need to do is install the client certificate generated on server 1 – ensure this is installed into the Local Computer certificate store so IIS can access it. Remember for security this certificate should not really include the private key.

After you have added your certificate click the edit button shown below

Then select the following “Require Secure Channel”, “Require Client Certificates” and “Enable Certificate Trust List”, then click New...

This Wizard will allow you to specify the client certificates which can be used to access the webservice. Click Add from store then select the client certificate installed above. Click next, give it a friendly name, next and Finish. Then select the certificate list:

You will probably have to restart IIS now before this works correctly (and I had to restart IIS every time I changed the certificate trust list).

And thats it! Transport security with client certificates DONE.

Here are some error messages you might get and how to fix them:

The adapter failed to transmit message going to send port "SendPort1" with URL "https://ServerName/TestWebserviceCommunication/TestWebserviceCommunication_ProcessMessage_SendAndRecieve.svc". It will be retransmitted after the retry interval specified for this Send Port. Details:"System.InvalidOperationException: Cannot find the X.509 certificate using the following search criteria: StoreName 'My', StoreLocation 'CurrentUser', FindType 'FindByThumbprint', FindValue '23D68B9999C21FC93702D259736DD0CC1C752B61'.
   at System.ServiceModel.Security.SecurityUtils.GetCertificateFromStoreCore(StoreName storeName, StoreLocation storeLocation, X509FindType findType, Object findValue, EndpointAddress target, Boolean throwIfMultipleOrNoMatch)
   at System.ServiceModel.Security.SecurityUtils.GetCertificateFromStore(StoreName storeName, StoreLocation storeLocation, X509FindType findType, Object findValue, EndpointAddress target)
   at System.ServiceModel.Security.X509CertificateInitiatorClientCredential.SetCertificate(StoreLocation storeLocation, StoreName storeName, X509FindType findType, Object findValue)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2.ApplyClientCertificate(ClientCredentials clientCredentials)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2.CreateChannelFactory[TChannel](IBaseMessage bizTalkMessage)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2.InitializeValues(IBaseMessage message)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2..ctor(IBaseMessage message, WcfTransmitter`2 transmitter)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfTransmitter`2.GetClientFromCache(String spid, IBaseMessage message)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfAsyncBatch`2.BatchWorker(List`1 messages)".

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

This error is because your client certificate is stored in your cert store, but not the cert store used by the biztalk host instance. Look at the step above showing how to run mmc as another user to fix this problem.

The adapter failed to transmit message going to send port "SendPort1" with URL "https://ServerName/TestWebserviceCommunication/TestWebserviceCommunication_ProcessMessage_SendAndRecieve.svc". It will be retransmitted after the retry interval specified for this Send Port. Details:"System.ServiceModel.Security.MessageSecurityException: The HTTP request was forbidden with client authentication scheme 'Anonymous'. ---> System.Net.WebException: The remote server returned an error: (403) Forbidden.
   at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
   at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelAsyncRequest.CompleteGetResponse(IAsyncResult result)
   --- End of inner exception stack trace ---

Server stack trace: 
   at System.ServiceModel.AsyncResult.End[TAsyncResult](IAsyncResult result)
   at System.ServiceModel.Channels.ServiceChannel.SendAsyncResult.End(SendAsyncResult result)
   at System.ServiceModel.Channels.ServiceChannel.EndCall(String action, Object[] outs, IAsyncResult result)
   at System.ServiceModel.Channels.ServiceChannel.EndRequest(IAsyncResult result)

Exception rethrown at [0]: 
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at System.ServiceModel.Channels.IRequestChannel.EndRequest(IAsyncResult result)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2.RequestCallback(IAsyncResult result)".

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

This error is because the service you have used was not trusted on the Server 2, ensure you have the correct client certificate selected in IIS.

Shrink a transaction log without doing a backup

noreply@blogger.com (Ross Dargan) — Sun, 15 Mar 2009 15:30:23 PDT

If you use Microsoft SQL 2005 (or 2000, and probably 2008) then Sometimes you just have to shrink the transaction log in a hurry especially if you get this error:-

The transaction log for database 'DB NAME' is full. To find out why space in the log cannot be reused, see the log_reuse_wait_desc column in sys.databases

I can never remember the command to truncate the log before you can use the inbuilt tools to shrink it.

In case you have the same problem here is the command:

BACKUP LOG [DBNAME] WITH Truncate_only

you should then be able to shrink the log without issue

P.S. There are reasons why you wouldn't want to do this (mainly around data recovery) so only do this on a test/dev environment.

Thanks

Ross

Useful Tools

noreply@blogger.com (Ross Dargan) — Fri, 07 Nov 2008 13:20:49 PST

Here are my five most useful tools

Live Mesh

A great tool that you probably all know about. Lets you keep files and folders sync'd across platforms (PC, Mac and smartphone at the moment). What I love most about mesh is the direction its heading, the sync bit is pretty cool (but foldershare was sorta doing that anyway) as James blog shows there will be four types of 'things' which can interact with the mesh.

Creators - These are things like digital cameras (even standard digital cameras using eye-fi could use this! how cool).
Consumers - devices like mp3 players and digital picture frames
Rich end points - Pcs and Macs are the main things here, but may also include printers as shown below - this I REALLY like!
Processors - These are more applications that run on the mesh, things like backup to another cloud app.

You can watch the recent PDC talk here: http://channel9.msdn.com/pdc2008/BB35/ on it.

You can also use this to remotely control your pcs from anywhere - its great as it traverses firewalls really well, much more secure than opening your pc remote desktop port to the world!

This is wild speculation but I wonder how long before microsoft's Office Live (http://www.liveside.net/main/archive/2008/10/31/office-in-the-cloud.aspx) will be integrated into your mesh live desktop, and you could print to your home printer which is also in your mesh... very exciting Im sure you will agree!

IDrive

I love this tool. It is essentially an online backup tool which just works. You get 2GB for free or 150gb for $50 (annoyingly the set up recurring billing, and they didn't email me to remind me it would be coming out - even though they swear they did!). The backup and restore speeds are pretty decent, and the "continues backup" mode is a life saver - the first time you recover a file you deleted by accident you know it was worth the £30 ish quid!

IDrive also supports a history mode allowing you to roll back to a previously version - very handy (and this also doesnt come out of your space allowance). Im very impressed with the security, and cant recommend this tool enough!

Acronis True image

This tool is similar to N orton ghost but a lot more user friendly and cheap! For those who dont know Norton Ghost this allows you to take and restore an image of your pc. If you do this once a month your machine will run very quick.

This help prevent issues with activating software multiple times (starting to become more essential with the DRM included in just about everything).

If you use this with Live mesh then after you restore your pc, live mesh will automatically sync your latest files to your pc from the cloud - ideal!

Google Reader

Keep yourself up to date with all your favourite web sites everywhere you go (you can read it on your mobile to!)

My Sip Switch

Im just starting to get into using VOIP at home, and I have loads to learn still, but My Sip Switch is proving to be a very good starting place. I aim to go into a lot more detail soon, but currently Im able to use my sip switch for the following setup:-

I have two numbers from Sipgate, My Sip Switch is registered with both of them. One of the numbers (my personal number) will ring the house phone automatically, the second number will only ring the house phone during allowed hours(I will use this number for companies I dont trust!).

Sipgate Also provides a voicemail service, which get email to you. I am going to use the following tool to be alerted when I get an email:-

maybe a bit tacky, but its very useful to know when you have an email!

My sip gate also allows you to notify of calls via Google Talk, so I can tell when the house phone has called (and who called) when Im at work!

Using Fring I can also receive and make the voip calls via my smartphone, and Phonerlite to get the calls via my PC. For my home Im looking at a few different possible solutions for my home phone including a SPA3102 and a SPA1001. Both cost under £50.

Still havent quite decided how Im going to use for outbound calls, but the great thing is about My Sip Switch is that you are not tied into a particular provider. You can pick which provider you use for which calls, whether you sent your Caller ID or not, and implement redundancy (if one provider fails to call out then you can fail over to another).

Finally you dont need to worry about emergency calls anymore, Sipgate allows you to make 999 calls after you set a home location.

Hope you find some of these tools useful - if you do let me know!

Patterns of Enterprise Application Architecture

noreply@blogger.com (Ross Dargan) — Thu, 11 Sep 2008 13:40:20 PDT

I have just started reading this book and was fascinated. What I loved most about was that I could start identifying patterns I was already using - just didn't know what they were called.

The book starts of as a conversation about the different types of patterns (but always gives a page number where you can get more information) as you get towards the end of the book each pattern is discussed in much more detail (including examples in mostly Java and sometimes .net)

Unfortunately Jave appears to be Martin Fowlers language of choice - the Table Data Gateway pattern has this bit of code:-

public void LoadWhere(String whereClause){
   String commandString=
       String.Format("select * from {0} where {1}", TableName,WhereClause);
   Holder.FillData(commandString, TableName);
}

Now I suspect most people reading this will spot the issue with the above instantly, but if not read up on SQL injection!!! (if you are unsure how devastating a sql injection attack can be then watch this video from TechEd - scary! http://www.microsoft.com/emea/spotlight/sessionh.aspx?videoid=989)

Perhaps I'm being picky, this is definitely only sample code - but it worries me how many people might just use this code and not notice the potential issue.

Anyway - the book is very good and I do recommend reading it - as Martin Fowler him self says

"Since patterns are common solutions to recurring problems, there's a good chance that you have already come across some of them... I'm not claiming to present anything new in this book. Indeed, I claim the opposite, this is a book of (for our industry) old ideas.... An important part of patterns is trying to build a common vocabulary, so you can say that this class is a Remote Facade and other designers will know what you mean"

The common vocabulary is exactly what I have found most useful about this book.

You can find the book on amazon here:

http://tinyurl.com/3ntvlm (ISBN: 0-32101274200)

Getting ASP.NET menus to work with Chrome

noreply@blogger.com (Ross Dargan) — Fri, 05 Sep 2008 02:17:14 PDT

The asp.net menu does a check to see if the browser the client uses can support JavaScript or not. But the check is not a very good one!

You can force the menu to work by overwrite the Page_PreInit method on the page and telling the page that the client target is a modern “uplevel” browser:-

protected void Page_PreInit(object sender, EventArgs e) 
{ 
    if (Page.Request.ServerVariables["http_user_agent"].ToLower().Contains("safari")) 
    { 
        Page.ClientTarget = "uplevel"; 
    }
}

Please note that you have to do this in the page class file, and not in the master page class file. Obviously this means you have to do it for every page – which is not ideal.

To get round this limitation simply make a class which inherits from System.Web.UI.Page, and change all your pages to inherit from this class instead (a find and replace is ideal here!).

RegisterDataItem can only be called during an async postback Error

noreply@blogger.com (Ross Dargan) — Thu, 26 Jun 2008 05:26:02 PDT

I received the above error whilst using a ajax popup extender. Originally the popup extender was within an update panel and worked fine, the second I removed the update panel I started to get the issue.

The problem turned out to be this line:-

PopupStatusImage.Cancel();

which basically tells the popup extender to hide. You can only call .Cancel() or .Commit() if your popup extender is within an update panel!

You can simply check the Script manager to see if its a partial post back if you need to keep the cancel in for some pages etc.

Hope this helps someone.

Learn a new Language

noreply@blogger.com (Ross Dargan) — Thu, 19 Jun 2008 06:56:06 PDT

I started coding in Java at uni about 4 years ago and since then I have been very Microsoft centric. Recently I have been reading Chris Dalbys blog (http://www.yellowpark.net/cdalby/) and bought my self a current cost meter, and am very very tempted to get a SLUG.

However I dont know Linux and really cant code in anything other than C#. So should I learn another language? If so which one!

Java: Know a bit but haven't touched in in nearly 4 years.
PHP: Again learned a bit at uni but not in any depth
C++: Did about 10 minutes at uni of this! maybe something to learn!
Ruby: Quite liked the idea of this dynamic language but was not totally sold on it!
Python: No nothing about python other than its a scripting language!

Last year I had a very interesting conversation with Dave Verner about Ruby on Rails and he said I should aim to learn a new language every year - at the time I thought he was a bit nuts, and he probably thought I was a lost cause - but I can see sense in what he was saying now!

I suspect Im going to try and learn a bit more about Linux so hopefully a language which will run happily in Java makes sense (though Im pretty sure all the above run happily under Linux).

Any suggestions??

Geek Test

noreply@blogger.com (Ross Dargan) — Fri, 13 Jun 2008 15:53:56 PDT

If you find this funny, you are a geek!

DDD Scotland: Session Five

noreply@blogger.com (Ross Dargan) — Sat, 10 May 2008 09:00:44 PDT

Red, Green, Refactor!

Ben Hall

Test->Code -> Refactor

Need to change your mindset

Because the tests have to be written we need to stop and think about what we have to implement. This encourages better design.

Code is a bit less of a black box

Test code 101

[TestFixture]
public class ObjectUnderTestTests
{
    [Test]
    public void MethodUnderTest_State_ExpectedBehaviour()
    {
        string result=Convert.ToString(true)
        Assert.IsTrue(tesult);
    }
}

Look at the frame works xUnit and MbUnit both contain more assertions and more attributes.

The TDD Mindset

Huge shift in development approach

ALWAYS write a test before writing code

always run your test

Fix failing tests as soon as possible

What do we test?

Everything! In isolation

Express how we want to interact with the code

small, isolated section of functionality

Single test should not be testing too much

What not to do!

Showed a demo from Scott Gutheries blog showing a page load doing everything. Not possible to test.

Showed how cruise control can be used to automatically run all tests

You need to have a controller set on top of the repository.

Testing in Isolation: Test Doubles

More flexibility for testing the application.

Dummy Objects, Stubs, Mocks, Fakes.

A Stub acts as a stand in

Mocks are powerful normally development using a framework.

Normal: Business Logic --> Data Layer

Test Dobule: Business Logic -> IData

IData -> DataLayer

IData -> Double (Pretends to be the data layer)

Sample:

We now want to send an email after successfully insertion.

We have an interface which implements SendInsertNotification

The controller then takes a interface in the constructor - the test service then is used.

You can get a hold of a realistic stub email service which will open port 25 etc.

Would we be creating a stub for the database? Recommended not to. But we do still want to be testing in isolation. So in the Business Layer we use the mocked data layer, and in the data layer we use the real implementation.

These could fail for other reasons, like network going down etc.

When testing the database you can use a tear down method which can clean the database and reseed all ids.

You can also create [Setup] code which will initiate everything

Code Coverage

NCover will use the profiler API to monitor what lines of code are being called an those which aren't.

Resouces

TDD By Exampl - Kent Beck

www.codebetter.com

webblogs.asp.net/rosherove

RhinoMocks

TestDrivent.Net

www.NUnit.com

DDD Scotland: Fourth Session

noreply@blogger.com (Ross Dargan) — Sat, 10 May 2008 07:40:20 PDT

Making the leap into advanced SQL

Tony Rogerson

Sub Queries

Sub queries (in the select statement) execute once per each row

Derived tables

select *
from (
    select type, name 
    from sys.objects 
    where type='U'
) as Syst ( type, name)

This can be used to overcome issues with unions etc.

Nested Functions in sql 2000 causes the function to be called twice.

Logically derived tables are a virtual table.

There are no statistics held on them.

Its self contained.

Common Table Expressions

Can be used for recursion

;with cteName (name, type, rowpos)
as (
     select name,type, rowpos = RowNumber() over (partition by type order by name)
     from sys.objects
)

Unlike derived tables this does let you self join e.g.

declare @HowMany int
set @HowMay = 10
with loopIT(i, descr)
AS (
    select i=1, descr=CAST('Anchor' as varchar(100))
    union all
    select i=i+1, descr = CAST('recuresed' as varchar(100))
    from LOOPIT
    Where i<@howmany
    )

select i, descr
from LoopID

Like derived table there are no stats, and they are expanded - but they do allow self join unlike derived tables. By default you can only recurse 100 times - you can extend this with MAXRECURSION.

Temporary tables/Table Variables

#Tables hold stats - you can tell it to refresh them and then the plan is more accurate.

Table Variables are only meant for < 1000 rows - so you get no stats but you do get the option to recompile - option (recompile). However it is not very accurate

The Sql plan uses the order of the column to compute selects not the most optimal index as it would use on a #table.

Table variable is not in memory, even local variables are stored on disk.

Hash tables do cause a compile lock so this causes an overhead, where as table variables dont - so they are better!

Cumulative Totals

You can use sub queries for this

select *,
tytd = (select sum(accounts) where xxx),
mytd = (select sum(accounts) where xxx)
from accounts a

The one pass approach uses the case expression

select *,
tytp = sum(case when xxx then amount else 0),
mytp = sum(case when xxx then amount else 0)
from accounts a

You can use pivots tables also, and CTE tables

Performance

You shouldn't use Duration - it is has a high duration but a low number of reads that means there is a problem somewhere. Reads, CPU are the two columns you should read.

Sub query = 6.5k reads

1 pass = 417 reads

union all = 1200 reads

and the pivot 1201 reads

Running Totals

CTE can do it by adding a row number, then self join look for the previous row and sum as you go along. However the query gets copied into each query + it doesnt filter rows correctly so this costs a lot in reads

Better approach is to copy it into a temporary table, index it then make your CTE read from the temporary table. goes from 20k reads to 7k.

DDD Scotland: Third Session

noreply@blogger.com (Ross Dargan) — Sat, 10 May 2008 05:05:25 PDT

Integration Service in Sql 2008

Allan Mitchell www.SQLDTS.com\ www.SQLIS.com

Behind the scenes: Threading

Multicast Transform generates n copies of the input. This is the same in 2008 but all output are on the same execution Path instead of the same thread.

A raw file means that IIS doesn't have to translate information - this means that its ideal for staging data.

To work around issues with SSIS 2005 you use the union all component. This works because the union all works asynchronously. This causes another execution tree (Another thread). This is not needed in 2008.

Pipeline Limiter

This doesnt sound good because we are moving data! Limiter exists in 2005 so its not going to be slower in 2008 - just you didnt know about it. In 2008 they tell you about it so you now get performance counters.

Why would I want to restrict the pipeline?

Date = buffer = memory.
Memory is reused when the buffer terminates
push back from a component = no reuse of memory = run out of memory!

Dont use select * because it will need to allocate memory in the buffer for something we wont use.

To capture these event in the Feb CTP you need to check the log messages for an attribute starting with "The Component%" Expect this to change tho

You can now use C# in the script component to write/manipulate data.

Lookup Component

Caching Options

Full: Before its used everything gets cached - Slow startup
Partial: Lookup component will try for a match in the cache first, then check the file.
None - The last query is cached but that it. Try to order your data. This is useful for volatile data.

Full cache is fasted but might take longest to cache, and a large amount of memory.

In 2005 these cannot be passed around - so if you want to loop through a directory then it will be cached for every file! This is also not transferable.

In 2008 there is now something called a Cache Transform. This can be used as a destination.

You can cache to:

A file - Its a Raw File
In Memory

This allows the reference to be passed around.

Because you can doesnt mean you should - by default the 2005 way is used.

Row Redirection

If a lookup component gets no match then by default it fails. You have to configure a redirect down the error output to ignore it if this should not be an error.

Now in 2008 we have a no match output option.

Points to note:

Lookup transform is case sensitive when done in cache. Obviously this is not the case in SQL

Data Profiling Task

Profiles data in your database (can do quick setup or verbose). only SQl2000 or above can be profiled.

Side Note: If you are going to Sql server then use the OLE Connection manager or the sql connection manager

This is fantastic for seeing how rubbish the data is.

Useful for identifying distribution of values.

Microsoft have provided a date profile viewer to read the xml produced by the above component. Which is really powerful - you can drill into it quite easily.

This is useful for checking data sent from a provider.

Change Data Capture (CDC)

Billed as an ETL function.

Gives increased efficiency - incremental extractions.

You can identify changed rows and columns.

Essentially transaction replication. It uses the log reader.

A lot of people uses triggers - this doesnt!

Use a sp called sp_cdc_enable_table with parameters to define the table you want to do the change data capture on.

Supports net changes/all changes so you can decide how you need your data (and potentially save space)

DDD Scotland: Second Session

noreply@blogger.com (Ross Dargan) — Sat, 10 May 2008 03:46:04 PDT

[Apologies because my laptop blue screened and windows live writer doesn't do auto recovery:( I REALLY need a new laptop now! - everything up to abusing threads is from memory]

How to Write Crap Code in C#

Ben Lamb www.benlamb.com

Started the demo by showing an application which analyses all off Shakespeare's books to count every word and figure out how often the words are used. Initially he runs it and it takes 2 minutes 1 second which he then explains has a sleep.timer in there set to 2 minutes

Initial program: 1 second

Abusing Exceptions

Initially he checked a dictionary to see if a word existed before updating it. This was then changed to just try and update the dictionary, and if it doest exist handle the exception to add it in.

When he ran this he showed us the CPU maxing out, and performance monitor looking at the number of errors thrown a second (175000 was how many)

Exception flowing: 41 seconds

Ben 1 - Redmond 0

Abusing Threads
You can get a thread by:

Using async callbacks
Threadpool
manually using Thread()

to really screw up a program you have to use Lock()

modify the program so we have one thread for each text file, and then we lock on the dictionary.

Multi Threaded analysers 1 second

Ben 1- Redmond 1

Misuse of the heap (Fun with Strings)

Value objects get stored on the stack (ints etc), reference objects get stored on the head.

Avoid Boxing/UnBoxing
Use Generic classes
Cache Locality
Prefer code that works to fast code.

Garbage collection is designed to clean up code - it can use reference counters. This causes the counter to be locked which can cause performance problems. .net uses mark and sweep which walks through an object tree and looks and everything and object it comes across is marked because its being used. Anything not marked gets removed. Then it sweeps everything to group together memory (defraging basically).

Every time an object goes through a process like this its generation gets marked as either generation 0/1/2 - the ones which are old (gen 2) they get put to the bottom so the garbage collection doesn't worry about it too much. Pinned objects are objects which cant be moved because its unmanaged code - this confuses things a bit.

Garbage collection is not deterministic - but don't bother calling it yourself you just make things worse. Plus things like the virus checker could cause more issues.

Showed a normalise line which basically loops through an array of chars to check it is a letter or a space. String builder is then used to append the values together. He changes this to a string which concats itself with the char.

Ben explains that he uses system.diagnostics.stopwatch to show performance details.

Consider putting timer metrics into apps so you can track down issues.

There are some counters showing garbage collection information. % of time in GC is really what you are after. any more in 10% then you have issues.

String concatenation takes: 1 minute 24 seconds.

Ben 2 - Redmond 1.

DDD Scotland: First Session

noreply@blogger.com (Ross Dargan) — Sat, 10 May 2008 05:32:29 PDT

Web Services: We dont need no stinking web server

Barry Dorrans http://idunno.org/

Agenda

What is WCF
WCF Vs Remoting
SOA, Services, Interfaces
Proving it

Remoting is going, WCF is an open standard (WS*) so much better, Also transport neutral so we can quickly switch from HTTP to TCP/MSMQ etc.

Roman Kiss has a null transport module which allows you to just communicate in memory

WCF Vs Remoting

WCF requires less permissions, runs in medium trust (mostly).
Easier versioning with WCF.
Two way communications (callback contracts)

SOA approach. Write the contracts first.

ABCs of WCF: Address (where to talk), Binding (how to talk), Contract (what we are talking about)

A server endpoint is constructed of Address binding Channel

A service can have multiple bindings - defines the shape of the service and the security.

A contract is defined for the service, data and message - can also create one for errors etc.

MEP: Message Exchange Protocol

Request - Reply
One Way
Duplex

Service Contracts are decorated with the attribute [Service Contract] and the methods with [Operation Contract]

Data contracts are decorated with [DataContract] and properties/functions with [MemberContract] - this gives us control of the message body or header - but ordering allows us to create optional components or to order them.

Step by Step instructions will be published

Create a WCF service library

Visual studio 2008 includes a test client for the library project. This even lets you inspect your request/response header.

Shows a simple?! wcf configuration file. Right clicking on it bring up the WCF configuration editor. This allows you to really easily change settings like the bindings.

WCF doesn't require a webserver to host HTTP web services

With windows 2008 it gets better. In XP/Vista we have to write a host if we dont use IIS with WAS we can host a service under any protocol underneath IIS.

Running through a demo called Rumour. Talks about using a constant file to define the namespace.

Name spaces should include a date to support versioning to prevent breaking changes

showed how you can set isrequired=false but did this on an int... not sure how WCF can handle a value non nullable type not being passed in?: Barry said that this would result in the int being set to a 0.

Barry recommends using Visual Studio 2008 because 2005 doesn't have a test harness.

What have we learned

We need data contracts for versioning. WCF will not send items down unless members are marked as a data contract. Barry claimed that even if you mark data as serializable it wont send it down the link... Im not sure this is correct as im sure I have done this... so I will chase it up

[LIVE DEMO]

Showed the MEX endpoint exposing meta data information.

To stop the service you must do:-

internal static void StopService()

{

if(myServiceHost.Stae == CommuncationState.Faulted)

MyserviceHost.Abort();

else if (myServiceHost.State!= CommincationState.Closed)

MyServiceHost.Close();

}

The reason we need this is because you cant use the using statement. Disposing of the WCF service calls close but if it is in a faulted state then it will throw an exception.

Do try and host WCF in IIS.

Replacing ASMX - double decorate contracts what the action & message names, rename the service and only use basic http binding.

Demonstrated creating a service host by passing in the base address

AddressAccessDeniedException - the process does not have rights to the name space, so that needs to be registered... however you can just run as admin.

This command allows you to register the address:

netsh http add urlacl url=http:\\+:8080\RumourService user=builtin\users

This should be done at install time.

Getting service host information:

endpoint.Address

endpoint.Binding.name

endpoint.Contract.name

Add service reference and discover is clever enough to check local projects (if they have an app.config file). Does this require the mex data?

The app.config then needed to be renamed to stop an error from happening

[My battery ran out even after a full charge - need a new one! so had to resort to manual blogging (pen&paper) - apologies if if this not as accurate]

Error Handling

Need to turn on error propagation on in the web.config. This then send the error message through but still sends it as a fault exception.

You can declare a data context which is decorated with the fault contract which allows the correct error to be send to the client and handled.

This then marshals everything into a soap fault

Barry then did a very quick run through of duplex working.

NO SPECTATORS, ONLY PARTICIPANTS

noreply@blogger.com (Ross Dargan) — Thu, 01 May 2008 03:24:11 PDT

There are loads of great conferences going on in may

Where: Glasgow
How Much: Free!

First up on Saturday the 10th of May is DDD Scotland (mentioned here). This is a free community event with some great tracks and speakers including Barry Dorrans (covering WCF - cant wait), Colin Angus Mackay and whilst I havent heard of Gary Short before he perhaps has the best Bio I have ever read (Gary Short was employed as a canoeist, but was presumed dead after disappearing under mysterious circumstances, only to reappear 5 years later. Gary had used this time to secretly re-train as a software Jedi and now plies his trade throughout the known galaxies, where he sells his skills to the highest bidder. Outside of his dog, his laptop is his best friend; inside of his dog, it's too dark to use it.)

Where: Newcastle
How Much: £599 + VAT

Next up is Thinking Digital run by CodeWorks on the 21st to the 23rd... not so much development - although Steve Clayton will be there talking about Microsoft new Mesh technology - even though he didn't give me an invite!

Where: Newcastle
How Much: Free*

*=May cost a demo/presentation

And finally Bar Camp: North East straight after thinking digital on the 24th and 25th.

The Rules of ~~Fight Club~~ Bar Camp are pretty simple, you have to take part... and that worries me a bit! At work we hold regular knowledge sharing seminars and its a great way to pass on new skills and anyone who knows me will tell you how much I love going on about some new technology - but running a demo/session outside of work is a different kettle of fish!

So I don't really have any idea what will be covered here, but Im really looking forward to meeting other developers from the North East!

Looks like I am going to have a busy May!

DDD Scotland - registration open

noreply@blogger.com (Ross Dargan) — Mon, 17 Mar 2008 04:32:59 PDT

This looks like its going to be a very good community event with some really interesting tracks. You can see the agenda here and sign up here.

Really looking forward to this, just wish there where more events like this in the North East!

Idea for some software

noreply@blogger.com (Ross Dargan) — Sat, 15 Mar 2008 03:59:10 PDT

I have just seen Will Shipley's latest Blog posting (http://wilshipley.com/blog/2008/02/ted2008-part-1-gossip.html) and he says he has had his mac book stolen but he knows his MAC code, so he wants people to look out for it.

Well this got me thinking - what if everyone subscribes their laptop mac code to a website, and if it goes missing they flag it as stolen. Then working with people who provide WIFI services we have a little app which basically checks the list. If a stolen mac address is spotted then the police could be notified.

Just a thought - but I think this could be a really useful tool against laptop theft!

Ross

Internet Explorer 8 Beta 1

noreply@blogger.com (Ross Dargan) — Thu, 06 Mar 2008 08:26:03 PST

As announced at the Mix Keynote speech IE 8 beta 1 is now available here

Everything looks pretty good but I couldn't get the web slices to work with Facebook - it just kept asking me to login.

Also spotted an issue with the asp.net menu control. It throws the following JavaScript exception "htmlfile: Not implemented" when trying to expand a menu item. The bit of code failing is trying to call "item.cells[0]" on a table row (TR). Suppose it is just a beta tho!

[Update]

There actually appears to be quite a few JavaScript issues with this version... defiantly not stable for sites reliant on JavaScript

[/Update]

Im a Dad!

noreply@blogger.com (Ross Dargan) — Wed, 05 Mar 2008 12:16:13 PST

Sorry for not posting for so long... but I am now a dad to Adam William Dargan!

I just had to do a quick post after watching the mix 08 keynote speech (Thanks Paul!).

Some of the highlights included "Deep Zoom" which you can use here: http://memorabilia.hardrock.com/ - Very impressive!

The other topic mentioned was IE 8. I'm really excited about the "Web Slices" technology - I'm glad that Microsoft chose to use MicroFormat technology to implement this rather than some new dodgy tag - shouldn't be too long until someone creates a firefox plugin for this!

Silverlight for Nokia is pretty cool too... but I thought there was going to be another bigger announcement since this was released yesterday!

Read Duncan Rileys live blog of the event over at TechCrunch

Hopefully Ill get back to blogging about WCF soon as possible!

Windows Live Writer... a follow up

noreply@blogger.com (Ross Dargan) — Sun, 17 Feb 2008 10:11:48 PST

Yesterday I blogged about Windows Live Writer and after posting about wcf I have to say I love it! This is a must have program for anyone using Blogger! It handles images no problem at all (which Im sure it had issues with during the beta) - also is pretty nifty with maps!

The only thing missing was a way to insert code easily. Well thankfully Mike Ormond has come up with a great little plugin (http://blogs.msdn.com/mikeormond/archive/2008/02/15/updated-source-code-formatter-for-windows-live-writer.aspx) This makes it incredibly easy to add code and still works with the javescript code viewer I wrote a little while ago.

To add the CSS in your blog to make it work correct take a little peek here: http://blog.the-dargans.co.uk/2007/01/syntax-highlighting-in-blogger.html, and if you want the javascript viewer then take a look here: http://blog.the-dargans.co.uk/2007/02/javascript-code-viewer.html.

Cheers

Ross

WCF: First Steps

noreply@blogger.com (Ross Dargan) — Sun, 17 Feb 2008 02:48:47 PST

WCF is basically a way of allowing applications to communicate. I know that's a very broad description but that is pretty much what the technology allows - the communication method (the bits that actually go down the line) is up to you (http/tcp/Named pipes - or even custom), the message you send is up to you to - normally XML is used... but that is your call, and finally the way the "conversation" takes place is up to you, be it full duplex (like talking on a phone), half duplex (like a walkie-talkie) or One way (like sending a letter).

Getting Started

So lets get started with the first hello world message!

Firstly lets get the plumbing in place. Fire up visual studio and create a console project called client. Now if you are using vista you need to start visual studio again but this time it must run as an administrator (right click on the icon and click run as administrator) - if you are using XP then you can just run it normally. Again create a console project but this time call it server.

Now for both projects add a reference to System.ServiceModel (right click on references, then click add reference

Then select System.ServiceModel and click ok:

And now for simplicity add a "Using System.ServiceModel;" to the top of both your client app and the server app.

Now we are ready to start coding!

Contract

As with Webservices we need to define what is actually going to be transferred. The contract should not change (but there are certain exceptions such as adding method but Ill cover that later).

A WCF contract is simply an interface with a few attributes added. For this simple example we are going to return a message sent to us, so the contract is going to be:-

    [ServiceContract]
    interface IHelloWorldObject
    {
        [OperationContract]
        string ReturnMessage(string strMessage);
    }

Note how we have decorated the interface with a Service Contract and the method with the Operation Contract.

This interface must be copied to both the client and server app (yes a bit manual for now... but we will get to WSDL soon!)

The Implementation

So now we have our contract we must implement it! Nothing to complicated so Ill just copy my implementation - feel free to vary it, as long as you implement this interface the sample code below will still work!

    class HelloWorldObject : IHelloWorldObject
    {
        public HelloWorldObject()
        {
            Console.WriteLine("Object Created");
        }    
        public string ReturnMessage(string strMessage)
        {
            Console.WriteLine("Message Received: " + strMessage);
            return "Return: " + strMessage;
        }
    }

This code is just for the server.

The host (& the bindings)

The host orchestrates the calls to everything but its the bindings that specify how data flows. The default bindings are:-

But then you always have the option of implementing your own custom binding!

(If you want to know more about these bindings then take a look here http://msdn2.microsoft.com/en-us/library/ms731092.aspx)

For this sample we will start using the basic http binding as its perhaps the easiest.

static void Main(string[] args)
{
    Console.WriteLine("Server Started");         
    ServiceHost oHost = new ServiceHost(typeof(HelloWorldObject));
    BasicHttpBinding oHTTP = new BasicHttpBinding();
    oHost.AddServiceEndpoint("Server.IHelloWorldObject", oHTTP, "http://localhost/HelloWorld");
    oHost.Open();
    Console.WriteLine("Server Listening...");
    Console.ReadLine();          
}

Lets walk through this a little. WCF makes a lot of use of reflection to create the object above (HellowWorldObject) so all we do is pass into the service host method the type so it can instantiate it as and when a call comes in.

The binding part is pretty simple - just the default implementation.

Then we add the binding as an endpoint to the host. The method AddServiceEndpoint accepts a string stating the contract this endpoint implements (HelloWorldObject must implement this contract to!), a referance to the binding and finally the address it will listen to - in this sample we will just listen to "http://localhost/HelloWorld"

Then all we do is open the connection. When you run this application you may see this error if you didnt start the server application as an administrator (under Vista):-

HTTP could not register URL http://+/HelloWorld/. Your process does not have access rights to this namespace (see http://go.microsoft.com/fwlink/?LinkId=70353 for details).

Simply close the project and restart it running as an administrator.

The Client

The client makes use of a channel factory - and again this code is pretty straight forward:-

static void Main(string[] args)
{
    BasicHttpBinding oHTTPBinding = new BasicHttpBinding();
    ChannelFactory<IHelloWorldObject> oChannel = new ChannelFactory<IHelloWorldObject>(oHTTPBinding);
    IHelloWorldObject oHi = oChannel.CreateChannel(new EndpointAddress("http://localhost/HelloWorld"));

    string strMessage = "Hello World";
    Console.WriteLine("Sending Message: " + strMessage);
    string strResponse = oHi.ReturnMessage(strMessage);
    Console.WriteLine("Message Received: " + strResponse);
    Console.ReadLine();
}

Again we add the Basic http binding (both client and server have to "talk" the same language).

The channel factory is a generic object which uses the interface so it knows what the contract is. We then create an instance of the interface by calling createchannel and passing in the endpoint we are going to listen to.

Then all we do is call the return message method - and that's it!

The Results

And if we run the client again we get the following from the server:-

Nothing too surprising there except perhaps the fact that the object is create twice. Ill cover how to stop this and manage session in a later tutorial.

The End

Ok that is a very simple look at WCF. I'm hoping this will be the start of a series looking and some more complicated examples but we will see how many people are interested in this first!

You can grab the code form here if you want to download it http://www.box.net/shared/vf0z8er4o8

Windows Live Writer

noreply@blogger.com (Ross Dargan) — Sat, 16 Feb 2008 02:16:44 PST

I figure that given I have just ditched Hotmail completely I should try and swap it for another Microsoft product... so here I go trying Live Writer.

I have no idea if this will work but this interface is very nice (I love how it still brings through your existing tags from Blogger). Also the web preview is just fantastically clever! I am very impressed!!

Feel free to try it yourself from here: http://get.live.com/writer/overview

And it is great to see Software+Services working together so nicely!

Ross

North Pack

noreply@blogger.com (Ross Dargan) — Tue, 09 Dec 2008 02:37:23 PST

Just a quick post to say hello to the guys over at http://northpack.co.uk/ (Thanks Mitch for the heads up!). This is a community to serve the North and whilst all the events at the minute appear to be Leeds and Manchester focused hopefully it wont take long to get something more Newcastle way.

They describe themselves as:

NorthPack is an ambitious project which aims to unite web related
professionals based in the North of England and shine a spotlight on the
Northern community, providing a showcase for talent and the great work that goes
on "up North".

So if that describes you why not sign up and add your blog to there RSS syndicate.

Ross