Firedancer Unleashed!

Arduino UNO: Proto Shield Review

2016-07-31T19:01:00.001+08:00

If you have just gotten yourself an Arduino UNO and wondering "what's next?" - Well, a good start will be to light up some LEDs (a.k.a. The "Helloworld" of electronics). You will of course need to purchase the components (such as LEDs, Resistors, Breadboard, Jumper Wires etc.) separately but if you do not want too much hassle, enter the world of Arduino Shields. You can quickly get started with basic electronic concepts using this Proto Shield designed for the Arduino UNO, which you can get it off cheap from eBay or AliExpress.

This is a very basic (and cheap) version that comes with:

2 x LEDs (wired-up to 2 resistors)
1 x Button
5 x extra 5V female pins
5 x extra GND female pins
1 x 170 pin breadboard (separate piece)

Note: There are of course more sophisticated ones (with more features) that you can find online but they usually cost more. Some vendors may provide this as a kit and you will need to solder everything yourself. This one I got was fully assembled on arrival.

The center of the shield allows you to place a tiny 170-pin breadboad for prototyping or solder in your own components to make things more permanent. Take note that the center design of the shield may vary from vendor-to-vendor and is not standardized. I would recommend that you do not stick the breadboard onto the shield and also to make sure that the breadboard you are placing onto the shield does not have its back metal strips exposed (or you will risk shorting out your shield or worst - the UNO).

A caveat that you should know about this shield is that it is not stackable. If you noticed, the space between the rows of pin headers at the top and bottom are not aligned for the next shield (because the Chinese did not only cloned the board, they also cloned the problem). You can probably fix it by de-soldering the pin headers and replacing them with the ones you desire or get a version of the shield that allows stacking. Since I am a noob, I will have to live with the defective design of this shield.

Testing the Shield

Before you start thinking (like me initially) that you can immediately blink LEDs on this, take note that you will still need a few male-to-male Jumper Wires to connect the LEDs to the pins. Noticed the tiny little holes marked with the labels - LED1 and LED2? That's where you need to poke the jumper wires.

There is also a tiny hole with the label + beside the S1 push button for you to learn inputs (like clicking a button in your app). Don't get confused with the RESET button - most Arduino Shields comes with a reset button for you to "reboot" your board.

#1 - Basic Test

To test out the shield, upload the following sketch to your UNO. It is just a simple program that turns LED1 on and then swap to turn on LED2 instead when you press the push button. When you release the button, it will switch back to turn on LED1.

#define LED1 13
#define LED2 12
#define BUTTON 11

static int _buttonState = 0;

void setup() {
// Setup pins.
pinMode(LED1, OUTPUT);
pinMode(LED2, OUTPUT);

// Pull-up Button pin.
digitalWrite(BUTTON, HIGH);
}

void loop() {
// Read button state.
_buttonState = digitalRead(BUTTON);

// Swap lightings.
digitalWrite(LED1, _buttonState);
digitalWrite(LED2, !_buttonState);
}

Disconnect the UNO from the computer and attach the shield. Then proceed to make the following hardware connections:

LED1 -> Pin 13
LED2 -> Pin 12
+ (S1) -> Pin 11

Once it is done. Connect the UNO to your computer and test the shield. LED2 should light up and LED1 will turn off when you press the button.

#2 - Chase LEDs

Let's try out something a little bit more interesting. We will wire-up 4 LEDs onto the tiny breadboard with some resistors and create the 'Chase LED' effect. Uploading the following sketch to your UNO:

#define LED1 13
#define LED2 12
#define LED3 11
#define LED4 10
#define TOTAL_LEDS 4

#define BUTTON 7

#define ON LOW
#define OFF HIGH

static int LEDS[] = { LED1, LED2, LED3, LED4 };
static int _pos = -1;
static bool _forward = true;

void setup() {
// Setup pins.
for(int i = 0; i < TOTAL_LEDS; i++)
pinMode(LEDS[i], OUTPUT);

// Pull-up Button pin.
digitalWrite(BUTTON, HIGH);
}

void loop() {
if (!digitalRead(BUTTON)) {
setDirection();
blink(100);
}
}

void setDirection() {
// Increase _position if have not reached the end.
if (_forward && _pos < TOTAL_LEDS - 1)
_pos++;

// Decrease _positon if have not reached the beginning.
else if (!_forward && _pos > 0)
_pos--;

// Set to Forward if reached beginning.
if (_pos <= 0)
_forward = true;

// Set to Backward if reached end.
else if (_pos >= TOTAL_LEDS - 1)
_forward = false;
}

void blink(int pause)
{
digitalWrite(LEDS[_pos], HIGH);
delay(pause);
digitalWrite(LEDS[_pos], LOW);
delay(pause);
}

Disconnect the UNO and proceed to make the following connections onto the tiny breadboard:

Blue LED (Anode +ve) -> Pin 13
Green LED (Anode +ve) -> Pin 12
Yellow LED (Anode +ve) -> Pin 11
Red LED (Anode +ve) -> Pin 10

Important! Make sure all the LEDs are connected to resistors of at least 220 ohm.

You will now discover that the lack of a power rail on the tiny breadboad is a challenge for you to connect the LEDs to the GND pin. Fortunately, the Proto Shield came with a row of GND Pin Header that you can use. Connect all the LEDs to GND.

Blue LED (Cathode -ve) -> 220 ohm Resistor -> GND
Green LED (Cathode -ve) -> 220 ohm Resistor -> GND
Yellow LED (Cathode -ve) -> 220 ohm Resistor -> GND
Red LED (Cathode -ve) -> 220 ohm Resistor -> GND

Finally, connect the Proto Shield button to Pin 7:

+ (S1) -> Pin 7

Your completed wiring should look something like the following picture:

Check all your connections to make sure that they are correct. Connect your UNO to your Computer and press the button on the Proto Shield to run the chase effect.

Summary

That's all to it for the Proto Shield. If you are a seasoned electronics person or have some experience with the Raspberry Pi, you will noticed that the Arduino's pin design is somewhat not very convenient for external breadboads without longer jumper wires, due to the positioning of the pins along the edge of the board. Therefore, you may find the Proto Shield useful as well for little projects.

Category	Ratings	Comments
Easy to Learn		Extremely easy to use even with basic electronics knowledge. No rocket science required.
Availability of Resources		Resources and samples are abundance but may be different from vendor-to-vendor.
Built-In Features		A button and 2 LEDs with matching resistors. Nothing fancy.
Customizability		Can solder your own components onto it but space to put breadboard for prototyping maybe too small.
Extensibility		Not stackable but exposes pin headers to connect to other modules and components.
Length of Excitement		Can learn all the features quickly. Will get bored very fast.
Price		Good value and quite cheap.

Origin: I suspect this shield originated from Adafruit's earlier creations. Since Adafruit published the designs for people to use, I guessed anyone can actually manufacture the similar thing. But you will notice that this shield has a few missing items from the original design i.e. the two 0.1uF Capacitors. Adafruit no longer produces this shield but instead sells an updated version that is stackable.

Beginning Arduino for Software Engineers

2016-07-24T14:30:00.000+08:00

While exploring the realm of electronics, I discovered that, like in the software industry, there were many camps and one of the camps that caught my attention was Arduino. It is an open source hardware company that existed for more than 10 years, much longer in existence than the Raspberry Pi. Its most successful and famous products to-date are the Arduino UNO and MEGA.

The main difference between the Arduino boards and Raspberry Pi is that the Arduino boards are Microcontrollers (MCU), whereas, a Raspberry Pi is a Single-Board Computer (SBC). It is often seen that both Arduino and Raspberry Pi can complement each other in an IoT solution, with the Arduino boards communicating with sensors, motors and the environment, while the Raspberry Pi acts as a gateway to consolidate telemetry data from the Arduino and sends them to the Cloud.

Being a MCU means that the Arduino board doesn't have enough processing power or storage capacity for you to load an operating system onto it (Yep! You can't load Windows 10 IoT onto it). You program directly onto its chip which makes it perform more efficiently without all the overheads of an operating system.

One good thing about Arduino is that it is open source hardware. You maybe wondering what does that mean? It simply means that all the designs and schematics of the hardware are published (in what is known as Eagle files) and available to the public. Any interested manufacturer (or even yourself) can use the designs to come out with own versions of Arduino boards and that's the reason why, you see a lot of Arduino boards out there with names like something-something-uino, especially from China which I think contributed to the popularity and wide-spread of the UNO and MEGA boards.

In my perspective, Arduino is like Android where everything is open-source and everyone is free to create their own version and sell it, creating lots of fragmentation and confusion, whereas, Raspberry Pi is like Apple, everything is controlled by a single organization and not to mention having lots of expensive accessories.

So far, we have just been talking about Arduino as the hardware. Arduino also comes with an IDE that we can use to develop our programs. The programming language of choice is Arduino C, a variant of C which is not too difficult to learn. I somehow have difficulties in mastering the C programming language - so I'm still struggling with it right now. >_<

Compared to Visual Studio (the best IDE on the planet), the Arduino IDE looks very primitive but it gets the job done. You just need to tell yourself that you are using a slightly better notepad and that you are debugging like Classic ASP. You can immediately learn a lot of things from the samples that comes with the IDE (provided you know how to wire-up the electronics part first). You also need to know that there are no emulators for you to work on unlike mobile development, so every time you make a coding mistake, you need to burn your program again to the chip. (After too many burns, the chip will meet its end-of-life).

Adding more confusion to Arduino is that the founders are currently fighting over legal trademarks and the rights to use the Arduino brand. There is actually an Arduino LLC that runs Arduino.cc and Arduino SRL that runs Arduino.org. Both of them used the same logo. Arduino.org also forked the Arduino IDE creating a lot of confusion to people. To me, I just think that the founders should just be friends and not fight - fighting is bad for consumers.

You will realized that the UNO and MEGA are like 6 years old boards and after their release, the other boards like DUE, ZERO and etc. do not have much uptake. Anyway, some people said that it is because the newer boards are all 3V3 (like the Raspberry Pi) and the UNO/MEGA boards are still 5V which is easier to work with and has more compatible components out there. There are also some who said that maybe China has not gotten their hands on the Eagle files yet (i.e. the ZERO).

Nevertheless, I hope the Arduino folks can get their act together and focus on their innovation because I'm quite interested in the Arduino PRIMO and OTTO by Arduino SRL. Hopefully, everything will be settled and we will have better toys to play with.

[Update: On 1st October 2016, both the Arduino companies have reached a settlement with each other and have agreed to bring their innovations to the market together.]

OK, after all that long introduction, I would like to say that it is pretty difficult, confusing and quite expensive to get an Arduino board in Malaysia. Also, since we are outside the US market, we are supposed to get Genuino boards. You can get them from some local online sites if they carry stock.

Most of the people I know, get those cheap unbranded version from ebay or AliExpress for less than RM15 (a.k.a Cinanos). Those usually come with CH340G chip (instead of the ATMEGA chip) which you may need to go through some installation acrobatics to get it working. I have not heard complaints from anyone yet but I think they serve as a good platform for learning - you know, just in-case you plug something in wrongly and see blue smoke coming out.

For me, I bought my UNO and MEGA from SainSmart (also China). The reason I bought from them is because they are still using the ATMEGA chip, uses same USB port-type as the original and they have put in some modifications onto the board by providing (minor) extra features and pins which is good for my learning. The board also look very stylish but the cost is higher than the Cinanos though.

Whatever Cinano version that you are getting, just make sure that there is a poly-fuse beside the USB port to protect your PC from overcurrent. Older versions do not have them and you will risk frying your PC's USB port if you plug in something wrongly.

In my opinion, I felt that it is much easier to work-on and pickup electronic concepts with the Arduino as compared to the Raspberry Pi. The only challenge with the Arduino right now is the complexity you need to go through to enable it for IoT. There are WiFi shields (ESP8266 and YUN) and modified boards that you can use, but they all look like "patches" or "add-ons" instead of a truly integrated system. However, this may change in the future with the PRIMO.

I hope I gave you enough insights to get started on Arduino.

Azure IoT SDK for Python on Raspberry Pi

2016-07-07T17:52:00.001+08:00

After playing with my Raspberry Pi 2 for a while, I figured that it may be good to take it to the next level by looking into the Internet of Things (IoT). The concept IoT is pretty simple which is basically to upload data from sensors (Telemetry data) to the cloud for monitoring and analytics purposes. So, I just need to be looking at how I can upload some JSON data to a cloud storage.

Since I have some basic working knowledge with Azure and since Microsoft has started supporting other platforms on Azure, I was thinking to maybe to skip Windows 10 IoT and try to get Azure working on my Pi2 with Python and Raspbian instead.

Fortunately, I found out that there is an Azure SDK for Python from Microsoft. Although it is still in RC, it is already quite usable. Installing the Azure SDK for Python is straight forward on the Pi2. Simply enter the following in a Terminal:

sudo pip3 install --pre azure

Note: I used pip3 because I am using Python 3.5.1.

I believe with the Azure SDK installed, we should somewhat be able to fulfill the basic requirements of uploading data to Azure storage. I have yet to test it because during the course of my research, I had found something even more awesome. Microsoft has actually released something called Azure IoT Suite and Azure IoT Hub. Not only does it allow you to upload telemetry data to the cloud but it also allows you to manage/control your devices from the cloud. To my delight, there is also an Azure IoT Device SDK for Python.

Going through all the hands on tutorials on the Azure IoT Hub site, I was convinced that I could use this on my Pi2. I tried to install the Azure IoT Device SDK on my Pi2 and this time it was not really that straight-forward anymore because most of the instructions were either for Windows or Ubuntu but fortunately, I got it working and here's how you can do it...

Step #1 Clone the Repository

Create a folder in your home directory on your Pi. I called mine Sources. Open a Terminal and navigate to the folder i.e. ~/Sources, then type:

sudo git clone --recursive git://github.com/Azure/azure-iot-sdks.git

After it is done, you will see an azure-iot-sdks directory inside your Sources.

Step #2 Increase Swap File size

This is an important step. You need to do this or the compilation will fail.

The swap file configuration is located in /etc/dphys-swapfile. You can edit it using any text editor. I will be using nano to do it.

sudo nano /etc/dphys-swapfile

locate the line:

CONF_SWAPSIZE=100

and change it to:

CONF_SWAPSIZE=1024

Save the file.

Note: As I am a noob in Linux, I do not know what other side-effects the above settings will cause. If you are worried about it, you can always change it back after everything is done.

After you have messed with the settings, you need to restart the swap:

/etc/dphys-swapfile stop
/etc/dphys-swapfile start

Note: It will take some time and it will look like everything is hanging and not responding but please don't panic.

Step #3 Build the Azure IoT Device SDK for C

Yes, you read it correctly. You need to build the sdk for C first. Make sure you have cmake version 3.x or higher and gcc version 4.9 or higher. If you are working on a new Raspberry Pi and have just installed Raspbian, you should be good to go.

Change to the ~/Sources/azure-iot-sdks/c/build_all/linux directory

Run:

./setup.sh

and after that, run:

./build.sh

You can get these instructions from here.

Step #4 Fix the make file.

If you proceed to follow the instructions given on the github site to compile the python iothub_client, you will get the following error:

CMake Error at /usr/share/cmake-3.0/Modules/FindBoost.cmake:1198 (message):
Unable to find the requested Boost libraries.

Boost version: 1.55.0

Boost include path: /usr/include

Could not fine the following Boost libraries

boost_python-py35

No Boost libraries were found. You may need to set BOOST_LIBRARYDIR to the directory containing Boost libraries or BOOST_ROOT to the locations of Boost.

I have filed it as an issue. I'm not sure if you will get the error if you are using python 2.7. I didn't bother to try. The error will mislead you into thinking you have a bad libboost installation but trust me, it is not.

To solve the problem, edit the CMakeLists.txt file by loading it in a text editor:

sudo nano ~/Sources/azure-iot-sdks/python/device/iothub_client_python/CMakeLists.txt

The line you are looking for is (Line 41 at time of writing):

find_package(Boost COMPONENTS "python-py${boost_python}" REQUIRED)

Change it to:

find_package(Boost COMPONENTS "python" REQUIRED)

Save the file.

Step #5 Build the Python IoTHub Client

Change to the ~/Sources/azure-iot-sdks/python/build_all/linux directory:

Run:

./setup.sh --python-version 3.5

Then:

./build.sh --build-python 3.5

You can find the detail instructions here.

Get Started!

Now you can go to the ~/Sources/azure-iot-sdks/python/device/samples and start your journey into IoT.

If you find all the electronics stuff in Raspberry Pi too intimidating and just want to jump straight to IoT, I will advice that you get a Raspberry Pi Sense HAT. It comes with all the basic sensors which is sufficient enough for you to explore Azure IoT and make your audience go wild in IoT presentations.

You can get it from element14.

Good luck and have fun!

Getting Started with Raspberry Pi

2016-07-06T18:46:00.000+08:00

A couple of months back, I wasn't sure what hit me but I got myself a Raspberry Pi 2. It was also quite silly because my Pi 2 came two days after the released of Raspberry Pi 3 *ARGHS!* Anyway, I had started playing with it since then and now I would like to share some of my experiences with other Software Engineers who maybe interested to get started with the Raspberry Pi.

Short introduction - A Raspberry Pi is a low cost, low-power, credit card-sized computer that is developed by the Raspberry Pi Foundation in the U.K., a non-profit organization with the noble goal of advancing computer science knowledge in adults and children. It is supported by the University of Cambridge Computer Library and Broadcomm.

So what can a Software Engineer do with a Pi? Well, short-off trying to use it like a small computer, you can use it for various home automation projects, learn low-level programming, create little irritating autonomous robots (no vacuum cleaners though) and best of all, use it to delve into the world of Internet of Things (IoT). IoT is a big buzzword nowadays (together with Data Science) and it is set to boom in the upcoming years so it might be a good idea to equip yourself with some skills now. *HeHe*

By now if you have googled for Raspberry Pi, you will see that it resembles a small mobo with familiar HDMI and USB ports, and being a Software Engineer, you would probably think it is no sweat! Here are some things I feel that you should know:

Don't Waste It As A Cheap PC

Many people I knew thought to use the Raspberry Pi as cheap substitute for a PC. My advice is, if you can afford a PC, please forget about thinking to use the Raspberry Pi as a PC because at its processing power today, you will be frustrated (unless you don't mind the slowness). Some people also told me they want to use it to learn another operating system - fine, but if you wanna learn linux (yup, the official OS is a Linux variant), why not just virtualize a full-fledged environment to learn? (Ok, maybe it is the smaller size that makes it less scary).

Knowledge of Electronics

The awesome thing about the Raspberry Pi is not really that it is a small PC but it is the row of pins you see on the board - the 40 GPIO pins. GPIO stands for General Purpose Input-Output and those pins are the key to the world of physical computing (and awesomeness). The GPIO pins will allow you to light up LEDs, read data from sensors, power up motors and servos and do all kinds of tok-kong stuff (interesting things).

To make use of the GPIO pins, you will need some basic electronics knowledge which can be somewhat a challenge to some Software Engineers, particularly - Googlepers.

Know Your Stuff

It is very common for Software Engineers to just google for code to get things done. What happens when you found bad code? - Just delete it and start over with zero risks. Your computer ain't gonna explode just because you googled an endless loop. Now things are a bit different when it comes to electronics - you need to know your stuff well.

I spend a lot of time reading and re-reading before I even poke my first pin to the Pi. The reason for all these paranoia is because when you wire-up things wrongly, you will "fry your Pi" (and yes, it rhymes). You will also need to be careful when you google for stuff because there are times, the information you get could be different (i.e. different pins for your version of sensors) and you could just see smoke coming out from your precious Pi.

Running Windows 10 IoT

As I mentioned earlier, the official supported OS for the Raspberry Pi is Raspbian, a debian-based linux and you will be using Python to program it. But what about Windows? Yes, you can install Windows 10 IoT on it (in fact, I got to know about Raspberry Pi after Microsoft made a big hoo-hah about it). However, the Windows 10 that you are going to get on the Raspberry Pi ain't gonna be the Windows that you are familiar with (and NO it is not just a command prompt like Windows Server Core - it is something more different).

But if you are curious to try it out (because Visual Studio is the most awesome IDE on the planet and you love C#), head over to the Windows 10 IoT site to get the image or you can wait for me to talk about it later.

Buy the Pi

Now that I have pre-empt you and you are still interested, the first thing to do is to get a Raspberry Pi 3 and all its required accessories i.e. Power Adapter, HDMI to VGA adapter etc.. You can get them from:

Element14 (recommended)
RS Online

If you are worried about not having the right components to get started, I would recommend that you get a Starter Kit. I got myself a Raspberry Pi 2 Ultimate Starter Kit from Canakit via Amazon.com. You can also checkout Adafruit, Sparkfun and Sunfounder. By now you must be thinking - Why everything overseas wan, got any local sites or not? Ok, for local sites you can try Ubitap (recommended), Cytron or MyDuino.

Ok, I know that face, that's the - Walaueh! Why so expensive wan? look. Well, if you want to save up some $, you can be daring a-bit and go try out eBay or AliExpress. But don't say I never warned you, you probably get your stuff after weeks or months or maybe .... never.

So what's next after you got your Pi. You will need a micro-SD CARD. The specs say you should be getting something Class 10 (or if there is something higher) with at least 8GB. Now, Class 10 micro-SD cards also have multiple variants. If you can spend some $, try to get some high-end Samsung cards like the EVO+ or even better, the PRO series if you have deep pockets.

The Pi doesn't have a hard-disk like a PC, so you load the OS and store stuff on the micro-SD card (and that's why you need to have a fast card). If you beh-song and don't want to use a micro-SD card, you can checkout Western Digital's PiDrive.

Installation

To install your OS, first you need to download the OS image (or images). The best way to do it is from NOOBS (New Out-Of-Box Software). Recall earlier I mentioned about Windows 10 IoT? If you try to install from NOOBS, you can actually select multiple OSes to dual, triple, quardro boot and Windows 10 IoT will be one of the option. For me, I will go with Raspbian.

Just extract the downloaded .zip file and copy the files over to your micro-SD Card. Put the card into your new Pi and power it up. Follow the simple installation steps, make some coffee and then everything should be done.

Go here if you need more detail instructions.

Installation Caveats

Now, some of you may be a bit kiasu (like me) and go buy a big micro-SD Card i.e. 64GB. You will then noticed that the Pi will not recognize the card. The reason is because the larger cards are usually formatted in exFAT. Follow the instructions here to format your card to FAT32.

Post-Installation Tips

Usually most people will tell you that you are now ready to go and say bye-bye, but I would like to give you a few more post installation tips. All the following should be done on your Raspbian OS.

Tip #1 Get Up-to-date

Make sure you are connected to the Internet. Open up a Terminal and enter:

sudo apt-get update

then

sudo apt-get upgrade

You will see that almost everywhere, people will ask you to do that all the time. As to why they didn't script that on OS boot-up, I don't really know. These are something like your Windows Update but in command line style.

Tip #2 Change the Keyboard Layout

Before you start to think your keyboard is spoilt, you should go and change the keyboard layout to US. By default, I think it is default to U.K. How to do it? Own self go find in the settings la. Take this chance to explore the OS a bit.

Tip #3 Install Remote Desktop

I find it more convenient (and productive) to remote in to the Pi to do work. To enable remote desktop, open up a Terminal and enter:

sudo apt-get install xrdp

After you have installed that, you will be able to Remote in to the Pi using Remote Desktop Connection or (mstsc.exe) from your Windows machine.

Tip #4 Share Files

After a while, you may have written sibeh a lot of python programs or have some stuff you have to backup to your PC. You may also want to copy things from your PC to the Pi. In order to do this, install samba.

The installation is a bit lengthy, you can visit here for full instructions.

Tip #5 Date Time Format

Not sure if it bugs you, but when I look at the date time at the taskbar, I feel sibeh boh-song. I changed the format to

%a %x %I:%M %p

which will display something like Wed 7/6/2016 6:20 pm.

Where to change it? Again, you own self go find in the settings la.

Conclusion

I hope you have find it informative (and entertaining) so far and I hope you can do something awesome with the Raspberry Pi (since nowadays, even 8 year old kids are starting to play with it). Now, if you somehow bought the Pi, took pictures of it and put on facebook to fish likes and then felt that it is too difficult to work on, you can always convert it into a Media Centre - How to do it? You own self google la. ;)

Visual Studio Code & ASP.NET 5 on Mac OS X

2015-05-07T10:44:00.002+08:00

Microsoft has recently announced the availability of Visual Studio Code and I have been eager to test it out on my Mac. Before you get too excited, Visual Studio Code is just a ~~text~~ code editor and is not an IDE. (If you are running on Windows and looking for an IDE, I will recommend you to get Visual Studio Community Edition instead but you need to get the 2015 version).

Since Visual Studio Code and ASP.NET 5 are still in the preview stage, getting everything setup may not be that straight-forward. I will share with you the ~~pain~~ steps that I have gone through in order to get them working on a Mac.

Step 1 - Download and install Visual Studio Code.

Head over to https://code.visualstudio.com/ and download VS Code.

Note: You will only get a zip file. There is no installer. Extract it to a location of your choice.

This is what you will see when you first load VS Code.

If you are accustomed to Visual Studio you will be asking "Is that it?". That's because it looks pretty much just like a notepad. There is no File -> New Project or whatsoever (because it is not an IDE). But don't under-estimate it, after all it is still part of the Visual Studio family!

To put it to good use for .NET development, here's ~~where the pain starts~~ what you need to do.

Step 2 - Setup Pre-requisites

.NET development nowadays isn't like it used to be in the good old days. To have .NET working on your system, you will need to ~~invite other non-related guests~~ install other pre-requisites. The most important pre-requisite that you need is npm and that comes with nodejs.

Step 2.1 - Setup nodejs

Download and run the nodejs installer.

Follow through the steps to get it installed.

Step 2.2 - Install grunt

Open up a Terminal and type in the following:

npm install -g grunt-cli

If you are not running as administrator, remember to use the sudo command i.e. sudo npm install -g grunt-cli

Step 2.3 - Install bower

Type the following in the same Terminal:

npm install -g bower

Again, please remember to use sudo if you are not running as administrator.

Step 3 - Setup ASP.NET 5

Once you have done with all the pre-requisites, it is now time to install ASP.NET 5.

Step 3.1 - Install Homebrew.

Wait, you can't just install ASP.NET like that, To get it, you will need Homebrew. Type the following in the Terminal:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Step 3.2 - Install ASP.NET 5

Now you can proceed to install ASP.NET 5. Follow the steps as stated in the ASP.NET 5 github. Type the following in the Terminal:

brew tap aspnet/dnx

brew update

brew install dnvm

Note: You have just installed the .NET Execution Environment (DNX) and .NET Version Manager (DNVM).

Step 3.3 - The missing stuff in bash_profile

At the time of this writing, there are some missing steps in the official installation guide to make things a little more ~~challenging~~ interesting for Mac users. You will need to enter some stuff into your bash_profile.

Don't panic if you cannot find this file, I was told that it does not exist in most Macs.

At the Terminal, type in:

touch ~/.bash_profile

open ~/.bash_profile

This will create the file and open it in a text editor. Type this into the text editor.

export MONO_GAC_PREFIX="/usr/local"

export MONO_MANAGED_WATCHER=false

source dnvm.sh

Save the file.

Warning: If you missed these steps, you will get into a pull-hair session later i.e. lots of command not found.

Step 4 - Scaffold a Web Application

We are almost there. To get started with a web application template, we can use a scaffolding tool to do it.

Step 4.1 - Install Yeoman

At the Terminal, type in:

npm install -g yo

Step 4.2 - Install ASP.NET Templates/Generators

That's just the tool, you will need the generator/templates. At the Terminal, type in:

npm install -g generator-aspnet

Step 4.3 - Create the Web Application

You are now finally ready to create your web application *claps*. At the Terminal, type in:

yo aspnet

You will get something like the following:

Select Web Application and press enter. Give your web project a name i.e. WebApplication and see the magic happens.

You can open your web project in Visual Studio Code. Just go to File -> Open and open the project folder i.e. WebApplication.

Step 5 - Running the Project

Here's the bad news - there is no F5 to run.

Step 5.1 - Restore nuget packages

Before running the application, you will need to restore the nuget packages. You can open up a Terminal and go to your project folder or you can press ⌘P from Visual Studio Code to open the Command Palette. Type in:

dnu restore

Step 5.2 Hosting the Web Application in kestrel

Finally, you can now host and run your beautiful Web Application. At the Command Palette, type in:

kestrel

or at the Terminal, cd to your project folder and type in:

dnx . kestrel

Once kestrel is started, you can launch any web browser and point it to:

http://localhost:5001

Note: If you can't get it on port 5001, try port 5004.

Wrap-up!

If you have followed me this far, then you should give yourself a pad on the shoulder. Getting all these to work is surely exciting but it is really a big hassle compared to the traditional Visual Studio experience. But by working through all these, we get to know what is going on behind the scene and will learn to appreciate the Visual Studio ~~spoon-feeding~~ experience more.

If you do not want to go through all the hassle just to start coding a Hello World ASP.NET Web Application, I would recommend that you just virtualize a Windows operating system, install Visual Studio 2015 and get on with it (I did that but my friends call me a cheat).

Have fun trying out ASP.NET 5.

Converting Struct with Strings to Byte Array

2014-08-31T00:36:00.000+08:00

There are times when there is a need to convert a struct to a byte array (and vice-versa) when working with memory-mapped files. Although the MemoryMappedViewAccessor provides methods to read and write structs, they will fail when the structs contain string fields such as the following:

public struct Sequence

{

public string Name;

public long Minimum;

public long Maximum;

public long Current;

}

You will encounter the error - 'The specified Type must be a struct containing no references.' when you try to use the Write() method. To solve the issue, you will need to use the WriteArray() method instead and this is when you will need to convert the struct to a byte array.

To achieve that, we will need to perform some unmanaged memory acrobatics. The following code will convert the struct to a byte array and use the WriteArray method to write it to the memory-mapped file.

private static void WriteSequence(MemoryMappedViewAccessor accessor,

Sequence seq)

{

// Get size of struct

int size = Marshal.SizeOf(typeof(Sequence));

byte[] data = new byte[size];

// Initialize unmanaged memory.

IntPtr p = Marshal.AllocHGlobal(size);

try

{

// Copy struct to unmanaged memory.

Marshal.StructureToPtr(seq, p, false);

// Copy from unmanaged memory to byte array.

Marshal.Copy(p, data, 0, size);

// Write to memory mapped file.

accessor.WriteArray<byte>(0, data, 0, data.Length);

}

finally

{

// Free unmanaged memory.

Marshal.FreeHGlobal(p);

p = IntPtr.Zero;

}

And to read the struct, we need to convert the byte array back to the struct using the following code which reverses the above process:

private static Sequence ReadSequence(MemoryMappedViewAccessor accessor)

{

Sequence seq;

// Get size of struct

int size = Marshal.SizeOf(typeof(Sequence));

byte[] data = new byte[size];

// Initialize unmanaged memory.

IntPtr p = Marshal.AllocHGlobal(size);

try

{

// Read from memory mapped file.

accessor.ReadArray<byte>(0, data, 0, data.Length);

// Copy from byte array to unmanaged memory.

Marshal.Copy(data, 0, p, size);

// Copy unmanaged memory to struct.

seq = (Sequence)Marshal.PtrToStructure(p, typeof(Sequence));

}

finally

{

// Free unmanaged memory.

Marshal.FreeHGlobal(p);

p = IntPtr.Zero;

}

return seq;

}

Before you can use the two methods, you will need to make some slight changes to the struct as follows:

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]

public struct Sequence

{

[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 50)]

public string Name;

public long Minimum;

public long Maximum;

public long Current;

}

Note: You must marshal the string with the proper type and constant size. Failing to do so, you will get the error - "Attempted to read or write protected memory. This is often an indication that other memory is corrupt." when you try to read the memory mapped file.

Once you have completed all the above steps, you are now able to read and write structs with strings from and to memory mapped files.

WCF: Windows Authentication with basicHttpBinding

2014-08-26T10:01:00.002+08:00

Most developers when developing Windows Communication Foundation (WCF) services, will take security for granted and deploy their web services unsecured. In this post, I'm going to discuss how we can easily configure a WCF web service to use Windows Authentication.

In the services config file, configure the security settings for the binding as follows:

</security>

</binding>

</basicHttpBinding>

</bindings>

Take note that I'm using basicHttpBinding for my web services.

Make sure that the service is using the binding configuration.

<service name="SecureDemo.Services.MyService">

<endpoint name="basicHttpMyService"

address=""

binding="basicHttpBinding"

bindingConfiguration="basicHttp"

contract="SecureDemo.Services.Contracts.IMyService" />

</service>

WCF requires that both the consumer and service configurations to be the same. Therefore, at the client config file, include the same binding configuration and then configure the client to use it.

<endpoint name="basicHttpMyService"

address="http://localhost:8888/MyService"

binding="basicHttpBinding"

bindingConfiguration="basicHttp"

contract="SecureDemo.Services.Contracts.IMyService"

</client>

That's all for the configuration. Now when you call the service, simply supply the credentials like the following example:

var proxy = new MyServiceClient();

proxy.ClientCredentials.Windows.ClientCredential.UserName = "HelloKitty";

proxy.ClientCredentials.Windows.ClientCredential.Password = "meowmeow";

proxy.ClientCredentials.Windows.ClientCredential.Domain = "cutieland";

You can substitute the hard-coded credentials with values retrieved from your login screen if you want to authenticate by individual account and not from an application service account.

Hope this was helpful. :)

Windows Server 2012 R2 on MacBook Pro

2014-07-12T16:03:00.001+08:00

I have just gotten a MacBook Pro for work and like all the other notebooks before it, the first thing I would do is to install Windows Server on it. Afterall, a notebook is only worthy if it can run Windows Server. ;)

There were very little resources on installing Windows Server 2012 R2 on a MacBook Pro and that's the reason I'm writing this. Being a Windows user for a long time, this is an exciting experience for me. I have successfully installed Windows Server 2012 R2 (with update) on my 15" MacBook Pro with Retina display so you can be assured that it works (or atleast most part of it).

I'm a follower of official recommended practices, therefore I used Apple's Boot Camp for this. Make sure you have downloaded the Windows Server .iso, prepared an external USB keyboard and two USB flash drives for stand by.

The Boot Camp Assistant instructions are pretty straight-forward. I will just highlight some of the extra things that you need to do in order to have a smooth installation. Otherwise, you may spend a good amount of time trying to figure out whadahell-is-wrong like me.

Important! Before you begin, make sure you reset the NVRAM. Follow the steps here. If you did not complete this step, the installation of Windows will run until finish and then fail with the following error:

"Windows could not update the computer's boot configuration. Installation cannot proceed."

After that, your Mac will restart with a white screen and shutdown itself - a.k.a. White Screen of Death (WSOD). You will no longer to be able to boot up from this point. If you are unlucky enough to hit such situation... don't panic! Restart your Mac and hold on to the Option (alt) key. This should allow you to boot back to your OS X and start all over again.

1. From OS X, launch Boot Camp Assistant (you can type "Boot Camp" in Spotlight)

2. Select all the 3 tasks available:

Create a Windows 7 or later version install disk
Download the latest Windows Support software from Apple
Install Windows 7 or later version (Screenshot showing remove because I already have Windows installed)

Tip: You need to use the first option to let Boot Camp create the bootable USB. I tried creating it manually from a Windows machine and it doesn't work.

3. Provide the path to the Windows Server .iso image and select one of the USB flash drives to copy it into. The other one will be used to store the Boot Camp drivers.

4. Create a partition for Windows.

5. The system will reboot and Windows installation will commence.

6. During the setup of Windows, make sure you select the partition named BOOTCAMP to install Windows. Do not mess with the other partitions!

7. Once the Windows Server installation is completed and properly configured, you will be presented with an interesting problem - The press CTRL+ALT+DEL to login screen. There is no way you can do that at this point in time - you can try it if you don't trust me. :p This is where your external USB keyboard is required. Plug it in and login. The Boot Camp drivers should automatically be installed (and if not, you can launch it from the USB flash drive).

Tip: You can disable the CTRL+ALT+DEL screen if you want. Follow instructions here.

8. Once everything is done, reboot and you are ready to go!

9. Before enabling Hyper-V or installing Visual Studio 2013, reboot to OS X then go to System Preferences -> Startup Disk and select the the BOOTCAMP disk. Restart the system. If you skip this step you will receive an error message stating that "Hardware Virtualization is not enabled in the BIOS" (and you can't access the BIOS on a Mac).

Note: I have heard interesting myths from friends telling me that the keyboard is pretty messed up in Mac, as in the keys are different from Windows i.e. can't do CTRL+C and CTRL+P to copy-paste. This is not entirely true. Once you have the Boot Camp drivers installed, the keyboard will function like normal and you can even do CTRL+ALT+DEL or WinKey (replaced with command key) +L to lock screen.

Extra Tip: OH NOES! How do we do screen capture since there isn't a Print Screen button on the keyboard? The key combo is Fn+Shift+F11.

10. To enable right-click, go to Control Panel -> Boot Camp -> Trackpad and set Secondary Click to Bottom Right Corner.

11. The WiFi may not be working. To fix the problem, download the Boot Camp Support Software from Apple's site and run the Broadcom Wireless x64 drivers in the Drivers\Broadcom folder.

There you go! No you have a fully functional Windows Server 2012 R2 on your MacBook Pro that you can play with.

Layered Architecture: Partitioning the Code

2014-04-18T12:44:00.000+08:00

Developing an enterprise layered application can be a daunting task especially when the layered samples provided, contains a barrage of Visual Studio projects. So, here's a brief description of what each folder and projects are for in the samples found in Layered Architecture Sample for .NET and also those that were unfolded by the Layered Architecture Solution Guidance (LASG) Visual Studio extension.

Business

This folder should contain all projects and assets that are related to the business layer. If you have any classes that are only shared between business layer components, you can create a [Project-name].Business.Framework project to hold them in here.

[Project-name].Business This project should contain all the main business components. The classes here should contain all the business processing and validation logic.

[Project-name].Workflows This project should contain all the WF workflow components that orchestrates the business flows. The classes here are stored as .xaml and will be compiled into a dll later. This is not a .xamlx project.

[Project-name].Workflows.Activities This project should contain all the workflow activity and related classes. Workflow activities facilitate the easy construction of workflows. Composite activities should reside in this project as well even if they are with the .xaml extension.

[Project-name].Workflows.Designers This project should contain all the workflow and activity designer classes. The classes here are to facilitate the designer experience of Visual Studio and should not be deployed to the application server.

Data

This folder should be used to group all the projects and assets that are related to the data layer. Any classes that calls external web services or accesses other data storage such as the file system, SharePoint lists or message queues, the should also be located here. If you have any classes that are only shared between data layer components, you can create a [Project-name].Data.Framework project to hold them in here.

[Project-name].Data This project should contain all the components that interacts with the database. Place all CRUD code in this project.

[Project-name].Data.AgentsThis project should contain all the components that interacts with other storage systems such as file, SharePoint lists or Queues. This project does not unfold by default in LASG.

[Project-name].Database This is a SQL Server Data Tools (SSDT) project that is used to hold database scripts and publish them to the database server. You can also use this to keep track of design and schema changes by checking it into a version control system such as Team Foundation Server (TFS). This project does not unfold by default in LASG.

Hosts

This folder should contain all the host projects that will be deployed onto the application server. You can create as many host projects as needed depending on your hosting needs.

[Project-name].Hosts.Web This project should contain all the assets and configuration for hosting the services back-end in the application server. This is the entry point to the application server.

Presentation

This folder should contain all the projects and assets that are related to the presentation layer. If you have any classes that are only shared between presentation layer components such as UI element manipulation code, you can create a [Project-name].UI.Framework project to hold them in here.

[Project-name].UI.ProcessThis project should contain all the user interface process components (or controllers). Session, state management and page control classes should also reside here.

[Project-name].UI.Web This project should contain all the pages, scripts and user interface assets that will be deployed to the web server. This is the main entry point to the layered web application.

Services

This folder should group all the projects and assets that are related to the service layer. All services related code whether they are ASMX, WCF or WEB API (REST), should reside in this folder. If you have any projects that extends the capabilities of the service technology, they should also reside in this folder.

[Project-name].ServicesThis project should contain all the WCF service implementation classes.

[Project-name].Services.Contracts This project should contain all the services and message contracts for the WCF services. For non-WCF services, this project can serve to hold wrapper classes for Request and Response message classes that are used for communication.

[Project-name].Services.HttpThis project should contain all the ASP.NET WEB API related service implementation classes. This project is not currently unfolded by LASG.

Shared

This folder contains all the projects and assets that can be shared among any of the projects in the layers.

[Project-name].EntitiesThis project should contain all the entity classes that are shared among all other projects in the layer.

[Project-name].Framework This project should contain all the cross-cutting components such as Authentication, Authorization, Auditing, Logging, Configuration and etc.

Tests

This folder should contain all the test projects for the components in the layers. You can create as many test projects in here based on your testing needs.

LeaveSample.TestThis project is created for you to place your tests (unit test, load test, web tests etc.).

This is how everything it looks like in Solutions Explorer. I have omitted some of the non-essential folders such as Scripts and Solution Items from the explanation.

What if I have multiple modules?

The above structure is simple when we are building a medium size application but how do we go about partitioning our projects when the applications becomes larger? There are two approaches that we can adopt.

Partition by Layer

The most common approach would be to partition the projects by layer. Each layer will contain the projects belonging to each module. Visual Studio provides the option to create Solution folders and you can use them to organize your projects neatly.

Take note that the Solution Folders aren't physical folders and if you want the physical folder structure to match the one shown in Solution Explorer, you will have to do it manually.

Partition by Module

The next approach is to partition all the projects into each Module's Solution Folder. I personally would recommend to use this approach as it allows you to selectively compile the entire module independently from the others. You can also selectively unload a particular module and keep yourself focused on the one that you are working on.

Can I have something simpler?

Frequently, Layered Architecture has been labelled as only suitable for large enterprise applications and overkill for smaller applications. This is understanding is inaccurate. Layered Architecture is an architecture pattern that should be inculcated in every business application developer, whether they are developing small or large applications.

The reason is that even the smallest business application has the potential to grow into something large or complex and Layered Architecture is merely a practice to logically partition the code. At the simplest level, we can still achieve layering with the following project structure.

The Entities project should be separated into a separate project for sharing reasons. This also applies if there is a Framework project available. Take note that the Contracts that is embedded inside the host will rely on Visual Studio's Add Service Reference dialog to re-create the service contracts on the presentation side. If a clean implementation is desired, the code in the contracts folder should also be separated into an individual project. LASG does not support this project structure.

You can check out the other posts in the series if you are interested to learn more about layering:

ASP.NET MVC with SignalR - Client Undefined Error

2014-04-09T09:07:00.000+08:00

While working on a new ASP.NET MVC layer sample with SignalR, I have constantly encountered the following error when I tried to access the site directly with the full path to the View i.e. http://localhost:13618/Home/Index

0x800a138f - JavaScript runtime error: Unable to get property 'client' of undefined or null reference

This error does not occur when I access the path to the site i.e. http://localhost:13618/

My Views are using a layout and it has all the SignalR javascript references defined.

Index.cshtml

ViewBag.Title = "Apply Leave";

Layout = "~/Views/Shared/_Layout.cshtml";

}

_Layout.cshtml

<head>

    <meta name="viewport" content="width=device-width" />

    <title>Layered Architecture Sample for .NET</title>

    <script src="~/Scripts/jquery-2.1.0.min.js"></script>

    <script src="~/Scripts/jquery-ui-1.10.3.min.js"></script>

    <script src="~/Scripts/jquery.signalR-2.0.3.min.js"></script>

    <script src="signalr/hubs"></script>

    <link href="~/Content/themes/base/jquery-ui.css" rel="stylesheet" />

    <link href="~/Content/Site.css" rel="stylesheet" />

</head>

It turns out that I was using the wrong way to reference the hub scripts in ASP.NET MVC. The problem can be easily resolved by changing the following line in the _layout.cshtml page from

<script src="signalr/hubs"></script>

*Slaps myself for not reading the documentation properly*

Serializing Enums in ASP.NET SignalR

2014-04-08T20:58:00.000+08:00

Was working on a new layered architecture sample that uses ASP.NET SignalR and found out that it couldn't serialize the enums properly by default.

As you can see from the screenshot, the LeaveStatuses enum has been serialized as a byte value instead of the text value. A quick solution will be to decorate the Status property with the [JsonConverter(typeof(StringEnumConverter))] attribute in the Leave entity class but I was reluctant to have my entities hard-coding that.

Fortunately, the problem can be easily solved with an alternative by including the following lines of code in the Startup class.

public class Startup
{
    public void Configuration(IAppBuilder app)
    {
        // Create JsonSerializer with StringEnumConverter.
        var serializer = new JsonSerializer();
        serializer.Converters.Add(new StringEnumConverter());
    
        // Register the serializer.
        GlobalHost.DependencyResolver.Register(typeof(JsonSerializer), () => serializer);

        app.MapSignalR();
    }
}

With that piece of code, the enums can now be serialized to their text values and I don't have to modify the properties in the entities.

Layered Architecture Solution Guidance (LASG) FAQ

2014-03-26T08:45:00.000+08:00

Over the years, I have collected a number of frequently asked questions while working on Layered Architecture Solution Guidance (LASG) and I think it will be good if I put them in a FAQ.

1. What versions of Visual Studio are supported?
The latest version of LASG targets Visual Studio 2013 but you can still download the older versions for Visual Studio 2012 and 2010. Older versions will contain lesser features and development on them has been discontinued.

2. What editions of Visual Studio are supported?
LASG is designed to run on Visual Studio Professional, Premium and Ultimate editions. The Modeling project only appears in Visual Studio Ultimate.

3. Is non-English version of Visual Studio supported?
As of version 1.0.1.0, LASG can only run on English versions of Visual Studio. A version that is supposed to support non-English Visual Studio is currently under works and hope to address the issue. The labels however, will still display in English even if it runs on non-English Visual Studio.

4. Can the Inertia code-generator support non-Microsoft SQL Server databases i.e. Oracle?
No. Inertia uses the SQL Server Management Objects (SMO) library and therefore, can only support Microsoft SQL Server.

5. Can I use Microsoft SQL Server Express edition?
Yes.

6. Does the Momentum code-generator supports Entity Framework?
No. Momentum only generates code that targets Enterprise Library Data Access Application Block (DAAB).

7. Are there plans to support Entity Framework in future?
No. There are no plans to support Entity Framework currently, but there were some thoughts on it.

8. How can I use LASG with Entity Framework?
Depending on your choice, you can use Inertia to generate the entities as POCOs and write your own CRUD methods in the Data Access Components that uses EF. You can then continue to use the LASG generators from Business Layer onwards and things will still work.

9. Does the Momentum code-generator supports Stored Procedures?
No.

10. Are there plans to support Stored Procedures in future?
No. Some experimental work was done to support Stored Procedures but it was shelved due to the massive effort required to re-design the internals of Momentum.

11. Why the code-generators have such code-names?
They were initially part of a separate pet-project called Project Newton before being merged into LASG. Hence, they all have code-names taken from articles describing the Newton Speed Theory.

If you have more questions, please feel free to put them in the comments and I will try my best to answer them. :)

Layered Architecture Solution Guidance (LASG) Tutorials

2014-03-24T08:49:00.002+08:00

I've finally completed writing all the tutorials for Layered Architecture Solution Guidance (LASG). Here's the full list for easy reference:

During the development of these tutorials, I have spotted some UI inconsistencies in some of the code-generators and I have decided to fix them in the next release. Hopefully, these articles are useful in helping you to adopt LASG.

How-To: Create UI Process Components with Impulse in LASG

2014-03-18T11:32:00.002+08:00

This is the final part of the 7-series tutorial on Layered Architecture Solution Guidance 2013. In this walk-through, we will be looking at the last code-generator in the family, the Impulse: User Interface Process Component Generator to create UI Controller components. Creating UI Controllers is also the final step in the LASG Code Generation process.

Before beginning this guide, please make sure that you have completed the previous tutorials on Creating Service Layer Components with Velocity in LASG and Creating Workflow Services and Activities with Vector in LASG. Please make sure that your solution is fully compilable without any errors before starting. This walk-through also uses the LeaveSample database that is part of the Layered Architecture Sample for .NET but you may use your own database to follow through.

Before launching the code-generator, the Host project will need to be started. In Solution Explorer, expand the Tutorial solution, go to the Hosts folder, right-click on the Tutorial.Hosts.Web project and select Properties.
At the project properties page, select the Web category and uncheck the Enable Edit and Continue check box. To understand why we need to do this, please refer to this post.

Note: I actually took this screenshot from another post. The actual port number for my project is 2381 and not 13619 (the port used in layer sample). Your port number could be different.
Save the setting.
In Solution Explorer, right-click on the Tutorial.Hosts.Web project, select Debug and then select Start new instance.
Your browser should launch with a HTTP Error 403.14 - Forbidden page. This is normal.
To test the WCF service, simply append LeaveService.svc to the end of the address i.e. http://localhost:2381/LeaveService.svc. To test the workflow service, append the address with LeaveWorkflowService.svc instead i.e. http://localhost:2381/LeaveWorkflowService.svc. You should get a result page similar to the following for both the address.

Note: Clicking on the url on this page will lead you to the WSDL of the service. (Whole page full of XML)

Tip: If you are curious how the host was configured, the answer lies in the web.config. As you clicked through the code-generators, bits and pieces of the settings were registered into the web.config.
Stop debugging but ensure that IIS Express is still running in the background. You can confirm this by checking the system tray for the IIS Express icon.

Note: The browser may shut down.
Once you have confirmed that your services are running, we can proceed with the code-generator. In Solution Explorer, expand the Tutorial solution, go to the Presentation folder. Right-click on the Tutorial.UI.Process project to invoke the context menu and click Add User Interface Process Components...
The Impulse: User Interface Process Component Generator will launch. Impulse is divided into 3 sections.
By default, Impulse loads the service contract assembly for the WCF services from the centralized bin folder located under the solution (i.e. Tutorial\bin). In this exercise, the Tutorial.Services.Contracts.dll should be automatically loaded.

Note: If the service contract project cannot be loaded, please exit Impulse and return to Visual Studio to ensure that there are service contracts in the Tutorial.Services.Contracts project and all dependent assemblies are located in the centralized bin folder. Please make sure that the project can be built successfully. You can also manually load the business project assembly by clicking the Load button and selecting the .dll file.
Like previous the code-generators, Impulse supplies us with default values in the 1. Define User Interface Process Component section.
Verify that the WCF Services radio button is selected.
Verify that Leave is in the UI Process name text box. The default Suffix for UI process components is Controller. Impulse will create a component called LeaveController for us.

Tip: Take note that although the UI process components are called controllers, they are in no way related to ASP.NET MVC controllers.
All the service contracts that are available in the service contract assembly will be listed in the Services dropdown list at the 2. Select Methods to Call section. Verify that ILeaveService is selected.
All the methods in the service contract will be shown in the list box under the Available operations check box. Check the Available operations check box to select all service operations.
Click on the button to add them to the Selected operations grid.
Click the Add to Code Generation button to queue the service definitions.
Return to the 1. Define User Interface Process Component section and click on the Workflow Services radio button to load the Tutorial.Workflows.dll.

Note: You should know the drill by now, if the workflow assembly fails to load, please exit Impulse and return to Visual Studio to make sure that there are workflows in the Tutorial.Workflows project and all dependent assemblies are located in the centralized bin folder. You will need to have at least one ReceiveActivity in your workflow to have something show-up. A blank xaml file will not do. Also make sure that the workflow project can build successfully. You can manually load the workflow project assembly by clicking the Load button and selecting the .dll file.
Verify that Leave is still in the UI Process name text box and the default Suffix is still Controller. Our LeaveController component will call both the backend WCF and Workflow services.
At the 2. Select Methods to Call section, ensure that LeaveWorkflowService is selected in the Services dropdown list.
All the methods defined in the Receive activities in the workflow xaml will be listed in the list box below the Available operations check box. Check the Available operations check box to select all the workflow service operations.
Click on the button to add them to the Selected operations grid.
Click the Add to Code Generation button to queue the definitions.
Notice the reminder about Add Service Reference? We can manually Add Service Reference from Visual Studio or have Impulse launch it for us. Because we are adding both the WCF services and workflow services at the same time, we will require Impulse to launch the Add Service Reference dialog for us once for our WCF service, and we will need to manually launch it again from Visual Studio to add the Workflow service reference.

Check the Launch Add Service dialog check box in the 3. Generate UI Process Components section. You will notice the reminder is gone.
By now, you should have the following:
Click the Generate button to generate the UI controller class.
Notice that you are once again reminded with instructions for Add Service Reference at the Code Generation Completed dialog, click OK.
The Visual Studio Add Service Reference dialog will launch.
In the Address bar, type in the url to your LeaveService i.e. http://localhost:2381/LeaveService.svc and click GO. You should be able to see the details of the LeaveService in the Services and Operations list boxes.

Note: If you cannot see the service, that means the host project is not running. Please make sure the host is running.
Enter LeaveService in the Namespace text box.
Click the Advance... button to open the Service Reference Settings dialog.
In the Collection type dropdown list, select System.Collections.Generic.List. Click OK.
Click OK again to let Visual Studio generate the service reference code.
To manually invoke the Add Service Reference dialog again for our workflow service, expand the Tutorial.UI.Process project, right-click on References and click Add Service Reference...
In the Address bar, type in the url to the LeaveWorkflowService i.e. http://localhost:2381/LeaveWorkflowService.svc and click GO. You should be able to see the details of the LeaveWorkflowService in the Services and Operations list boxes.
Enter LeaveWorkflowService in the Namespace text box.
Repeat step 31-32 for the LeaveWorkflowService.
Click OK to let Visual Studio generate the service reference code.
The Tutorial.UI.Process project should contain the generated controller class.

Ready to Go!

You can now start developing your Web UI. At this point in time, LASG does not provide any UI generator. You can easily hook up the code to your WebForm like this:

var upc = new LeaveController();

GridView1.DataSource = upc.ListLeaves(0);

GridView1.DataBind();

But before you run the Tutorial.UI.Web project, you will need to copy the configuration that was generated by Visual Studio in the Tutorial.UI.Process project. LASG provides a convenient way for you to do it.

Right-click on the Tutorial.UI.Process project and select Update Service Configuration to Client Projects.

Summary and Conclusion

We have finally completed the tutorial for the last code-generator in LASG, the Impulse: User Interface Process Component Generator. As with other code-generators in the family, you are welcomed to change the generated code but take note that any changes made in your code will not be reflected in the definitions stored in Impulse. Therefore, practice caution when you are regenerating code on existing code file.

Tip: In case you are wondering why all the fancy code-names for the code-generators, they were actually names mentioned in Newton speed theories. The code-generators was initially part of my own project called Project Newton before being merged into LASG.

This ends our 7 part series of walking through with LASG. I hope you find the information useful in adopting LASG. Thank You :)

How-To: Create Workflow Services and Activities with Vector in LASG

2014-03-17T11:40:00.002+08:00

This is the sixth in the series of the Layered Architecture Solution Guidance 2013 walk-throughs where we will be exploring the Vector: Workflow Services and Activity Generator to create workflow services and workflow activity components. Creating workflow services is the fifth step in the LASG Code Generation process. Before beginning this guide, please make sure that you have completed the previous tutorial on Creating Business Components with Motion in LASG. Please make sure that your solution is fully compilable without any errors before starting.

This walk-through also uses the LeaveSample database that is part of the Layered Architecture Sample for .NET but you may use your own database to follow through. It is also important that you have followed the steps in the sample to create the Workflow Persistence Store database. Otherwise, follow the instructions here to locate the SQL scripts and then execute them on a newly created WorkflowInstanceStore database.

In Solution Explorer, expand the Tutorial solution, go to the Business folder.
Right-click on any one of the Tutorial.Workflows, Tutorial.Workflows.Activities or Tutorial.Workflows.Designers project and click Add Workflow Activities... on the context menu.

Tip: Vector generates code to these 3 projects accordingly. The workflow service will be generated as an empty .xaml file to the Tutorial.Workflows project, activity classes will be generated to the Tutorial.Workflows.Activities project and their respective designer classes will be generated to the Tutorial.Workflow.Designers project. The classes in the Tutorial.Workflows.Designers project are only required for workflow design time purposes and do not need to be deployed onto your server.
This launches the Vector: Workflow Services and Activity Generator which contains 4 sections. Take note that Vector is designed to generate Windows Workflow Foundation (WF) services and activities, therefore, you are encouraged to read-up on WF and WorkflowServices to understand the terminologies used.

Note: It is always difficult to classify Workflow Services in the Layered Architecture due to their technological nature of being both a business component and also a service. In LASG, Workflow Services are still classified under the business layer even-though they are considered peers to the WCF services.
By default, Vector will attempt to load the business project assembly from the centralized bin folder located under the solution (i.e. Tutorial\bin) and populate the details in the 1. Define Workflow Service section. In this exercise, the Tutorial.Business.dll should be automatically loaded.

Note: If the business project assembly is not loaded, please exit Vector and return to Visual Studio to make sure that there are business components in the Tutorial.Business project and the solution is compilable without errors. Also please ensure that all dependent assemblies are located in the centralized bin folder. You can also manually load the business project assembly by clicking the Load button and selecting the .dll file.
All business components that exists in the assembly will be listed in the Business components dropdown list. Verify that the LeaveComponent is selected.
Vector will suggest the Workflow service name for us. By default, it will replace the Component suffix from the business component with a WorkflowService suffix. Leave the default suggestions and let Vector create a LeaveWorkflowService component for us.
Vector will also suggest the service contract name for us. Verify that ILeaveWorkflowService is entered into the Service contract text box.
By default, the service namespace is http://tempuri.org/. You can change the value by typing into the Service namespace text box or click the button to get Vector to suggest one for you. For this exercise, we will use the default value.
Since most of the values were automatically filled in for us, please ensure that the settings that you have are somewhat like the following:
Define an endpoint for the new service. This only needs to be done once when a workflow service is first created. At the 2. Define Endpoint Configurations section, make sure basicHttpBinding is selected in the Binding dropdown list and leave the default values in the Endpoint name and Address. Click the button to add it to the Service endpoints grid.

Tip: You can create multiple endpoints for your services based on WCF's guidelines. You may be wondering why WCF is mentioned here. The reason is because WorkflowServices are actually a fusion of WF and WCF. Therefore, some of the WCF settings will apply to WorkflowServices as well. For this exercise, we will only use one endpoint.

Note: The Address is only used in non-IIS/WAS hosts i.e. your own Console host. Since most enterprise WorkflowServices are usually (and recommended to be) hosted on IIS/WAS, this setting is seldom used and is just retained for backward LASG compatibility.
Creating the Apply workflow activity. At the 3. Define Workflow Activities section, select Apply in the Business methods list box.

Note: You can actually select more than one business method to be included in an activity but that is not recommended. It is always advisable to have one business method to be exposed as one activity whereby, you can later use the granular activities to compose more complex activities.
Make sure that CodeActivity is selected in the Base activity type dropdown list.
Leave the Activity name text box value as Apply. By default, Vector uses the name of the business method as the activity name.
Make sure None is selected in Emit tracking record dropdown list.
Select ReceiveAndSendReply at the Messaging wrapper activity type dropdown list. This will create a messaging wrapper activity that will wrap your activity between a pair of ReceiveActivity and SendReplyActivity.

Tip: Selecting Receive will create an wrapper class that only contains a ReceiveActivity which will only allow asynchronous calls to your workflow activity.
Verify that the value ApplyReceiveSend is entered into the Messaging wrapper activity name text box.
The Apply activity is the first activity in our workflow (as in a User applies for a leave). To allow our activity to start the workflow, we will need to check the Can create instance check box.
Check the Persist before send check box.

Tip: Persist before send will ensure that all our values in our workflow will be saved before returning the result to the caller.
Click the button to add the Apply activity to the Activities grid.

Tip: To remove an activity from the grid, right-click on the row to invoke a context menu and click Delete.
Create an Approve activity. Select Approve in the Business methods list box. Make sure ReceiveAndSendReply is selected in the Messaging wrapper activity type dropdown list and click the button to add it to the Activities grid.

Tip: Notice the Can create instance check box has been automatically unchecked? This is to prevent us from accidentally creating multiple activities that could start the workflow.
Create a Cancel activity. Select Cancel in the Business methods list box. Make sure ReceiveAndSendReply is selected in the Messaging wrapper activity type dropdown list and click the button to add it to the Activities grid.
You should now have 3 activities in the Activities grid.
Click on the Add to Code Generation button to queue the definitions.
By now, you should be familiar with the tool-bar in the 4. Generate Workflow Activities section, so we will skip repeating the explanations.
At the Generate group box at the bottom of the section, check on the Workflow xaml check box to instruct Vector to create a blank xaml file for us.

Warning! You only need to do this for the first time you are creating the workflow service. If you already have a workflow service xaml with the same name, you will be prompted whether to overwrite it. Take extra caution on this!
Make sure the Base wrapper activity check box is checked. This will instruct Vector to create the base class that provides basic functionality to the wrapper activity classes. Your code will fail to compile if you do not check this.

Warning! You only need to do this once when you create your first workflow activity. You can safely omit this on subsequent workflow activity creation.
Check the Service configuration check box to let Vector register the endpoints in the Host project config file.
If you have created the WorkflowInstanceStore database before beginning this guide, you can check the Persistence store check box and click on the Connect button to connect to your SQL Server.
At the Connect... dialog, Enter the Server name and provide the relevant authentication details to your SQL Server. Click the Connect button to connect.
Back at the 4. Generate Workflow Activities section, the WorkflowInstanceStore database should be available for selection in the dropdown list. Select WorkflowInstanceStore.
Click the Generate button to generate the workflow service and activity classes.
At the Code Generation Completed dialog, click OK.
Vector will generate code to all 3 of the workflow projects - Tutorial.Workflows, Tutorial.Workflows.Activities and Tutorial.Workflows.Designers.

Post Code-Generation Steps

After generating the code, you can begin designing the workflow using the Visual Studio workflow designer.

Open the LeaveWorkflowService.xaml in Visual Studio and design the workflow accordingly.
Drag a Flowchart activity from the Visual Studio Toolbox pane into the workflow designer surface.
Drag the ApplyReceiveSend activity that we just code generated into the flowchart.

Tip: If you are curious and look at the code of the ApplyReceiveSend activity, you will notice that it is just a design time activity that instructs the designer to wrap a ReceiveActivity and SendReplyActivity around the Apply activity. You can also achieve the same result by dragging the activities manually and hooking them up, but Vector just made it simpler for you.
Connect the Start to the ApplyReceiveSend activity.
One important step that needs to be done with the activities generated by Vector is to set the CorrelationID.
Expand the ApplyReceiveSend activity by double-clicking on it in the Workflow Designer.
Click on the Apply Receive activity.
At the properties pane in Visual Studio, locate the CorrelatesOn property. Click on the ellipses [...] button.
At the CorrelatesOn Definition dialog, define a key by selecting the CorrelationID from the Leave entity. The XPath will be automatically created for you. Click OK and you are done.

You can continue to complete your workflow following the ASPNET-WCF-DAAB sample found in the Layered Architecture Sample for .NET. If you do not complete the workflow with the Approve and Cancel activities, you will still be able to proceed with the next tutorial.

Summary

In this tutorial, we have gone through how to use the Vector: Workflow Services and Activities Generator to create a workflow service and activities. Vector attempts to make the development workflows faster and easier by providing a means to quickly expose your business methods as activities. You are welcomed to change the generated code but take note that any changes made in your code will not be reflected in the definitions stored in Vector. Therefore, practice caution when you are regenerating code on existing code file.

Coming up next - How-To: Create User Interface Process Components with Impulse in LASG.

How-To: Create Service Layer Components with Velocity in LASG

2014-03-13T18:03:00.000+08:00

This walk-through will get you familiarized with the Velocity: Service Layer Component Generator that is available in Layered Architecture Solution Guidance 2013 to create service layer components. We are now at the fourth step in the LASG Code Generation process where we will be exploring the creation of service layer components.

Before beginning this guide, please make sure that you have completed the previous tutorial on Creating Business Components with Motion in LASG. Please make sure that your solution is fully compilable without any errors before starting. This walk-through also uses the LeaveSample database that is part of the Layered Architecture Sample for .NET but you may use your own database to follow through.

In Solution Explorer, expand the Tutorial solution, go to the Services folder.
Right-click on any one of the Tutorial.Services or Tutorial.Services.Contracts project and on the context menu, click Add Services and Service Contracts...
This will launch the Velocity: Service Layer Component Generator which contains 4 sections. Take note that Velocity is designed to generate Windows Communication Foundation (WCF) services and therefore, you are encouraged to acquire some knowledge of WCF to understand the terminologies used.
By default, Velocity will attempt to load the business project assembly from the centralized bin folder located under the solution (i.e. Tutorial\bin) and populate the details in the 1. Define Service section. In this exercise, the Tutorial.Business.dll should be automatically loaded.

Note: If the business project assembly is not loaded, please exit Velocity and return to Visual Studio to make sure that there are business components in the Tutorial.Business project and the solution is compilable without errors. Also please ensure that all dependent assemblies are located in the centralized bin folder. You can also manually load the business project assembly by clicking the Load button and selecting the .dll file.
All business components in the assembly will be listed in the Business components dropdown list. Make sure that LeaveComponent is selected.
Velocity will attempt to suggest the Service name for us. By default, it will replace the Component suffix from the business component with a Service suffix. Leave the default suggestions and let Velocity create a LeaveService component for us.
By default, the namespace is left blank but you can enter one of your choice it in the Namespace text box or click the button to get Velocity to suggest one for you. For this exercise, we will leave the namespace as blank.

Note: WCF will automatically give a service namespace of http://tempuri.org if the namespace is left blank.
You can also set the Instance Context Mode and Concurrency Mode of your service. Verify that Instance Context Mode is PerCall and Concurrency Mode is Multiple.

Tip: For more information, read Session, Instancing, and Concurrency.
Velocity allows you to create custom fault contracts which you can extend later, you can check the Generate fault contract check box and specify the name of the fault contract. All fault contracts will be suffixed with Fault. For this exercise, we will not create any fault contracts.
Since most of the values have been automatically filled in for us, please ensure that the settings that you have are somewhat like the following:
At the 2. Select Business Methods section, the Business methods grid lists all the business methods that are available in the business component. Select the ListLeaves and GetLeave methods by checking on the their check boxes in the first column.

Note: Velocity exposes business methods as service methods in a one-to-one mapping.
Define an endpoint for the new service. This only needs to be done once when we first create a service. At the 3. Define Endpoint Configurations section, make sure basicHttpBinding is selected in the Binding dropdown list. Leave the suggested Endpoint name and Address. Click the button to add it to the Service endpoints grid.

Tip: You can create multiple endpoints for your services based on WCF's guidelines. If you are using a HTTP based binding and want to use SSL, you can also check the SSL/HTTPS enabled check box to allow Velocity to generate the basic configuration settings for you. For this exercise, we will not create any extra endpoints or use SSL.

Note: The Address is only used in non-IIS/WAS hosts i.e. your own Console host. Since most enterprise WCF services are usually (and recommended to be) hosted on IIS/WAS, this setting is seldom used and is just retained for backward LASG compatibility.
Click on the Add to Code Generation button to queue the definition.
Like the other code generators, you can perform various action from the tool-bar at the 4. Generate Service Components section. You can Preview the code that will be generated, set Options, reposition the methods, Reset the definitions so that they will be code-generated again or Delete the definitions.

Warning! Deleted definitions will not be recoverable.

If you Preview the code, you will noticed that the preview window now opens up in split view to show you the code of the service interface and the service implementation classes.

Note: All definitions will be marked as code generated (indicated by the IsUpdated property) after code generation. They will continue to appear in the list but will not be code generated the next time. Clicking Reset will change one or more definitions' IsUpdated value to true and mark them for code generation again.

You can also Delete a method, a service component definition or all service component definitions from the tool-bar. Clicking on the Options icon will open the Code Generation Options dialog that will allow you to set a few standard options. Leave the settings alone and click OK.
Make sure the Generate service configuration check box is checked.
Click the Generate button to generate the service layer component classes.
At the Code Generation Completed dialog, click OK.
The Tutorial.Services project should contain a generated LeaveService code file and the Tutorial.Services.Contracts projects should contain a generated ILeaveService code file. Take note that this is WCF's requirement that all services must be accommodated by a service contract (or interface).

Summary

We have basically gone through how to use the Velocity: Service Layer Component Generator to create service layer components for our layered application. As you can see, Velocity is very easy to use as it provides most of the default values that are needed for code generation. Since Velocity generates WCF service components, it is recommended that you do some read-up on WCF to fully understand what is going. You can still change the generated code but take note that any changes made in your code will not be reflected in the definitions stored in Velocity. Therefore, practice caution when you are regenerating code on existing code file.

Coming up next - How-To: Create Workflow Services and Activities with Vector in LASG.

How-To: Create Business Components with Motion in LASG

2014-03-12T13:21:00.002+08:00

With the Entities and Data Access Component classes generated, it is now the time to look at the Motion: Business Component Template Generator that is available in Layered Architecture Solution Guidance 2013 to create business components. Creating business components is the third step in the LASG Code Generation process. Before beginning this guide, please make sure that you have completed the previous tutorial on Creating Data Access Components with Momentum in LASG. Please make sure that your solution is fully compilable without any errors before starting.

This walk-through also uses the LeaveSample database that is part of the Layered Architecture Sample for .NET but you may use your own database to follow through.

In Solution Explorer, expand the Tutorial solution, go to the Business folder, right-click on the Tutorial.Business project and click Add Business Components...
This launches the Motion: Business Component Template Generator that comprises of 3 sections.
At the 1. Define Business Component section, enter Leave in the Create new component text box and leave the default Suffix as Component. Motion will create the class with the name LeaveComponent for us.

Tip: After you have defined business components and successfully compile them in the project, they will appear in the Use existing component dropdown list the next time you launch Motion.
By default, Motion will attempt to load the data project assembly from the centralized bin folder located under the solution (i.e. Tutorial\bin). All projects unfolded by LASG (except the Host and Web projects) will target this folder. In this exercise, the Tutorial.Data.dll should be automatically loaded.

Tip: All the data access components and their methods will be listed in the Data access component methods tree-view in the 2. Select Data Methods to Call in Business Method section if the data assembly is loaded correctly. Take note that it is normal for the tree-view to be disabled.

Note: If the data project assembly is not loaded, please exit Motion and return to Visual Studio to make sure that there are data access components in the Tutorial.Data project and the solution is compilable without errors. Also please ensure that all dependent assemblies are located in the centralized bin folder. You can also manually load the data project assembly by clicking the Load button and selecting the .dll file.
Create the Apply business method. Enter Apply in the Business method name text box. The Data access component methods tree-view should be enabled now.

Tip: It is recommended that you use business-like terms (i.e. Apply, Submit, Cancel, Approve, List etc.) to name your business methods instead of technical terms such as Insert, Update, Delete and Select. For example, it makes more sense to name your business method as Register for adding a user instead of calling it InsertUser.
Expand the LeaveDAC and LeaveStatusLogDAC nodes in the tree-view. If you remembered, these are the two DACs which you have created in the previous tutorial. Check on the check boxes beside the Create method of the LeaveDAC and the Create method of the LeaveStatusLogDAC.
Click the button to add them to the Called methods grid.

Tip: You can also drag-and-drop methods from the Data access component methods tree-view to the Called methods grid. You can't select or check a DAC. If you have made any mistakes and want to start over, click on the Reset button.
Check on the check box in the Return grid column for the Create method of the LeaveDAC component. This will return the updated entity after insertion.

Tip: You can use the Top and Bottom arrow buttons to re-arrange your called methods.
Click on the Add to Code Generation button to queue the definition.
Create the Approve business method. Enter Approve in the Business method name text box.
Check on the UpdateById method of the LeaveDAC and the Create method of the LeaveStatusLogDAC.
Click the button to add them to the Called methods grid.
Uncheck any check boxes in the Return grid column.
Click on the Add to Code Generation button to add the definition to the code generation queue.
Create the Cancel business method. Enter Cancel in the Business method name text box.
Repeat the same steps from step 11 to 14 for the Cancel method.
Create the business method to list leaves. Enter ListLeaves in the Business method name text box.
Drag the SelectPage method under the LeaveDAC node to the Called methods grid. Verify that the check box at the Return grid column is checked.
Click on the Add to Code Generation button to add the definition to the code generation queue.
Create the business method to retrieve a Leave. Enter GetLeave in the Business method name text box.
Drag the SelectById method under the LeaveDAC node to the Called methods grid. Verify that the check box at the Return grid column is checked.
Click on the Add to Code Generation button to add the definition to the code generation queue.
By now you should have 5 business methods in your LeaveComponent queue - Apply, Approve, Cancel, ListLeaves and GetLeave.
Next, we will make some modifications to the methods. At the 3. Generate Business Components section, select the Apply method in the list below the LeaveComponent and locate the CallMethods property in the property grid. Click on the ellipses [...] button to invoke the MethodModel Collection Editor dialog.
At the MethodModel Collection Editor dialog, select the Create method that belongs to the LeaveStatusLogDAC. Locate the Parameters property in the property grid on the right and click on the ellipses [...] button to invoke the ParameterModel Collection Editor dialog.

Tip: If you want to wrap your methods in a TransactionScope, you can set the TransactionBegin property to true for the first method in the scope and set the TransactionEnd property to true for the last method in the scope. Please take note that it is your responsibility to arrange the sequence of the method calls correctly. For this exercise, we will not define any transaction scope.
At the ParameterModel Collection Editor dialog, select the leaveStatusLog parameter and set its IsInternallyDeclared property to true. This will indicate that this parameter will be declared in code and not at the method signature.
Click OK to close the ParameterModel Collection Editor dialog.
Click OK to close the MethodModel Collection Editor dialog.
Repeat these steps (24 to 28) for the Approve and Cancel methods.
Like the other code generators, you can perform various action from the tool-bar at the 3. Generate Business Components section. You can Preview the code that will be generated, set Options, reposition the methods, Reset the definitions so that they will be code-generated again or Delete the definitions.

Warning! Deleted definitions will not be recoverable.

Note: All definitions will be marked as code generated (indicated by the IsUpdated property) after code generation. They will continue to appear in the list but will not be code generated the next time. Clicking Reset will change one or more definitions' IsUpdated value to true and mark them for code generation again.

You can also Delete a method, a BC definition or all BC definitions from the tool-bar. Clicking on the Options icon will open the Code Generation Options dialog that will allow you to set a few standard options. Leave the settings alone and click OK.
Finally, click the Generate button to generate the business component classes.
At the Code Generation Completed dialog, click OK.
The Tutorial.Business project should contain the generated LeaveComponent code file.
Extra Tip: If you are regenerating code and the code file already exist in the project, you will be prompted to either Append, Backup, Overwrite or Cancel the process.

Append will append whatever new methods to the bottom of the class. Take note it is not an intelligent append and duplicate methods may be appended to your class. You also need to make sure that you do not have any comments or commented code at the end of your class files.

Backup creates a copy of your code file with a .bak extension and requires you to manually include its contents into your project.

Overwrite will replace your existing code with the newly generated one. Any previously defined methods that were not included in the latest code generation will be lost.

Cancel will discard the generated code from memory and preserves the existing code file.

You can choose to apply the same option for all the files by checking the Do the same for all occurrence check box.

Adding Business Logic

By now the template of your business component should have been generated and you can proceed to implement your own business logic.

As an example, we can modify the Apply method to contain some logic:

public Leave Apply(Leave leave)

{

Leave result = default(Leave);

leave.DateSubmitted = DateTime.Now;

leave.Status = 0; // Pending

// Data access component declarations.

LeaveDAC leaveDAC = new LeaveDAC();

LeaveStatusLogDAC leaveStatusLogDAC = new LeaveStatusLogDAC();

// Step 1 - Calling Create on LeaveDAC.

result = leaveDAC.Create(leave);

var leaveStatusLog = new LeaveStatusLog()

{

LeaveID = result.LeaveID,

Status = result.Status,

Date = DateTime.Now

};

// Step 2 - Calling Create on LeaveStatusLogDAC.

leaveStatusLogDAC.Create(leaveStatusLog);

return result;

}

For a more complete implementation of the business logic, you can refer to the ASPNET-WCF-DAAB sample from Layered Architecture Sample for .NET

Summary

We have basically gone through how to use the Motion: Business Component Template Generator to create business components for our layered application. Take note that the methods generated only contain the basic structure without any logic. You will still need to add-in your own business processing logic to them but take note that any changes made in your code will not be reflected in the definitions stored in Motion. Therefore, practice caution when you are regenerating code on existing code file.

Coming up next - How-To: Create Service Layer Components with Velocity in LASG.

How-To: Create Data Access Components with Momentum in LASG

2014-03-10T15:52:00.002+08:00

This is the third in the series where we will get you started with the Momentum: Data Access Component Generator that is available in Layered Architecture Solution Guidance 2013 to create data access components. Creating data access components is the second step in the LASG Code Generation process. Before beginning this guide, please make sure that you have completed the previous tutorial on Creating Entities with Inertia in LASG.

This walk-through also uses the LeaveSample database that is part of the Layered Architecture Sample for .NET but you may use your own database to follow through.

In Solution Explorer, expand the Tutorial solution, go to the Data folder, right-click on the Tutorial.Data project and click Add Data Access Components...
This launches the Momentum: Data Access Component Generator. The code generator is divided into 4 sections.

Tip: Notice like all the code generators, Momentum is colour coded to the code generation guide.
At the 1. Connect to Database and Select Data Source section, click on the Connect button to connect to a Microsoft SQL Server.
In the Connect... dialog, enter the Server name and select the appropriate authentication method. Click Connect to connect to the SQL server.

Note: LASG uses the SQL Server Management Objects and therefore, only Microsoft SQL Server is supported. You can use any editions of SQL Server including the Express edition.
Select the LeaveSample database from the database list located beside the Connect button. All the tables available in the database will be listed in the Sources dropdown list.
Select the dbo.Leaves table from the Sources dropdown list.
Momentum will attempt to suggest the Entity name and Data Component name for you based on the selected table's name. By default, the components are given a "DAC" suffix. You may change that to your own naming convention (i.e. Repository) if desired.

Tip: If your tables have plural names, the Singularize class names option will attempt to suggest singular entity class names for you.
At the 3. Define Data Access Methods section, you can select the Method type to create. For this tutorial, we will be creating many methods. First, select Insert | Update | Select from the Method type dropdown list.
Click on the Columns check box to select all the columns.

Note: If you have previously changed the name of the properties during the creation of your entity, you will also need to change them here to match it. Similar to when you are creating entities, you can click on the bar above the Columns grid to launch the Define Column Mappings dialog to fix any abbreviated column names.
Click on the Add to Code Generation button to queue the data access component definition to the 4. Generate Data Access Components section. You should see 3 methods in the list - Create, UpdateById and SelectById under the LeaveDAC. Their properties are shown in the property grid below the section. You may make any last moment changes here.

Caution! Tampering with the properties in the property grid may cause unexpected results or errors in code generation. Make sure you understand the behaviour of the code generator before messing with the values in the property grid.
Create another method. From the Method type dropdown list, choose Select (Paging). Leave the method name as SelectPage.
You will need to specify an Order column for the SelectPage method. Click on the check box in the Order grid column for DateSubmitted.

Tip: If you are creating many Select methods, you may want to standardize them to use one common method to load the entities. You can achieve this by checking the Use Load method check box and entering the method name. To generate the method, check the Generate Load method check box for the first Select method you create and uncheck it for subsequent methods. For this tutorial, we will leave the checkboxes unchecked.

Warning! Checking these check boxes incorrectly will cause duplicate Load methods to be generated and will result in your code not compiling.
Click the Add to Code Generation button to add the method to the LeaveDAC definition.
We will need the SelectPage method to return the results ordered by descending. To do this, select the SelectPage method in the list below the LeaveDAC definition. Locate the OrderByColumns in the property grid and click on the ellipses [...] button beside it.
At the OrderColumnModel Collection Editor dialog, make sure DateSubmitted is highlighted and locate the OrderDirection property in the property grid on the right. Change the value to Descending and click OK.
Create another data access component for the LeaveStatusLogs table. Select dbo.LeaveStatusLogs in the Sources dropdown list.
Choose Insert in the Method type dropdown list.
Check the Columns check box to ensure all columns are selected.
Click the Add to Code Generation button to queue the definition.
Create a Select method for the LeaveStatusLogDAC. Choose Select in the Method type dropdown list.
We will define a filter for this method. Check the check box in the Filter grid column for LeaveID.

Note: Notice that the Referenced tables dropdown list is now enabled and the Reference columns list shows all the columns that belongs to the referenced table. If you have borrowed columns from other tables previously in Inertia, you will need to specify them again in here for your DAC. Momentum will automatically create a JOIN clause in the generated SQL statement for you.

To add a borrowed column, select the reference column and click the [ + ] button below the Reference column list. To remove any borrowed columns, simply select it in the Columns grid and press the Delete key on the keyboard. For this exercise, we shall skip adding any borrowed columns.
At the 4. Generate Data Access Components section, you can perform various action from the tool-bar. You can Preview the code that will be generated, set Options, reposition the methods, Reset the definitions so that they will be code-generated again or Delete the definitions.

Warning! Deleted definitions will not be recoverable.

Note: All definitions will be marked as code generated (indicated by the IsUpdated property) after code generation. They will continue to appear in the list but will not be code generated the next time. Clicking Reset will change one or more definitions' IsUpdated value to true and mark them for code generation again.

You can also Delete a method, a DAC definition or all DAC definitions from the tool-bar. Clicking on the Options icon will open the Code Generation Options dialog that will allow you to set a few standard options. Leave the settings alone and click OK.
Check the Add connection string to host config file check box to instruct Momentum to create a connection string entry in the config file of your host project. Also make sure the Generate base class check box is checked for the first time you generate classes for your data access component project.

Note: Momentum supports Enterprise Library DAAB 5.0 and 6.0. By default, it will generate code to use the SqlDatabase class but if you prefer to use the DatabaseFactory.Create method, you can check the Use factory check box. Make sure you also check the Use EntLib 6.0 check box if you are using Enterprise Library DAAB 6.0 when using factory.
Before proceeding, verify you have defined the following DAC definitions in the 4. Generate Data Access Components section:

LeaveDAC - Create, UpdateById, SelectById and SelectPage methods.
LeaveStatusLogDAC - Create and Select methods.
Finally, click the Generate button to generate the data access component classes.
At the Code Generation Completed dialog, click OK.
The Tutorial.Data project should contain the generated code files for the data access components.
Extra Tip: If you are regenerating code and the code file already exist in the project, you will be prompted to either Append, Backup, Overwrite or Cancel the process.

Append will append whatever new methods to the bottom of the class. Take note it is not an intelligent append and duplicate methods may be appended to your class. You also need to make sure that you do not have any comments or commented code at the end of your class files.

Backup creates a copy of your code file with a .bak extension and requires you to manually include its contents into your project.

Overwrite will replace your existing code with the newly generated one. Any previously defined methods that were not included in the latest code generation will be lost.

Cancel will discard the generated code from memory and preserves the existing code file.

You can choose to apply the same option for all the files by checking the Do the same for all occurrence check box.

Summary

We have basically gone through how to use the Momentum: Data Access Component generator to create data access components for our layered application. Take note that the methods generated are just the basic logic. You can still make any necessary changes to the code as you please but take note that any changes made in your code will not be reflected in the definitions stored in Momentum. Therefore, practice caution when you are regenerating code on existing code file.

Coming up next - How-To: Create Business Components with Motion in LASG.

How-To: Create Entities with Inertia in LASG

2014-03-05T16:04:00.001+08:00

This walk-through will get you started with the Inertia: Business Entity Component Generator that is available in Layered Architecture Solution Guidance 2013. The first step in creating layered applications after designing the database is to create the entity classes. Before beginning this guide, please make sure that you have completed the previous tutorial on Getting Started with LASG.

This walk-through also uses the LeaveSample database that is part of the Layered Architecture Sample for .NET but you may use your own database to follow through.

In Solution Explorer, expand the Tutorial solution, go to the Shared folder, right-click on the Tutorial.Entities project and click Add Entities...
The Inertia: Business Entity Component Generator will launch. The code generator is divided into 3 sections.

Tip: Notice the code generator is colour coded to the code generation guide.
At the 1. Choose Data Source and Columns section, click the Connect button to connect to a Microsoft SQL Server.
In the Connect... dialog, enter the Server name and select the appropriate authentication method. Click Connect to connect to the SQL server.

Note: LASG uses the SQL Server Management Objects and therefore, only Microsoft SQL Server is supported. You can use any editions of SQL Server including the Express edition.
Select the LeaveSample database from the database list located beside the Connect button.
In the Source tab, select the Leaves table from the Sources dropdown list. All its columns will be listed in the list box below the Columns check box.

Tip: Inertia will attempt to suggest the entity class name for you based on the selected table’s name. If your tables have plural names, the Singularize class names option will attempt to suggest singular entity class names for you.
If you are not using the LeaveSample database tables and have abbreviated column names i.e. hr_wkflowID, PrdName, StdMercName etc., you can create meaningful property names for them by clicking on the bar above the column list. This launches the Define Column Mappings dialog. Make any necessary changes to the Property Name and click OK when done.

Note: It is recommended that you have well defined meaningful column names in the tables than to rely on the column mappings.
Check on the Columns check box to select all the columns and click on the button to add them to the 2. Define Entity Settings section.

Tip: You can also individually select or deselect columns. Press the Ctrl key while clicking on the columns to select/deselect individual columns or use the Shift key to select/deselect a range of adjacent columns.

Note: Not all data-types are supported. Complex data-types are currently not supported.
The Reference To tab will show all the tables that the current table is referencing to with a foreign key. You can choose to relate to the entire table as a reference object or individually select the columns which you want to include as part of the current entity (a.k.a. borrowed columns). Once you are done with your selections, click on the button to add them to the Entity properties grid. For simplicity, we shall not add anything from here in this tutorial.
The Reference By tab shows all the tables that is referencing the current table with a foreign key. You can choose to relate the tables here as a collection of Entities. Click on the button to add them to the Entity properties grid. You will also need to generate the Entity classes individually for the tables you selected here later. For simplicity, we shall not add anything from here in this tutorial.
The columns that you choose will appear in the Entity properties grid under the 2. Define Entity Settings section as properties to be generated for the Entity. You can make adjustments here i.e. changing the comments or marking a property to accept nulls.

Tip: In case you want to remove any columns from the properties list, just select the row and press the Delete key on the keyboard or your can right-click on the selected row to invoke a context menu to delete it.
Once you are ready, click on the Add to Code Generation button to queue the definition to the 3. Generate Entities section. You can queue as many entities for code generation as you wish. To create another entity, simply click on the New button. For this tutorial, we shall create 2 entities - Leave and LeaveStatusLog.
Each queued entity definition will be listed in the 3. Generate Entities section. You can review the definitions and make any last moment changes in the property grid located at the bottom of the section. You can delete any entity properties by selecting the row and pressing the Delete key.

Caution! Tampering with the properties in the property grid may cause unexpected results or errors in code generation. Make sure you understand the behaviour of the code generator before messing with the values in the property grid.
You can perform various action from the tool-bar at the 3. Generate Entities section. You can Preview the code that will be generated, set Options, reposition the properties, Reset the definitions so that they will be code-generated again or Delete the definitions.

Warning! Deleted definitions will not be recoverable.

Note: All definitions will be marked as code generated (indicated by the IsUpdated property) after code generation. They will continue to appear in the list but will not be code generated the next time. Clicking Reset will change one or more definitions' IsUpdated value to true and mark them for code generation again.

You can also Delete an entity property, an entity definition or all entity definitions from the tool-bar. Clicking on the Options icon will open the Code Generation Options dialog that will allow you to set a few standard options. Leave the settings alone and click OK.
Finally, click the Generate button to generate the entity classes.
At the Code Generation Completed dialog, click OK.
The Tutorial.Entities project should contain the generated code files for the entities.

Summary

We have basically gone through how to use the Inertia: Business Entity Component Generator to create Entity classes for our layered application. You can make any necessary changes to the code but take note that any changes made in your code will not be reflected in the definitions stored in Inertia. Therefore, practice caution when you are regenerating code on existing code file.

Coming up next - How-To: Create Data Access Components with Momentum in LASG.

Boost Android Emulator Performance

2014-02-27T23:23:00.002+08:00

As a n00b trying to figure out Android development, I fell into the same experience of many who have wondered and complained about the slow performance of the Android Emulator. Fortunately, I was pointed to this article before a training. The article describes that we can actually download and install the Intel® Hardware Accelerated Execution Manager (HAXM) to boost the performance of the Android emulator on Windows machines.

Since I'm running on Windows Server 2012 R2, I downloaded the version which promises to be a hotfix for Windows 8.1 but when I try to install it, I got the following:

This computer does not support Intel Virtualization Technology (VT-x). HAXM cannot be installed. Please refer to the Intel HAXM documentation for more information.

I was puzzled by the error message since my machine has Hyper-V installed and I'm pretty sure that my VT-x was enabled in the BIOS. I proceeded to download the Intel® Processor Identification Utility just to be sure and I got the following:

What the ...

Anyway, after some research I found out that the hypervisor layer that Hyper-V installed was interfering with the tool. Now, I'm caught in-between. I am running on a Server workstation (is there such a term?) and I need Hyper-V for my VMs and Windows Phone 8 development. I don't want to un-install Hyper-V just because I wanna try out some Android development.

Fortunately, I discovered a trick which can let me boot my Windows without loading the hypervisor layer temporarily. To achieve this, some bcdedit acrobatics is required. Please take note that playing with bcdedit may lead you into disastrous results such as not being able to boot up your machine - so don't say I didn't warn ya. ;)

If you open up a command prompt (in Administrator mode please!) and type in bcdedit, you should get something like the following:

Notice the hypervisorlaunchtype is set to Auto. Now, let's create a backup entry of that for our no Hyper-V version (cos I don't want to screw up the original). Issue this command to clone an entry.

bcdedit /copy {current} /d "Windows Server 2012 - No Hyper-V"

You should get a message telling you that the current entry has been copied to a new entry with some identifier GUID. If you type bcdedit again, you should be able to see the new entry with the same hypervisor setting. Issue the following command to disable it.

bcdedit /set {the-identifier-GUID-of-the-entry-you-just-created} hypervisorlaunchtype off

If you have done it correctly, you should be able to get something like the following:

Now that the setting is off, you can reboot your machine (and pray) and when prompted, select to boot with the Windows Server 2012 - No Hyper-V entry. You should not feel any difference other than Hyper-V is being switched off. With this you can now install HAXM without any problems.

Note: When installing HAXM, you will be asked to allocate an amount of memory. I took the default 2GB.

Now stuff doesn't just stop here. After you have HAXM installed, launch the Android SDK Manager from your Android IDE i.e. eclipse or Xamarin. Make sure you checked on the Intel x86 Atom System Images for the Android API levels that you want and install it.

Also remember to scroll to the bottom and check the Intel x86 Emulator Accelerator (HAXM) to install it.

Once they are installed, launch the Android Virtual Device Manager and create a new Android Virtual Device (AVD). At the create screen, select Intel Atom x86 for the CPU/ABI and make sure you check Use Host GPU.

Note: If you decide to put anything higher than 768 for RAM here you will get a warning saying:

"On Windows, emulating RAM greater than 768M may fail depending on the system load. Try progressively smaller values of RAM if the emulator fails to launch."

You may ignore that and give it something larger. I gave mine 2048 but if you try to start the emulator now, you may encounter the following error:

To fix this, go to your user folder and locate the folder of the AVD that you are trying to start i.e.

C:\Users\Administrator\.android\avd\Note3.avd

Open up the config.ini file with a text editor and locate the hw.ramSize= entry. Change whatever value that it has to suffix with an 'mb'. i.e.

hw.ramSize=2048mb

Save it and whalla! Your Android Virtual Device can now be started and will run at awesome speed.

Get Asset Tag from BIOS

2014-02-27T10:11:00.001+08:00

It was that time of the year where the auditors are visiting again and I found out this handy command to retrieve the Serial Number and Asset Tag from the system BIOS.

Open a command prompt, and type in:

wmic SystemEnclosure get SerialNumber, SMBIOSAssetTag

... and the values will be shown.

Note: Somehow the BIOS on my Dell machine only allow up to 10 characters for the tag. If your company uses anything longer, then it won't fit :'( Darn! And I was hoping I don't have to flip my notebook the next time they wanna see it.

How-To: Get Started with LASG

2014-02-26T13:36:00.000+08:00

Welcome to the first tutorial in a 7 series walk-through to introduce you to the Layered Architecture Solution Guidance (LASG) visual studio extension. This tutorial will get you started in using the extension to unfold the basic structure for a layered web application and in the upcoming posts, we will be going through each of the code-generators individually.

Launch Visual Studio 2013 with administrator privileges.
Open the File menu, click New and then click Project...
At the New Project dialog, under Installed Templates on the left pane, expand Guidance Packages and click Layered Architecture Solution Guidance.

Leave the Layered Application solution template selected as default.
Enter a Name for the solution i.e. Tutorial and click OK.
At the Choose LASG Solution Template dialog, select Layered Web Application.

Note: The Project Namespace will follow the solution name that was provided earlier. You can also check/uncheck the projects which you want to add to/remove from your solution from the Project templates list.
Click OK, and wait for the projects to unfold.
For Layered Web Applications, the New ASP.NET Project dialog box will be displayed. This is Visual Studio 2013's default prompt for creating Web Projects. Choose the Empty template and check the Web Forms check box in the Add folders and core reference for.

Note: You may choose other predefined Visual Studio 2013 web templates if you wish.
Click OK to complete the unfolding process. Once completed, the Layered Web Application solution is now ready for use.

Tip: All the relevant project references were automatically added between the projects for your convenience.

Note: At this point of writing, LASG only supports English version of Visual Studio.
If you have not installed Enterprise Library - Data Access Application Block on your machine, you may need to NuGet it. Right-click on the Tutorial.Data project and select Manage NuGet Packages...
Search for Enterprise Library and select the Enterprise Library - Data Access Application Block package for installation.

Summary

We have basically gone through the simple steps of unfolding a layered web application using LASG.

If you would like to learn more about Layered Architecture concepts, please visit the following links:

Coming up next - How-To: Create Entities with Inertia in LASG.

Runtime Caching with SqlChangeMonitor

2014-02-24T16:35:00.002+08:00

I was looking for a cache solution similar to the one provided by System.Web.Caching for my services layer and I was introduced to System.Runtime.Caching by my associate (after he completed the research assignment which I gave him that is ;p ). New in .NET 4, System.Runtime.Caching provides an easy to use cache solution for non-ASP.NET Web applications and is just what I needed.

Sometime back I talked about Query Notifications & SqlDependency. Now, I'm gonna do something similar but I will attempt to wire-up a SqlDependency to the SqlChangeMonitor to allow our cache data to be auto-refreshed when there are changes to the database table. This is useful for caching reference data for services that are built on WCF (or WEB API).

I will show it with a very rough code example. Please refactor it to your needs.

Start by declaring a few global variables.

private static MemoryCache _cache;

private SqlChangeMonitor _monitor;

private SqlDependency _dependency;

private bool _hasDataChanged;

The static MemoryCache variable is defined for coding convenience so that we don't have to keep referring to MemoryCache.Default all the time.

At the constructor of the class, initialize the _cache and call the SqlDependency.Start() method.

_cache = MemoryCache.Default;

SqlDependency.Start(CONNECTION_STRING);

Next, create the main function that loads the data into a CacheItem:

private CacheItem LoadData(out CacheItemPolicy policy)

{

const string SQL_STATEMENT = "SELECT [ID], [Data] FROM dbo.TestData";

var data = new List<TestData>();

var db = new SqlDatabase(CONNECTION_STRING);

using (DbCommand cmd = db.GetSqlStringCommand(SQL_STATEMENT))

{

// Initialize SqlDependency

_dependency = new SqlDependency(cmd as SqlCommand);

_dependency.OnChange += dependency_OnChange;

using (IDataReader dr = db.ExecuteReader(cmd))

{

while (dr.Read())

{

// Create a new TestData

var testData = new TestData();

// Read values.

testData.ID = GetDataValue<int>(dr, "ID");

testData.Data = GetDataValue<string>(dr, "Data");

// Add to List.

data.Add(testData);

}

// Create a new monitor.

_monitor = new SqlChangeMonitor(_dependency);

// Create a policy.

policy = new CacheItemPolicy();

policy.ChangeMonitors.Add(_monitor);

policy.UpdateCallback = CacheUpdateCallback;

// Put results into Cache Item.

var item = new CacheItem("TestData", data);

// Reset the data changed flag.

_hasDataChanged = false;

return item;

}

Few things to note in this code-snippet:

The CONNECTION_STRING is a constant I defined for my connection string and its value is not shown in this example.
I'm using Enterprise Library Data Access Application Block (DAAB) but you can use standard ADO.NET.
I'm loading the data into an Entity called TestData with a custom GetDataValue method that checks for null before assigning the values.
I'm using a hard-coded SQL statement but you can change it to a stored procedure if you wish.

In the example, we pass the command into a new SqlDependency instance after creating it and subscribe to the OnChange event. After loading the data, we create a new SqlChangeMonitor and pass in the SqlDependency instance to its constructor. We then create a new CacheItemPolicy and add the SqlChangeMonitor to its ChangeMonitors collection. Next, we register to the UpdateCallback and create a new CacheItem to store the data. Finally, we set the _hasDataChanged flag to false before returning the CacheItem.

The OnChange event doesn't really do much other than setting the _hasDataChanged flag to true when there is a change in the TestData table.

void dependency_OnChange(object sender, SqlNotificationEventArgs e)

{

// DataChange Detection

_hasDataChanged = true;

}

The CacheUpdateCallback method contains a little more code to refresh the cache item in memory.

private void CacheUpdateCallback(CacheEntryUpdateArguments args)

{

// Dispose of monitor

if (_monitor != null)

_monitor.Dispose();

// Disconnect event to prevent recursion.

_dependency.OnChange -= dependency_OnChange;

// Refresh the cache if tracking data changes.

if (_hasDataChanged)

{

// Refresh the cache item.

CacheItemPolicy policy;

args.UpdatedCacheItem = LoadData(out policy);

args.UpdatedCacheItemPolicy = policy;

}

The _monitor and the OnChange event subscription will be cleaned up in this method and the _hasDataChanged flag is checked to see if the items in cache should be refreshed. Without doing this, the data in the cache will be invalidated and reset to null.

Finally, the method that we want to expose out to callers to call will be like the following:

public List<TestData> Select()

{

if (_cache["TestData"] == null)

{

// Create a policy.

CacheItemPolicy policy = null;

// Load data into Cache Item.

var item = LoadData(out policy);

// Set Cache Item into cache with the policy.

_cache.Set(item, policy);

}

return _cache["TestData"] as List<TestData>;

}

The method first checks the cache for any data and if none is available, it will query the database for it and sets it to the cache. If the data is available, it will be retrieved directly from the cache.

I hope this post is useful to you. Please make sure that you follow the rules and guidelines of SQL Server Service Broker (i.e. ALTER DATABASE LeaveSample SET ENABLE_BROKER) and Query Notifications to get it working. :)

Entity Framework 6 vs EL6 DAAB Performance - Rematched

2014-02-14T21:55:00.000+08:00

After I published the benchmark results for Entity Framework (EF) vs. DAAB, I was asked to verify whether the performance of EF can be improved by using Stored Procedures (SP). Logically speaking, using SP may help improve performance, but it should improve for both frameworks and not favour one over the other. Meaning if DAAB is shown to be faster than EF, any tuning on the database side should not be able to make EF faster than DAAB - at best, it can only make EF perform better than its own previous results.

To make sure there are no stones unturned, I have decided to do a rematch of the performance testing using the same methods and machine specifications from the previous test. The only differences are the code have all been converted to use Stored Procedures.

Here's a code snippet for one of the methods used in the DAAB code that has been converted to use SP:

public Leave SelectById(long leaveID)

{

const string SQL_STATEMENT = "GetLeaveById";

Leave leave = null;

// Connect to database.

Database db = DatabaseFactory.CreateDatabase(CONNECTION_NAME);

using (DbCommand cmd = db.GetStoredProcCommand(SQL_STATEMENT))

{

db.AddInParameter(cmd, "@LeaveID", DbType.Int64, leaveID);

using (IDataReader dr = db.ExecuteReader(cmd))

{

if (dr.Read())

{

// Create a new Leave

leave = LoadLeave(dr);

}

return leave;

}

And here's the equivalent method in EF:

public Leave SelectById(long leaveID)

{

using (var db = new DbContext(CONNECTION_NAME))

{

var paramLeaveID = CreateParameter("LeaveID", DbType.Int64, leaveID);

var result = db.Database.SqlQuery<Leave>("GetLeaveById @LeaveID",

paramLeaveID).ToList<Leave>();

Leave leave = result.Count > 0 ? result[0] : null;

// Return result.

return leave;

}

All the SPs were imported and mapped in the EF Model.

I was also tipped-off by some friends who have experience in EF to try to have these settings set to improvement performance.

db.Configuration.LazyLoadingEnabled = false;

db.Configuration.AutoDetectChangesEnabled = false;

db.Configuration.ValidateOnSaveEnabled = false;

However, they don't make much difference because the Entities that were used are Plain-Old-CLR-Objects (POCO) so change-tracking is not available, and they are not linked by navigation properties, so lazy-loading won't kick-in.

Now let the rematch begin.

Unit Test: Single Run

The ApplyThenApproveTest unit test method was chosen again for the test and the results are:

Method using Entity Framework 6.0.2 took 168 ms to complete 1 transaction.

Method using Enterprise Library 6.0 DAAB took 137 ms to complete 1 transaction.

Looks like the performance of EF has indeed been improved a little with SP but the performance of DAAB has dropped slightly. Nevertheless, the performance gap between EF and DAAB is almost consistent compared to the previous non-SP test.

Unit Test: Looping in 1000

Let's see how both of them fair in loops.

Method using Entity Framework 6.0.2 took 46 secs to complete 1000 transactions.

Method using Enterprise Library 6.0 DAAB took 32 secs to complete 1000 transactions.

Once again the results show that EF performs slower than DAAB and both seems to be 2 secs slower than their previous non-SP results.

Instrumented Performance and Diagnostics Profiler

Let's dig into the profiler and see some results:

Create method in data layer for Entity Framework 6.0.2 took an Avg. Elapse Time of 1,299.91. The EF methods DbContext.SaveChanges() took Avg. Elapse Time of 1,133.98 and DbSet.Add() took 150.57.

Create method in data layer for Enterprise Library 6.0 DAAB took an Avg. Elapse Time of 23.06. The DAAB methods Database.ExecuteScalar() took Avg. Elapse Time of 19.71.

Strangely, it shows that both EF and DAAB takes longer when SPs are used (could this be due to the extra processing needed to map to SPs internally?). From the call tree we can observe that EF is still slower than DAAB.

Load Test

Now for the grand finale! Let's see how they perform in load testing.

Entity Framework 6.0.2 completed with 6945 Total Test runs with an Avg. Test Time of 0.84 sec and gave 0 errors.

Enterprise Library 6.0 DAAB completed with 7310 Total Test runs with an Avg. Test Time of 0.79 sec and gave 0 errors.

Unlike the previous non-SP test, EF runs more stable this time and produces no errors under high load. Because of that, its performance has also been able to almost match the new results for DAAB. This could be due to the more optimized SQL statements which are used in the SPs as opposed to the ones auto-generated by EF's engine. The performance of DAAB somewhat fell when using SPs.

Behind The Scene

Let's take a look at what's going on behind the scene.

INSERT operation using Stored Procedure using Entity Framework 6.0.2

INSERT operation using Stored Procedure using Enterprise Library 6.0 DAAB

SELECT operation using Stored Procedure using Entity Framework 6.0.2

SELECT operation using Stored Procedure using Enterprise Library 6.0 DAAB

From SQL Server Profiler, we can see that both EF and DAAB now use the specified stored procedures to perform their operations. Curiously, EF still uses sp_executesql for the queries but you can see that it is no longer using the verbose auto-generated SQL statements previously.

Conclusion

It is common for us to think that using Stored Procedures (SP) may help improve the performance of our applications but it seems that RDBMS have came a long way and have evolved tremendously in providing us with the required performance. It should be noted that both native ADO.NET and EF uses sp_executesql internally which provides us the safety of preventing SQL-injection attacks (when used with named parameters) and cached execution plans. While sp_executesql can also be tuned further, it is suffice for general purpose usage.

The results in this rematch may not be able to prove that EF could perform better than DAAB by employing SPs but it does prove that EF is more stable and provides better query performance with properly written SPs. The only challenge I experienced is the rigid nature of SPs which makes it difficult to accommodate EF's dynamic nature i.e. difficult to support dynamic column sorting and parameter filtering, and I would certainly not encourage constructing dynamic SQL inside SPs.

For now, I will still need to stick with DAAB.

Disclaimer: The results are based on my own research and it is just a simple benchmark for EF and DAAB. It is not about performance tuning. If you are unhappy with the results, I would urge that you conduct your own tests in your own environment for verification. You never know, some things work differently in different environments.

Entity Framework 6 vs. EL6 DAAB Performance

2014-02-10T17:04:00.002+08:00

Lately, I have encountered several questions regarding the performance of Entity Framework (EF). I have been conducting feasibility study on EF since version 4.0 and followed through several versions but every time I see the performance results, I was not convinced to use it. But I can see performance improvements in every new version. Wanting to satisfy my curiosity and also to update my impressions towards EF, I have thought to conduct another round of performance test on it.

I realized that not only have I built an arsenal of samples but I have also created an avenue to use those samples for performance testing (all thanks to my Associate who reminded me by always ~~playing with~~ using the samples in demos and test runs). For the showdown, I will pit the ASPNET-WCF-EF sample against the ASPNET-WCF-DAAB sample, since both uses exactly the same code for all the layers but only differs in data access technology (all thanks to the swap-in-swap-out capability of Layered Architecture).

The Test Machine

Windows Server 2012 R2 x64
Intel Core i7-4800MQ CPU 2.7GHz (Quad Core HT)
16 GB RAM
500 GB Solid-State Hybrid Drive
Visual Studio 2013 Ultimate Update 1
Microsoft SQL Server 2012 x64

Preparation

To make use of the samples for the performance tests, few changes had to be made to both sets of code.

1. Disabling IsOverlap Check in Business Component

The samples contain an IsOverlap business logic check to prevent duplicate records from being inserted into the Leaves table. This will cause the unit tests to fail if high volume of test data were induced into it. To allow the application to continue to run even with overlaps, I commented out the line where the exception is thrown. You can locate the line of code in the Apply method of the LeaveComponent.cs.

// Check for overlapping leaves.

if (leaveDAC.IsOverlap(leave))

{

//throw new ApplicationException("Date range is overlapping with another leave.");

}

This will allow the query to continue to run but the exception will be suppressed.

2. Disable Enable Edit and Continue

Enable Edit and Continue needs to be disabled in the LeaveSample.Hosts.Web project to allow it to be running (See here for details) for the test. Alternatively, you can publish it to IIS if you want. For this test, I'm using the default IIS Express.

With this done, we can start the LeaveSample.Hosts.Web project in debug mode to launch the IIS Express instance and then close the browser that was launch. The Host will be running in the background.

Take note that this needs to be done for both samples and at any one time, only one of the samples can be opened and run in Visual Studio. This is to avoid any conflicts.

A Glimpse of the Code

In case you do not want to download the samples and wondering how the code looks like, basically, here are some code snippets to give you an idea. I will recommend you download the samples and play with it yourself.

The EF code looks like...

public Leave Create(Leave leave)

{

using (var db = new DbContext(CONNECTION_NAME))

{

db.Set<Leave>().Add(leave);

db.SaveChanges();

return leave;

}

and the DAAB code looks like...

public Leave Create(Leave leave)

{

const string SQL_STATEMENT =

"INSERT INTO dbo.Leaves ([CorrelationID], [Category], [Employee], [StartDate], [EndDate], [Description], [Duration], [Status], [IsCompleted], [Remarks], [DateSubmitted]) " +

"VALUES(@CorrelationID, @Category, @Employee, @StartDate, @EndDate, @Description, @Duration, @Status, @IsCompleted, @Remarks, @DateSubmitted); SELECT SCOPE_IDENTITY();";

// Connect to database.

Database db = DatabaseFactory.CreateDatabase(CONNECTION_NAME);

using (DbCommand cmd = db.GetSqlStringCommand(SQL_STATEMENT))

{

// Set parameter values.

db.AddInParameter(cmd, "@CorrelationID", DbType.Guid, leave.CorrelationID);

db.AddInParameter(cmd, "@Category", DbType.Byte, leave.Category);

db.AddInParameter(cmd, "@Employee", DbType.AnsiString, leave.Employee);

db.AddInParameter(cmd, "@StartDate", DbType.DateTime, leave.StartDate);

db.AddInParameter(cmd, "@EndDate", DbType.DateTime, leave.EndDate);

db.AddInParameter(cmd, "@Description", DbType.AnsiString, leave.Description);

db.AddInParameter(cmd, "@Duration", DbType.Byte, leave.Duration);

db.AddInParameter(cmd, "@Status", DbType.Byte, leave.Status);

db.AddInParameter(cmd, "@IsCompleted", DbType.Boolean, leave.IsCompleted);

db.AddInParameter(cmd, "@Remarks", DbType.AnsiString, leave.Remarks);

db.AddInParameter(cmd, "@DateSubmitted", DbType.DateTime, leave.DateSubmitted);

// Get the primary key value.

leave.LeaveID = Convert.ToInt64(db.ExecuteScalar(cmd));

}

return leave;

}

I know you are already screaming - "Holy Cow!!!"

Unit Test: Single Run

I noticed that the Unit Test in Visual Studio now provides execution times. This is very handy and I would like to take advantage of it. I chose the ApplyThenApproveTest unit test method since it simulates a complete Apply and Approval of a leave transaction which should give a good mix of INSERTs and UPDATEs to multiple tables and SELECT operations.

Running a single unit test on each sample produces the following results:

Method using Entity Framework 6.0.2 took 171 ms to complete 1 transaction.

Method using Enterprise Library 6.0 DAAB took 139 ms to complete 1 transaction.

Unit Test: Looping in 1000

That's pretty good for both since they completed in milliseconds. Let's raise the stake to loop 1000 times and see the results:

Method using Entity Framework 6.0.2 took 44 secs to complete 1000 transactions.

Method using Enterprise Library 6.0 DAAB took 30 secs to complete 1000 transactions.

Instrumented Performance and Diagnostics Profiler

Let's dig slightly deeper to see the time breakdown. Visual Studio Ultimate comes with an awesome Performance and Diagnostics Profiler. I used Instrumented profiling on both the samples and here are the results from the web-server's perspective. The result is from web-server to database.

Apply method using Entity Framework 6.0.2 took an Average Elapse Time of 1,124.97

Extra observation: Querying took an Average Elapse Time of 2,937.66

Apply method using Enterprise Library 6.0 DAAB took an Average Elapse Time of 432.23

Extra observation: Querying took an Average Elapse Time of 861.90

And here are the detail breakdown of how much time it took for instrumenting only the app-server side. Results are from app-server to database:

Create method in data layer for Entity Framework 6.0.2 took an Avg. Elapse Time of 517.30. The EF methods DbContext.SaveChanges() took Avg. Elapse Time of 477.78 and DbSet.Add() took 38.74.

Create method in data layer for Enterprise Library 6.0 DAAB took an Avg. Elapse Time of 9.27. The DAAB methods Database.ExecuteScalar() took Avg. Elapse Time of 4.96 and DatabaseFactory.CreateDatabase() took 0.02.

From the Call Tree we can observe that the functions using DAAB is considerably faster than EF.

Load Test

Finally, let's see how the two perform under load. I will use Visual Studio Load Test to load the single run ApplyThenApproveTest unit test method. I will use a constant load of 100 users and run the test for 1 minute.

Entity Framework 6.0.2 completed with 1692 Total Test runs with an Avg. Test Time of 3.41 sec but it gave 154 errors. The error thrown was 'An error occurred while reading from the store provider's data reader'.

Enterprise Library 6.0 DAAB completed with 8236 Total Test runs with an Avg. Test Time of 0.70 sec and gave 0 errors.

I was surprised (and concerned) by this result as it appears that EF may not be able to perform under high stressed load.

What is Going On Behind The Scene?

Out of curiosity, I fired-up SQL Profiler to see what is being sent to the SQL Server.

INSERT statement generated by Entity Framework 6.0.2

INSERT statement going through Enterprise Library 6.0 DAAB

SELECT statement with Paging generated by Entity Framework 6.0.2

Custom Paging SELECT statement going through Enterprise Library 6.0 DAAB

Summary and Wrap Up

From the performance test results it clearly shows that Entity Framework is still not as efficient as wrapper libraries such as DAAB which uses native ADO.NET. However, EF does make code a lot more easier to read and a lot less to write. The load test findings do raise some concerns for high performance applications that requires high concurrency.

Because of these findings, I would have to skip EF implementation again for this round :( and wait for more improvements to it. The choice of not being able to use EF for me is bind by the constraints I have in my environment that requires the applications to process millions of transactions per hour. This is the volume of processing in Telco environment.

Whether to use EF or not is entirely up to your environment and choice (as long as you know the limitations of the chosen technology). If your system does not require such crazy high loads, I don't foresee you will have any problems in using EF.

I hope this post has given you good enough insights into Entity Framework. If you have ideas for me to fix the EF exception, please feel free to post it in the comments.