In my mind two great inventions Microsoft came in last ten years were Windows Presentation Foundations (also Slilverlight) and Powershell. We in .NET used to jealous of Linux Shell Programming as something that powerful was not possible in Windows. Powershell changed all that, at present it is the most powerful scripting language and shell available ( btw: Its 10 times better than linux shells even).

I use PS (PowerShell) in everyday repetitive tasks. We use it to download source code from source code control, build code, release code, create release notes, email to sqa, mark task as fixed in task management system, run automated test scripts and in many more tasks. From now on I am going to share a few PS recipes that make my life easier on a regular basis. I use it as much as I use C#.

When doing automated build, its best to clear the old binaries, or even its good to clear the binary files when running from VS because old binaries can sometimes create unpredictable results.

Cleaning Up Binaries in folder

# Returns a list of files given the filter pattern
#  $sourcePath - Path which to recursively look for files
#  $pattern – File filter such as *.dll
#  $list – Ignore, used internally

function Get-Files ( $sourcePath, $pattern, $list )
{
    # Is this the root call
    $return_list = $false
    if ($list -eq $null)
    {
        # Create the collection that we will return
        $list = New-Object System.Collections.ObjectModel.Collection[System.IO.FileInfo]
        $return_list = $true;
    }
   
    $dir = New-Object System.IO.DirectoryInfo $sourcePath
    $files = $null
    $files = $dir.GetFiles($pattern)
    if ($files.Length -gt 0)
    {
        $files | % {$list.Add($_)}   
    }
    $dirs = $dir.GetDirectories();
    $dirs | % {Get-Files $_.FullName $pattern $list}
    if ($return_list -eq $true)
    {
        return $list
    }
}

Now we want to delete the dll and pdb files. We will create a function called Delete-Binaries. Get file function will return the list of files of a certain type. Then we can delete the files. See below

function Delete-Binaries ( $sourcePath )
{
    Write-Host "Deleting all dll and pdb files from path $sourcePath"
    Get-Files $sourcePath "*.dll" | % { $_.Delete() }
    Get-Files $sourcePath "*.pdb" | % { $_.Delete() }
    Write-Host "Deletion Complete"
}

 

Windows NT is a dynamic filesystems. It has certain cool capabilities such as file and directory linking, spurse files, alternate streams and many more. Today we are going to see what Alternate Streams can do for us.

Alternate Streams

We can programmatically open files in NTFS as file streams. NTFS supports alternate file streams for any file. We can open multiple file streams on a single file. To a normal user only the default file stream would be visible. This technique can be used to hide sensitive data in plain sight, or hide the encrypted data in hidden stream. Some viruses and rootkits use this technique to hide. Even two years ago many of the famous antiviruses did not use to scan for alternate file streams.
Before we go into programming details lets have fun exercise with alternate file streams.

  Step 1. Open up command prompt in windows
  Step 2. Type the following
    notepad myfile.txt
  Step 3. Type "Default text" in notepad then save the file
  Step 4. again in command prompt type
    notepad myfile.txt:hidden_data
  Step 5. Type "My gmail password : yoho" in notepad then save the file
  Step 6. Open the myfile.txt. What do you see?
 

You can see the "Default text" in the file. But the data you saved in the alternate stream named "hidden_data" is not visible. If you look in the explorer you will not find the file named myfile.txt:hidden_data, rather you will only see myfile.txt. Now if we type in the command again "notepad myfile.txt:hidden_data" then we will see that the text "My gmail password : yoho" is on notepad.

So, myfile.txt have an alternate file stream called "hidden_data". We can open and keep multiple such streams for the file. This is a cool feature, isn’t it?

 

.NET Does not support NTFS alternate strems

NTFS streams have been around for more than 10 years and we have .NET version 4.0 released. But unfortuntely .NET Framework yet do not have any support for alternative file streams. This is quite unexpected and sad.

I am quite disappointed in Microsoft for not providing such a basic file system manipulation has been in NTFS for over 10 years.

 

So? What is our option

You can write Alternative file stream class yourself by using the basic windows APIs. But it would be a lot easier to use a prebuilt library. Greg Duncan has already created a library. His code can be found here.

http://coolthingoftheday.blogspot.com/2008/09/accessing-ntfs-alternate-data-streams.html

Update : I am no longer using e72. I do not see symbian as preferred platform for developing smartphone applications. I have moved on to android which I personally find almost magical.

A little Background

A few days ago I got a Nokia E72 phone as a gift and I really loved the phone because of its 600 Mhz processor. I needed a phone with a full keyboard so that I can check and send emails easily from anywhere. I like windows mobile platform but did want to try raw processing power. As usual I tried writing .NET in Symbian ( usual Nokia phone OS ) and had fun. But I needed more functionality that is native to the OS and decided to go with python. Although Nokia has a cool gadget API, I wanted take advantage of the hardware on the phone. There is a python compiler for the S60 platform, I installed it and wrote my first tiny python code.

Can’t resist the urge to share the spec of E72 hardware. It has a 600 MHz ARM 11 Processor, 290 MB internal memory, 128 MB RAM, 16GB external mem capbility, Acceleretometer for motion detection, Digital Magnetic Sensor for compass, Ambient Light Sensor, Active GPS, 5 Megapixed cam, WLAN, Bluetooth, FM Radio, Optical Trackpad, HSDPA 10Mbps connectivity, Active noise cancellation etc.

Downloading and installing python on your S60 phone

Download the python for S60 from https://garage.maemo.org/frs/download.php/6448/PythonForS60_1.9.7.tar.gz.

Extract the contents then install the python runtime for your device. Then install the python shell script mine was S60 3^rd edition feature pack 2.

Video Demo

This is what the application does

A python primer for a c# (or any) developer

Unlike perl, which looks horrible (sorry if any perl developer is reading this, I mean you no harm, it is just my opinion), python looks pretty structured somewhat like Delphi/Pascal. My objective was to write a few programs for my phone and play around so I wanted to learn python fast in an hour. Before we proceed further we should know a few facts about python. Firstly Python is a dynamic scripting language that is interpreted. Then …

Without correct indentation python code will not run

Wait a second, what!?
Yes, python has specific code indentation rules that makes your write code in beautiful formatted indentation. Python indentation works like C# braces. For example let me demonstrate a hello world method.

def say_hello():

 print u”hello world”

See the indentation that you have to make for the function? Yep that makes most python code look nice at the same time

No variable Declaration

There is no variable declaration required to use a variable, just make up a variable and use it. See example

myname = “Shafqat Ahmed” 

print myname

Importing namespaces

Like .NET you can import namespaces ( called modules ) in python.

import math 

import appuifw

A primer on python for Series 60

Since my objective was to write code on python for Symbian, I am going to share few little tidbits.

For Python for S60 make sure you use Unicode everywhere

The Python S60 SDK for Symbian (I think other phones also have this same requirement) requires you to use unicode strings in most places. See below

myname = u“Shafqat Ahmed” 

print myname

See that “u” in front of the string declaration? That is how you make a unicode string.

Important modules

For a PyS60 app usually the first required module is the Application UI Framework for short appuifw.  The e32 module contains some system function, the contacts module lets us use the contacts on the phone, the messaging lets us use sms messaging, the telephone library lets us dial or hang up a phone number, the audio module lets us play music, or record voice note or record a phone conversation.

Using the Symbian Native UI

There are simple functions that lets the user use the native Symbian UI widgets. For example if we want to read values as input. We could write something like

import appuifw 

name = appuifw.query(u”What is your nam?e”, u”text”)

Code

Reading the Accelerometer data on PyS60 on Nokia E72

In order to access the accelerometer data we need to use the sensor module. Then we will setup a callback where the accelerometer data will be sent. Here is how the code looks like ( I created a class to organize code, so the actual code look a little different)

import sensors 

from sensors import * 



accelerometer = AccelerometerXYZAxisData ( data_filter = LowPassFilter()) 

accelerometer.set_callback ( data_callback=my_callback)

Then I can access the data in my callback function each time by accessing accelerometer.x for X axis data, accelerometer.y for Y axis data and accelerometer.z for Z axis data.

Managing Sensitivity

The E72 has a -60 to +60 range value for each axis. We could use a constant value to move the graphic object, then that would move evenly. But we want to the app to be more responsive human like. So if we take the value of the axis and devide it by a value, say 10 then I would get 1 for a 10 value from the accelerometer, a 3 for 30 on accelerometer, a 6 for 60 (max value) on accelerometer. Similarly I would get a -1 for -10. A value of +/-10 mins the device is a little tilted, +/-60 means highly tilted. So can move the object faster if the device is more tilted.

Also I have added a menu for controlling sensitivity. If I use 5 instead of 10 then the division would result higher values thus making the movement faster. The application code looks like this.

def my_callback(self):

  x_incr = (self.accelerometer.x / self.sensitivity) * -1

  self.x= self.x + x_incr

  y_incr = (self.accelerometer.y / self.sensitivity ) 

  self.y= self.y + y_incr

  if self.y < 0:

    self.y = 0

  if self.y > 220:

    self.y = 220

  if self.x < 0:

    self.x = 0

  if self.x > 300:

    self.x = 300

  self.draw()

Then we draw the graphic on to an image then blit it down to the drawing canvas.

Download

Please download the draw.py(2.0K) from here. And keep in your E:\data\python folder, use c: if you have installed python in the internal memory not the memory card.

Running the app

Run python shell, select ‘Run Script’ from the menu and run draw.py. To exit press the right softkey. You can select high sensitivity from the left softkey menu to have more fun.

J2ME has always been the standard in cross phone applications. Most java app are truly cross phone. The only phone we could write code in .NET was the Windows mobile or Pocket PC. A company called AppForge came up with a solution to run .NET code. But in 2007 the company was acquired by Oracle and shut down.

Now Red Five Labs have come up with a .NET based solution called Net60 which runs windows mobile based .NET code. Currently .NET 2.0 framework is supported and they plan to add support for compact framework 3.5. Now what made me exited is their licensing policy. With AppForge it was not possible to develop when the trial expired. With Net60 that is not not case. You can get a IMEI bound license for your non commercial application. Now, isn’t that wonderful? That means I can write my own apps to run on symbian and use them. That also paves way for .NET based open source apps. Lets face it, Nokia holds 41% market share for smartphone applications, which is a very large number (source).

I recently got myself a Nokia E63 phone, which is actually an economic version of Nokia E71 (only diff is 2Mpx cam instead of 5 Mpx cam and no GPS and plastic body instead of steel). It has the full qwerty keyboard. I spend a lot of time reading and writing mails as communication has become a large part of my role, so the full keyboard is really helpful. I was looking for a good way to write C# code and run it on my phone ( E71- Symbian OS S60 3rd edition feature pack 1).  After a few tries it worked very nicely. So here is what you need to do in order run .NET code on your Symbian phone.

Aquiring the SDK

1. Get the Net60 SDK from here. It is about 13MB. You would have to provide your phone IMEI no and email. That is the phone where the application would run. If you want a version where it will run on different phones, then you would have to get a commercial license.
2. Check your email to get the free non commercial express license  from Red Five Labs. ( I’d strongly suggest you see the video tutorial at their site, which is the next set of steps about)

Writing the Application

1. Create a compact framework 2.0 application (not CF 3.5)
2. After you are done with the application add a Genesis Setup Project to the solution
3. Point to the executable, fill up the necessary fields
4. Add Net60.sisx as dependency, also add the license that you received from RF Labs.
5.  Compile, install the output on your cell phone and run

Debug version vs Release Version

The debug version comes with logger and a runner tool. If you are developing, I would suggest that you use the debug version. Making a build and deploying each time you change the code can be pretty annoying. There is a NET60 Launcher tool that can run a .NET executable. So instead of building installation package and deploying each time you can just use that tool to run the exe. Make sure you copy the executable under c:\DATA\RedFiveLabs\Apps\ folder so that it shows on you NET60 Launcher.

Additional API

There are some additional API provided by RF Labs to control various features in Symbian OS. These are distributed over a few assemblies. These assemblies are

RedFiveLabs.Mobile.Audio
RedFiveLabs.Mobile.Bluetooth
RedFiveLabs.Mobile.Location
RedFiveLabs.Mobile.Messaging ( for SMS etc)
RedFiveLabs.Mobile.OpenGL ( oh yes open GL support)
RedFiveLabs.Mobile.Pim ( for contacts)  
RedFiveLabs.Mobile.Services
RedFiveLabs.Mobile.Vibration

Have fun .Netting on Symbian

A few days ago we wanted to how would our application behave under high stress scenarios and wanted to take as little time as possible. We thought we would be able to use Microsoft Web Stress tool. But it turned out that the tool location is not available anymore on the Microsoft web site. We tried an older version, but it seemed that it was not a proper tool to run load tests.  We have a custom Sharepoint application running in front of an application server. All of the core business logic goes through the application server. The web application calls the application server via a few web service calls. This is how the application is supposed to be deployed.

Since all the core requests are coming from web to the app via web services, and our goal was to to stress test the application server, we could simulate stress by calling the web service in the application server.

Issues

The first and basic issue is, that we had very limited time and in that time it was not possible write real life test scenarios for most of the use cases.

The second issue was, the test should be as realistic as possible. Writing tests may not be best way.

Solution

The solution was to record a real use case, and then play it again many times with a little varied parameters.

Implementation

So we injected a little code in the web service (we could do this with context handlers, but we did not have the time). Serialized each of the web requests sequentially, In the start of a use case we would clear the folder where we were serializing the web requests. After we have finished the use case then we would de-serialize the requests and inject some randomizing items to a few parameters that changes in real life and then run in multiple thread to simulate real stress. The diagram below shows the whole process. The process if repeated in multiple threads to simulate multiple clients and it worked.

So if any of you want to simulate real life scenario without writing too much code, give this a try. Will update this blog with sample code a few days later.

For teams located in different locations conference calling is a must. Most of these calling facilities have toll free 1-800 dial in numbers. Skype lets you dial in to the toll free 1-800 numbers without any cost. One annoying thing with these conference calls is that you have to dial in a pin code to enter. If you are dialing in to one conference only then you memorize it. But if you have to dial in several different conferences calls in a day it becomes a pain. You would probably have to lookup the outlook calendar entry and type it in, which is quite annoying and distracting. Recently I have been trying to do everything with Powershell. As you have guessed … … now we can dial into a Skype meeting with one powershell command.

You can download the powershell script : Download skype.ps1 (5.2K)

I just wrote a function ( just bare minimum, no error handling ) that calls a Skype contact or a phone number waits for certain time then enters the pin as DTMF codes. I have the different meeting and number to different  functions which calls this function. And now I can get into any meeting without hassle.

The function is called Call-SkypeContactSendDTMF. The first parameter can be a skype contact name or a saved phone only contact or a number to dial. The second parameter is the number of seconds to wait after the phone has been picked up by the conference bot. The third parameter is the DTMF codes. You can add pause by putting in “@” to pause for a second. Here is the calling syntax

Call-SkypeContactSendDTMF (ConferenceNo, SecondsTOWait, DTMFCodes)

So I have created different entries from the meetings like this

function call-conf1()
{
   Call-SkypeContactSendDTMF ("ConfNo",10,"7363784394#")
}

All I have to do is type in call-conf1 like this

call-conf1

I love powershell. Hope this helps someone.

btw: When executing this script remember that we want the function to stay in memory so you can either put this in the powershell profile or you can call it in the same context by adding a “. ” to it.

Recently I have been trying to do almost everything in Powershell. With the release of Powershell V2 RC and the awesome visual debugger that comes with Powershell called Powershell ISE, I have a Powershell window open at all times. I am getting my google calendar events form command line, doing recursive ftp uploads and download as well. Now I have started to even do outlook automation with PS.

The Problem

Any slightly efficient developer would write functions and reuse them later. So I tended to write functions add kept on adding them to my Powershell profile file. After a while that turned a little messy and hard to manage. So now I thought why don’t write different script files put my functions in there and use them when needed. So I created functions for Google Data API, Outlook functions. Now when I call them from the prompt, after execution of the script the functions are no longer there. It is because the scriptblock runs in a different execution context and after execution the context is removed. So the functions I have put the file no longer exist when I call them.

For example I have written a function in myfunc.ps1 to convert string into base64 string which looks like this

function tobase64 ( $asciistring )
{
  return [Convert]::ToBase64String([System.Text.Encoding]
    ::ASCII.GetBytes($asciistring))
}

Now after running the script .\myfunc.ps1 when I call the function, it no longer works.

Solution

After running around the internet for hours, looking for a solution to run scripts in the same executions context and trying out several different approaches, I was about to give up. Then I found the worlds easiest solution in Donn Felkers blog. All you need to do is to add a simple dot ( “.” ) at the start of the line to make it run the in the same context.

So if you were running the script like this,

.\myfunc.ps1 
or
c:\somefolder\myfunc.ps1

You should run it like this

. .\myfunc.ps1 
or
. ‘c:\somefolder\myfunc.ps1’

See the added dot? That does the trick. I wish this was advertized more, then I wouldn’t have had to search for hours to find a solution.

My blog is hosted at Typepad, which in my personal opinion takes the least time to manage than any other blog hosting service. The payment option from typepad has no Paypal support as of the time of my writing and I did not want to spend from my card. So I planned to move from typepad to self hosting. However that situation was resolved with help of typepad and I am still in Typepad,the best managed blog service.

I was planning to use BlogEngine.NET, a .NET based blogging platform to host and it supports importing from BlogML. Typepad is a blog hosting service that is based on MovableType blogging platform. MovableType exports blog contents into a plain text based format. I searched around the internet for a tool that converts typepad format to BlogML. Unfortunately there was none. So I had to write a converter to convert the typepad format.

Gotcha!

After converting the the blog content to BlogML I just found out that there is a click once installer that comes with BlogEngine.NET 1.4.5 and the application do not start. After a little investigation I had figured that the location of the installer is no longer valid. I then had to add support to directly import into BlogEngine by calling their webservice. So if you are converting the typepad blog exported please use the code below.

Download, Source and Usage

Download Typepad2BlogMLBinary
Download Typepad2BlogML

Usage: Typepad2BlogML.exe -i:[input file] -mode:[blogml|service] -o:[output file] -s:[service address] -u:[username] -p:[password]

There are 2 application modes, one for converting into BlogML and another for uploading to BlogEngine via a webservice.

For converting to blog ML you should use something like this:

Typepad2BlogML.exe -i:mytypepadfile.txt -m:blogml -o:out.blogml

For importing into BlogEngine 1.4.5 use command line code like this:

Typepad2BlogML.exe -i:mytypepadfile.txt -m:service -s:http://www.myblog.com/api/BlogImporter.asmx -u:username -p:password