Bizic Bojan

How to update OpenSSL lib on OSX

Fri, 27 Nov 2015 03:41:00 -0500

Last night one of my CI builds ran into error by executing CMake based build with following error message:

Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the system 
 variable OPENSSL_ROOT_DIR: Found unsuitable version "0.9.8y", but required is at least "1.0.0"

Running following command to get current version of installed OpenSSL

$ openssl version -a

Gave me

Even though build was pretty recent the version number was still lower from that what I needed.

Since I am using Homebrew as package manager I could install newer version with following command.

$ brew install openssl

This however, downloads and builds OpenSSL into Homebrew local folder and does not change symlinks for currently installed version. Running the following command should update symlinks in /usr/bin

$ brew link openssl --force

But running the

$ openssl version -a

Showed the same version again so I had to update symlinks manually.


$ sudo ln -s /usr/local/Cellar/openssl/1.0.2d_1/bin/openssl /usr/bin

If you are running latest version of OSX (10.11.1 El Capitan at this moment) chances are that "System Integrity Protection" is enabled thus not allowing you to edit content under /usr/bin. You can check if this protection is enabled by running

$ csrutil status

If this protection is enabled here you can read about how to disable it before executing these commands.

Visual Studio Online Build deploy to Azure VM

Thu, 25 Jun 2015 19:56:00 -0500

There are many different reasons why you would want to deploy your builds from Visual Studio Online build system directly to your Azure VM. You might have test or staging environment running on Azure and you want to setup contentious integration build to run every time someone of you teammates checks something in. After each build run is finished it would be nice to for latest changes to be deployed automatically so that your test team can get latest version as soon as possible without you doing the deployment manually.

In essence we want to make WebDeploy profile for our application and instruct MSBuild running on Visual Studio Online to execute that profile once build run is finished. At the moment of writing this there are some improvements to this process on VS Online portal so you might check that out but my approach might be better for simpler use-cases.

Preparing your VM

It doesn't necessarily have to be Azure Cloud virtual machine, it could be machine running anywhere even your local on premise server as long as it is publicly accessible and running IIS Web Deploy agent. For sake of this example I will use Azure VM.

First we need to create Web Deploy Publishing profile for our application. In solution explorer right click on project you want to deploy and click on Publish.

When dialog pops up expand it by clicking on More Options to reveal Microsoft Azure Virtual Machines. From there you can pick up existing VM running on Azure or create new one.

From this dialog you can create new VM and make sure to check "Enable IIS and Web Deploy" because it will make configuration a lot easier. If already have your existing machine running here are detailed instructions how to enable IIS and Web Deploy. If you are creating new VM pay attention to Visual Studio Output window because this process can take some time.

Configure Build Process

Now that we have our machine ready for hosting and accepting our deployments it's time to configure our Build. Once again I will use Visual Studio Online Builds but same method applies if you are running Team Foundation Server on premise or even if you are just building your solution from command line manually or using scripts. Point is, whatever method for compiling your application you are using it will directly or indirectly run MSBuild in background and MSBuild can run our Publish profile as part of the process.

At the moment of writing creating build process directly in VS Online portal in browser is still in beta so I will do it from local Visual Studio. First make sure that you commit at least one version of your solution to source control because you will need it to define build process. It doesn't matter if you are using Team Foundation or Git VCS, VS Online Build will work with both.

Switch to Team Explorer and click on Builds. From top of the window choose "New Build Definition".

First step is quite self-explanatory. Leave it at enabled and give it some meaningful name. Give it some name that will clearly describe what application suite it is building, what configuration, how often (like nightly, CI etc.) because that list can grow over time.

On Trigger tab choose how you want your build to run. Manually, on each check in etc. I want to have continuous integration, meaning on every source code change application will get built and deployed to my designated environment.

Source Settings tab setts where your solution is to be found on VCS. Settings for Team Foundation Server and Git use different notation but Visual Studio should detect this step automatically if your code has already been committed (checked in) to VCS.

On Build Defaults tab you can leave everything by default.

Now we come to most important step of all. Build processing.

Items to build setting needs to point to your solution or project file. This will probably be detected automatically. If not just click trough. Under advanced settings you need to enter additional MSBuild Arguments. This is where we set our deployment profile. Remember that we could technically configure multiple publish profiles for one single application. We could make one profile for test and another for staging environment. We could also build in Debug configuration for test and Release for staging environment. Since I am deploying for test I use following command:

/p:configuration="Debug;PublishProfile="bojan-webdeploy";AllowUntrustedCertificate=True;Password=PassowrdForWebDeployProfile;VisualStudioVersion=12.0

Settings are split with semicolon. Configuration is "Debug" in this case. PublishProfile is the name of publishing profile we set up in first step. You can find it inside application Properties/Publish Profiles.

It is the file with .pubxml extension. I found that "AllowUntrustedCertificate=True" and "VisualStudioVersion=12.0" are necessary for this process to work. Password had to be the one used by Web Deploy on your target server.

Hit Save and you should be able to see your new Build definition inside Visual Studio Builds in Team Explorer as well as in Visual Studio Online. Right click on it and then on "Queue New Build".

Your Build should kick of at this point and you should be able to see progress both in Visual Studio and VS Online Portal.

If everything went ok with the build your application should be successfully deployed to target environment. Double-clicking on Build in both VS and VS Online Profile allows you to see logs and eventually diagnose if something went wrong.

Deploying specific project in solution

There is a special case when you have two or more Web type projects in solution(Web Application, Web Site, Web API). Even though you define publishing profile in specific project in solution and it works if you publish using wizards and UI from Visual Studio it will not work from the command line and build systems for that matter. It will not know which project it needs to be deployed with that specific profile. To work around that you need to add custom Property Group inside .proj file of application you want to deploy.

Simple unload and edit your csproj file by adding something like following:

Then you add this custom property you defined in project to MSBuild configuration arguments list and set it to true.

/p:configuration="Debug;DeployThisOnAppOnBuild=true;PublishProfile="bojan-webdeploy";AllowUntrustedCertificate=True;Password=PassowrdForWebDeployProfile;VisualStudioVersion=12.0

Umbraco 7 Multi User Picker

Thu, 29 Jan 2015 06:45:00 -0500

Recently I had a chance to upgrade older web site based on Umbraco 6 to most recent version 7.2.2. Upgrade went more or less smooth as one would expect. Problem was upgrading third party components (when I say 3rd party i mean uComponents). I could live without uComponents but I also had custom Multi User Picker Data Type.

With Umbraco 7 you get User Picker Data type ( property editor) but that allows you to select just one User. I needed Data type that would give me list of CMS Users where I would be able select from (using checkboxes probably). At the time of writing there was no such components available for Umbraco 7.x so that meant building my own.

The whole idea behind these property editors in Umbraco is to simply show some values stored in data store to user in user friendly fashion and give them ability to edit those values. Before Umbraco version 7 it was possible to wrap different kinds of , let's call it code, into macro and use that as "property editor" for your data type. That was the time and version of Umbraco where those kinds of macros and property editors where in one way or another executed and effectively rendered on server side (XSLT, asp.net user controls etc). Umbraco version 7 was completely redesigned and everything that runs under your /umbraco/# part of the site was build as single page web application using AngularJS. This means if you want something injected and rendered in Umbraco back office you need to follow this pattern and build your Plugin (Data Type Property Editor) using AngularJS and available Umbraco Angular services and factories. Umbraco 7 documentation is not that great but at least they have basic introduction and some samples.

Custom Property Editor in Umbraco 7.x

First thing when build custom property editor is you need to create folder for it under App_Plugins folder. In my case it's called MultiUserPicker.

This folder will always consist of at least three files. Package manifest, describing your editor so that Umbraco knows how to deal with, AngularJS controller and some HTML file used as AngularJS template. For MultiUserPicker editor package.manifest is defined:

{   
    //you can define multiple editors   
    propertyEditors: [      
        {
            /*this must be a unique alias*/ 
            alias: "MultiUserPicker",
            /*the name*/
            name: "Multi User Picker",
            /*the html file we will load for the editor*/
            editor: {
                view: "~/App_Plugins/MultiUserPicker/MultiUserPickerEditor.html",
				valueType: "JSON"
            }
        }
    ]
    ,
    //array of files we want to inject into the application on app_start
    javascript: [
        '~/App_Plugins/MultiUserPicker/MultiUserPicker.controller.js'
    ]
}

This is more or less self explanatory. Important in this case to notice is valueType: "JSON". This tells Umbraco what the internal data storage type for this field is. You can find more about this types here. You could use INT, STRING, TEXT, DATETIME or JSON as value type. View is path your editor HTML file and the last line tells where our AngularJS controller is.

Once package.manifest is defined you are already able to use it in Umbraco as custom data type. In Developer section expand Data types and click on Create new.

You should be able to select our new Multi User Picker editor from the drop down.

Next we need a way to get Umbraco Users to our front end client side code. The way we do that is by creating specialized Umbraco WEB API controller.

namespace UmbracoProject.Controllers {
    public class UmbracoUser {
        public int Id { get; set; }
        public string Name { get; set; }
        public bool Selected { get; set; }
    }
    [PluginController("CustomEditors")]
    public class MultiUserPickerApiController : UmbracoAuthorizedJsonController {
        
        [HttpGet]
        public IEnumerable<UmbracoUser> GetAllUsers() {
            int totalUsers = 0;
            var allUsers = Services.UserService.GetAll(0, int.MaxValue, out totalUsers);
            var retVal = allUsers.Select(user => new UmbracoUser {
                Id = user.Id,
                Name = user.Name,
                Selected = false
            }).ToList();
            return retVal;
        }
    }
}

This controller exposes simple JSON API with one GetAllUsers method that returns list of all users in CMS. We will call this method (endpoint) from our AngularJS controller.

AngularJS controller code:

//adds the resource to umbraco.resources module:
angular.module('umbraco.resources').factory('multiUserPickerResource',
    function ($q, $http, $routeParams) {
        var apiCallsFactory = {};
        //the factory object returned
        //this cals the Api Controller
        apiCallsFactory.getAllUsers = function () {
            return $http.get("/umbraco/backoffice/CustomEditors/MultiUserPickerApi/GetAllUsers/");
        }
        return apiCallsFactory;
    }
);
angular.module("umbraco")
    .controller("multiUserPickerController", function ($scope, multiUserPickerResource) {
        // Fire when page is loaded and Plugin gets pulled in
        Reload();
        // Loads users from actual Umbraco page
        // Then calls Users API and adds missing users if one was added meanwhile
        function Reload() {
            //Bind Umbraco model to our local model
            $scope.Users = $scope.model.value;
            // Get Users from API
            var promiseGet = multiUserPickerResource.getAllUsers();
            promiseGet.then(function (pl) {
                var usersFromWebApi = pl.data;
                // Check if have some values comming from Actual Umbraco Fields value
                // if not just map all users from API to local model
                if ($scope.Users != "") {
                    angular.forEach(usersFromWebApi, function (usrFromApi, key1) {
                        var found = false;
                        angular.forEach($scope.Users, function (usrFromContent, key2) {
                            if (usrFromApi.Id == usrFromContent.Id) {
                                found = true;
                            }
                        });
                        if (!found) {
                            $scope.Users.push(usrFromApi);
                        }
                    });
                } else {
                    $scope.Users = usersFromWebApi;
                }
            },
            function (errorPl) {
                $log.error('failure loading Users from API', errorPl);
            });
        }
        // Map our local model to Umbraco page model to be autmaticaly saved to database
        $scope.SyncWithUmbracoModel = function (users) {
            $scope.model.value = users;
        };
    });

Basically what this code does is, on top of the listing I am registering my custom Angular factory that will be used for calling my custom .NET JSON API endpoint. Under the factory I am registering new controller within Angular application and passing in my factory as parameter (DI). Reload method is called on load, so each time page with our property editor loads we want to automatically show some values. When we defined our valueType in package.manifest as JSON we told umbraco to store values of this field in database as JSON (string) so anytime our controller gets instantiated we will get $scope object trough dependency injection. $scope.model.value is what holds value to be stored to database when the page is saved. When Reload method get triggered it assigns this value coming from Umbraco to my custom local Users model. This doesn't have to be this way necessarily, I could work only with $scope.model.value, but I found it more meaningful this way. Reload method then calls our .NET endpoint and gets the list of all Umbraco users defined on system. It adds new Users to local model if some users were defined since last page save. Removing of the users is intentionally left out because Umbraco doesn't let you delete any users once defined. Last method from controller simply maps local model back to $scope.model.value to be saved to Umbraco database when page where editor is gets saved.

Last piece of the puzzle is HTML template and it is defined as follows:

<div ng-controller="multiUserPickerController">
    <div>
        <label ng-repeat="usr in Users">
            <input type="checkbox" ng-model="usr.Selected" ng-click="SyncWithUmbracoModel(Users)" value="{{usr.Id}}" />
            {{usr.Name}}
        </label>
    </div>
</div>

What goes on in template is pretty straight forward Angular approach. Top element defines controller responsible for this context. Next we use ng-repeat to iterate trough all the users in $scope. ng-model on checkbox handles binding between control and the model and at last ng-click does the sync with Umbraco model to be saved to database.

Here is what end result looks like:

Here is the full solution if you need the code:

UmbracoProject.zip (8.9MB)

Let me know if it works for you as it did for me.

Set the SQL Server CE connection string for Entity Framework at runtime

Sun, 16 Nov 2014 10:10:00 -0500

Typically when you create your EF data model wizard will politely add all necessary settings in you app or web.config. That's all fine and dandy, but what if you have SQL Compact Database and you have multiple copies or database names could change and run-time. There is tone of other reasons why you might not want to store your connection strings in plain text config files. Basically you must pass connection string to your Entity Model at run-time.

First we need to extend our generated DbContext class with additional constructor that accepts connection string and pass it on to base class. At the moment of writing, this applies to EF 6.x and EF 5.x and I am using Database first approach, not sure if it works for Code First. Generated DbContext partial class can be found in solution tree if you expand .edmx file it is the one with EdmxFileName.Context.cs name.

public partial class DBEntities : DbContext
    {
        public DBEntities()
            : base("name=DBEntities")
        {
        }

        public DBEntities(string connectionString)
            : base(connectionString) {
        }
    
        protected override void OnModelCreating(DbModelBuilder modelBuilder)
        {
            throw new UnintentionalCodeFirstException();
        }}}

And this would be the method for getting connection string:

 public static string GetSqlCeConnectionString(string fileName, string password) {
            var connectionStringBuilder = new SqlCeConnectionStringBuilder();
            connectionStringBuilder.DataSource = fileName;
            connectionStringBuilder.Password = password;
            var entityConnectionStringBuilder = new EntityConnectionStringBuilder();
            entityConnectionStringBuilder.Metadata = "res://*/YourEdmxFileName.csdl|res://*/YourEdmxFileName.ssdl|res://*/YourEdmxFileName.msl";
            entityConnectionStringBuilder.Provider = "System.Data.SqlServerCe.4.0";
            entityConnectionStringBuilder.ProviderConnectionString = connectionStringBuilder.ToString();

            return entityConnectionStringBuilder.ConnectionString;
        }

Next, just pass that connection string to DBContext constructor:

using (var db = new DBEntities(connectionString)) {
....
}

If something goes wrong when connecting to database or accessing compiled resources this blog post can be very useful.

How to install-uninstall Qt 5.3 on openSuse 13

Wed, 13 Aug 2014 16:44:00 -0500

Installing Qt on Windows and OS X is quite simple. Download, run, click next few times. On Linux it’s not that simple, and like with most software installations on Linux it requires few steps more that are not so obvious. One option is to download entire Qt source packages and compile/install it by yourself, which many Linux users prefer. Source is available as Git repository or as tarball archive. Either way you get the same thing. I’ve already wrote about how to compile Qt and Qt projects.

In this post I’ll list simple steps how to install Qt 5.3 from .run installer. I did it on openSuse 13.1 because it’s mine peered distro but same goes for Ubuntu or any other distro supporting .run packages.

Step 1:

Download installation package form Qt-project.org.

Step 2:

Give the downloaded .run package execute permissions. Open the Terminal, navigate to your .run package and type in following command:

chmod +x qt-opensource-linux-x64-5.3.1.run

You can give full path to your package or navigate to folder where it’s downloaded and execute from there. Name of the file can vary based on version you downloaded form the Qt web site. At this point you are ready to start installation.

Step 3:

Start the installation package by simply clicking on your .run package or execute following command form the Terminal:

./qt-opensource-linux-x64-5.3.1.run

This window should pop up.

Step 4:

Follow the installer visual workflow. Select the installation path.

Then select the components you wish to install.

Choose license and accept.

And you are ready to go.

Uninstalling Qt

If you want to uninstall Qt and all of its components form the operating system navigate to installation directory you choose in Step 4 from your Terminal and execute following command:

./MaintenaceTool

This window should pop up.

Choose “Remove all components” , click next and you are done.

Generating the Julia Set using C++ and OpenCL

Sun, 03 Aug 2014 23:00:00 -0500

For the sake of keeping things short as possible I will skip redundant explanations what Julia Set and fractals are in general. There is plenty documentation on internet about it and let’s face it you are probably here just for the codez. Just to make sure we are on the same page this is the picture of Julia set and what we will generate with our implementation. Colors are completely arbitrary.

Task here is to generate 2D image of Julia set where each individual pixel in image is represented with following equation:

z_k+1 = z_k² + c

Z and c represent 2D points with (x,y) values. Each pixel is given an initial value for z and we will refer to as z₀. This initial value is determined by the pixel’s position in the range z spans.

The value of c is constant for all the pixels. The above equation is applied to each pixel and is evaluated iteratively (in a loop). In each iteration, z is updated according to the above equation (the list of values z_k refers to the orbit, or dwell of z). This creates a sequence of z values for each pixel. The first 5 values in a sequence are…

z₀

z₁ = z₀² + c

z₂ = z₁² + c

z₃ = z₂² + c

z₄ = z₃² + c

…

Once we’ve calculated a new value for z in a given iteration, we calculate the length of z. If the length is greater than some threshold (usually 2.0), we stop iterating and use the index of the current iteration (k) as an index to look-up an array of colours to determine the colour for the pixel being processed. If we never exceed the threshold and k reaches a given maximum number of iterations then we colour the pixel black.

The image plane (x, y) is actually a complex plane and z and c are complex numbers. Complex numbers are like 2D points (with x and y values as noted above), but the rules for multiplying complex numbers is different. Multiplying “real” numbers (as you’ve done in maths before) simply scales the number along the “number line”, that is, the real axis. However, multiplying complex numbers (2D points on the plane) actually rotates the points in the plane as well as scales their position.

So this is basic intro into what code actually does. I will assume reader is familiar with setting up OpenCL, both for ATI/AMD and Nvidia graphic cards. Very briefly, for AMD cards you should download OpenCL SDK from here and for Nvidia ones CUDA toolkit form here. If we are talking about Windows and Visual Studio as C++ IDE then for Nvidia cards you should add following to “Additional include directories”:

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v5.5\include

and

C:\Program Files (x86)\AMD APP SDK\2.9\include

for AMD based cards. This is where your OpenCL header files will live after SDK inhalation. Same goes for “Library directories”

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v5.5\lib\Win32

and

C:\Program Files (x86)\AMD APP SDK\2.9\lib\x86

Of course paths can differ if your version and installation path is different then mine.

Last “setup” thing to do is to link against “OpenCL.lib”, actual OpenCL library. This is the point where you are good to go and write, compile and execute your OpenCL kernels.

In my particular example I am using “Windows Imaging Component” to manage bitmaps (load, save to file , etc.). Basically entire application, if I may call it that, is split into three files, main.cpp where application logic is, julia_kernels.cl – OpenCL implementation for generating Julia set images and imageio.cpp which is basically wrapper around aforementioned “WIC”.

Application allows to execute code both on CPU and GPU to be able to measure speed gains when using OpenCL over standard linear CPU implementation.

main.cpp

#include 
#include 
#include 
#include 
#include "imageio.h"
#include "setup_cl.h"
#include 

#include 
#include 

using namespace std;


//
// support structs (reflect vector types in OpenCL)
//

struct float2 {

	float x, y;

	float2(const float _x, const float _y) : x(_x), y(_y) {}
};

struct float3 {

	float x, y, z;

	float3(const float _x, const float _y, const float _z) : x(_x), y(_y), z(_z) {}
};

struct float4 {

	float x, y, z, w;

	float4(const float _x, const float _y, const float _z, const float _w) : x(_x), y(_y), z(_z), w(_w) {}
};


struct pixel_color {
	float R;
	float G;
	float B;
	float A;
};

pixel_color colors[16];

void InitPixelColors()
{
	pixel_color black = { 0.0f, 0.0f, 0.0f, 255.0f };
	pixel_color blue = { 9.0f, 26.0f, 236.0f, 255.0f };
	pixel_color brown = { 163.0f, 107.0f, 7.0f, 255.0f };
	pixel_color green = { 8.0f, 240.0f, 6.0f, 255.0f };
	pixel_color magenta = { 250.0f, 32.0f, 130.0f, 255.0f };
	pixel_color orange = { 250.0f, 107.0f, 6.0f, 255.0f };
	pixel_color red = { 250.0f, 0.0f, 0.0f, 255.0f };
	pixel_color darkGrey = { 50.0f, 50.0f, 50.0f, 255.0f };
	pixel_color lightBlue = { 50.0f, 170.0f, 200.0f, 255.0f };
	pixel_color lightGreen = { 50.0f, 240.0f, 175.0f, 255.0f };
	pixel_color lightYellow = { 245.0f, 250.0f, 140.0f, 255.0f };
	pixel_color violetRed = { 245.0f, 150.0f, 180.0f, 255.0f };
	pixel_color ivory = { 255.0f, 250.0f, 240.0f, 255.0f };
	pixel_color yellow = { 235.0f, 255.0f, 15.0f, 255.0f };
	pixel_color cyan = { 0.0f, 240.0f, 240.0f, 255.0f };
	pixel_color lime = { 191.0f, 255.0f, 0.0f, 255.0f };

	colors[0] = black;
	colors[1] = blue;
	colors[2] = brown;
	colors[3] = green;
	colors[4] = magenta;
	colors[5] = orange;
	colors[6] = red;
	colors[7] = darkGrey;
	colors[8] = lightBlue;
	colors[9] = lightGreen;
	colors[10] = lightYellow;
	colors[11] = violetRed;
	colors[12] = ivory;
	colors[13] = yellow;
	colors[14] = cyan;
	colors[15] = lime;
}

pixel_color getPixelColor(int val){

	return colors[val];
}

void generate_julia_set_image(
	cl_context context, 
	cl_device_id device, 
	cl_command_queue commandQueue, 
	cl_program program, 
	float c_component_real_part, 
	float c_component_imaginary_part,
	int imageWidth,
	int imageHeight,
	int numIterations
	)
{
	cl_int err;
	cl_kernel kernel = 0;
	cl_mem outputImage = 0;
	BGRA8 *result;
	pixel_color pixelColor;
	pixelColor.R = 1.0;
	pixelColor.G = 1.0;
	pixelColor.B = 0.0;
	pixelColor.A = 1.0;
	cl_mem pixelColorBuffer;

	float2 C_component(c_component_real_part, c_component_imaginary_part);
	cl_mem c_componentBuffer;

	try
	{
		kernel = clCreateKernel(program, "generate_image", 0);
		if (!kernel)
			throw(string("could not create kernel"));

		// setup output image
		cl_image_format outputFormat;

		outputFormat.image_channel_order = CL_BGRA; // store a BGRA image
		outputFormat.image_channel_data_type = CL_UNORM_INT8;// Each component is 8 bits in the range [0, 1]

		outputImage = clCreateImage2D(context, CL_MEM_WRITE_ONLY, &outputFormat, imageWidth, imageHeight, 0, 0, &err);

		pixelColorBuffer = clCreateBuffer(context, CL_MEM_READ_ONLY | CL_MEM_COPY_HOST_PTR, sizeof(pixel_color), &pixelColor, 0);
		c_componentBuffer = clCreateBuffer(context, CL_MEM_READ_ONLY | CL_MEM_COPY_HOST_PTR, sizeof(float2), &C_component, 0);

		if (!pixelColorBuffer){
			throw(string("pixel buffer failed"));
		}

		clSetKernelArg(kernel, 0, sizeof(cl_mem), &pixelColorBuffer);
		clSetKernelArg(kernel, 1, sizeof(cl_mem), &outputImage);
		clSetKernelArg(kernel, 2, sizeof(cl_mem), &c_componentBuffer);
		clSetKernelArg(kernel, 3, sizeof(int), &numIterations);


		// setup worksize arrays
		size_t globalWorkSize[2] = { imageWidth, imageHeight };

		// setup event (for profiling)
		cl_event doneEvent;

		// enqueue kernel
		err = clEnqueueNDRangeKernel(commandQueue, kernel, 2, 0, globalWorkSize, 0, 0, 0, &doneEvent);

		// block until kernel finishes and report time taken to run the kernel
		clWaitForEvents(1, &doneEvent);
		cl_ulong startTime = (cl_ulong)0;
		cl_ulong endTime = (cl_ulong)0;

		clGetEventProfilingInfo(doneEvent, CL_PROFILING_COMMAND_START, sizeof(cl_ulong), &startTime, 0);
		clGetEventProfilingInfo(doneEvent, CL_PROFILING_COMMAND_END, sizeof(cl_ulong), &endTime, 0);

		double tdelta = (double)(endTime - startTime);

		cout << "time taken (in seconds) to create the image = " << (tdelta * 1.0e-9) << endl;

		// extract the resulting image from OpenCL
		result = (BGRA8*)malloc(imageWidth * imageHeight * sizeof(BGRA8));
		if (!result)
			throw("cannot create output image buffer to save to disk");

		size_t origin[3] = { 0, 0, 0 };
		size_t region[3] = { imageWidth, imageHeight, 1 };

		err = clEnqueueReadImage(commandQueue, outputImage, CL_TRUE, origin, region, 0, 0, result, 0, 0, 0);

		wstring filename(L"gpu_" + to_wstring(imageWidth) + L"x" + to_wstring(imageHeight) + L"_" + to_wstring(255) + L".bmp");
		saveImageBGRA8(imageWidth, imageHeight, result, filename);
	}
	catch (std::string err)
	{
		cout << "Error has occured: " << err << std::endl;
	}
	
	/* Deallocate resources */
	clReleaseMemObject(pixelColorBuffer);
	clReleaseMemObject(outputImage);
	clReleaseMemObject(c_componentBuffer);
	clReleaseKernel(kernel);
	
}



double GetComplexLenght(complex input)
{
	return std::sqrt(std::pow((std::real(input)), 2) + std::pow((std::imag(input)), 2));
}

void RunOnCPU(int imageWidth, int imageHeight, int numIterations, double c_component_real_part, double c_component_imaginary_part)
{
	std::chrono::time_point start, end;
	start = std::chrono::system_clock::now();

	cout << "\n\nRunnig  on CPU\n" << endl;

	BGRA8* imageBuffer = (BGRA8*)malloc(imageWidth * imageHeight * sizeof(BGRA8));

	complex Z(0.0, 0.0);
	complex C(c_component_real_part, c_component_imaginary_part);


	for (int i = 0; i < imageHeight; i++)
	{
		for (int j = 0; j < imageWidth; j++)
		{
			Z = complex((-imageWidth / 2 + j) / (imageWidth / 4.0), (-imageHeight / 2 + i) / (imageHeight / 4.0));
			int iteration = 0;

			for (int k = 0; k < numIterations; k++)
			{
				Z = std::pow(Z, 2) + C;
				iteration++;
				if (GetComplexLenght(Z) > 2)
				{
					break;
				}
			}

			int pixelIndex = j + (i * imageWidth);
			if (iteration == numIterations)
			{
				imageBuffer[pixelIndex].r = 0;
				imageBuffer[pixelIndex].g = 0;
				imageBuffer[pixelIndex].b = 0;
				imageBuffer[pixelIndex].a = 255;
			}
			else
			{
				pixel_color pc = getPixelColor(iteration % 16);
				imageBuffer[pixelIndex].r = (BYTE)pc.R;
				imageBuffer[pixelIndex].g = (BYTE)pc.G;
				imageBuffer[pixelIndex].b = (BYTE)pc.B;
				imageBuffer[pixelIndex].a = (BYTE)pc.A;
			}
		}
	}

	end = std::chrono::system_clock::now();

	std::chrono::duration elapsed_seconds = end - start;
	std::time_t end_time = std::chrono::system_clock::to_time_t(end);

	std::cout << "\nFinished computation on CPU at " << std::ctime(&end_time)
		<< "elapsed time: " << elapsed_seconds.count() << "s\n\n\n";

	wstring filename(L"cpu_" + to_wstring(imageWidth) + L"x" + to_wstring(imageHeight) + L"_" + to_wstring(numIterations) + L".bmp");
	saveImageBGRA8(imageWidth, imageHeight, imageBuffer, filename);
	free(imageBuffer);
}

int RunOnGPU(int imageWidth, int imageHeight, int numIterations, double c_component_real_part, double c_component_imaginary_part)
{
	cout << "\n\nRunnig  on GPU\n" << endl;
	// 1. Create and validate OpenCL context
	cl_context context = createContext();

	if (!context)
	{
		cout << "cl context not created\n";
		shutdownCOM();
		return 1;
	}

	// 2. Get a list of devices associated with the context
	size_t deviceBufferSize;
	cl_int errNum = clGetContextInfo(context, CL_CONTEXT_DEVICES, 0, 0, &deviceBufferSize);
	cl_device_id* contextDevices = (cl_device_id*)malloc(deviceBufferSize);
	errNum = clGetContextInfo(context, CL_CONTEXT_DEVICES, deviceBufferSize, contextDevices, 0);

	// use the first device in the list - should be GPU since we requested this during context setup
	cl_device_id device = contextDevices[0];

	// 3. Create and vailedate program object based on julia_kernels.cl
	cl_program program = createProgram(context, device, "julia_kernels.cl");

	if (!program)
	{
		cout << "Could not create program object";
		shutdownCOM();
		return 1;
	}

	// 4. create and validate the command queue (for first device in context)
	// Note: add profiling flag so we can get timing data from events
	cl_command_queue commandQueue = clCreateCommandQueue(context, device, CL_QUEUE_PROFILING_ENABLE, 0);
	if (!commandQueue) {

		cout << "command queue not created\n";
		shutdownCOM();
		return 1;
	}

	// call function to carry out task
	generate_julia_set_image(context, device, commandQueue, program, c_component_real_part, c_component_imaginary_part, imageWidth, imageHeight, numIterations);

	/* Deallocate resources */
	clReleaseCommandQueue(commandQueue);
	clReleaseProgram(program);
	clReleaseContext(context);
}

void Test_800x600_255(double c_component_real_part, double c_component_imaginary_part)
{
	int imageWidth = 800;
	int imageHeight = 600;
	int numIterations = 255; 
	RunOnCPU(imageWidth, imageHeight, numIterations, c_component_real_part, c_component_imaginary_part);
	RunOnGPU(imageWidth, imageHeight, numIterations, c_component_real_part, c_component_imaginary_part);
}

void Test_1024x768_255(double c_component_real_part, double c_component_imaginary_part)
{
	int imageWidth = 1024;
	int imageHeight = 768;
	int numIterations = 255;
	RunOnCPU(imageWidth, imageHeight, numIterations, c_component_real_part, c_component_imaginary_part);
	RunOnGPU(imageWidth, imageHeight, numIterations, c_component_real_part, c_component_imaginary_part);
}

void Test_1680x1050_255(double c_component_real_part, double c_component_imaginary_part)
{
	int imageWidth = 1680;
	int imageHeight = 1050;
	int numIterations = 255;
	RunOnCPU(imageWidth, imageHeight, numIterations, c_component_real_part, c_component_imaginary_part);
	RunOnGPU(imageWidth, imageHeight, numIterations, c_component_real_part, c_component_imaginary_part);
}

void Test_6400x3200_255(double c_component_real_part, double c_component_imaginary_part)
{
	int imageWidth = 6400;
	int imageHeight = 3200;
	int numIterations = 255;
	RunOnCPU(imageWidth, imageHeight, numIterations, c_component_real_part, c_component_imaginary_part);
	RunOnGPU(imageWidth, imageHeight, numIterations, c_component_real_part, c_component_imaginary_part);
}

void TestScalability()
{
	int imageWidth = 6400;
	int imageHeight = 3200;
	double c_component_real_part = -0.805;
	double c_component_imaginary_part = 0.156;
	int numIterations = 255;

	while (RunOnGPU(imageWidth, imageHeight, numIterations, c_component_real_part, c_component_imaginary_part) == 0)
	{
		imageWidth += 1000;
		imageHeight += 1000;
	}
}

void RunAutomaticTests()
{
	cout << "Runnig automatic tests!" << endl;
	double c_component_real_part = -0.805;
	double c_component_imaginary_part = 0.156;

	// Run speed comapre tests
	Test_800x600_255(c_component_real_part, c_component_imaginary_part);
	Test_1024x768_255(c_component_real_part, c_component_imaginary_part);
	Test_1680x1050_255(c_component_real_part, c_component_imaginary_part);
	Test_6400x3200_255(c_component_real_part, c_component_imaginary_part);

	// Test Scalability - How big images can we create
	TestScalability();

	cout << "Test run finished!" << endl;
}

int main(int argc, char **argv)
{
	// initialise COM so we can export image data using WIC
	initCOM();

	InitPixelColors();

	// clear screen
	system("cls");
	cout << "***************************************************************\n";
	cout << "*             Julia set generator                             *\n";
	cout << "***************************************************************\n\n\n";

	cout << "Do you want to run automatic tests both on GPU and CPU (y/n)? ";
	char answer;
	cin >> answer;

	if (answer == 'y' || answer == 'Y'){
		RunAutomaticTests();
	}
	else
	{
		cout << "Enter image width: ";
		int imageWidth = 800;
		cin >> imageWidth;
		cout << "\nEnter image height: ";
		int imageHeight = 600;
		cin >> imageHeight;

		float c_component_real_part;
		cout << "\nEnter real part of C component: ";
		cin >> c_component_real_part;

		float c_component_imaginary_part;
		cout << "\nEnter imaginary part of C component: ";
		cin >> c_component_imaginary_part;

		int numIterations = 255;
		cout << "\nEnter the number of iterations: ";
		cin >> numIterations;

		RunOnCPU(imageWidth, imageHeight, numIterations, c_component_real_part, c_component_imaginary_part);

		RunOnGPU(imageWidth, imageHeight, numIterations, c_component_real_part, c_component_imaginary_part);
	}
	//done
	shutdownCOM();

	return 0;
}

And this is actual OpenCL kernel implementation:

// Julia set structs and kernels
typedef struct pixel_color {
	float		R; 
	float		G; 
	float		B;
	float		A;
} pixel_color;

//2 component vector to hold the real and imaginary parts of a complex number:
typedef float2 cfloat;

#define I ((cfloat)(0.0, 1.0))
#define M_PI 3.14159265358979323846

/*
 * Return Real (Imaginary) component of complex number:
 */
inline float real(cfloat a){
     return a.x;
}
inline float imag(cfloat a){
     return a.y;
}

/*
 * Get the modulus of a complex number (its length):
 */
inline float cmod(cfloat a){
    return (sqrt(a.x*a.x + a.y*a.y));
}

// Add two complex numbers
inline cfloat cadd(cfloat a, cfloat b){
	return (cfloat)( a.x + b.x, a.y + b.y);
}

/*
 * Get the argument of a complex number (its angle):
 * http://en.wikipedia.org/wiki/Complex_number#Absolute_value_and_argument
 */
inline float carg(cfloat a){
    if(a.x > 0){
        return atan(a.y / a.x);

    }else if(a.x < 0 && a.y >= 0){
        return atan(a.y / a.x) + M_PI;

    }else if(a.x < 0 && a.y < 0){
        return atan(a.y / a.x) - M_PI;

    }else if(a.x == 0 && a.y > 0){
        return M_PI/2;

    }else if(a.x == 0 && a.y < 0){
        return -M_PI/2;

    }else{
        return 0;
    }
}


/*
 * Multiply two complex numbers:
 *
 *  a = (aReal + I*aImag)
 *  b = (bReal + I*bImag)
 *  a * b = (aReal + I*aImag) * (bReal + I*bImag)
 *        = aReal*bReal +I*aReal*bImag +I*aImag*bReal +I^2*aImag*bImag
 *        = (aReal*bReal - aImag*bImag) + I*(aReal*bImag + aImag*bReal)
 */
inline cfloat  cmult(cfloat a, cfloat b){
    return (cfloat)( a.x*b.x - a.y*b.y, a.x*b.y + a.y*b.x);
}

/*
 *  Square root of complex number.
 *  Although a complex number has two square roots, numerically we will
 *  only determine one of them -the principal square root, see wikipedia
 *  for more info: 
 *  http://en.wikipedia.org/wiki/Square_root#Principal_square_root_of_a_complex_number
 */
 inline cfloat csqrt(cfloat a){
     return (cfloat)( sqrt(cmod(a)) * cos(carg(a)/2),  sqrt(cmod(a)) * sin(carg(a)/2));
 }

 inline pixel_color getPixelColor(int val){
	pixel_color colors[16];
	pixel_color black = { 0.0f, 0.0f, 0.0f, 1.0f };
	pixel_color blue = { 9.0f, 26.0f, 236.0f, 1.0f };
	pixel_color brown = { 163.0f, 107.0f, 7.0f, 1.0f };
	pixel_color green = { 8.0f, 240.0f, 6.0f, 1.0f };
	pixel_color magenta = { 250.0f, 32.0f, 130.0f, 1.0f };
	pixel_color orange = { 250.0f, 107.0f, 6.0f, 1.0f };
	pixel_color red = { 250.0f, 0.0f, 0.0f, 1.0f };
	pixel_color darkGrey = { 50.0f, 50.0f, 50.0f, 1.0f };
	pixel_color lightBlue = { 50.0f, 170.0f, 200.0f, 1.0f };
	pixel_color lightGreen = { 50.0f, 240.0f, 175.0f, 1.0f };
	pixel_color lightYellow = { 245.0f, 250.0f, 140.0f, 1.0f };
	pixel_color violetRed = { 245.0f, 150.0f, 180.0f, 1.0f };
	pixel_color ivory = { 255.0f, 250.0f, 240.0f, 1.0f };
	pixel_color yellow = { 235.0f, 255.0f, 15.0f, 1.0f };
	pixel_color cyan = { 0.0f, 240.0f, 240.0f, 1.0f };
	pixel_color lime = { 191.0f, 255.0f, 0.0f, 1.0f };
	
	colors[0] = black;
	colors[1] = blue;
	colors[2] = brown;
	colors[3] = green;
	colors[4] = magenta;
	colors[5] = orange;
	colors[6] = red;
	colors[7] = darkGrey;
	colors[8] = lightBlue;
	colors[9] = lightGreen;
	colors[10] = lightYellow;
	colors[11] = violetRed;
	colors[12] = ivory;
	colors[13] = yellow;
	colors[14] = cyan;
	colors[15] = lime;

	pixel_color normalizedColorVal = { colors[val].R / 255.0f, colors[val].G / 255.0f, colors[val].B / 255.0f, 1.0f };

	return normalizedColorVal;
 }

__kernel void generate_image(const global pixel_color *pixelColor, write_only image2d_t outputImage , const global float2 *c_component,  int numIterations)
{
	// get id of element in array
	int x = get_global_id(0);
	int y = get_global_id(1);
	int w = get_global_size(0);
	int h = get_global_size(1);

	pixel_color pc = *pixelColor;
	//pc.R = c_component[0].x;
	//pc.G = c_component[0].y;

	cfloat Z = { ( -w / 2 + x) / (w/4.0f) , ( -h / 2 + y) / (h/4.0f) };
	cfloat C = { c_component[0].x, c_component[0].y };
	int iteration = 0;

	while (iteration < numIterations)
	{
		 cfloat Zpow2 = cmult(Z, Z); 
		 cfloat Zn = cadd(Zpow2, C);
		 Z.x = Zn.x;
		 Z.y = Zn.y;
		 iteration++;
		 if(cmod(Z) > 2)
		 {
			break;
		 }
	}

	// threshold reached mark pixel as black
	if (iteration == numIterations)
	{
		pc.R = 0.0f;
		pc.G = 0.0f;
		pc.B = 0.0f;
	}
	else
	{
		pc = getPixelColor(iteration % 16);
	}

	// RGBA
	float4 color = (float4)(pc.R, pc.G, pc.B, pc.A);

	write_imagef(outputImage, (int2)(x, y), color);
}

Here is the example application running.

I don't want to get into details about how each line of code works. I’ve tried to comment on important parts of code. If you have any questions about some parts of the code hit me in comments and I’ll be glad to clear things up.

Here is the entire solution available for download.

JuliaSet.zip (19.74 kb)

LINQ: Finding out duplicates in the list with multiple fields

Sat, 19 Jul 2014 05:10:00 -0500

I needed very quick simple “one line” solution to filter out duplicates in my list. So I have a List<T> where T is complex object with multiple properties. I wanted to filter out non unique items based on two fields. In database analogy my list is table with two columns based composite primary key and I need to select non-distinct values.

Here is sample code:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace LinqDuplicates
{
    public class Item
    {
        public int ID { get; set; }
        public string Name { get; set; }
    }

    class Program
    {
        static void Main(string[] args)
        {
            List items = new List();
            items.Add(new Item { ID = 0, Name = "John" });
            items.Add(new Item { ID = 1, Name = "Thorsten" });
            items.Add(new Item { ID = 1, Name = "Thorsten" });
            items.Add(new Item { ID = 2, Name = "Rapunzel" });
            items.Add(new Item { ID = 3, Name = "Rapunzel" });
            items.Add(new Item { ID = 3, Name = "Grinch" });

            var duplicates = items.GroupBy(x => new {x.ID, x.Name})
                            .Where(g => g.Count() > 1)
                            .Select(g => g.Key);

            foreach (var val in duplicates)
            {
                Console.WriteLine(val.ID + " " + val.Name);
                Console.ReadLine();
            }

        }
    }
}

So this code will select just one duplicate I have in list and that is Item with ID=1 and Name=”Thorsten”.

Another way around to get all distinct values would be:

var distinct = items.Select(item => new { item.ID, item.Name }).Distinct();

This results in list with all the distinct values based on two fields (composite key) criteria.

Node.js Blueprints book review

Mon, 16 Jun 2014 00:26:00 -0500

Couple of months ago I did a book review for PacktPub for a Node.js beginner level book. Today I got a great news the book is finally published and available. It is a very simple 270 pages read and follow trough book. It touches up on many other popular frameworks and tools like AngularJS, Socket.io, BackboneJS, EmberJS, GruntJS. I would definitely recommend it for someone looking for quick start with Node.js.

OS X 10.10 Yosemite on Virtual Machine

Tue, 03 Jun 2014 21:31:00 -0500

Apple just announced next version of OS X and I wanted to give it a try. Beta will be available for first million subscribers or you can get it from third party sites. I couldn’t wait for notification mail from Apple so I got my copy from here. Since this is a beta I didn’t want to mess up my current system so virtual machine was better choice. I’ve used Oracle Virtual Box for OS X but it works with VMware Fusion or Parallels as well. Basically it’s an upgrade of you current OS X system so you will need running OS X virtual machine. There is a ton of tutorials how to install OS X on virtual machine. I’ve had OS X 10.8 and upgrade went smoothly. Simply extract and copy installer to your virtual machine and run it. It will restart two times and that’s about it.

I’ve used my 2011 iMac as a host with Intel Core i5 processor so experience is not that impressive. It is obvious that system is still in Beta. On the other hand UI looks pretty nice in my opinion but with all that translucency it’s starting to look very much like Windows.

Here are some screenshots of running system.

1. Login Screen

2. Apple ID

3. Desktop & Screen Saver

4. System Info

5. Safari

6. Spotlight

7. Lunchpad

Qt: Detect in the .pro file if compiling on 32 bit or 64 bit platform

Sun, 25 May 2014 23:18:00 -0500

Couple of months earlier I wrote how to compile Qt projects (.pro files) from command line both on OS X and Windows. Basically it comes down to calling qmake executable and passing path to your .pro file and path to make spec file for specific compiler. It goes something like this:

C:\QT5_build\bin\qmake.exe [Path to your .pro file] 
-r spec C:\QT5_build\mkspecs\win32-msvc2010

I am using shared .pri configuration files which I later include in my .pro files to have consistent settings across different project. In my base .pri settings files I define where are shared libraries, include files, platform specific compiler flags.

One of the things I want to control is where should output go, namely DESTDIR. I wanted to for my 32 bit builds to land in Output folder and 64 bit in Outputx64, both on Windows and OS X. Here is my shared configuration .pri file:

BUILD_TARGET = SimpleProjectName

win32{
	PROJECT_FOLDER = ..\\..
}
else{
	PROJECT_FOLDER = ../..
}


macx-clang{
    OUTPUT_FOLDER = $$PROJECT_FOLDER/Outputx64
}

macx-clang-32{
    OUTPUT_FOLDER = $$PROJECT_FOLDER/Output
}

win32{
	SOURCE_FOLDER = $$PROJECT_FOLDER\\Source
	OUTPUT_FOLDER = $$PROJECT_FOLDER\\Output$$(Platform)
	TEMP_FOLDER =   $$OUTPUT_FOLDER\\Temp
	MODULE_TEMP_FOLDER = $$TEMP_FOLDER\\$$BUILD_TARGET
	GENERATED_FOLDER = $$OUTPUT_FOLDER\\Generated
	MODULE_GENERATED_FOLDER = $$MODULE_TEMP_FOLDER\\Generated
}
else{
	SOURCE_FOLDER = $$PROJECT_FOLDER/Source
	TEMP_FOLDER = $$OUTPUT_FOLDER/Temp
	MODULE_TEMP_FOLDER = $$TEMP_FOLDER/$$BUILD_TARGET
	GENERATED_FOLDER = $$OUTPUT_FOLDER/Generated
	MODULE_GENERATED_FOLDER = $$MODULE_TEMP_FOLDER/Generated
}

CONFIG(debug, debug|release): CONFIGURATION = Debug
CONFIG(release, debug|release): CONFIGURATION = Release

win32{
	CONFIGURATION_OUTPUT_FOLDER = $$OUTPUT_FOLDER\\$$CONFIGURATION
	CONFIGURATION_MODULE_TEMP_FOLDER = $$MODULE_TEMP_FOLDER\\$$CONFIGURATION
}
else{
	CONFIGURATION_OUTPUT_FOLDER = $$OUTPUT_FOLDER/$$CONFIGURATION
	CONFIGURATION_MODULE_TEMP_FOLDER = $$MODULE_TEMP_FOLDER/$$CONFIGURATION
}

# set intermediate directories
win32{
	UI_DIR = $$GENERATED_FOLDER\\Qt\\$$BUILD_TARGET
	RCC_DIR = $$CONFIGURATION_MODULE_TEMP_FOLDER
	MOC_DIR = $$MODULE_GENERATED_FOLDER
	OBJECTS_DIR= $$CONFIGURATION_MODULE_TEMP_FOLDER
}
else{
	UI_DIR = $$GENERATED_FOLDER/Qt/$$BUILD_TARGET
	RCC_DIR = $$CONFIGURATION_MODULE_TEMP_FOLDER
	MOC_DIR = $$MODULE_GENERATED_FOLDER
	OBJECTS_DIR= $$CONFIGURATION_MODULE_TEMP_FOLDER
}

DESTDIR = $$CONFIGURATION_OUTPUT_FOLDER

With this configuration my project files are normally under "Source" folder and then I will get to right next to it Output or Outputx64 depending on -arch I am using. Inside Output folder there will be Debug or Release folder, based on configuration I am compiling. Next to Debug and Release there will be folders for Temp and Generated files. Something like this:

Key lines for mac OS X are number 11 and number 15. macx-clang is 64bit build configuration and macx-clang-32 is 32 bit build configuration. On line 21 I set Windows build configuration based on $$(Platform) variable. This will in case of 64 bit build add “x64” to “Output”. This flag works only for windows as far as I know.

Note: In order to be able to build both 32 and 64 bit executables with Qt on OS X platform Qt must be also compiled for desired architecture. To be able to build Qt on OS X for 32bit architecture Qt build must be configured with -platform macx-clang-32 flag before calling make command. For example:

./configure -opensource -confirm-license -platform macx-clang-32 -debug-and-release -no-compile-examples ….