Less pain in your's development hands - Oleksii's Blog

Cordova (PhoneGap) iOS Security

2012-12-27T00:00:00-08:00

Almost a year ago I wrote an article about securing PhoneGap source files on iOS platform.

There were a lot of questions, and private emails, that’s why I have actualized the description a bit, and made new manual.

NOTE: using this method will not secure your javascript code 100%, but will help a lot to protect it from a lot of hackers. This methid ONLY works for iOS.

The idea

The main idea at the moment contain 2 steps:

Prepare single-file application inside index.html and encrypt it with key.
Put decryption mechanism into XCode application while loading the html.

At step #1 better to embed all CSS, JS and even images into one file, so it will a hacker less holes to get in.

At step #2 your encryption key will be stored in binary mode, so only very experienced hackers will try to find it.

Preparing index.html

Once you have developed your application with cordova, we need to encrypt it. For this purpose I have made a small tool, as previously a lot of things was not working in right way.

To install the tool, you should install NodeJS and NPM like it described here.

Then you should install my small tool corc through Mac OS terminal:

npm install -g cordovacrypt

Maybe, you will need to use sudo.

To run encryption you should execute:

corc KEY FILEIN [FILEOUT]

Where:

KEY - your key without spaces and any bash illegal symbols.
FILEIN - path to the index.html file which should be encrypted.
FILEOUT - optional parameter where to save encrypted file. If not specified, original will be rewritten.

Example:

corc mySecretKey index_original.html index.html

Now you have prepared index.html file.

Making changes to XCode application

Make sure, you have created a new Cordova application like it is written here, and make next steps:

Once the application is ready, open project inside XCode, and go to the Project Navigator.
There you will see the www item, *YourProjectname*.xcodeproj, etc.
Unfold .xcodeproj item and than Classes > Cleaver and select CDVViewController.m.
In the center you will see the source code of CDVViewController.m.
Find the line - (void)viewDidLoad. This is a method on view loading.
Scroll down to the end of this method (the last if statement), where will be the line: if (!loadErr) {.

You should remove everything inside this if statement, except body of else statement. And paste next code into if statement:

Please, enable javascript, to see the source code.

This code will hook the index.html load and give it a different content.

Now after the method viewDidLoad (or before, it is up to you), insert 2 more service methods, which will actually decode the encrypted index.html:

Please, enable javascript, to see the source code.

In line 17 you may see a strange string ffffffffffffffffffffffff - this is your encryption password. Replce it with yours.

The last thing you should add into this file are two includes. Go to the top of the file, and after all #import directives add:

#import <CommonCrypto/CommonCryptor.h>
#import <CommonCrypto/CommonDigest.h>

At this point you can compile the code and see that it is working!

Belz. Using AppHarbor as .Net worker mule

2012-05-10T00:00:00-07:00

We are living at a time, when everyday brings something brand new and fantastic. For a lot of time already I am keeping an eye on AppHarbor service. Shared .Net hosting became free and usable. More than that, AppHarbor allows you to run not only websites, but daemons (services/workers), which are created from simple console application.

Lately, I have discovered Iron.IO, which provides stable distributed messaging protocol, and even dynamic workers module. Unfortunately, they do not support .Net for workers yet. This article provides an information on how to implement almost free daemon for your needs.

Design

When you need a background process to process some jobs, the main parts of it should be:

Queue of messages;
API calls to messages queue to enqueue/dequeue messages;
Daemon application.

Today, all these parts could be made/used with almost no cost involved.

Queue of messages

With all today tools and services, there is no need to develop bicycle again and again. There already are numerous count of services, which provides queue messaging with even no cost for starting project.

Everything I have investigated, are:

IronMq;
CloudAMQP (RabbitMQ code inside)

Both of them are almost the same, but I recon API of IronMq is a bit better.

API calls to messaging queue

To manage the queue, we need an API connection to it. IronMQ provides such functionality with next methods:

Enqueue message(s);
Dequeue message(s).

More than that, after dequeue from IronMq, message is not deleted from server actually, but waits a timeout to be returned back (for example your code has an exception). It will be removed from server only after deletion of message.

NOTE: IronMQ is a paid service, where 10000 requests costs 1 US cent (the price for today). If you are requesting queue to check for messages every second, it will cost you about 3 US dollars per month. If you are processing a lot of messages during each request - that is not a price for good service.

Daemon application

This is the most sweet part of whole article. You can create a separate windows service, which will work as daemon. Inside daemon, it will be good to create several parallel threads for workers. Best way would be, if your daemons are set up on different servers, to scale the work. And surely, you need to host it somewhere.

Let’s code some worker, which will calculate fibonacci value and push the result to your iPhone, through Prowl. My proposal is using AppHarbor as daemon hosting and IronMq as message queue service. As daemon kernel, we will use Belz project.

10 minutes of development

First of all you have to register an account at AppHarbor.

Now login to the system and inside “Applications” section create an application “Belz Try”.

Go to “Addons” section, and select IronMQ addon to be installed. Inside “Configuration variables” section, you will find settings for your IronMq queue project.

Now a bit of coding. Copy the repository url and make git clone from it.

Next step to create a new solution inside cloned directory from Visual Studio.

After creation of solution, be sure, you have NuGet installed (this will make your life easier). Create console application inside solution, and add next NuGet packages:

Belz.QueueProvider.IronMq;
Belz.QueueProvider;
Belz;
Prowlin (for prowl notifications);

Now, modify your Program.cs to look like this:

Please, enable javascript, to see the source code.

Don’t forget to change API keys and tokens to use app.

After this step, commit and push your AppHarbor daemon to server.

And you are done.

Testing

I have created a sample application to put some work into queue:

ConsoleApplication1.exe 13

This command will put number 13 to be calculated for fibonacci by our daemon, and sent to iPhone through Prowl service.

The code of application is simple:

Please, enable javascript, to see the source code.

It require next NuGet Packages:

Belz.QueueProvider.IronMq;
Belz.QueueProvider;

Big notice

Don’t forget to switch off your test daemon, because in sometime if you reach out free requests, IronIO will bill you for this.

Extending Migrator.NET with logic scopes

2012-03-12T00:00:00-07:00

Migrator.NET (or at GitHub) is a well-known migration tool for .Net Framework. For already a years this lib is stable and much better than even today’s Entity Framework migrations. It is also very hard to add something new to the already good thing, but some time ago I have a need to extend it’s functionality.

The Aim

You should know, that the usual way of working with db migrations is manual upgrade of db scheme. So, when your project is big enough and has no unpredicted parts, the Migrator.Net is what you really need.

In my case, there was a project, which should contain different db structures in different modules. Modules may be written by different people, and you can understand what a nightmare can be to allow all of them using 1-2-3 versioning system, which is default today. For example:

we have 2 modules A and B, which are supported by 2 different people;
A has got a migration 2012031200 (timestamp for the version YYYYMMDDHH) on Monday;
the developer of B saw it and added a migration 2012031300 on Tuesday to module B;
on Wednesday the developer of A has added new migration 2012031400;
when system is launched, it should run all latest migrations to make a db scheme actualized.

You must notice, that migrations of both modules are not changing each other tables, but only their own needed structures.

And here will be a problem, if Tuesday migration failed, Wednesday migration will not run at all. That’s why we should dedicate separate logical spaces for each logical module.

The Proposal

First of all the task was to remain backward compatibility, so for the end developer there will not be a difference with scope or without it.

Than I have added a new parameter to migration attribute:

Please, enable javascript, to see the source code.

And after this, if you want to migrate to latest, you have to specify the scope name of the migrations. A list of scopes can be fetched with Scopes property of Migrator class.

If you wish to apply all migrations from all scopes, use MigrateAllScopesToLastVersion method inside Migrator class.

Modified version of Migrator.NET is available at GitHub.

Easter Egg

If you specify a scope as empty string, code will not detect scope automatically, but will return the default scope (32 zeroes).

Using MvcMiniProfiler with NancyFX

2012-03-05T00:00:00-08:00

Lately, some new interesting web-development frameworks appeared based on mvc concept. Some of them are totally new experience, but others are using the best experience from different technologies and languages.

Ruby On Rails Clones

Sinatra - is one of the smoothest and intelligent frameworks I’ve ever seen. When Ruby became famous, a lot of things were ported from this language, to others. I’ve even investigated a PHP port named Slim Framework, and it is really fantastic as for PHP.

NancyFX has appeared at the end of 2010 and day-by-day they have damage the brain of the C# coding society (the web development part). With this, many interesting tools as MiniProfiler, should be also available for use with NancyFX.

NancyFX and MiniProfiler

On some stage of coding with NancyFX, I’ve got a need to measure the performance of it. There were no doubts, that MvcMiniProfiler will do the job. As I have not found a working solution for this, I’m posting it here.

All you need to do, is create a custom bootstrap for NancyFX. How to make it, please read here.

Than you have to override some methods of custom bootstrap, so the profiler will be injected into code.

Please, enable javascript, to see the source code.

MvcMiniProfiler is describing how to inject the header into template and how to run tests very good on their site, so I am not touching this theme.

The example of profiler usage:

Please, enable javascript, to see the source code.

PhoneGap (newely Cordova) Application Encryption

2011-09-20T00:00:00-07:00

This article is deprecated, please read the fresh one here.

This time I am writing about PhoneGap Application security.

It happened that I was involved into a simple iPhoneApp development. The reason, why we have selected PhoneGap is simple - nobody knows Objective C. So, during a month team was developing an iPhone application with PhoneGap API. And when we have got a first compilable and functional sample, it occurs, that “app” file could be unpacked as zip archive, and all our javascript is visible and readable.

There were 2 solutions we could jump into:

Move to Objective C
Secure the javascript code

Item #1 for our team was complicated and long in time to learn.

Item #2 could be a way, but there was a need to read secured code from PhoneGap-based application. So, here was also 2 ways - secure with javascript encryption like JJEncode, or to encrypt javascript, which will be inside app file, and decrypt it in memory on application lunch and put into WebView context. And we have took the second way.

The Way

Now let me explain the method, how in theory the encryption is made.

We have some build scripts, and nobody is working in XCode with javascript and html. All of us are using favorite source editors for it. XCode is used only to build a project for deployment and test. So, we have build scripts:

Script is parsing index.html, and include all references to javascript/css/images inside this one html file. In this way, we have bundled application inside one index.html file.
Script is encrypting index.html with AES 256bit encryption and put this file into XCode project as resource (please, note this).
XCode build is running and generating app-file.

PhoneGap Patch

The PhoneGap library was patched to support decoding of the index.html with provided key and load this memory block into WebView. I have shared the modifiew PhoneGap sourcecode at Github. To use it, you should manually build the library from sourcecode and install it. Don’t forget, that PhoneGap library now is trying to search for an encrypted “index.html” resource file.

Providing Key

Sure, you need to provide key for patched PhoneGap framework to decrypt your source code.

Inside your AppDelegate.m file, add a method:

Please, enable javascript, to see the source code.

Here you see, that your’s 256bit key is 32 f symbols.

You see the finish

Yes, that is all. Now your’s html will be decrypted and loaded into WebView on iApp start, and there will not be files unsecured insude your app.

That is right, that someone can decompile objective c code to find out the key, but this is a bit harder than if you show all your sources from zip.

P.S.

At the end, maybe you will need a method to encrypt your html file. I have it only for php:

Please, enable javascript, to see the source code.

Another RestAPI accessor for .Net

2010-07-03T00:00:00-07:00

Some time has passed after I have launched the open-source library to access Google Storage Rest Api. By that time there was a problem accessing the REST service of google. The solution I have found was a great library named RestSharp. On the first glance it came, that library is very handy and lightweight. For sure, developers of it are great people and professionals. But, when it came to customize the RestClient for GS api needs, it found out that it is very hard and not intuitive to extend the flexibility of this library.

That’s why I have developed some small and light solution for simple RestApi requests. Have named it Rest4Net.

The body

The most of the functionality you need to do, is to inherit your class from Rest4Net.Connector.

And that’s almost all. You need to implement constructor and methods, your client class will return.

If response from the server is Xml or Json, this library can parse response into object (like deserialisation). If you need some array field, mark it as IList<> please.

For example, I have made an implementation of Bit.ly service client here.

How to use the service demo application is here.

Hope, this library will help someone. ;)

Google Storage API with C#. SharpGs. Try it

2010-06-17T00:00:00-07:00

Today I have great news for you - SharpGs, is a first open-source C# library to access Google Storage API.

Why have I done that - maybe for fun and interest of Google products. But the real idea was that now I have something new to share and give to society, and became alive in my blog :).

How to use

Here will be the small instruction on how to use the library. On the GitHub you will find a demo project with the main function calls. So, here is the instruction:

First of all you need to get invite or right to access this beta-testing launch of SharpGs.
Here you need to create new access key pair (open key and a secret to crypt verification data).
In your code create a new instance of SharpGsClient class and pass key and secret, you have got from step 2. And you are done :)

To get the list of buckets from the server, use the Buckets property of SharpGsClient instance object. This property provides the enumeration of IBucket interface objects, which has properties of bucket information or methods to operate with bucket objects.

Each call of the method could throw an exception if something goes wrong. The list of supported exceptions will be increased in a time.

I think this library is too simple to make tons of examples, so, if there will be questions, I will answer them in comments or other post.

Sorry, for leak of comments inside the code, hope they will appear there soon.

Have a nice try!

AcroDB. Business Logic

2010-04-13T00:00:00-07:00

Previously I have talked only about entity creation and maintain. But what if we need to add some business logic to this entity. For example, we have a table of Pages, and you need to fetch all cities of some country?

Preparation

First of all we need never to forget, that all AcroDB entities are lazy-loaded and each of them contains a reference to the AcroDataContext, that created them. And if you want to save an integrity, you need to use the same AcroDataContext when fetching new entities. So, how will we do?

Entity creation

The first step was written before is the entity creation:

Please, enable javascript, to see the source code.

Business logic

Under the interface we have small problem, which is we can’t implement methods. So, we need to create separate classes of business logic, or to write extension methods. I used to implement extension method for the interface:

Please, enable javascript, to see the source code.

You may notice the extension method AcroContext(). To use it you need to reference AcroDB.Helpers namespace. This method returns the link to data context that was attached to entity.

The result

So, in result you could call:

Please, enable javascript, to see the source code.

And all children of the page will be fetched from DB.

AcroDB. How to use

2010-03-26T00:00:00-07:00

Hi, all! Almost a week have passed, after I have posted my new flexible provider article. Maybe, nobody commented it, because of article compicity… So, today I will try to show you, how to use AcroDb.

The model

Usually, when you start to design your future application, you need to describe 3 levels of MVC, and first of it goes the model layer. For the simple projects, the model layer consists of simple tables/entities. Let’s say we have to build a model for only one entity type, to store cars.

Car entity description

What fields should this entity have? Let them be:

ID - the unique Guid of car record (primary key).
Name - car name string (if we need to specify the limit of this field, let it be maximum 80 chars).
YearOfAssembly - the year, when this car starts to drive (integer value).
Description - the text with notes about the car (infinite string).

Car entity prototype in AcroDb

Now let me describe, how this entity will look like in code using AcroDb library.

Please, enable javascript, to see the source code.

And if you have setup AcroDataContext class settings correctly as it was described in first post, after application start you will have a Car table in your database/no-relational system. Or if you will change properties of this interface, table schema will be updated, thanks to SubSonic.

So, AcroColumnStringLength attribute is used to specify the maximum length of the column in db. The AcroColumnStringLong attribute marks te column as infinite (MAX) size. For all column attributes, please, read here.

Working with entity

Now I will show the parts of code, how to work with our entity.

Creating and saving an object

Please, enable javascript, to see the source code.

Querying a db

Here is a sample of Linq query for multiply objects:

Please, enable javascript, to see the source code.

Almost the same example for single object and count function:

Please, enable javascript, to see the source code.

As you can see, here you can use usual Linq methods applying to the Query property, as Query property is IQueryabe type.

Updating and deleting an entity

Please, enable javascript, to see the source code.

The end for today

As you can see, I have tried to save the simplicity in working with objects as in Linq2Sql, but have added a multi-context support for it, so you are not limited only for MS SQL…

Also, you don’t need to work with db directly. All tables/objects will be created for you automatically.

If you have any comments/proposals please, write them in comment form. Next time I plan to describe, how to use simple business logic layer with AcroDb and relations between entities.

AcroDB. The beginning

2010-03-20T00:00:00-07:00

The big day

After a year of hard work I begin to share new opesource projects with Internet community. Hope you will love it and use it in your development. Also, if someone will want to help me providing the future of this small creatures, please write me a line to email box.

New .Net-based CMS (the Content Management System) that is FREE

Ok, first of all about my ideas and projects. During last 3 years I was thinking of new stable and reliable CMS system/framework that will be easy to work with for end users like websites owners, same as for developers. Please, don’t think it is one more CMS system, that is real shit… Nope, we have given all love to this fantastic system, and all people who were testing it in internal testing, have already made good reviews about it.

Why have we (my best friend - designer and I) done that?

While we were working at Acropolium Studio since 2003, and our team already have done about 4-5 engines that were CMS-like systems. Some of them were plain, some of them - moduled and flexible, some - quick like a lightning. But all of those systems were developed on PHP. It is good technology and even now it has big popularity, but maybe because of my laziness, soon I have moved to C#. From the very beginning it was incredible language, same as .Net was a perfect technology.

After some time developing and supporting web applications based on .Net framework, I have made a research on existent ASP.Net CMS systems. There were not a lot of them by that time. Also, almost all of them were shareware. That’s why we have begin to think about new, fresh concepts in CMS building. And when the MVC.Net library appeared in RC1, I began to implement internal architecture of new, fresh, opensource cms. There were about four initial architecture implementations that have been gone to recycle bin. The only idea that have survived through last 2 years was the Data Access Layer provider (hope all of you are working with MVC pattern, so it was Model). And part of this layer that is separate thing now, I will describe today. Hope, other articles will appear once/twice per week.

AcroDB

The result you will see today is approximately 99% modified from the 2 years old version. 1% left the idea of the layer simplicity. About 60% of code were rewritten this week, after SubSonic AutoMigrations appeared (many thanks to Rob Conery and Co). Before, it was the work of Microsoft SMO - tons of code and same result as now (the only thing there were relations, but now we support no relation dbs and relations are not needed in simple data provider - one more “hi” to Rob’s team for their NoRM).

The goal

There are plenty tasks in developer life, where he need to make simple access to some data source quickly. For example, it is needed to make a temporary local list of visitors, or some cache in data source. What steps do developer needs to make to support this?:

Select the data source provider, and if needed - install it.
Create simple table structure to store data at later time.
Create ORM (we will work with linq2sql) prototype classes, that will work with this db.
Create Linq DataContext to work with source.
Create simple data layer to access data through LINQ entities.

For harder projects this steps count could grow… Also, if in future developer will need to change data source (for example move from SQLite to MS Sql), he will need to remake steps 2-4, and sometimes even step #5. Also, if we need to work with entity interfaces in project, than we will need to add step #2a to create prototype interfaces and change step #3 to inherit those interfaces by linq classes. If you will use Nolics ORM you will need to write some more same text like prototype classes and interfaces…

So, what was the purpose of the AcroDB development? First of all to remove all non-needed steps and make them as internal work for DBProviderFactory. And as a result developer will got to write several lines to get access to rows in different DB system. Here is the list of tasks we have thought to achieve:

Move abstraction of DB entity from classes to interfaces;
Implement unique interface for add, edit, delete, query operations for any data provider (SQL db, No Relation DB, collection);
Automatic tables creation/modifying (migration) in data provider, based on described entity interface (if it is SQL-oriented);
Automated settings fetching from the loaded assemblies.

I have wrote a lot of bad English text already, so let me put some code now:

Please, enable javascript, to see the source code.

This is all things you need to do. Not very simple on first view, but you will not need to create something else to store data.

So what we’ve got here?

AcroDbEntity Attribute

This attribute applies to interfaces, to mark them for data context factory to be inherited at runtime. The line AcroDataContext.ScanAssemblyForEntities calls a static method to scan assembly for interfaces marked with AcroDbEntity attribute. Than AcroDataContext makes a class that inherits this interface in memory using IL code generation. So, at the end, this class will be the endpoint for linq operations.

DataContextFactory

It is an abstraction for any data context, which could be anything, like relations-based db, non-relation db, files, memory, etc. The only thing if you need to implement own data source context, you need to code 3 classes:

data provider (add,edit,delete,query commands implementation);
data context (link to data source and transactions until commit changes);
data source registrator - the class that will describe your’s custom provider, and context (later we want to remove it and replace by attribute for data context).

About implementing custom data source provider I will try to write an article asap.

DefaultDataContext

This static property of AcroDataContext is determining the default datasource, with which the entities will work.

AcroDataContext.Go

The helper method to create the instance of default data context to work with. Please, note, that data context implements IDisposable to free all used resources quickly.

Working with entities

Inside using block you can see the creation of new object and it’s storage to data source. While using the generic method of data context named Provide, AcroDB internally maps the interface to runtime created class and creates it’s instance.

As you can see at the end of using block, to ensure, that data was commited to data source, you need to call SubmitChanges method like in simple LINQ context. It is not needed for all data source types, but to make system work with different providers, don’t forget to call it.

The end of the day. Source code

Thank you for reading this article. I will appreciate feedbacks and comments. If there were some not clear aspects of code, or text here, please comment about that. The source code of the library could be find at github. Don’t forget to download subsonic and NoRM (if you want to run mongodb provider) libs.

P.S.: Migration

Almost forget about auto migration. Simply call AcroDataContext.PerformMigrations() after loading all entities, to change datasource to latest entity description automatically. Once more thanks to Rob Conery.

Also, there are a lot of internal sugar, that was not described here. But I will work on this, if it will be interesting.

PHP Client for Java-based webservice

2007-08-22T00:00:00-07:00

While developing one commercial product I have met this problem, and for the research in it’s solvation have spent about 2-3 days. Maybe, this article will help some people, that are new to SOAP programming. So, If you will find it useful – I will be happy.

Some time ago I have met a problem to connect the PHP script to Java-written webservice. It was a task, when the server-side part was developed as the collection of different webservices. For their good performance and memory on-fly caching support we have selected Java language and tomcat server as the main tools of development.

On the other hand, a lot of already developed websites were written with the help of PHP language, because it is simple and powerful for small sites. Also, it is running good on the Linux/Unix platforms, which are much cheaper.

So, we have to develop stubs (php classes that are giving access to remote webservice methods).

By that time I have already worked with connections to the remote webservices using .Net Framework. There you need only to fill in the remote WSDL file path and Visual Studio will automatically generate stubs for you. So, there we even have no problems of cross-language remote connection.

After developing all webservices that our company needed we have tested them with the help of small C#-written utility. As I have mentioned earlier – in VS (C#, .Net) there is no problem to create link to any webservice.

The next task was to make PHP client to Java service. After some search in Google, have found that new version of PHP (version 5.0+) supports the SOAP connection as a client and as a server. Also, have found, that for the PHP version that is younger than 5th there was a library called NuSOAP (please, notice that this lib is not working with new versions of PHP).

So, I have made an instance of SoapClient class in PHP and give it a WSDL URL as parameter. Than have tried to make some remote call… Here the problem appears. The failure was with remote call parameters, just because Java webservice framework (JAXb, JAXr) converts simple collection of parameters to the one object with variables that are real parameters. Also, all argument names are renamed to the template argX, where X is a number of argument starting from 0.

Also, there left some problem with passing null value to remote call.

For example, we have remote call to Login method and it receives two parameters – username & password. So, the stub in PHP will be:

Please, enable javascript, to see the source code.

In PHP code the remote call will be like this:

Please, enable javascript, to see the source code.

Do you see how nice it is?

Now, try to imagine, that this is only one remote call, and what will be if there are about 100 such calls? You will need 100 different stubs. Also, please, notice what will be the difference between these stubs. Let’s show one more example of more complicated call (RegisterUser). Let’s imagine that there are such parameters – username, password, email, full name.

So, the stub will be:

Please, enable javascript, to see the source code.

And the code will be:

Please, enable javascript, to see the source code.

Using the reflection technology, we can develop the class that will be as unique stub for all remote calls and it will dynamically create internal class variables for the arguments if needed. Here is the code of it:

Please, enable javascript, to see the source code.

The example of call is here (for login stub):

Please, enable javascript, to see the source code.

That’s all. And the object will be filled with all needed argX.

Now, let’s unify the SoapClient class. All we need is to catch the call of external method with all parameters and dynamically create the stub and generate the remote call. Here is the code sample (please, note that here there is add-on information stored about logged in user session ID for future remote calls without passing this ID to the parameter list by developer):

Please, enable javascript, to see the source code.

Not big part of code, but makes a lot of work for us. Now, let’s see how is it used:

Please, enable javascript, to see the source code.