Nicolais Personal blog

Weeklies 2 - Documentation

2016-02-15T22:58:00.001+01:00

So I spend this weekend writing up some documentation for the flexbillet-api. I have done no research what-so-ever, and this makes for a nice point!

While not-researching, I actually wrote some documentation. My thoughts kept coming back to "there must be a tool to automate some of this", but instead of exploring, or even start writing something up, I just did it. And there it is: https://github.com/nwillems/node-flexbillet-api

So why was the documentation important? 1. It would suck to put something out there, and then have nobody able to use it, 2. The feels - giving something back, and then proudly publishing to NPM. Yes its on NPM, this little gem(ahem..), https://www.npmjs.com/package/flexbillet-api

This is my first published module on NPM and it feels really great. Thinking about it, it seems completely unreasonable, but it feels good.

So the actual documentation, if you look at the commit history I did quite some iterations. Recently I've been introduced to squashing commits, but this time I felt that it would defeat the purpose. It sort-of documents the documentation process. The documentation is split in three parts: Installation, Simple usage and reference docs.

The installation is easy, it is also the shortest part, it also makes a lot of assumptions, you should be somewhat familiar with the terminal and the node ecosystem, or at least be able to translate it.

The "simple usage" is the most straight forward part, also the hardest part for me to write, simply put it should boil down the module to a single piece of code that shows you that it works, something you could copy-paste and have working within 5-minutes. I think it succeeds in that.

Finally the "reference doc" should be a comprehensive list of the methods provided and their expected behaviour. This is the time-consuming part to write, first ensure that the method is tested(and fails), then implement it correctly, then have the test produce output, now add the inputs and outputs to the documentation, factor out common objects(See Generally defined objects), and lastly contemplate wether you should start an effort to structure the returned output differently, or maybe make it more of an exploration for the user...

And this is where I leave you, with a list of next weeks topic - testing.

EDIT: This post should have had a GIANT kudos to the NPM people, https://docs.npmjs.com/ is an awesome site, I never thought it would be so easy to publish something to NPM. Awesome job guys!

Weeklies 1 - Writing a wrapper API

2016-02-07T23:25:00.003+01:00

At the moment I'm trying to get moving on a small project, it's called DigiKreds(Yes, like DigiMon, but instead of monsters it's for local Scouts organisations, in danish "Kreds").

The first thing to note, is that we recently got access to a new system for handling trip-invitations and payment(https://flexbillet.dk/). And I got access to their API description, ya great customer service.

First of all, my language of choice is Javascript, so its going to be a node module. Second, I would like the interaction to be as clean as possible, meaning that you should configure the module once and then use the methods provided. And speaking about methods, the API is sort of REST, which is nice. But then again, this is no advanced system, thank you developers :-).

So I got off to writing some code, which was the goal. Wee, lets hope I can push myself to do something similar during next week, and suddenly "a wild language binding appeared". For now the code is just a bunch of failing tests and no actual outside world calls, so its nowhere near being public, but in a short few iterations its going to be.

While looking into this, I started my internal monologue about RPC vs REST. There are certainly cases where RPC makes more sense, or at least its easier to implement. Just thinking about the classic calculator example, do you POST to the add-resource, or how? Please feel free to leave a comment on this topic, I'm eager to learn.

Weekly 2: documentation

EDIT: Remove sign-of and add link to following article

Notes on installing Archlinux

2014-04-29T13:36:00.000+02:00

This blog post is a collection of notes, for installing a modern Archlinux system.

Networking

I had a minor problem with the networking. It simply required restarting the dhcpd service for the given interface:

systemctl restart dhcpd@enp2s0.service

Partitioning

The system has a 250G drive(Shows as 232G), and is GPT with the following scheme:

Size	Type	Mount point
200M	EFI System	...?
50G	Linux	/
80G	Linux	/var
102G	Linux	/home

Suggestions found at: https://wiki.archlinux.org/index.php/partitioning

Base package installation

pacstrap -i /mnt base base-devel

Needs to remove reiserfsprogs(maybe also xfsprogs,jfsutils) after installation.

Maybe stupid question but why libldap, libsasl and openssl ? openssl is pretty basic yes, but why in the base system?

Generating /etc/fstab

Should /boot come before /var and /home or does is at all matter to spend time mounting boot when using UEFI? Of course it should be mounted when doing sysupgrades, but otherwise?

More updates to come!

The toughest job aka creating server/client certificates

2013-12-29T19:22:00.000+01:00

The reason why is pretty unimportant, and frankly I don't know what I did to have it working. But suddenly it did.

The task is to have an apache server use SSL with client-certificates for authorization. No problem, you just create your own CA, issue a server certificate, issue a client certificate and convert to PKCS12-format for consumption in your favorite browser, right? Not so fast!

So I made a gist walking through the steps: https://gist.github.com/nwillems/8172932

My starting point was this: https://code.google.com/p/migrid/wiki/ServerInstallNotes#Certificates_-_Manually

And intermediary process has been a lot of googleing seeing all sorts of procedures to accomplish something similar, and then not yet. Some resources has been:

http://lakm.us/logit/2013/01/https-server-client-certificate-pair-1-generate-openssl/ final resource
http://gagravarr.org/writing/openssl-certs/ca.shtml
http://gagravarr.org/writing/openssl-certs/general.shtml#cert-convert - didn't really work, but pointed in the right direction.

That pretty much sums it up - have fun authenticating with your new certificates.

Have a happy new years :-)

/Willems

An app idea, pre configured cooking timer

2013-05-03T00:58:00.002+02:00

Today I was making coffee and I had this idea about an app.

Because making coffee is a "long" process, you most often don't want to stand and wait for it to finish. So what if you could just once record the time it takes, and then next time you make coffee you just press the "I'm making coffee"-button, and your phone will beep when coffee is ready.

For my coffee "machine"[0] it takes about 11 minutes, another thing is the coffee machine at work and a third thing could be boiling an egg.

It could replace your good old cooking timer[1], and save you the trouble of turning it too little to make a loud and clear ring.

Letting the ideas flow, I thought you could make it such that a picture of what you are "timing" would be a sufficient lookup. Such that the flow would be something like, take a picture of coffee maker, stop timer when coffee is done, next time you make coffee you just snap a picture of the coffee maker and voila it starts the timer. I'm no image-recognition guy, so I have no idea if this is feasible on a regular smartphone, but sure seems like a nicer way, instead of having to either type or scroll through a list.

The idea is of course up for grabs, if anyone feel for making such a thing. I would be proud if you mentioned this post.

/Willems

[0]=One like this: https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0_ZdeXoyWhyWIWEJOEBTVuVSz8mSOnUJGLIFRmhl_TOFm_ijUjvoklsQ2C-_AT7wvuRD77of9l5B7gyWRkBd_8eX8b4HswDCgDxjiWHiU9qyWAyMFk9FydZJ0hYpKf_cqUq257UaYhKs3/s1600/NewMokaExpressLarge.jpg
[1]= http://img.alibaba.com/img/pb/080/061/279/279061080_599.jpg

Managing a project on Github.

2013-04-23T14:30:00.002+02:00

A short post on managing code via Github. They probably have a thousand articles and this has been written a million times before, but I'll write it to remind my self, and as a nifty reference for my group in "Company project"(I guess there will be some sort of post about that).

The post is a work in progress, so feel free to add tips & tricks, corrections and whatnot in the comments, or drop me an email or a message on G+.

The Project

The current project, and the one I'll refer in this post, is a LaTeX project for the report, that we are supposed to write. It contains 5 sections at the moment and a single appendix(there are more to come), its a total of 9 pages and growing.

The overhead of the below described process is probably growing over our heads before we know - but it makes things more manageable and adds structure to the process - which can probably help us.

The Meats

When developing with Github you have a bunch of useful features, what is mainly explored in this post is the pull-request(shortened to PR) feature.

Branches

The master branch is supposed to be a stable version of whatever you are doing, in our case it is supposed to be a compile-able report that we can ship off to our mentor at any point in time.

To achieve that goal, branches come in very handy. When you write something new, or change an existing part, a branch helps you keep your changes in one place. Thereby not polluting the master-branch with your unfinished or non compile-able work.

The part about keeping master clean, could very well be achieved with tags - this would require close coordination between team members about when what happens, when to make a tag, how many commits and so on. With the branches approach, you can keep your changes isolated and do not have to worry about what the rest of the world is doing to the code base.

In conclusion branches helps you keep sets of changes isolated from the rest of your project, without the coordination efforts required when working in the same branch.

Commands: create a new branch git branch <branch-name>, switch to a branch: git checkout <branch-name>

Merging

When you've create a set of changes, and you are ready for the world to know that you have something, you create a PR. Actually, if you go to the Github project page right after pushing your latest changes, there should be a button allowing you to create a pull request.

Put in a short name and a description of what has been done, and the world is ready to check it out. This process doesn't seem more fit than sending an email to the team, telling them that the branch is now ready for review/merging.

But of course its smarter, why else would I write a post about it. Your changes are nicely lined up, in a separate tab of the PR's view in Github, that's a first. Secondly, when you hover your mouse over a changed line, a little speech balloon pops up to the left of the change, if you click this you can comment on that specific change. This allows the team to review your changes, and possibly rule out spelling mistakes, faulty environments, forgetfulness and the likes.

And when the team says go, you can press the green merge button and your changes are merged into master(if you choose master as your target). Also the green button won't appear if your changes are in conflict with other changes in the project(More about this later).

In conclusion pull requests are a nice way of making sure that the team knows what is changed, and that every part of your project is keeping up to some agreed upon standard.

Commands: Only Github UI action relevant.

Conclusion

For me this is a very natural way of working with a project, that being a report in LaTeX or a codebase of several thousand lines. Keeping track of changes, makes the team aware of whats going on, thereby not leaving anyone behind and unknowing of the projects current status.

The Nifties

When you experience a yellow button, and your pull request cannot be merged, it is often caused by other PR's being merged into master, and therefore conflicting with yours. This can be solved by rebasing your changes on top of master(For a reference, see http://git-scm.com/book/en/Git-Branching-Rebasing).

git checkout <your-branch>;

git rebase master

Thats all for now, if you have anymore nifty tricks please add them in the comments.

Pacman short usage

2012-08-09T11:27:00.002+02:00

Finding pkgs:
pacman -Ss | less

Removing pkg:
pacman -Rnsu

Install
pacman -S

Do YOU need more short commands? Put them in the comments and I'll follow up.

PS: I know this is written from the blogger interface - my own is broken :(

Google blog editor - now finished

2012-07-10T14:51:00.003+02:00

So you have been waiting? I guess not. But I'm happy to announce that, my own Google blogger frontend is now finished. It is created completely without any server-side code, and relies completely on client-side scripts. It is implemented with the Google API, jQuery, wymeditor, soyutils and bootstrap. If you just wanna go, have a look at: http://nwillems.dk/blogging, you can log-in with your Google Account

Actually, why this post is so important to write, is because it is the first one written from the site. Yeah - that's about it.

All I can tell you is that the above mentioned tools, really do rock! So check them out. And if you like, check out the code for blogging at: https://github.com/nwillems/gBlogging-interface

Have a great summer

LaTeX lists

2012-04-17T11:36:00.001+02:00

After hitting it off with LaTeX for a while now, I have gathered a collection of normal goto places(the ones that show up first in my google searches). And I would like to share them with you, therefore the following list.
But first a word on the sites, they are not complete references, nor static structures to support your work, rather they are dynamic places where you can search information on specific problems and find like-minded people.

I present to you, The list:

http://texblog.org/2008/10/16/lists-enumerate-itemize-description-and-how-to-change-them/
http://en.wikibooks.org/wiki/LaTeX
http://stackoverflow.com/questions/tagged/latex
http://imf.au.dk/system/latex/bog/ (Danish)
Comprehensive Tex Archive Network http://www.ctan.org/
http://www.howtotex.com/
http://www.artofproblemsolving.com/Wiki/index.php/LaTeX:Symbols
Last but not least - your memory.

If you have some nice resources yourself - please add them in the comments and I'll add them to the post in a "Externals" list.
That's about it. Have a great day.

PS: This is another ugly post, I'm sorry, the blog editor is still in the making.

Changelog:
0 - initial
1 - Added comment on adding links in comments
2 - Added ctan to list
3 - Added "how to tex" to list
4 - (28/08 2014) Added AoPS math symbols

The way to code, these days

2012-02-17T13:38:00.001+01:00

Hi

Recently I've been developing the blogger editor thingy. And I have discovered some nice tools to help me out.
Since I decided to make it completely client-side, everything(dynamic) needs to be done using JS.

Therefore I went out on the interwebs and found:

In the following posts, I will try to highlight their use in the project. So this is just a teaser.

Bonus library: http://twitter.github.com/bootstrap/ - I might redo the project with this library, so have patience.

Happy coding

Blogger WYSIWYG editor - A bit of complaints

2011-10-11T18:12:00.002+02:00

As some might have noticed, I blogged a little lately. And found some problems with the editor for blogger. Maybe I should go write a wrapper for the editor.

To the actual problem.

How on earth do you expect me to respect an editor for HTML that instead of treating a double line break as a new paragraph just makes another line break. HTML has fine markup to make things just a bit semantic, why ruin them?
The editor start by inserting a div-element, fine enough, but no paragraph element. And where did headings go? I can chose between 5 different text sizes, and yes you guessed correct, it doesn't use the heading-tags.
Another thing I noticed, while writing this post, when putting angle-brackets into the editor, while in compose-view, switching to HTML-view, and back again, it converted my angle-brackets to real html code instead of maintining the brackets as < and > that made me sad, said another way, the composer, doesn't convert characters to HTML-entities, if needed.

My solution:

As a good person, instead of just rambling about how things are wrong. I hereby suggest some improvements, that I believe will be enough.

When pressing the enter button, a line break(<br />) should be inserted. Then a check should occur, checking to see if the text before the cursor is a line break. If it is a line break, then replace, the old and the new line break with paragraph-end followed by paragraph-start. This would cause the first occurence of a double line break to break HTML-validity. This is simply solved by assuming paragraphs, and the clean post would looks something like the following:

<div ...="">
<p>
|Cursor|
</p>
</div>

Furthermore, it should convert characters that needs it to their respective HTML entities, the only place, where this would cause problems is in between pre-tags, but that is a rare case, and probably could be fixed by having a boolean value expressing whether you are inside pre tags or not.

Developing software - thoughts

2011-10-11T16:48:00.000+02:00

Go on shoot me, for writing something you already know.
But this just came to me. As a computer science student I often think about improving the world, with software and computers of course! The theme has been spinning around my head lately, and I'm really empty for ideas. So thinking to myself, perfect time to pick up some of these old projects that never went flying. But then it struck me, as lightning from a clear blue sky.

My motivation for making software, has always been to make something that I would like better than existing products. Like as in working with.
Example: a database system, that has SQL support(over HTTP-probably) and maybe RESTfull also, but is more like a document oriented database, not dependent on a fields existence. And instead of returning traditional row oriented data, maybe XML, or even better JSON data, since this document store should be object oriented.

Alright, this might seem complicated, it is - if you were to develop the whole thing yourself. A good datastore, not dependent on fields, and containing objects(probably related to different parent records), not easy(at least from my point of view). A server wrapping this datastore library, furthermore containing SQL optimizer and parser, REST supporting stuff, JSON/XML serializer, and some sort of effective request handling. Daunting task, and this is only my 5 minute overview of the example.

Back from the example - what struck me was, that developing software can be more simple than that. SHOULD be more simple than that. The philosophy of doing everything yourself in the first go, is absurd, probably unreachable! The solution is: Use existing libraries to get the base functionality, and then use your favorite language to wire things together. When more speed is required, find bottlenecks and write the library/program yourself, integrate it tightly with your existing wire-code, and when it works, generalize so you can provide the functionality to your other projects.

Then Repeat the bottleneck-finding-rewrite-code-process until software is completely written by yourself, release to manufacturing, sell product and support, profit!, sell company, PROFIT!
Well allright, you might not be writing for money, then just give your code and libraries to the community, such that I can read your code and make it myself afterwards :-)

PS.: Coming up, complaint about Google bloggers wysiwyg editor.

Restoring your initramfs

2011-10-09T13:57:00.000+02:00

So a couple of days ago I ran a system upgrade on my arch-box(also my primary laptop), it told me that I needed to upgrade to 3.0 kernel. Cool I thought and went through. And after fixing some driver problems, I accidentally crashed my initramfs - no booting for me. I guess you know the problem, since your here, so read on and get the answer.

I must say, this article wouldn't have been, if it weren't for the Archlinux wiki and the IRC channel. Thanks to falconindy, Hattara-pilvi, brain0. Andd thanks to d_garbage for finding them in his logs.

How you doing?

Once again I had some problems with my wireless driver, so I followed normal procedure. Copy old driver .so-file to new kernel folder. Everything was all dandy, so I rebooted and BAM! no wireless networks for me.

After fixing things and such, i recreated my initramfs, with mkinitcpio. Rebooted and wham bam thank you mam, "You have junk in your initramfs" - sorry to those of you who misses the original text, but I forgot to write it down.

So how am I going to rescue my computer? Well the obvious answer is recreate your initramfs. But how are you going to do this, when it doesn't even get to some console/tty.

Here's a short list with each step, explanations later:

Make a bootable device with arch
Mount the necessary partitions
Run mkinitcpio

Bootable device - USB

# Naive way
dd if=image.iso of=\\?\Device\Name\Windows\Way\ugh
# More sophisticated, but slow. And --progress is specific for "dd for windows"
dd if=iamge.iso of=\\?\Device\Name\Windows\Way\ugh --progress
# Faster, more sophisticated, same comment as above
dd if=iamge.iso of=\\?\Device\Name\Windows\Way\ugh bs=4M --progress

Sadly my only available machine was a windows-box(But I trained it well). Reading the arch-wiki[1] I found out about "dd for windows"[2]. Made my bootable USB, but when booting from it, it said it didn't have isolinux.bin.

I was just about writing my query to google, booted it again to get the correct error message, and wham it started correctly. Morale, try more than once when you boot from USB's.

When you see the Arch menu - just choose install. This will get you to a console where you can do more than what is needed.

Mounting partitions

I'm not saying my setup is complicated, it just has some quirks :-)

The necessary partitions in this case is /(root) /dev /sys and /proc and in my case boot is on another device/partition, so that also needed some mounting. Also remeber that /dev, /sys and /proc should be bind mounts[3]

Remember that you don't need no sudo, since you already are logged in as root in the arch install.

mount /dev/sdb2 /mnt
mount --bind /dev /mnt/dev
mount --bind /proc /mnt/proc #TBH I'm not sure i used this
mount --bind /sys /mnt/sys
mount /dev/sda1 /mnt/boot

(Re)Running mkinitcpio

You have two options when it comes to this, either chroot, or don't. I chose to chroot, it seemed easier.

chroot /mnt

When running mkinitcpio - remember to specify a preset

mkinitcpio -p linux
# Below commands are untested, and used when not chrooting(as far as I understand)
mkinitcpio -b /mnt -p linux

If all succeeds, you should have a running system by now. Try rebooting(remember to remove the CD/USB - I know stupid warning, but it helps), if all works, celebrate. If it didn't work, please go to the nice and very friendly Archlinux IRC channel on freenode.

Footnotes

= https://wiki.archlinux.org/index.php/Install_from_a_USB_flash_drive
= http://www.chrysocome.net/dd (On the wiki page its under the cygwin section)
= I'm sorry I can't explain what it(bind mount) means, but feel free to leave a comment.

Percent internet users affect the fertility rate

2011-02-28T19:43:00.000+01:00

At the moment I'm playing with Google Public Data Explorer - so I tried making a graph for Fertility rate, Life expectancy at birth and Internet users. And it shows that the percent of internet users has an effect on fertility rate. See for your self below:

Step one of my website

2011-01-11T23:46:00.005+01:00

This post focuses on the initial step I took to have my new website up and running. The domain part.

What is assumed here is a little understanding of Linux and a some drive towards getting things done.

First I had to find out how my provider(GratisDNS) would like the data - luckily for me it turned out that I only have to provide very few parameters in a URL String and then it would automigaclly read the IP from where I sent it. All set with the information, now on to creating a little script.

Knowing that I had to call a "simple" URL, and by reading on the forums(very supportive forum over at GratisDNS), I found that using curl is by far the easiest. So my script looks like this:

curl -s "https://ssl.gratisdns.dk/ddns.phtml?..params.."

curl is a nice little program, that can request a server by HTTP[1]. The "-s" option makes it run silent and then theres the URL, remember the double quotes around the URL. That all there is to making the script - remeber to save it in a location where the user you are planning to run it as, has execute-rights.

To the crontab - this is the abstract part. First we need to gain an understanding of the software. I do this by asking: What does it do? it schedules another piece of software to run. What software? Anything that you can run on a command line. So that should give a basic understanding.

How do we do it then? we utilize the crontab command it has three options L(ist), E(dit) and R(emove) so you wan't to edit the crontab - cool "crontab -e" opens up your editor and you are ready to rumble.

I have taken the liberty to borrow some of this nice graphic from http://www.crontabrocks.org/:

______________________ 1. Minute - Minutes after the hour (0-59)
|  ____________________ 2. Hour - 24-hour format (0-23).
| |  __________________ 3. Day - Day of the month (1-31)
| | |  ________________ 4. Month - Month of the year (1-12)
| | | |  ______________ 5. Weekday - Day of the week. (0-6, where 0 indicates Sunday)
| | | | |
* * * * * /some/script/or/command

To explain it, the first part(the star-ish one) is the scheduling part, the other part is your command - this could be anything, in our case a shell script.

The interesting part is the first one, the scheduling. First of all there is a special value, identified as an asterisk(*), this tells cron to do it every-something. And as stated by the descriptive labels, each of the starts represents a time unit.

The minute(s) within the hour
The hour(s) within the day - 24 hour clock, zero based.
The day(s) within the month
The month(s) within the year
The day(s) within the week, better known as weekday.

Specifying a value for any of them would result in the command being executed every time there's a match.

For example one could write:

0 0 1 * * /path/to/script --parameter

This would be run on the first day of every month the zeroth minute of the zeroth hour.

So by playing around with cron and getting to know the format, I decided that my script should run hourly, or more precise, it should run on the 30'th minute of every hour. So the cron expression looks like this:

30 * * * * /home/nwillems/gratisdns_ddns.sh

I belive that hourly is fine, and then setting it to that minute seems like a bit odd, instead of just setting it to the zeroth minute. I was wondering though, if I should make it run half hourly, then the cron expression would look like this:
0,30 * * * * /home/nwillems/gratisdns_ddns.sh

What with the coma, you ask? That is cron's way of defining a list of time-unit. So for example "0 0,12 * * *" would mean something running each half a day(midnight and noon).

That really was all that needed to be done - and off you go with your own "custom" made Dynamic DNS.

If anyone cares out there - I found a special little article, I found it quite amusing reading: http://gizmodo.com/5716226/so-you-wanted-to-be-a-computer-programer

/* EDIT _Fixed pre environment around needed places */

[1]=If your interested you should go read the man page - http://curl.haxx.se/docs/manpage.html

My personal website

2010-12-29T06:49:00.000+01:00

Last couple of days, I have been working on my website(http://nwillems.dk). As a little Christmas project, I decided to fix my website. So here is the steps I took to make it.

For a while, I have been running my "old" computer as a server. Its running Ubuntu Server Edition, and doing good. I have had a DynDNS hooked up to my router so I could access my home from anywhere.

In this post I will just describe what I have done, not any code. That I will show off in the next couple of posts.

But then I found out that my DNS provider(GratisDNS) is offering a service similar. Eager to try it, I worked it out with a little cron job and all, very nice. All it took was the cron job to run half hourly and then pointing to a script with a simple curl command.

Next up, making something useful. At work I have been playing around with jQuery and Blueprint, so these tools was the obvious choice. The former site on the server just displayed a dumb map of my latitude, and for a long time I have wanted to do something cooler. Also I would like to incorporate this blog into the site.

So the requirements suddenly grew, to this:

Show a little personal information
Show latitude - in some cool way
Make blog available
Show off - probably just links to projects i work on/in.

Layout was done quick and easy with blueprint, three columns with some fancy headings. Adding the content is a bit harder, since I'm no writer, its still on the todo.

I had been struggling with the latitude public location badge JSON, so I started of with my blog. I found a nifty little article[1] about making a feed widget with jQuery, and found something I have to write more about, YQL(Yahoo! Query Language). This smart feature provided by Yahoo! is so nice, and I almost instantly had my blog feed on the site.

Almost blown away by awesomeness, I immediately attacked on the latitude public location badge. Thinking, since jQuery tells me that JSON has no problem with cross-site security, i can just fetch it as I do with my browser through jQuery. But NO!, and I still don't know why, after ½ a days work on the irritating latitude thingy. I decided to use my new best friend YQL, and boom shagalang, it was there. Then I extracted the latitude and longitude from the response and added to the map.

And voila - I now have a decent website.

Merry Christmas and a Happy New year.

[1] = http://tutorialzine.com/2010/02/feed-widget-jquery-css-yql/

Installing debian from USB-stick

2010-10-21T21:40:00.000+02:00

So to "steal" the title of another very helpful blog post, thats very mature, huh?

No actually I'm in the middle of doing it, so I thought if I link to the post it will get better ratings on google. So here it is: http://h0bbel.p0ggel.org/installing-debian-from-a-usb-stick

Just one note about it. When using dd for windows - at least on my Windows 7 laptop - don't have your explorer(the file browser thingy) open, it "locks" the usb device, So before starting close Explorer windows and happy copying.

UnitTesting - Mocking for MyBatis

2010-09-24T23:15:00.001+02:00

Since I've recently dived into developing with MyBatis, I also started doing unit-test for my library.

I was ignored by two facts

The test ran really slow. Like 1 sec pr. test
My data changed during the day - testing in a shared test-environment

So I sat down and read some of the MyBatis source code for the classes I use. And found out I just needed to create two Mock classes and I would be good. So I did, following is the code, some of the methods for implementation was removed since they only throw a "UnsupportedOperationException":

MockSqlSession

package com.zylinc.zydata.mybatis.mocking;

import org.apache.ibatis.session.SqlSession;

/**
 *
 * @author Nicolai Willems
 */
class MockSqlSession implements SqlSession {

    public void close() {
        // Do nothing
        // Maybee destruct object
        // Naah who cares??
    }

    public  T getMapper(Class type) {
        String mockClass, interfaceName, packageName, mockClassName;
        Class typeToInstantiate;
        T result = null;

        interfaceName = type.getName();
        int lastIndex = interfaceName.lastIndexOf('.')+1; // Plus one to get of the '.' again
        packageName = interfaceName.substring(0, lastIndex);
        mockClassName = interfaceName.substring(lastIndex);

        if(mockClassName.startsWith("I")){
            mockClassName = mockClassName.substring(1);
        }

        mockClassName = "Mock"+mockClassName;

        mockClass = packageName+mockClassName;
        try{
            typeToInstantiate = Class.forName(mockClass);
            result = (T)typeToInstantiate.newInstance();
        }catch(ClassNotFoundException e){
            System.out.println("Class not found: "+e.getMessage());
            System.out.println("Cause was: "+e.getCause().toString());
            e.printStackTrace(System.out);
        }catch(InstantiationException e){
            System.out.println("Class instantiation failed: "+e.getMessage());
            System.out.println("Cause was: "+e.getCause().toString());
            e.printStackTrace(System.out);
        }catch(IllegalAccessException e){
            System.out.println("Illegal Access exception: "+e.getMessage());
            System.out.println("Cause was: "+e.getCause().toString());
            e.printStackTrace(System.out);
        }
        return result;
    }
}

MockSqlSessionFactory

For readability I have removed all of the other openSession-methods, since they just do "return this.openSession();"

package com.zylinc.zydata.mybatis.mocking;

import org.apache.ibatis.session.Configuration;
import org.apache.ibatis.session.SqlSession;
import org.apache.ibatis.session.SqlSessionFactory;

/**
 *
 * @author Nicolai Willems
 */
public class MockSqlSessionFactory implements SqlSessionFactory {

    private Configuration configuration;

    public SqlSession openSession() {
        return new MockSqlSession();
    }

    public Configuration getConfiguration() {
        return this.configuration;
        //throw new UnsupportedOperationException("Not supported yet.");
    }

}

If you have any questions felle free to contact me, either in the comments field or on the MyBatis user list.

MyBatis - how to do it simple

2010-09-22T23:57:00.001+02:00

As I wrote yesterday I'm working with software development. Currently I'm working on a common database layer for our portals.

I found that many of the current frameworks for ORM(Object-Relational-Mapping) like Hibernate and openJPA doesn't support "custom" SQL. That basically means that your database should 100% reflect your application model or you should have views instead of actual tables, this would be livable for me since I don't need to do any updates on the data I'm retrieving, but that might be an issue later on, and as you probably guessed I don't have any views available, I have Stored Procedures. And that where the custom SQL comes in, none of them supports this feature and that left our current application a coding mess of SQL commands in strings and parameter binding on SP's with JDBC directly in code, not abstracted and generalized for use by other apps/portals.

So is this just a whiner post or what? Absolutely not. I'm going to show you how you simply can make use of your current SP's in your Java code. I recommend everyone looking for this kind of functionality to take a look at MyBatis and read their user guide[1], it contains tons of information you never thought you would use.

Configuration
First you need to configure MyBatis to access your database. A simple XML Configuration file is needed, and if you want some flexibility you use a *.properties file togheter with the XML file.

<!-- DOCTYPE LEFT OUT -->
<configuration>
  <environments default="yourcrazyenvironmentname">
    <environment id="yourcrazyenvironmentname">
      <transactionManager type="JDBC" />
      <dataSource type="POOLED">
        <property name="driver" value="${driver}" />
        <property name="url" value="${url}" />
        <property name="username" value="${username}" />
        <property name="password" value="${password}" />
      </dataSource>
    </environment>
  </environments>
  <mappers>
    <mapper resource="path/to/your/insane/file/for/mapper.xml" />
  </mappers>
</configuration>

The above configuration makes room for the earlier mentioned properties file, you can however hardcode the values in your myBatisConfiguration, just replace the values encapsulated by ${...} and their encapsulation with your own values.

Mapping
The you create your mapper file. This is one of the simplest yet most challenging tasks when starting out MyBatis. Sadly I'm no creative soul, so I won't start making examples here, instead I will refer to the excellent section in the user guide about this part of the process.

Mappings is basically a piece of XML code where the body-content of the tag is a piece of SQL code. This is interesting from the ORM perspective but from a coding perspective the next part is much more fun

The interface
This is where you make your application go typesafe, forget about the other mentioned method in the book, this is the shit. For me an interface might look like:

package dk.nwillems.data.mappings

import ...;

public interface ExampleMapper{
    public Collection getCrazyDataType();
    public Collection getCrazyDataTypeEasyFilter(String param);
}

This looks straight forward. but if you would like a function called

getCrazyataTypeAdvancedFilter(String param1, String param2);

That wouldn't work. why you ask? This sounds like the holy grail of ORM, sadly its not. To make the above piece of code work, assuming that you have multiple parameters for your SP or query, you should do the following:

getCrazyataTypeAdvancedFilter(
@Param("param1NameInMapper")String param1,
@Param("param2NameInMapper")String param2);

And now you should be good to go with the rest of the user guide, I spent a long time finding this "issue" about the annotations. Now I've spelled it out, I hope someone out there can use this know-how for some constructive work.

In future blog posts I will check out some of the other features and probably write a more in-depth post about MyBatis and how-to get started. This post was just to get my confusion out and put some knowledge in the world.

Happy coding - now with MyBatis
Nicolai

[1] = MyBatis user guide http://mybatis.googlecode.com/svn/trunk/doc/en/MyBatis-3-User-Guide.pdf

Its time or what?

2010-09-22T00:06:00.000+02:00

So I started working for real. I got offered a full-time job where I was employed as a student-worker. So I've gone full-time working and left the studies for a year. So currently I'm working for Zylinc A/S, danish company in IP-Telephony making switch-boards.

So today while I was working, there was some problems with time formatting in my project. The project is a common data access layer for a bunch of Java portals. And since my main focus is statistics I work with dates and time. All date's and times in this post is in 24hr and date/time format is as following YYYY-MM-DD HH-mm-ss[1]

First things first
I found out that MySQL handles date/time strangely, compared to the SQL standard[2]. According to the standard TIMESTAMP is a type containing date and time information in a format like "2010-09-21 22:21:20". This seems obvious, furthermore two types are definied, DATE which consists and contains only the date part(2010-09-21) of a TIMESTAMP value, and the second being TIME containing only the time component(22:21:20).

All this is fine and looks good. But what about the real world? Thats where MySQL comes in. Imagine you have a stored procedure, let's call it foo it takes two parameters(eg two sets of date and time) and returns the recorded statistics in the interval. So based on our knowledge from before we write a procedure like this:

Delimiter $$
DROP PROCEDURE IF EXISTS `foo`$$

CREATE PROCEDURE `foo`(
    param1 TIMESTAMP,
    param2 TIMESTAMP)
BEGIN
    SELECT BeginTime, EndTime, ... FROM Statistics
    WHERE BeginTime >= param1 AND BegindTime <= param2;
END PROCEDURE$$

This seems like the correct way to do it, but not surprisingly it isn't. See this discussion: http://stackoverflow.com/questions/409286/datetime-vs-timestamp. So this means my procedure definitions should look like this instead:

Delimiter $$
DROP PROCEDURE IF EXISTS `foo`$$

CREATE PROCEDURE `foo`(
    param1 DATETIME,
    param2 DATETIME)
BEGIN
    SELECT BeginTime, EndTime, ... FROM Statistics
    WHERE BeginTime >= param1 AND BegindTime <= param2;
END PROCEDURE$$

All right, so this works for MySQL, does it work for MSSQL to? So it seems(I haven't studied this further, since it isn't something I'm going to use - but soon as I need it, I will write about it).

Enough about types, what does a DATETIME value accept? I've found several ways of interacting with it
For example you can pass in this value:
"2010-09-21" - goes fine, a warning should be raised
"2010-09-21 22:21:20" - expected, all goes well
Theese above values seem logic, but then comes:
NOW()-0(=20100921222120.000000) - Strangest value I ever seen.

What does this mean? Can I use a value like 1285107680(Epoch time for the example date) or would that mean: 1285-10-76 80:00:00 or 0000-12-85 10:76:80 or how? I'm seriously confused about this. Not that I found a real solution to this problem, I'm now just aware of this behavior. As I wish you now are and will take this into consideration when developing with MySQL.

Second comes java
First it is important to know what the Epoch is.What I'm talking about is the UNIX Epoch, better known as seconds since 1970-01-01 00:00:00 UTC. Everybody's talking about it, it is the universal time representation for all sorts of things, even windows developers uses this format. But in the Java world, seconds is not good enough, understandable somehow.

Drawing on the good old examples milliseconds can be quite important doing stock trading but come on, at least make it compatible with the rest of the world, an approach where integer values are treated as UNIX Time and the decimal part(if using a float value) as milliseconds, and if you like, you could have nanoseconds(hows that for your stock trading needs?). I spent half a day figuring this out, looking at output from stupid tests being confused. Documentation should state this more clearly[3].

So thats almost everything from an exciting day at work - unless some extremely interesting stuff comes by, I plan on writing a little jump start tutorial for MyBatis[4] tomorrow, I found some interesting things they didn't put in the manual. Btw: I also did a contribution to Stripes web framework[5], I hope that when getting a little deeper into MyBatis I can make some contributions there to.

Have a great evening - and happy coding out there.

[1] = See this for reference http://download.oracle.com/javase/1.4.2/docs/api/java/text/SimpleDateFormat.html
[2] = ANSI SQL http://en.wikipedia.org/wiki/SQL#Date_and_time
[3] = So now I've done it, hope google catches this.
[4] = http://mybatis.org/
[5] = http://www.stripesframework.org

Adventure with LFS

2010-04-14T14:46:00.004+02:00

In this post I will explain what I'm doing this time in my vacation. A rather interesting project I think.

So LFS, what that? It's Linux From Scratch, building your own Linux, all from the bottom.

It has been my dream for ages to do this, and right now, I'm doing it. At the moment I'm compiling gcc second pass.

It really feels good, even though it sounds intimidating. A little History:

First I booted up a virtual machine and attached the Arch live ISO, but it didn't have GCC, and after hours of downloading packages I died a little. Then I took my good old laptop(IBM Z60m) and installed a fresh Arch Linux, with everything needed to get online, and everything needed for the LFS build(The host system requirements).

And off I went, until I encountered my first doubt, and thought OH NOES! and I was thinking a hell lot, until I remembered IRC, went online joined the LFS channel on freenode, saw that LFS has its own IRC-Network, connected and asked my question. These people are more helpfull than the ubuntu channel on freenode. So don't worry, theres a lot of help out there.

You can check it out at http://www.linuxfromscratch.org/. I'm planning on setting up me webpage, which have been bad and not really existing for a long time. And then publish a source and a patch list, which I will try to keep up to date. Besides that I'm thinking of a build script, that automates the process. But first I will compile and make my own system, then we'll see how the build script looks.

If you have already started compiling, then theres only one thing to say: Happy compiling fellow LFS'er.

/Nicolai

My new favorite server

2010-03-06T10:11:00.005+01:00

While developing my own application server I've discovered several other wonderful examples of this kind of applications.

Lets not wait anymore, my favorite is so far Apache Tomcat. It has a nice modular architecture that provides you with the ability to develop new interfaces, it already has AJP and HTTP implemented, but you could just implement what ever you wanted, thats nice.

The configuration is straightforward and the community is good(I've tried the IRC, a bit slow, but really good).

I won't go into examples, because I have not yet developed one. But be assured that this is a good product, can't wait till the new version comes out(v7)

Next up, is another geeky post, about my adventure with LFS.

Article: Four phases...

2010-02-01T21:02:00.003+01:00

Besides researching for OAS, I'm writing a short article about software development. It's called "The four phases of software development" - a short outline is given - then You can probably research the rest yourself or wait for the paper/article to come out.

Four phases:

Research
Spaghetti code
Agility
Refactor

I will not explain details, but I think I will write a short summary of each of theese chapters as they become done - since each chapter should begin with this short summary.

I believe that these words has been written before, but they have been written in retrospect, where as I'm just starting as a developer for a danish company, starting out first year project and researching/developing OAS. I'm in the eye of the storm, which I hope will give a different perspective on these matters.

Short post, this late evening. I have some design issues I'm working on visualizing and writing in words for OAS. And since I found a different application server named Open Application Server - I guess I need a new name for the final product.

Developing an application server/foundation - Follow up

2010-02-01T13:00:00.000+01:00

So I see that I forgot too mention my readings in the last post about OAS. You shouldn't bee left out on that.

So my bible at the moment, is The Object-Oriented Database System Manifesto. This is wonderful reading, it tells you exactly what is needed in an object oriented database management system. It can be found free of charge right here.

Another thing I'm reading at the moment with great pleasure, is the documentation of JDO. It is a nice knowledge base, build with "simple" documentation marks in code. See the javadocs here: http://db.apache.org/jdo/api23/apidocs/index.html

Also this documentation helped me understand the need for docs in projects - since nobody would have been able to understand anything if they haven't documented well.

There probably will be some edits, but for now thats it.

Time - Exam and development

2010-01-26T12:14:00.005+01:00

Last Friday(Jan 22) my exam in Object Oriented Programming and Design(OOPD) was released. Interesting assignment, due on the coming Friday(Jan 29). We are supposed to create a maze game, where you(the user) or the computer(based on an algorithm) direct an Explorer through the maze to a goal.

Why is this interesting? I tell you, as a developer[1] I see challenges instead of problems - challenges to be solved. And this was an easy task, because you had a recipe. But when you don't have one, for example with OAS thats a real challenge. So thats why I alongside with designing and developing OAS is writing a paper on the software development cycle. It's called "The four phases of software development" and is a personal guide on how to run development - but as with the blog, I'm releasing it to the public, at the moment I find Lulu[2] the best, since I've found a lot of the information I'm searching in general on Lulu.

Also I'm writing a follow up post on the original OAS post - with some litterature I'm reading.

As a side note I'm looking for a footnote "plugin" for blogger(be it browser-, offline- or web-based - fx an editor(on/off line) any good suggestions?

[1]= I prefer developer over programmer or software engineer - see http://www.youtube.com/watch?v=8zEQhhaJsU4

[2]= Lulu.com