Java, the Programming, and Everything

My encounter with a small bug in Hibernate

noreply@blogger.com (pawelstawicki) — Sun, 08 Jan 2012 23:58:00 +0000

The problem

At work, I needed to use entities with @DiscriminatorColumn inheritance. It means all types are kept in the same table, with value in this column showing what type given row is of. It's not recommended way to handle inheritance, but for some reasons we needed to use it. In developement, locally, I was using PostgreSQL database. When I tried to store this entities, I was receiving strange errors. Saying I cannot store an entity because entity with such id is already in database. It was quite strange, I was trying to store vanilla new entity. Test case to show this error is very short, so I'll include it here:

//Parent entity
@Entity
@Inheritance(strategy = SINGLE_TABLE)
@DiscriminatorColumn(name = "CLASS_ID", discriminatorType = INTEGER)
public abstract class ParentEntity {
  @Id
  @GeneratedValue(strategy = IDENTITY)
  private Long id;
}

//Child entity with discriminator  
@Entity
@DiscriminatorValue("1")
public class InheritingEntity extends ParentEntity {
}

//Test
public class PersistChildEntitiesWithDiscriminatorTest extends BaseCoreFunctionalTestCase {
  
  @Test
  public void shouldPersistTwoEntities() {
    Session session = openSession();
    session.beginTransaction();
    InheritingEntity child1 = new InheritingEntity();
    InheritingEntity child2 = new InheritingEntity();
    session.save(child1);
    session.save(child2);
    session.getTransaction().rollback();
  }
}

The cause

This test throws exception on second save, but only on PostgreSQL. Why is that? Well, when you save new entity to persistence context, Hibernate issues SQL call to database instantly. Other queries, like updates, are cached, and sent to database on em.flush or em.commit. But inserting of new entities is not cached and there is a reason for that. When we save new entity, Hibernate needs to assign ID to it, and this is taken from database. Most databases return ResultSet with one row and one column after insert, and it contains newly assigned ID. However, PostgreSQL behaves a bit differently. It returns whole inserted row (of course, with ID filled in). In most cases it works, because ID is the first column in this row, so when Hibernate takes value from the first row and the first column, it is the correct one. However, in case of classes with discriminator, ID is not the first column. Discriminator is the first column. So first insert is correct, ID 1 is assigned to child1, but then when we try to store child2, Hibernate also tries to assign 1 to it's ID, and complaints that there already is another entity with it.

The solution

So there was a bug in Hibernate. Can I solve it? I asked this question to myself, but to answer I couldn't do anything else than try ;) So I forked hibernate repository (yes, it's on github!) and... I was quite overwhelmed by the mass of code there. First challenge was to try to open it in my IDE, with all the subprojects and their interdependencies configured correctly. Thankfully there is gradle task for creating project files for IntelliJ IDEA, the IDE I'm happy user of. Next task was configuring Hibernate tests to use my PostgreSQL database. It turned out quite easy after one or two emails on hibernate-dev list. Now I had to change the code assigning IDs to entities to take it not always from first column first row, but sometimes from column of given name. So I had to get the name of column keeping IDs, which I did with a little help from other developers on the dev list.

The contribution

Now I commited fix to my forked repository on github, issued a pull request, got some comments, fixed files formatting... We'll see if it's accepted.

UPDATE: It is accepted :)

Easy way to convert file encoding

noreply@blogger.com (pawelstawicki) — Wed, 28 Dec 2011 17:10:00 +0000

Easy way to convert text files to UTF-8 on Ubuntu. Install package enca and you are able to:
enconv -L pl -x UTF-8 myfile.txt
Where after -L is language specified (necessary for enca to recognize file encoding before conversion) and after -x destination encoding.
Warning: enconv overwrites existing file, so better create backup copy before.

UPDATE: Another way, useful if you know source file encoding:
iconv -f ISO-8859-2 -t UTF-8 source.txt > utf8.txt

Play! with Heroku

noreply@blogger.com (pawelstawicki) — Wed, 30 Nov 2011 22:18:00 +0000

Recently, inspired by some talks on Devoxx, I decided to check Play framework and Heroku.

Play is another java web framework, but this one is heavily influenced by Ruby on Rails. Convention over configuration, little code necessary etc. Looks like Play is gaining momentum, version 2 (now beta) supports Scala and now it became part of Typesafe stack.

Heroku is another cloud. Well, quite different than AWS or GAE (actually, as far as I know, it works on AWS). Heroku is not only running applications, it is also able to build and deploy them. So to deploy your changes, it's enough to make git push.

So first, download Play 2 beta from this site. Unpack, add dir to your $PATH, so that you could use play command from anywhere.

Go to dir where you want to create the app and issue play new helloapp (where helloapp is your app name). Play should create helloapp directory. No go into it. You can notice that Play nicely added .gitignore file there, so when you create git repository here, unnecessary files won't be added. Play created fully functional, basic web application. You can issue play run in helloapp directory and the server is going to be started and application deployed. You can now see it on localhost:9000.

Now you'll need account on Heroku. Create one on heroku.com, it's easy as creating email or forum account. Then just activate it by email sent to you from Heroku.

For your app to be able to run on Heroku cloud, you'll need to add one more file. In the helloapp dir create Procfile file with content like this:
web: target/start
If you created your app with Play 1.x instead of 2.0, Procfile should be quite different:
web: play run --http.port=$PORT $PLAY_OPTS

To interact with the cloud you'll also need heroku-toolbelt. Get one from here.

Ok, you have your app and all the tools necessary to deploy it to the cloud. You'll also need git, which I assume you have already installed ;) As I mentioned on the beginning, it's necessary to push your app's code to Heroku.

Create git repository in the helloapp directory. Enter the directory and issue the following comands.
git init
git add .
git commit -m init
This commands created git repository in the directory, added current directory (helloapp) to the git index, and commited the changes to the local repository.

Next thing to do is creating new application on Heroku: heroku create --stack cedar from helloapp dir. This not only creates new application, but also nicely adds remote git repository. Heroku has few stacks. Stack is OS and stuff installed on it for our applications to run. For now, the only stack that supports Play is called Cedar. It's not the default one, so you need to tell Heroku explicitely that it should be used for your application.

It is also going to ask about your credentials to authenticate you on Heroku. It is done only once, and then your ~~credentials~~ key and token are stored on disk. If you want to delete them, just heroku auth:logout.

Ok, so you have application on Heroku, with git repository added to your remote ones, and you have one app locally which you want now to deploy to the cloud. Well, it couldn't be simplier:
git push heroku master

From now on it only takes some patience. Deployment takes time. When it's finished, it gives you the address, something like http://blooming-stone-5863.herokuapp.com deployed to Heroku in the console.

Go to this address and you'll see your app. Have fun!

Warsjawa 2011

noreply@blogger.com (pawelstawicki) — Thu, 20 Oct 2011 06:56:00 +0000

Dear reader. This post is about Polish event, organised by Polish programmers for Polish programmers. Speaking only Polish. So if you don't speak Polish, I don't think you'd be interested. Therefore, this post is in Polish.

W miniony weekend wybrałem się na warsztaty Warsjawa 2011. Wybrałem się z ekipą ze Szczecina jak zwykle pociągiem, co okazało się mieć taki minus, że pora roku jaka już nadeszła zapewnia że w pociągu może być albo zimno, albo gorąco. My wybraliśmy zimno. Jakoś jednak dojechaliśmy rankiem do Warszawy. Ja udałem się załatwić jeszcze parę prywatnych spraw, i nie bez drobnych komplikacji dotarłem do budynku przy Nowowiejskiej na tyle wcześnie, że mogłem jeszcze pomóc nosić ławki i krzesła.

Z czterech dostępnych ścieżek wybrałem DDD i CqRS, prowadzone przez Sławka Sobótkę i Rafała Jamroza. Warsztaty miały części prezentacyjne, i stricte warsztatowe. O ile prezentacjom niczego nie zabrakło (przynajmniej dla mnie, ale obie widziałem już wcześniej), to część warsztatowa jakoś się "rozlazła".

Z początku wstęp do DDD, ciekawy i świetnie poprowadzony. Dowiedzieliśmy się co to takiego to DDD, do czego się nadaje i z czym to się je. Po wstępie Sławek "oprowadził" nas po wcześniej przygotowanej na warsztaty aplikacji, pokazując "klocki DDD", z których jest zbudowana.

W pierwszej części warsztatowej zadaniem uczestników było rozszerzenie funkcjonalności aplikacji w duchu DDD. Zaczęliśmy prawidłowo, od testów. Z braku czasu testy pozostały na etapie wstępnym, tzn. takim, w którym wystarczyło w klasie testowanej z każdej metody zwrócić true żeby przechodziły. Więcej chyba było w tej części dyskusji, pytań i odpowiedzi niż kodowania, ale nie uważam tego za jej wadę. Wszak programowanie to przede wszystkim komunikacja. I tak czas upłynął nam do obiadu.

Na obiad była pizza, zgodnie z obietnicami organizatorów nie zabrakło :)

Po przerwie obiadowej znowu miała miejsce część prezentacyjna, w której Sławek pokazał co to takiego ten CqRS, i jakie ciekawe triki można stosować w celu poprawienia wydajności aplikacji. Np. mieć osobną bazę danych do zapisu i osobną do odczytu.

Prezentacja dość płynnie przeszła w kolejne warsztaty, które jednak chyba więcej miały z prezentacji. Nie żeby mi to przeszkadzało, może nawet tak miało być :) Dowiedzieliśmy się czegoś więcej o kilku bardziej skomplikowanych wzorcach, jak Saga czy Specification.

W ostatniej części Sławek pokazał nowe dla mnie narzędzie JBehave. Musze powiedzieć, że zrobiło na mnie wrażenie, i zamierzam poświęcić trochę czasu na bliższe poznanie tegoż. Zauważam też spore podobieństwo do Cucumbera, ciekawym czym się różnią.

Po wszystkim cała nasza ekipa czuła się pozytywnie naładowana informacjami, czuło się entuzjazm do dalszego zgłębiania poruszonych tematów, a to przecież najważniejsze. I choćby to świadczyć może o sukcesie tych warsztatów.

Jeśli miałbym szukać minusów to może za mało było napojów, no i nie było kawy. Aczkolwiek nie był do duży problem, bo w budynku były automaty zarówno z kawą, jak i z zimnymi napojami.

First CodeRetreat in Poznań

noreply@blogger.com (pawelstawicki) — Thu, 20 Jan 2011 21:53:00 +0000

On 15th January I had a pleasure to participate in first Code Retreat in Poznań. The event was organised by Adam Dudczak and Poznań JUG. If you don't know what Code Retreat is, take a look here. It is a great opportunity to meet other coders, see how they work and exchange knowledge and experience. There were 5 sessions, and the pairs were changing each time, so everyone coded with 5 different people.

Idea of Code Retreat is not only to code in pairs but also to use TDD. After lunch, during break, there were two discussion groups, one discussing BDD, and the other pair programming. I joined BDD group. First it was explained what BDD is, then we were discussing. The group was very active, many people had questions but there were also many answers :)

Programming task was standard for Code Retreats - Conway's Game Of Life. It's nice to watch development. During the first session we didn't finish our task, like most pairs. We barely started when the time was over. After the third session, most pairs had Game Of Life algorithm finished. Of course together with tests :)

Such an event is not only opportunity to learn, but also to meet people. Some I already knew from twitter, blogs etc., some from other events, and some were completely new to me. There was time to talk, not only during quite short breaks between sessions, but also during lunch and "afterparty". For lunch there was not pizza, but some decent dish, huge and tasty.

I have to write also about the place the event was held at. It was in Cognifide offices, and it was one of the nicest offices I've ever seen! Lunch and afterparty was in the basement, but this basement looked like a decent pub. Brick walls and ceilings, bar, comfy sofas and tables. There was also refrigerator full of beer, and it was used during afterparty :)

For me this event was a big success. Congrats, organizers!

Scala script to find duplicate files

noreply@blogger.com (pawelstawicki) — Fri, 24 Dec 2010 23:07:00 +0000

Here is simple Scala script finding duplicate files and moving them to another directory. It searches album-with-duplicates for duplicates of files in main-album. All duplicates found are moved to copies directory in user's home.

If you have some ideas how to improve it, I'd appreciate if you share it in comments.

MD5 algorithm taken from here.

package com.blogspot.pawelstawicki.remove.duplicates

import java.security.MessageDigest
import java.io.{FileInputStream, File}
import org.apache.commons.io.{FilenameUtils, FileUtils, IOUtils}

/**
 * @author ${user.name}
 */
object App {
  
  def main(args : Array[String]) {
    val dir1 = new File("/photos/main-album,");
    val dir2 = new File("/photos/album-with-duplicates");

    val dir1Content = getAllFiles(dir1)
    val dir2Content = getAllFiles(dir2)

    var dir1Map = Map[String, File]()
    dir1Content.foreach(f => {
      val md5 = md5SumString(IOUtils.toByteArray(new FileInputStream(f)))
      println("md5 for " + f.getPath + ": " + md5)
      dir1Map = dir1Map + (md5 -> f)
    })

    var dir2Map = Map[String, File]()
    dir2Content.foreach(f => {
      val md5 = md5SumString(IOUtils.toByteArray(new FileInputStream(f)))
      println("md5 for " + f.getPath + ": " + md5)
      dir2Map = dir2Map + (md5 -> f)
    })

    for(md51 <- dir1Map.keys; md52 <- dir2Map.keys) {

      if (md51.equals(md52)) {
        val suspectedDuplicate = dir2Map(md52)
        val original = dir1Map(md52)

        if (checkDuplicate(original, suspectedDuplicate)) {
          println(suspectedDuplicate.getPath + " is duplicate of " + original.getPath)
          val copiesDir = new File(FileUtils.getUserDirectory + "/copies/" + FilenameUtils.getPathNoEndSeparator(original.getAbsolutePath()));
          println("Moving to " + copiesDir.getPath)
          FileUtils.moveFileToDirectory(suspectedDuplicate, copiesDir, true)
        }
      }
    }
  }

  def checkDuplicate(f1: File, f2: File): Boolean = {
    val bytes1 = new Array[Byte](1024*1024)
    val bytes2 = new Array[Byte](1024*1024)

    val input1 = new FileInputStream(f1)
    val input2 = new FileInputStream(f2)

    var bytesRead1 = input1.read(bytes1)
    while(bytesRead1 > 0) {
      val bytesRead2 = input2.read(bytes2)

      if (bytesRead1 != bytesRead2) {
        return false;
      }

      //Bytes read number the same
      if (!bytes1.sameElements(bytes2)) {
        return false
      }

      bytesRead1 = input1.read(bytes1)
    }

    //bytesRead1 is -1. Check if bytes read number from file2 is also -1
    if (input2.read(bytes2) == -1) {
      return true;
    } else {
      return false;
    }
  }

  def md5SumString(bytes : Array[Byte]) : String = {
    val md5 = MessageDigest.getInstance("MD5")
    md5.reset()
    md5.update(bytes)

    md5.digest().map(0xFF & _).map { "%02x".format(_) }.foldLeft(""){_ + _}
  }

  def getAllFiles(dir : File) : List[File] = {
    var l = List[File]()
    dir.listFiles.foreach(f => {
      if (f.isFile) {
        l = f :: l
      } else {
        l = l ::: getAllFiles(f)
      }
    })

    l
  }

}

JSF2.0 component for cross-field validation

noreply@blogger.com (pawelstawicki) — Mon, 20 Dec 2010 23:02:00 +0000

Have you ever had problems with cross-field validation in JSF? Me too, so I created this component. You can validate few UIInput components and have their values as List in validator. The component is in softwaremill-faces library. To use it in maven project, add repository:

<repository>
  <url>http://tools.softwaremill.pl/nexus/content/groups/smlcommon-repos/ </url>
  <layout>default</layout>
  <releases>
    <enabled>true</enabled>
  </releases>
  <snapshots>
    <enabled>true</enabled>
  </snapshots>
</repository>

and dependency:

<dependency>
  <groupId>pl.softwaremill.common</groupId>
  <artifactId>softwaremill-faces</artifactId>
  <version>43-SNAPSHOT</version>
</dependency>

Now you can use multiValidator component. First add namespace to your .xhtml page:
xmlns:v="http://pl.softwaremill.common.faces/components"

Then just wrap components you want to cross-validate in <v:multiValidator>. If you attach validator to this component, value parameter that goes to validation method is List of values of UIInput components inside tag.

E.g. if you want to validate two checkboxes. Each can be checked or unchecked, but at least one has to be checked.

<v:multiValidator id="multi" validator="#{bean.validationMethod}">
  <h:selectBooleanCheckbox value="#{bean.check1}" />
  <h:selectBooleanCheckbox value="#{bean.check2}" />
</v:multiValidator>
<h:message for="multi" />

Validation method in bean:

public void validationMethod(FacesContext context, UIComponent component, Object value) {
  List<Object> values = (List<Object>) value;
  //value is list of values of both selectBooleanCheckboxes
  Boolean firstChecked = (Boolean) values.get(0);
  Boolean secondChecked = (Boolean) values.get(1);

  if (! (firstChecked || secondChecked)) {
    Message message = new FacesMessage(FacesMessage.SEVERITY_ERROR, "Check at least one checkbox", null);
    throw new ValidatorException(message);
  }
}

If none checkbox is checked error message is displayed in <h:message for="multi"> tag.

Source code of this component is on github.

Any suggestions, opinions or questions regarding this component are welcome. Have a good time using it :)

GWT table row as UiBinder

noreply@blogger.com (pawelstawicki) — Tue, 02 Nov 2010 22:51:00 +0000

Some time ago I wanted to dynamically add rows to a table in GWT, but I wanted to define row template using UiBinder. Programatically it is no problem. You can create FlexTable and add widgets to it. The problem arises when you want to do it using UiBinder. You can create a FlexTable in UiBinder, and you can create widgets, but you can't create element which renders to html tag <tr>. Kazik Pogoda found a way to do it.

First we need two widgets which renter to <tr> and <td>. The code for TR:

public class TrElement extends ComplexPanel {
  private TableRowElement tr;

  public TrElement() {
    Document doc = Document.get();
    tr = doc.createTRElement();
    setElement(tr);
  }

  @Override
  public void add(Widget child) {
    add(child, (Element) tr.cast());
  }
}

The TD is analogous, just change TableRowElement into TableCellElement and doc.createTRElement() into doc.createTDElement(). We can now create UiBinder xml file which renders to single table row (TR element is it's root):

<!DOCTYPE ui:UiBinder SYSTEM "http://dl.google.com/gwt/DTD/xhtml.ent">
<ui:UiBinder xmlns:ui="urn:ui:com.google.gwt.uibinder"
      xmlns:g="urn:import:com.google.gwt.user.client.ui"
      xmlns:my="urn:import:pl.my.gwt.client">
 
  <my:TrElement>
    <my:TdElement>
      <g:HTMLPanel>
        <g:Label ui:field="icon" styleName="Icon"></g:Label>
        <g:Label ui:field="title" styleName="Title"></g:Label>
        <g:Label ui:field="description" styleName="Description"></g:Label>
      </g:HTMLPanel>
    </my:TdElement>
    <my:TdElement>
      <g:Label ui:field="price" styleName="Price"></g:Label>
    </my:TdElement>
  </my:TrElement>
</ui:UiBinder>

Here is how we can use it to add to a table (FlexTable):

TableRowView trView = new TableRowView();
table.getElement().appendChild(trView.getElement());

Warsjawa 2010. Impressions.

noreply@blogger.com (pawelstawicki) — Fri, 29 Oct 2010 23:45:00 +0000

My impressions after this year's Warsjawa are very positive. Well, in fact I could expect this :) Presentations were interesting, meeting other IT people is always good too. I met Bartek Zdanowski, who wanted to meet me. It's quite strange feeling when you meet somebody who knows you, but didn't meet you before. Strange but nice :)

First presentation was about Play framework by Wojciech Erbetowski. I was a little late, but I've seen enough to notice that Play is framework different than all the others.

I was a bit disappointed that Sławek Sobótka didn't make it there (he fell ill). He's topic was the most interesting for me. However, Paweł Lipiński worthily replaced him. Paweł was talking about what in programming he was taught by... his own children, kindergarten, Roomba etc. I never noticed that when there are 2-3 people in a project, it is much cleaner, but it's becoming a mess when there are more than 5 people. I never noticed it, but now it's very clear to me. It seems Paweł is a man who can learn and get knowledge from anything. Ah, and he mentioned me :)

This year on Warsjawa there was also something from our Szczecin JUG. Darek "Lock" Łuksza presented Git version control system and it's Eclipse plugin EGit. Darek did great job, and even found one bug during presentation :) It seemed a bit embarassing for him, but the rest was very good. He knows the topic very well (he's EGit commiter) and he showed many things coding live. Overall presentation was very good.

After Darek's presentation pizza arrived. It disappeared quite quickly, maybe there was a bit too few. We run out of beverages too. Anyway I didn't hear somebody died of hunger or thirst, so it wasn't that bad :)

Next interesting presentation was about Clojure. This time there were two presenters. First Marcin Rzewucki showed us some theory, then Jan Rychter told about practical use of Clojure in Fablo. He showed few interesting features of this language, like STM. Thanks to STM in Fablo they are able to replace customer's database without stopping the service. Pretty impressive.

Did you know that in Java you can have two methods with the same name and arguments, differing only by return type? Compiler can't compile this, but if you write it in bytecode it is perfectly valid! This and others interesting things about bytecode were shown by Adam Michalik. I wonder if any company needs "bytecode programmer"? I like such low-level stuff. At the university I always liked assembler.

The last presentation by Rafał Rusin was about Apache HISE. I have to admit I was too tired and the topic was not very interesting for me, so I didn't remember much from this one.

Generally I think this was very nice conference. Meeting new people, and some older friends, is always nice. Listening to interesting people talking about interesting things is also nice. It seems one hour for presentation is very good duration. All the presenters could do all they wanted/needed (at least it seemed so). I remember on GeeCON there was only 45 minutes and it was too little. If only there was more pizza (or maybe something better?) and beverages it would be perfect.

Convert JME application to Android

noreply@blogger.com (pawelstawicki) — Wed, 13 Oct 2010 08:58:00 +0000

Here is the story:

I bought Android phone. Earlier I was using Windows Mobile phone, and I had one j2me application on it that was very important to me. It was "TokenGSM", application which serves the same purpose as RSA tokens, but installed on my phone. Very good decision of my bank that they created it so users don't have to carry additional device with them (users have choice, you can also get RSA token if you prefer it). So this application is necessary if I want to log in to my bank account. Pretty vital.

But, as you probably know, there is no Java ME on Android :( Here is what I did (this forum was useful):

Call the bank and ask for new GSM token. Token is somehow bound to the phone, so if you change phone, you need new token.
Write down URL from WAP Push SMS. Do it before opening it. On my phone (Android 2.2, HTC Desire) if you don't do it but just try to open URL, it's lost. Browser can't open it, but you don't even have chance to see it. Browser just blinks and returns to home screen :| It's not in history, nor in downloads.
Go to http://www.netmite.com/android/ and download App Runner. I went there from my phone and downloaded directly to it. I'm not sure if this point is really necessary, but some say it is.
Click "Convert existing j2mes into apk & upload to Android Market." I don't think it really uploads to Market.
Enter your written down URL and click "Get Apk". You have your Android TokenGSM :)
Transfer the file to your phone and install it.
Now you have to activate the token with code you received from the bank.

Flash scope in JSF 2.0 (nothing about Adobe Flash)

noreply@blogger.com (pawelstawicki) — Sun, 26 Sep 2010 21:38:00 +0000

In JSF 2.0, amongst Request, Session etc, there is the new scope called Flash. (However there is no annotation @FlashScope, and you can't put bean in this scope). It's concept is taken from Ruby on Rails. Data in Flash scope are available for current and for next request, but not for subsequent requests. Usage is very simple.
On first page (e.g. index.html):

<h:form>
  <h:inputtext value="#{flash.text}">
  <h:commandbutton action="page?faces-redirect=true" value="To page">
</h:form>

And this is the page we are redirecting to, page.xhtml:

<h:form>
  <h:outputtext value="#{flash.text}">
</h:form>

Why would you need such scope? Imagine you want to enter data on one page, then redirect to another page, and display this data. If not for redirect, we could use Request scope. But with redirect value is sent with request, then browser is redirected to another page, and makes another request. Request scoped value is gone. In such case Flash scope becomes useful. We can put value in this scope with one request, and when another request is made, the value is still there. But it is not stored in session, and is not available for subsequent requests.

However, there is a way to make data in Flash scope alive a bit longer. It is enough if in page.xhtml we change #{flash.text} to #{flash.keep.text}. This way data is not removed from the scope after first request, but is available for one more.

In managed beans, you can get flash scope by

Flash flash = FacesContext.getCurrentInstance().getExternalContext().getFlash();

JSF 2 templating

noreply@blogger.com (pawelstawicki) — Fri, 27 Aug 2010 23:22:00 +0000

In this post I want to share my knowledge about JSF 2 templating. Most of this post are source codes, but if you are novice and don't know how to start, and where to put the source code, here you go.

0. Download Netbeans and setup the project.

Install Netbeans and run it. I use version 6.9. In Netbeans, go to File -> New Project... and then choose Java Web and Web Application. Click Next.

In next window choose project name, optionally directory where to store it, and set the project as the main project.

In the next window leave <None> in enterprise application and choose web server. Default should be ok, the same about context path.

Check JavaServer Faces. In Libraries tab select Registered Libraries and JSF 2.0.

1. Simple template and it's client. <ui:insert> and <ui:define>

Ok, we'll create simpliest template possible. In Web Pages create new .xhtml file, let's name it headerFooter.xhtml. It's our template containing header, footer, and place for some content between them.

You can let Netbeans create it for you along with nice CSS files, you can also select one of predefined template layouts. Click File -> New File -> JavaServer Faces -> Facelets Template. Our template code:

<?xml version='1.0' encoding='UTF-8' ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:ui="http://java.sun.com/jsf/facelets"
      xmlns:h="http://java.sun.com/jsf/html">
    <h:head>
        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
        <title>Header-Footer Template</title>
    </h:head>
    <h:body>
        <div id="top">
            Here goes our header
        </div>
        <div id="content" class="center_content">
            <ui:insert name="content">Content</ui:insert>
        </div>
        <div id="bottom">
            And here footer
        </div>
    </h:body>
</html>

Now create template client. It is a page which uses the template. Let's call it templateClient.xhtml:

<?xml version='1.0' encoding='UTF-8' ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<ui:composition xmlns:ui="http://java.sun.com/jsf/facelets"
                template="./headerFooter.xhtml">
    
    <ui:define name="content">
        content
    </ui:define>

</ui:composition>

In template there is <ui:insert> tag. This is a named place where content from template client goes, and it's name in this case is content. In client, there is <ui:composition>. It uses template given as tags attribute, and has <ui:define> tags, with the same name as <ui:insert> in template. So what's in <ui:define> tag is inserted in place of <ui:insert> with the same name. You can run your project (in Netbeans right-click on project and select Run) and see the templated page at http://localhost:8080/JsfTemplating/faces/templateClient.xhtml Any content surrounding <ui:composition> tag is ommitted.

Template client can also be a template. Just put some <ui:insert> tag inside of <ui:define> block. E.g.

<ui:define name="content">
    <div class="info">
        Here goes some additional info, which is not in the header nor in the footer, but we need it on some pages.
    </div>
    <ui:insert name="innerContent">
</ui:define>

2. Decorate

We can also insert parts of markup in our pages. If you have piece of code which you put on some pages and it's always the same, you can separate it to a file and just insert it in the pages. <ui:decorate> tag is used for this:

<?xml version='1.0' encoding='UTF-8' ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:ui="http://java.sun.com/jsf/facelets">
    <body>
        Some page text
        <ui:decorate template="./disclaimer.xhtml"/>
    </body>
</html>

Content of disclaimer.xhtml is injected where <ui:decorate> tag is. Don't add <?xml ... ?> declaration or doctype there, or it is going to be inserted in the middle of your page, which is not desirable. You can also use standard templating in disclaimer.xhtml. You can put <ui:insert> there and <ui:define> in page which uses it.

3. Include

Another method to attach some piece of markup is <ui:include>. This one also lets us pass some parameter to the piece of code we want to include, and this parameter can be managed bean. Lets say we have bean with information about some documents:

@ManagedBean
public class Paper {

    private String author;
    private String title;

    //getters and setters omitted
}

Piece of code to display information about a book is prepared in book.xhtml file:

<div xmlns="http://www.w3.org/1999/xhtml">
    Book "#{book.title}" by #{book.author}
</div>

Now in any other JSF page we can use it:

<ui:include src="./book.xhtml">
    <ui:param name="book" value="#{tomSawyer}" />
</ui:include>

I hope this post is going to be helpful to somebody. At least to me. Happy templating with JSF!

JPA Puzzle. When your entity misses some fields.

noreply@blogger.com (pawelstawicki) — Mon, 09 Aug 2010 21:11:00 +0000

Today I encountered quite interesting problem. It wasted few hours of my life, so maybe if you read this you can save some. Example is simplified as much as possible.

@Entity
@Table(name = "PERSONS")
@NamedNativeQueries({
    @NamedNativeQuery(
        name = "getPersonBasic",
        query = "SELECT p.name, p.surname FROM persons where p.surname = ?",
        resultClass = Person.class),
    @NamedNativeQuery(
        name = "getPersonDetails",
        query = "SELECT p.name, p.surname, p.age, p.street, p.city FROM persons where p.surname = ?",
        resultClass = Person.class)
})
public class Person {
    
    private String name;

    @Id
    private String surname;

    private int age;

    private String street;

    private String city;

}

Now, in my application I am getting Person with surname "Stawicki" without details, by "getPersonBasic" query. Then I need detailed person in the same transaction, so I execute "getPersonDetails". And... I am getting Person without age, street and city. This three fields are null. Why? Do you already know? If not, look below for answer.

The problem is in cache, and in @Id which is only on surname field. First person is retrieved without details and stored in cache. Then we try to get person with details, but JPA doesn't understand difference between our queries. It queries the database, and identifies that if it builds entity from resulting ResultSet, it is going to have the same @Id as entity which is already in cache. If @Id is the same, logically it is the same entity. So it just returns it without building whole entity from ResultSet.

Review: Real World Java EE Patterns Rethinking Best Practices, by Adam Bien

noreply@blogger.com (pawelstawicki) — Tue, 27 Jul 2010 20:57:00 +0000

"Real World Java EE Patterns" is a book targeted rather for developers with some experience with JEE. If you are a beginner, you can miss some context. If you have some experience with JEE, in this book you'll probably find solutions to problems that are familiar to you.

Adam Bien is great at explaining difficult topics. Difficult? I didn't find anything difficult in this book ;) E.g. transactions isolation is explained very clearly.

The book is very good catalog of JEE Patterns. Each pattern is described separately in similar manner. Each chapter has subchapters: "Problem", "Forces", "Solution", "Testing", "Documentation", "Consequences" and "Related Patterns". In "Problem" a reader can find short description of a problem the pattern should solve. "Forces" shows features that solution should have. "Solution" contains description of pattern, what classes it consists of and what is their responsibility. Usually accompanied by very clear and simple pieces of code. In "Testing" and "Documentation" author highlights what should we test when we use certain pattern, and what should be documented (quite obvious, isn't it?). In "Consequences" we can read about what are pros and cons of the pattern. "Related Patterns" is self explanatory. Most interesting subchapter is "Solution", and it also has sub-subchapters. One of them is "Rethinking". It is good part for experienced JEE developers. Adam shows why some patterns are obsolete. It doesn't mean you should never use it, but in most cases it is no longer necessary in JEE5 or 6. Some patterns, when moved from EJB2 to EJB3, are not adding any value, but instead are adding layer of abstraction and unnecessary complicating the system.

What I like about Adam Bien is that he is not only writing and talking about programming, but he's also programming. While reading the book, one can feel that the author has real experience with the topic. Sometimes he advices not to use what is common "best practice", when it is not necessary and is not adding any value. Good programmer should be able to balance pros and cons of possible solutions, not just blindly follow common practice.

There are small mistakes in the book, but only editor ones, like misspellings and formatting mistakes, single lines of code on next/previous page etc. Nothing really annoying, but there is room for improvement on this field.

Thank you Adam for this book!

Transaction isolation level

noreply@blogger.com (pawelstawicki) — Sun, 30 May 2010 22:23:00 +0000

Recently I have read very good explanation of transaction isolation levels in "Real World Java EE Patterns. Rethinking Best Practices." by Adam Bien.

Transactions were introduced for two reasons. To make complex operations atomic (all or nothing) and to give possibility to concurrently use resources. Without transactions it is possible that two users (or more specifically actors, this can be some parts of the system, not necessarly live users) read/write the same resource simultaneously, and changes made by one of them are lost and overwriten by the other one.

Consider following example document:

Cheesecake bottom
250g crunchy bisquits
100g butter

Cheesecake top
900g cottage cheese
250g mascarpone cheese
1 lemon

Imagine one user gets the document and the other user also is getting the same document. Both users have the same data. Now both of them are editing it. One likes sweeter cake bottom, so adds "2 spoons of sugar" to the recipe, and writes changed recipe back to the database. In database now it looks like this:

Cheesecake bottom
250g crunchy bisquits
100g butter
2 spoons of sugar

Cheesecake top
900g cottage cheese
250g mascarpone cheese
1 lemon

Meanwhile second user adds "2 eggs" to cheescake top recipe and saves his version:

Cheesecake bottom
250g crunchy bisquits
100g butter
2 spoons of sugar

Cheesecake top
900g cottage cheese
250g mascarpone cheese
1 lemon
2 eggs

Change made by the first user is lost. In some cases it can lead to inconsistent data.

So that's why transactions were introduced. Transaction isolates changes made by one user from changes made by the other. There are 4 levels of transaction isolation.

Serializable

Transaction locks all necessary resources. If user wants to read or write some resource, it is locked and no one else can read or write it. This level of isolation can easly lead to deadlocks:

User Ann opens transaction. Ann needs resource A, transaction locks it.
User John opens another transaction. John needs resource B, transaction locks it.
Ann needs resource B, but it is locked, so Ann's transaction needs to wait until it is released
John needs resource A, but it is locked. John's transaction waits until it is released. Both users wait endlessly.

If John didn't need A, he finishes his transaction, B is released and Ann can finish her transaction too.

Repeatable reads

Guarantees that the same query will return the same results if executed in one transaction. Even if other transaction modifies resource meanwhile. Exception is adding - new rows can appear in query result. If another transaction deletes existing rows or modifies them, this changes are not visible.

Ann opens transaction. She reads names of java4people organizers: "Stawicki" and "Gruchała".
Bob opens transaction and deletes "Stawicki". Bob commits changes and closes transaction.
Again Ann reads names. She gets "Stawicki" and "Gruchała".

If Bob added some name, Ann would read it too.

Read commited

The same query can return different result even in one transaction if another transaction makes changes and commits them.

Ann opens transaction and reads names of java4people organizers: "Stawicki" and "Gruchała".
Bob opens transaction and deletes "Stawicki".
Ann reads names again. She gets "Stawicki" and "Gruchała".
Bob commits his transaction
Ann reads names again (still in one transaction). She gets "Gruchała".

Read uncommited

Like no transactions at all. It only gives us atomic operations. User can rollback all the changes in transaction. Changes are visible to other transactions even before commit. Of course, if transaction that made changes is rolled back, changes are not visible any more.

After GeeCON 2010

noreply@blogger.com (pawelstawicki) — Sat, 15 May 2010 10:02:00 +0000

This year GeeCON took place in Poznań which is much closer to Szczecin than Cracow. I expected it to be as good as a year before, maybe I expected too much. I don't mean it was bad, but previous year it was so great that maybe it was hard to keep this level. Or maybe I was not choosing right presentations. There were three tracks this year so choice was sometimes difficult.

Like a previous year there was "University Day" before the conference. University Day is day of workshops on various topics. I have chosen Gradle, like many other people. Too many unfortunately. Hans said there are too many attendants to make exercises for everyone, so only he coded live. I missed coding myself.

First interesting concept I heard about was Object Teams. It is a new idea of modularizing our programs, based on entities, collaborations and roles. All three look pretty much like java classes. In OOP there are objects which are data and methods. We can use objects to modularize our applications, but often connections between modules are becoming complicated. Creators of Object Teams concluded that modularization and OOP is not about modules/objects itself but about connections between them. So there are entities, which represent data. There are collaborations, that represent operations on data, and roles. In a collaboration, each entity plays some role. Everything looks pretty and neat, and I wonder if it is going to become popular in coming years. I must admit I haven't heard about it before.

Quite interesting presentation was about JSF 2.0 by Ed Burns. Nothing quite new for me, but if someone thinks JSF didn't change much since 1.2 version he should attend this presentation (or watch on GeeCON's channel on parleys.com when it becomes available ;)). Guys who created version 2.0 addressed all the problems developers were complaining about with previous versions. I remember Seam was framework which was built on JSF and addressed such problems. Now, when it is solved in JSF itself, I wonder what are differences between Seam and JSF 2.0.

On the second day Jonas Bonér was talking about actors, agents, STM and other solutions making concurrent programming much easier than threads with shared state and locks. Next time I'll need to do some concurrent programming in Java I'll definitely use something from Akka.

Vaclav Pech's presentation was quite similar. He also talked about concurrent programming with actors, but in Groovy. As usual Vaclav gave very good presentation.

Vaadin is the next thing I want to use. At work I use GWT and I don't really like it. GWT compilation is very long and asynchronous callbacks are making code more complicated. Besides most of work is done on the server anyway. Programming in Vaadin is very similar to programming in GWT, but everything is server side. Vaadin uses GWT for presentation, so it is possible to write custom components for Vaadin in GWT, but then compilation is needed only once. Damn, this could save many hours of my life.

Bruno Bossola's presentation was very good and funny. No suprise here :) Bruno talks about things that many developers really need. I agree somewhere we lost real Object Oriented programming. Bruno also mentioned a book which I feel is unfairly forgotten. "Applying UML and Patterns: An Introduction to Object-Oriented Analysis and Design and Iterative Development" by Craig Larman. When I started working for NCDC more than few years ago, my boss put this book to company's library and recommended it to us. I read it and I feel it made me a better developer. If you put all your code into one huge class or even method, buy or borrow this book and read it. Even if it's old it's worth it. Some things don't change even in IT ;)

If I can have some hint for the organizers I'd like to suggest to make presentations longer. Many presenters didn't manage to show everything they wanted. I think 1.5 hour is minimum.

I heard food was also a problem on the first day. It didn't suffice for everybody, but I managed to grab my plate so I don't complain :)

SzJUG member, Filip "Filus" Pająk presentation on TestNG

noreply@blogger.com (pawelstawicki) — Fri, 23 Apr 2010 22:20:00 +0000

Recently (last Thursday) I had pleasure to attend Szczecin JUG meeting at which Filip presented TestNG. Presentation was great. I knew Filip is a perfectionist, and when he does something it is just perfect :), but I didn't know he has such great speaker talent! I felt he really cares about being understood.

Filip was very well prepared. A lot of knowledge and a lot of great examples. Examples were prepared before and were short and simple. Just what you need to understand some issue, no less and no more. I think it is better than coding "live" when something can blow up. Show example, run it and show results.

Filip claimed he is no "guru" and doesn't know TestNG very well, but as far as I remember, he knew answer to every question :). I hope it was not his last presentation at Szczecin JUG. Or maybe he'll present somewhere else now? Who knows?

Szczecin JUG Meeting. Filip "Filus" Pająk on TestNG.

noreply@blogger.com (pawelstawicki) — Sat, 17 Apr 2010 17:42:00 +0000

On 22nd of April 2010 Filip "Filus" Pająk will present TestNG. If you are around Szczecin, feel invited. It's free :)

Filus is experienced Java programmer and tester. When he does something, you can be sure it's going to be good. I think he'll make great presentation.

I am very glad Szczecin JUG is organising another meeting. It's been some time since the last one.

Maybe I should also present something? JavaFX? Scala?

JavaScript training

noreply@blogger.com (pawelstawicki) — Mon, 22 Mar 2010 21:13:00 +0000

This weekend I was able to take part in JavaScript training by Damian Wielgosik . As I was not very experienced JavaScript developer, I have learned a lot. I also got convinced that in JavaScript everything is possible :)

I am impressed by the trainer. I was quite surprised when I found out that it was the third training he conducted. I thought he is much more experienced. He had great "contact with the audience", atmosphere was very friendly, nobody was afraid to ask questions or discuss. Thank you Damian!

Two days full of training is quite a much time, so there was also place for exercises. This was very good, nothing consolidates knowledge like some exercises and because Damian was always there to help if we were stuck, we were not wasting time. Doing exercises in groups was also very good idea. We worked as a team and helped each other. I never doubted in efficiency of pair programming :)

Scope of topics was impressive. I learned a lot about prototypes and constructors, which was a bit strange for Java programmer (but not so much if he was also learning Ruby ;)) I think we covered pretty much everything in JS, from basics (which we run through quite fast) to optimization.

I would also like to thank other participants for great atmosphere, discussions, questions, help and info about "pierogarnia" :) I wish I could attend more such good trainings.

Review: Programming Scala, by Venkat Subramaniam

noreply@blogger.com (pawelstawicki) — Fri, 19 Mar 2010 22:28:00 +0000

Courtesy of The Pragmatic Bookshelf I was able to read "Programming Scala" by Venkat Subramaniam. If you are a Java programmer, this book is a great introduction to Scala for you. It is packed with knowledge. There was not even one chapter after reading which I could say "well, this was quite obvious, I could expect this". Book is targeted for experienced Java programmers, so author does not cover things we already know. This was great for me, my time was not wasted learning what I don't need. Instead, I was led from easier topics in Scala (like classes, typing, closures) to more difficult ones (pattern matching, actors and concurrency) at a nice pace. Most topics are explained quite clearly and understandable. There are few which I had difficulties understanding and I needed to fiddle with some code to get it. But fiddling with code is fun for programmer, isn't it? One of such topics were actors, which is a great feature in Scala and something completely new for Java programmer (unless he knows Erlang too ;)). It took me some time with IDE to understand actors, but it was also fun.

The book not only teaches how to program in Scala, but also shows tools to run these programs. Knowing that Scala doesn't necessarly need to be compiled to run and that it can be run as a script can be very useful if you need to do some simple job on your system. Author shows how to use it as a scripting language. I also liked the last chapter in which author creates Scala application using various features covered in the book. It was a good way to sum up.

The only weak point of this book are examples. Especially at the end of the book they become more and more difficult and not always clearly explained. Another disadvantage of examples is their little connection with reality. Have you ever need to count letters in blog URL? In some cases they even show bad practice like throwing general Exception. I know it's only example, but educational book should not promote bad practice even there.

Said all this, I would recommend this book to every Java programmer who wants to learn Scala. If you are not afraid to fiddle a little with code (and you should not be, you are a programmer after all;) ) it is a very good book for you and you are it's perfect target.

Last but not least, not only author, but also editors deserve praise. I have PDF edition and I can login on pragprog.com page anytime and download it. I don't have to worry that if I loose it - it's lost for good. Moreover there are .mobi and .epub formats available. I could read the book on my phone on a bus or a tram which was very convenient. And there is even more - the books are DRM free, which shows publishers not only care for readers comfort, but also trust us. I appreciate that.

Simple AJAX with JSF 2.0

noreply@blogger.com (pawelstawicki) — Tue, 02 Mar 2010 22:42:00 +0000

In JSF 2.0, there are tags added to make AJAX calls easy. Without further ado:

<?xml version='1.0' encoding='UTF-8' ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"
 xmlns:ui="http://java.sun.com/jsf/facelets"
 xmlns:h="http://java.sun.com/jsf/html"
 xmlns:f="http://java.sun.com/jsf/core">
<h:head>
   <h:outputScript name="jsf.js" library="javax.faces" target="body"></h:outputScript>
   <title>Ajax</title>
</h:head>
<h:body>
   <f:view contentType="text/html; charset=UTF-8">
       <h1>Hello Ajax World!</h1>
       <h:form id="form">
           <h:inputText id="name" value="#{nameMBean.name}"/>

           <h:commandButton value="Reverse name via Ajax!">
               <f:ajax execute="name" render="reverseName"/>
           </h:commandButton>
           <h:outputText id="reverseName" value="#{nameMBean.reverseName}"/>
       </h:form>
   </f:view>
</h:body>
</html>

First things first. To use AJAX in JSF 2.0, we have to attach jsf.js file (line 8). Then we are ready.

Most interesting is line 18. <f:ajax> added to command button makes AJAX action fire when button is pressed. We could as well put it into <h:inputText> element, then it would fire on change of text inside (onchange event).

Parameter execute tells JSF which parts it should send to server for model update. This could be also form, for all the parameters in the form, or list of element ids separated by space. Parameter render denotes which parts of view should be rerendered after receiving AJAX response. Here we can also put more than one.

We can put more parameters to <f:ajax>, like event, where we can specify on which event AJAX action should occur (not necessarly click event), or listener, where we can call some method on managed bean.

Make Ubuntu faster

noreply@blogger.com (pawelstawicki) — Thu, 11 Feb 2010 21:22:00 +0000

There are few very simple methods to make Ubuntu faster. Some of them work not only on Ubuntu, but on other linux systems as well.

Change dynamic linking to static one

When compiler generates binary file, elements which should connect with external libraries are only stubs. When running application for the first time (in a session), it is dynamically linked, which means stubs are replaced with real calls to libraries. This process takes some time. Open Office on my machine runs about 5 seconds for the first time, and 1-2 seconds every next time.

Fortunately there is a program called prelink, accessible from Ubuntu repositories.

sudo apt-get install prelink
sudo prelink -amR

After the second command prelink scans binaries specified in /etc/prelink.conf and modifies them replacing dynamic calls with static ones, but in binary files, so this change is permanent. Disadvantage of this solution is that after every update of processed program we need to run prelink again.

We can always go back to dynamic linking (undo prelink):

sudo prelink -au

You HAVE TO go back to dynamic linking if you want to remove prelink.

Get now what I'll need in a few miliseconds

Next useful program fastening Ubuntu is preloader. It uses magic (complicated algorithms) to predict what data is going to be necessary next and loads it before application asks.

You only need to install it:

sudo apt-get install preload

It should run automatically, but if for some reason it is not running:

sudo /etc/init.d/preload start

Koziołek says it consumes a lot of RAM, but I didn't notice :)

Run services in paralell

If you have more than one core, you can set your system to start services in paralell during startup. This is potentially dangerous, as there might be services which depend on each other.

Edit /etc/init.d/rc and find line CONCURRENCY=none. Change it to CONCURRENCY=startpar.

This post is here courtesy of Bartłomiej "Koziołek" Kuczyński, and is translation of his post in Polish.

Partial mocks in Mockito - Mock only what you need, left the rest to the original class

noreply@blogger.com (pawelstawicki) — Tue, 12 Jan 2010 09:35:00 +0000

In Mockito you can not only create "regular" mocks, but also partial mocks. Let's assume we need to use instance of class A, and we want to mock it. We can do mock:

A aMock = Mockito.mock(A.class);

Sometimes, we want to use instance of real class A, but mock only part of it. Only one or few methods. With Mockito it is possible:

A a = Mockito.spy(new A());

So we spy real object, we can verify it's method calls, but we can also do that:

Mockito.when(a.methodCall()).thenReturn(1);

We can mock some of A's methods, leaving the rest to the real A instance.

For me it was useful while I was creating test involving user. User object's have ID, but there is only getter for it. ID is always set by database, and should never been changed by programmers, so there is only getter. Thanks to partial mocks, I mocked only getId() method, without need to add setId() just for testing purposes.

CouchDB

noreply@blogger.com (pawelstawicki) — Sun, 13 Dec 2009 23:32:00 +0000

CouchDB is a new kind of database. It's not a relational database, it's not objective database. CouchDB just stores documents. A document is something like java map, it has keys and values. Values can be strings, numbers, dates, lists, maps, also binary attachments. Documents are accessible by pure http, REST service, in JSON format. CouchDB is written in Erlang, so it is well scalable for many processors/cores. (At least this is what it's authors claim ;)) Let's give it a try.

Installation

On ubuntu issue:

apt-get install couchdb

During installation, couchdb user was created. Now you can run database as a background process:

sudo -i -u couchdb couchdb -b

Ok, fire up your favorite browser and open page http://localhost:5984/ You should see something like: {"couchdb":"Welcome","version":"0.10.0"}

Not very useful, is it? Try going to http://localhost:5984/_utils/ This is Futon, CouchDB admin tool. From here you can create new databases and documents and manage them.

Simple operations by Futon

Create database tryme and some document. Field _id is generated automatically, add two more fields:

firstname: "Pawel"
surname: "Stawicki"

Click "Save document". You can see that field _rev was added automatically. Every change of every document is remembered, and you can always get the old version.

Protection

So there is one document in the database, let's protect it. Go to /etc/couchdb/local.ini and edit [admins] section. Add:

admin = <admin_password>

Password will be automatically hashed after run.

If you want to enable access from any machine, change bind_address from 127.0.0.1 to 0.0.0.0

Operations by RESTful HTTP interface

CouchDB can be accessed by RESTful http interface. Curl is one nice tool for it, so install it if you don't have it already on your system:

sudo apt-get curl

Now issue:

curl http://localhost:5984

You should see known welcome message. Try

curl http://localhost:5984/tryme

and some data about tryme database should appear. We can also list all existing databases:

curl http://localhost:5984/_all_dbs

Ok, all this stuff above you can do by the browser. But you can also create documents and databases in CouchDB by REST interface. Try this:

curl -X PUT http://localhost:5984/tryme/1 -d '{ "firstname": "Leszek", "surname": "Gruchała" }'

You just created a new document. In curly braces is document in JSON format. It's id however is not long stream of characters, but just "1". Much more predictable and repeatable. UUIDs are generally better. CouchDB can generate one for you:

curl http://leonidas:5984/_uuids

It generates one id. If you need more, you can pass "count" parameter:

curl http://leonidas:5984/_uuids?count=3

So we've seen that CouchDB uses JSON as documents format, and we can do operations by HTTP. However, from now on, I'll be using Futon because it is just easier. Futon is CouchDB administration tool (http://localhost:5984/_utils).

Creating view by map/reduce

JSON is JavaScript format, and it allows to put also functions into the document. Map functions (from map/reduce) are useful to create something similar to SQL view.

Let's add another document to our database, with following fields:

firstname: "Leszek"
surname: "Kowalski"

Now choose "Temporary view" from selection box on the right. Here you can write map and reduce functions. For view, map is enough, let's leave reduce empty for now. Create function:

function(doc) {
  if (doc.firstname == 'Leszek') {
    emit(doc.firstname, doc.surname);
  }
}

It filters documents to those with firstname "Leszek" only.

So we can filter documents to only those which interest us. What about joins? Can we get some document, and relevant documents data in one query? It's also possible. Let's add addresses to those guys in database. First, we need to distinguish documents related to persons from the ones related to addresses. For that, add type field to each person, and as value enter "person". Now, we'll need their id's, so copy/paste it somewhere.

Create address as a new document, and add those fields:

type: "address"
person_id: <id_of_person>
city: "New York"

The same for two remaining persons. Put some another city there. Ok, now our map function is a little more complicated:

function(doc) {
  if (doc.type == "person") {
    emit([doc._id, 0], doc);
  } else if (doc.type == "address") {
    emit([doc.person_id, 1], doc);
  }
}

This way we have person always next to her address.

We learn another CouchDB feature here - result of map is always sorted by key. In this case key is 2-element array, in which first element is the person id, and second one is 0 in case of person, 1 in case of her address. It's not like SQL join, we don't get all the data in one document, but still it answers our needs - we have person and it's address.

If we are interested in specific person, we can add parameter to GET request. But first we need to save our view in "desing document". Click "Save As..." button, fill in design document name (e.g. "reader") and view name (e.g. "person-address") and voila.

Design documents are special documents in database for storing views. You can have different design documents e.g. for readers and for writers or administrators. In desing document you can set a lot of stuff telling CouchDB how to render it's content (means how to change it into nice html). If you want to learn more about design documents, look into CouchDB book.

GET parameters for narrowing search

Now when our view is saved, we can use GET request to get it's content, and set parameters to limit what we get to person which we are interested in. Such URL: http://localhost:5984/tryme/_design/reader/_view/person-address?startkey=["1",0]&endkey=["1",1] will return person with _id=1 and it's address documents.

Another useful parameters are:

key - for specific key (not range, like in case of startkey, endkey)
descending=true - by default view is ordered by ascending key, set this parameter to order it descending
group_level - sets reduce level. Default is 0.
group=true - behaves like group_level set to maximum
revs_info=true - lists revisions of specified document. Applicable only to document selected by id, not to view

To understand group_level, we'll need reduce function. First, let's add some more parameters to our persons. Let's add "position" and "salary":

To one person add:

position: "manager"
salary: 5000

To second one:

position: "developer"
salary: 3000

and to third one:

position: "developer"
salary: 3500

Now create new view, with map and reduce functions:

//map:
function(doc) {
  emit([doc.position, doc._id], doc);
}

//reduce:
function(keys, values, rereduce) {
  var salaries = 0;
  for(i = 0; i < values.length; i++) {
    salaries += values[i].salary;
  }
  return salaries;
}

Save this view as "salaries". Now open URL http://leonidas:5984/tryme/_design/bla/_view/salaries and output is:

    {"key":null,"value":11500}

All values are grouped, and key is ignored.

Add parameter: http://leonidas:5984/tryme/_design/bla/_view/salaries?group_level=1 and it returns

{"key":["developer"],"value":6500},
{"key":["manager"],"value":5000}

Values are grouped for first level of key (first element of key array).

group_level=2 returns

{"key":["developer","61fe9c6c226b978f74b76329191806b3"],"value":3000},
{"key":["developer","eb3873f48bb581df13762324b8ec0313"],"value":3500},
{"key":["manager","1"],"value":5000}

Two elements of array are taken into account. In this case, it is the same like group=true, which takes all elements of key array.

What is rereduce for?

As you noticed, reduce function has third parameter rereduce

Result of map function is kept as sorted B-tree. Let's assume we want to summarize some value from each document (tree node). Just like salaries.

First, there are keys and whole nodes (because value of our map function is whole document) passed to reduce function, and rereduce is set to false

Look at the picture, we can reduce branch of the tree into 17, another branch into 4, and third one into 2. Assume each branch has common key at specified group_level (e.g. all people from first branch are developers, second are managers and third are administrators, if group_level is set to 1).

Then reduce function is called again, with following parameters:

function(["developer", "manager", "administrator"], [17, 4, 2], true);

There is common key for whole branch, reduced scalar value for that branch, and rereduce parameter set to true.

Important thing here is to remember that not always values are whole nodes, sometimes it can be already reduced values, and in such case rereduce parameter is helpful to distinguish this situation. In fact, our reduce function wouldn't work for big amount of nodes, because it handles only nodes (only rereduce=false case), not reduced values. To work with a lot of data, we should handle also scalar values:

function(keys, values, rereduce) {
  if (rereduce) {
    return sum(values);
  } else {
    var salaries = 0;
    for(i = 0; i < values.length; i++) {
  }
}

Now it should count and sum all the salaries.

Mercurial installation and remote server interaction by hg-login

noreply@blogger.com (pawelstawicki) — Wed, 18 Nov 2009 20:25:00 +0000

I want to have mercurial on my machine, but I also want to have "central repository" for continuous integration builds. I chosen mercurial because it is fast (like git), and it has good support in NetBeans (at least I heard it has ;)).

To push/pull changes to/from server, I use hg-login. Ssh is also necessary. Ok, so here are the steps:

1. Install mercurial on both client and server.

apt-get install mercurial

2. Create user mercurial on the server, and disable his shell.

sudo adduser mercurial

Edit /etc/shadow and set his password to "*".

3. Go to server (ssh will suffice) and switch to user mercurial.

sudo su - mercurial

4. Go to /home/mercurial and create repositories directory.

cd /home/mercurial
mkdir repositories

5. Create file hg-login in /home/mercurial. Copy script there. Script can be found on http://mercurial.selenic.com/wiki/HgLogin

mcedit hg-login

It can be necessary to change the script. In my case I needed to change /usr/local/bin/hg to /usr/bin/hg. Change to where your hg command is located.

6. Go back to client machine. Generate ssh keys pair for yourself.

ssh-keygen

Add private key to your ssh agent.

ssh-add

This way when you connect by ssh from client, you are automatically authenticated.

7. Go to server again. In /home/mercurial/.ssh create file authorized_keys.

cd /home/mercurial/.ssh
touch authorized_keys

In this file put line:

command="/home/mercurial/hg-login franek",no-port-forwarding,no-X11-forwarding,no-agent-forwarding [key]

franek is user name. Replace [key] with content of your .ssh/id_rsa.pub file from client. Put whole content there (with ssh-rsa in the beginning and <user>@<host> at the end).

8. Create repository in /home/mercurial/repositories/.

cd /home/mercurial/repositories
mkdir myapplication
cd myapplication
hg init

9. Allow franek to access myapplication repository. In /home/mercurial/repositories create file myapplication.allow. In this file list users who has to have access to myapplication repository. In this case franek, so the file looks like this:

franek

10. On client machine, go to directory where your application is: cd <yourdir>/myapplication

11. Create local mercurial repository.

hg init

12. Add all what is in /myapplication directory to local repository.

hg add

13. Commit added files.

hg commit

You'll need to edit commit comment.

14. Push changes to remote repository.

hg push ssh://mercurial@yourserver/myapplication

While creating this recipe, hg-login site was very helpful.

As I mentioned before, I chosen mercurial because of Netbeans. But I don't find Netbeans support of mercurial satisfactory. I am used to eclipse and it's great support for CVS. In eclipse, if there are some changes in CVS and I don't have them (incoming changes), I can see what has changed (file by file diff) before I update my code. I failed to do this in Netbeans with mercurial.