Lotus Shade of Yellow

How to use TLS and SHA-2 certificates in Domino Web Service Consumer

2014-10-23T11:01:00.000+02:00

We have a solution that is used by thousands of public sector workers.
The solution relies heavily on interacting with various web services provided by the tax authority, social welfare and health authorities and others.
So we have a lots of Domino web service consumers and providers that require both SSL and authentication using certificates.
Like many others we were aware of Domino's lack of support for TLS and SHA-2, but the web services providers were not insisting on them, so everything worked fine. Until last Friday, when one after the other turned off support for SSL v3. We could just watch as all our consumers stopped working.
Despite the recent technotes (1418982 and 1687167) stating that the fix will be available in "the next several weeks", we needed a working solution immediately.
We are familiar and have been using reverse proxies before. But it didn't seem they could be used in our case. A reverse proxy is often positioned in front of a server (provider), but in this case our Domino servers were clients (consumers).
But, it turns out that you actually can use reverse proxy even in this use case.
What happens is that our Domino web service consumers first contact the Apache reverse proxy using SSL, but without any authentication. Then Apache contacts the web service provider using the certificate that Domino used previously. Since Apache supports SHA-2, we can use new certificates to authenticate ourselves.
After Apache and web service provider finished handshake (using TLS) and authentication, it is free for the web service consumer in Domino to do its stuff.
This means that Domino doesn't need to do any of the SHA-2 and TLS stuff. It only needs to communicate with the Apache proxy using SSL v3, which it supports.
The setup is easy. You'll need an Apache server (obviously), we installed our in a CentOS virtual machine:

yum install httpd mod_ssl

Once you have Apache installed, open the configuration file in a text editor:

vim /etc/httpd/conf/httpd.conf

Scroll down to the end of file and enter the following:

<VirtualHost *:8443>
    # Turn off forward proxy
    ProxyRequests Off

    # Communication with Domino is using SSL, so we need SSL support   
    SSLEngine On
    SSLCertificateFile /etc/pki/tls/certs/localhost.crt
    SSLCertificateKeyFile /etc/pki/tls/private/localhost.key

    # This is necessary for authentication to work.
    SSLProxyEngine On
    # This is Domino certificate including private key saved as unecrypted pem file.
    SSLProxyMachineCertificateFile /etc/httpd/certs/domino-cert.pem
    # This is list of CA certificates necessary to authenticate the provider.
    SSLProxyCACertificateFile /etc/httpd/certs/provider-cert.pem

    # Redirection rules are in this case very simple - redirect everything that comes
    # to the proxy to the web service provider address.
    ProxyPass / https://ws.provider.com/
    ProxyPassReverse / https://ws.provider.com/

    # Allow only connections from Intranet.
    <Proxy *>
        Order deny,allow
        Deny from all
        Allow from 172.20.20.0/24
    </Proxy>
</VirtualHost>

Just a few things to mention here:

You should be able to use certificate and key installed by default with Apache, as they are only used to secure communication between the Domino and the proxy.
Domino key and certificate must be in unencrypted pem format. Use openssl to convert if necessary. If you should get error message about missing or encrypted private key, open your pem certificate and confirm that it includes RSA in lines -----BEGIN RSA PRIVATE KEY----- and -----END RSA PRIVATE KEY-----.
openssl sometimes generates certificate without the word RSA and then Apache won't be able to use it.

That concludes the Apache configuration. The only thing that remains is to modify the web service consumer - find in your code the line where you set endpoint address, something like
https://ws.provider.com/ws/getTemperature
and change it to
https://proxy.mycompany.com:8443/ws/getTemperature

And that's it. We now have working solution for using Domino web services together with TLS and SHA-2 certificates. And we can calmly wait for IBM to implement support for this in Domino.

A few parting thoughts - the above configuration is using Apache version 2.2.15 and CentOS 6.5. If you are using Debian-based or Red Hat 7 based distribution, the configuration will be somewhat different, but this should put you in the right direction.
I also posted this question on Stack Overflow, you might want to keep an eye on it, maybe there is some even easier and better solution.

Making drag and drop work in XPages - Part 6: Data Attributes

2014-01-05T20:37:00.002+01:00

There is a demo site accompanying this series of posts on drag and drop. The text is mostly the same, but you get working examples on the demo site.

Enabling the key | label functionality

This is the first blog post in this year — and it is the last part of this drag-and-drop series.
So far, we have learned how to create fairly advanced drag-and-drop controls. In this last part we'll learn another technique, which, while not directly related to drag-and-drop, can help in developing applications.
It is often necessary to display one, often more verbose, information to application users, while in the back-end we work with other, more terse and precise, information. For example, we might display a product name to the user, while we prefer to work with the SKU of the product (because it is unique). Or, we display a contact's name and address, but we work with UNID in the background — because UNID makes it easy to get that contact's document from the database.
In other words, we work with key | label pairs.
If you are familiar with the "classic" Notes development, you have probably used key | label pairs for Combobox, Listbox, Dialog list, Checkbox or Radio button fields. Some of the XPages controls, such as Combo Box and List Box, also allow for the use of key | label pairs.
So, how do we use key | label pairs with our drag-and-drop controls? The answer is by using HTML5 data attributes. The HTML5 data attributes allow you to specify application specific attributes that you can assign to HTML tags, while maintaining validity of the HTML markup.
Let's take an example. Assume you are creating an application that allows users to follow stock price of their favourite car manufacturers. You can use a drag-and-drop control for that, but instead of sending the name of the car manufacturers back to the server, you want to send their stock symbols. Something like this:

Here is the basic HTML code:


<div class="sideBySide">
  <div class="left">
    <ul class="source connected">
      <li data-stock-symbol="BMW">BMW</li>
      <li data-stock-symbol="DDFAIF">Daimler</li>
      <li data-stock-symbol="FIADF">Fiat</li>
      <li data-stock-symbol="F">Ford</li>
      <li data-stock-symbol="POAHF">Porsche</li>
      <li data-stock-symbol="TSLA">Tesla</li>
      <li data-stock-symbol="VLKAF">Volkswagen</li>
    </ul>
  </div>
  <div class="right">
    <ul class="target connected">
    </ul>
  </div>
</div>

The majority of the code should be familiar by now. The only new thing is the data-stock-symbol attribute, which, in fact is an HTML5 data attribute.

The data attributes consist of a mandatory prefix data- and a lowercase name. By convention, hyphens are used instead of spaces. The content of a data attribute can be anything that can be represented as a string (such as JSON).

The content can be read and set using three different techniques. For the sake of simplicity, let's take the following example:


<div id="manufacturer" data-stock-symbol="DDFAIF">Daimler</div>

The most natural way would be to use the native HTML5 dataset API. You would use it like this:


var manufacturer = document.getElementById("manufacturer");
// get value
var stock = manufacturer.dataset.stockSymbol;
// set value
manufacturer.dataset.stockSymbol = "DDFAIF";

As you can see, you access the data attribute by stripping the data- prefix, removing hyphens from the attribute name and converting it to camel case, so that data-stock-symbol becomes stockSymbol as in the example above.

Unfortunately, Internet Explorer 10 and below do not support the HTML5 dataset API, so you'll probably want to use one of the other two techniques.

The jQuery data() method will be the best option for the most uses. It is used like this:


var manufacturer = $("#manufacturer");
// get value
var stock = manufacturer.data("stock-symbol");
// set value
manufacturer.data("stock-symbol", "DDFAIF");

Finally, there are the standard JavaScript getAttribute() and setAttribute() methods, that you could use in case you don't have jQuery available:


var manufacturer = document.getElementById("manufacturer");
// get value
var stock = manufacturer.getAttribute("data-stock-symbol");
// set value
manufacturer.setAttribute("data-stock-symbol", "DDFAIF");

If you want to learn more about the HTML5 data attributes, I recommend you start with this blog post.

Applying it to XPages

Now, let's take a look at how to apply data attributes in an XPages application. Once again, let's use the stock-choosing drag-and-drop control.


<ul class="source connected">
  <xp:repeat id="repeat2" rows="30" var="manufacturer" removeRepeat="true">
    <xp:this.value>
      <![CDATA[${javascript:
        return [["BMW", "BMW"], ["Daimler", "DDFAIF"], ["Fiat", "FIADF"], ["Ford", "F"], ["Porsche", "POAHF"], ["Tesla", "TSLA"], ["Volkswagen", "VLKAF"]];
      }]]>
    </xp:this.value>
    <xp:text escape="true" id="sourceLI" value="#{manufacturer[0]}" tagName="li">
      <xp:this.attrs>
        <xp:attr name="data-stock-symbol" value="#{manufacturer[1]}"></xp:attr>
      </xp:this.attrs>
    </xp:text>
  </xp:repeat>
</ul>

The code is very similar to the code we used in our first example.

You should note the following changes:

Once again, a static array is used as a data source for the repeat control. This time, it is a two-dimensional array containing manufacturer's name and stock symbol. In a real application, you would probably use a view and itterate over documents containing necessary data.
Computed text control has been extended with <xp:this.attrs>...</xp:this.attrs> which allows us to define data attributes and set them programmatically.

The JavaScript code for reading data attributes is also very similar to the code we have already used. This is the adjusted code:


var items = [];
$("ul.target").children().each(function() {
  var item = {manufacturer: $(this).data("stock-symbol")};
  items.push(item);
});
var jsonData = JSON.stringify(items);

Instead of reading the text of each li element, we read the value of the data attribute. The rest of the implementation is the same as already described above.

Epilogue

I hope that you have found this blog series useful and that it will help you use drag-and-drop in your applications.

Please let me know if you should have any questions, comments or suggestions.

How to backup your Mercurial server to Amazon S3

2013-12-26T12:17:00.002+01:00

In the last couple of years, I've seen a clear trend towards using established software development methods and techniques when developing Domino applications, such as the use of revision control systems like SVN, Git and Mercurial.

It seems that majority of developers is using distributed revision control systems like Git or Mercurial. When working in a team, developers use a central repository (i.e. server) to share the code. The services like Bitbucket and GitHub can be used for that purpose, but they cost and, when working on a commercial product, there are always privacy and security concerns.

A better solution is to setup your own server. And it is fairly easy to do: a while ago I wrote instructions on how to set up a Mercurial server running on a Red Hat Linux server.

One issue that you have to deal with when running your own server is backup.

One of the simplest, cheapest and most reliable solutions is to use Amazon S3 storage for backups. I have already shown how to use it to back up Domino server, and now I want to show you how to backup your Mercurial server.

First of all, you'll need two things:

the backup script s3backup.sh,
s3cmd utility for uploading files to Amazon S3.

Note: The script currently works only on Red Hat-based distributions. If you are running a Debian-based distribution (e.g. Ubuntu) and are interested in the script, let me know and I'll make necessary modifications.

You can get the script from either the Bitbucket (Mercurial) or GitHub repositories.

You'll find the s3cmd utility at the S3 Tools site, together with installation instructions. Once you have installed the utility, run the configuration by executing:

# s3cmd --configure

Enter your access key and secret key when requested. If you do not plan on encrypting the backups, leave the encryption password blank and accept the default path to GPG. Choose whether you want to use HTTPS (I recommend you answer Yes here). Let the utility test the settings and if everything is OK, save the configuration when asked.

With that out of way, let's configure the script.

First of all: you'll need to run the script as root user, because the script needs permissions to stop and start the HTTP server service.

Put the script in a suitable directory — I keep mine in /root/bin. Make sure the script is executable:

# chmod u+x s3backup.sh
# ll s3backup.sh
-rwxr--r--. 1 root root 4774 Dec 25 15:47 s3backup.sh

Now, open the script in your favourite text editor and set these variables:

SOURCEDIR: path to the directory on the Apache server containing your repositories. In my case it is /var/www/vhosts/hg.cs-computing.com/repos — you specified this location when you installed the Mercurial server, check your installation in case you forgot it. The script will backup everything it finds in this directory.
BACKUPDIR: path to a directory where backup archives will be created and from where they will be uploaded to Amazon S3. Use whatever location you want (providing you have full access to it). You don't need to create this directory, it will be created automatically.
LOGDIR: path to a directory where you want to keep the log files. I keep mine in /var/log/s3backup, but again, use whatever suits you best. This directory will be created automatically as well.
DAYSTOKEEP: number of days to keep the log files. The log files older than DAYSTOKEEP days will be deleted. 14 by default.
HTTPSERVER: name of your HTTP server service. If you are using Apache on Red Hat this will be httpd, which is also the default value.
S3DEST: name of the bucket on Amazon S3 where you want to store backups. You can specify the bucket name like this: s3://your-bucket-name/. If you have and want to use a folder inside the bucket, specify it like this: s3://your-bucket-name/folder-name/. Make sure not to forget the trailing slash. It is mandatory in this case!

To run the script simply execute it:

# cd /root/bin
# ./s3backup.sh

The script works like this:

If the HTTP server is running, it is stopped. I don't know if this step is necessary: it feels safer to first stop the server and then copy the files (in order to avoid copying files being modified by someone else). On the other hand, stopping the server in the middle of a changeset upload may corrupt the repository as well. I am open for any suggestion on this.
Once the HTTP server is stopped, all files (repositories) from the SOURCEDIR are copied to the BACKUPDIR.
As soon as the copy operation is finished, the HTTP server is started again.
The copied repositories are packaged and compressed in a .tar.gz archive. When the archive is created, the copied repositories are deleted.
The archive is uploaded to the specified Amazon S3 bucket. If the upload is successful, the archive is deleted. Otherwise, it is left in the directory and the script will try to upload it again the next time.
The script checks if some of the log files are older than DAYSTOKEEP days and deletes them.

If you specify -v (or --verbose) option, the script will output all log entries to the console as well, which is useful when you run the script manually:

# ./s3backup.sh -v

The recommended way to run the script is by scheduling its execution using cron. To schedule the execution, run the following command:

# crontab -e

This will open (possibly empty) list of cronjobs in your default text editor. Add the following line:

15 1 * * * /path/to/script/s3backup.sh > /dev/null 2>&1

15 1 * * * means that this cronjob runs daily at 01:15. Then, the path to the script follows. Finally, you may choose to log the output of the cronjob to a file, or simply trash it like I did in the above line.

If you want to learn more about the cron, here is a nice introduction.

I plan on updating the script so it can be used more generally—for example, I plan to add option to skip stopping / starting the web server. Then you can use it to backup any folder to Amazon S3. Let me know if you have comments or suggestions.

Making drag and drop work in Xpages - Part 5: Advanced example

2013-08-12T22:12:00.000+02:00

There is a demo site accompanying this series of posts on drag and drop. The text is mostly the same, but you get working examples on the demo site.

Controlling output

If HTML5 Sortable satisfies your needs, you are good to go. But what if you need better control of your drag and drop?

Let's consider the first example. It has two features that are impossible to implement in HTML5 Sortable (without changing its source code):

notice how elements are copied between the lists, instead of moved as in HTML5 Sortable;
notice how you can delete an item from the target list — this is implemented by writing custom code for dropped elements.

And for that you'll need jQuery UI widgets. But, let's start by examining the HTML code for the first example:


<div class="sideBySide">
  <div class="left">
    <ul class="source">
      <li>Alfa Romeo</li>
      <li>Audi</li>
      <li>BMW</li>
      <li>Ford</li>
      <li>Jaguar</li>
      <li>Mercedes</li>
      <li>Porsche</li>
      <li>Tesla</li>
      <li>Volkswagen</li>
      <li>Volvo</li>
    </ul>
  </div>
  <div class="right">
    <ol class="target">
      <li class="placeholder">Drop your favourites here</li>
    </ol>
  </div>
</div>

As you can see, the code is almost identical to the code we used for the HTML5 Sortable example — we no longer use the class connected, we use an ordered instead of an unordered list for the target list and we have a single item in the target list with class placeholder.

As expected, it is the JavaScript code that makes all of the difference:


$(".source li").draggable({
  addClasses: false,
  appendTo: "body",
  helper: "clone"
});

$(".target").droppable({
  addClasses: false,
  activeClass: "listActive",
  accept: ":not(.ui-sortable-helper)",
  drop: function(event, ui) {
    $(this).find(".placeholder").remove();
    var link = $("<a href='#' class='dismiss'>x</a>");
    var list = $("<li></li>").text(ui.draggable.text());
    $(list).append(link);
    $(list).appendTo(this);
    updateValues();
  }
}).sortable({
  items: "li:not(.placeholder)",
  sort: function() {
    $(this).removeClass("listActive");
  },
  update: function() {
    updateValues();
  }
}).on("click", ".dismiss", function(event) {
  event.preventDefault();
  $(this).parent().remove();
  updateValues();
});

I use only a small subset of options and attributes that are available for each widget. Consult the official documentation for the full list.

The first thing (lines 1–5) we need to do is to make list items draggable. We select all line items li having the class of .source and attach draggable(). We then set some options to make draggable() work to our liking:

addClasses We use our own CSS classes, so we set this to false. Set it to true if you want to use jQuery UI CSS styles.
appendTo This defines the bounding box for the control. Most often, you'll leave this at the default setting of body. However, there might be times when you want to limit the area inside which a user can drag elements. Simply set the attribute to #myContainerName or similar.
helper Tells the widget to create a clone (copy) of the element to use in drag and drop operations.

Sometimes, you might encounter issues with overlap/z-index when you set appendTo: "body". If that happens, try to downsize the bounding box by changing the selector to something else, as described above.

Thereafter we have a long code that starts by selecting the target container (in our case an ordered list ol) by using the class selector .target. Then we chain droppable() and sortable() to it. on("click"…) has nothing to do directly with drag and drop and we'll come to it later.

We set the following options for droppable():

addClasses We use our own CSS classes, so we set this to false. Set it to true if you want to use jQuery UI CSS styles.
activeClass This is the class that gets added to your target container, each time you start dragging an element. In our example, it makes the target border extend downwards to highlight the drop zone.
accept This option defines what elements are accepted by the target control. This could be useful if you have multiple drag and drop controls and want to prohibit mixing of elements. At any case, don't remove the value ":not(.ui-sortable-helper)", since it prevents helper elements generated by sortable() from appearing.
drop This function is called when you release the dragged item on the target control. It is here that we create an element to be displayed within the target control. But, first of all (line 12) we remove any element with class .placeholder. In our example, this is the class responsible for showing the list item with the text "Drop your favourites here".Then, we create a variable that holds HTML code for a link that we use to remove a list item (line 13). After that, on line 14, we create another variable to hold HTML code for a list item and attach to it the text from the dragged item (draggable is a jQuery object representing the element being dragged).Finally, we append the link to the list item (line 15) and then append the list item to the parent — our sorted list (line 16). This results in HTML code similar to this:
```
<li>
  Volvo
  <a class="dismiss" href="#">x</a>
</li>
```
Line 17 calls the function updateValues() that we that we created in an earlier post. This function will update the backend each time you drop an element onto the target control. Depending on your design (e.g. you prefer to manually update the backend by clicking a button) you may want to remove this function call.

And this is how we set sortable():

items Allows us to specify which elements inside the target control will be sortable. We only want to exclude the placeholder.
sort This function is called during sorting. The only thing we do here is to remove the class listActive from the sorted element. This class gets unintentionally added because of the interaction with droppable().
update This is the function that get called when the sort order is updated. Here we call the function updateValues(), but once again, it is up to you whether you want to update the backend with each change.

And finally, the on("click"…) function. We append this function to each link in the list item in order to make it possible to remove the list item. We target the links by using the class selector .dismiss (see generated HTML code above) and we override the click behaviour: when user clicks on the link, we first prevent normal link behaviour (line 28) and then we remove the link's parent, i.e. the list item element (line 29). Since we want to send all of the sort changes to the backend immediately, we end the function by calling updateValues() (line 30).

These are the CSS styles specific to this implementation (the others, such as source and target are the same as in the HTML5 Sortable example):


body > li {
  width: 177px;
  margin: 5px;
  padding: 5px;
  -webkit-border-radius: 4px;
  -moz-border-radius: 4px;
  border-radius: 4px;
  text-shadow: 0 1px 0 rgba(255, 255, 255, 0.5);
  list-style-type: none;
  list-style-position: inside;
  border-width: 1px;
  border-style: solid;
  border-color: #ccc !important;
  background-color: #fafafa !important;
  color: #bbb !important;
}
.listActive {
  border: 1px solid #ccc;
  background-color: #fcfcfc;
  padding: 0.5em 0 3em 0 !important;
}
.placeholder {
  list-style-type: none;
  text-align: center;
  font-style: italic;
  border: 1px dashed #ddd !important;
  background-color: #fff !important;
  color: #aaa !important;
}
.dismiss {
  float: right;
  position: relative;
  top: -8px;
  line-height: 20px;
  font-size: 12px;
  font-weight: bold;
  text-decoration: none !important;
  color: #468847;
}

The only really interesting thing here is the first style, body > li. It is used to style the element that is being dragged. While being dragged, this element (li in our case) is a child of the element (body in our case) we stated in the appendTo option (see draggable() above) and this is the only way to target and style it.

Next time

Next time - using data attributes for key | label functionality.

LOT-408 Assessment Test (Notes and Domino 9.0 Application Development Update)

2013-08-06T21:29:00.000+02:00

I have recently taken assessment test LOT-408 IBM Notes and Domino 9.0 Social Edition Application Development Update, just to see what I can expect in the real certification.

I would say that the assessment was as difficult as the 8.5 Application Development Update certification. The focus in the assessment test was heavily on the Extension Library, going into very fine details of the various controls.

There were far too many (for my taste) questions of the type "What menu item do you need to navigate to in order to do this or that?" I hate those kind of questions and I never can answer them. I really see no point in them. I like questions where there is a part of code and I need to find an error in it or complete it.

As I plan to do my update certification by the end of September, I guess I'll have to start browsing those Extension Library controls immediately.

As a side note, I am preparing for Red Hat Certified System Administrator (RHCSA) as well. That exam is completely performance-based, meaning that you perform a variety of tasks on a live system, instead of answering scripted questions. You have access to all help resources that come with the system, but there is no Internet access.

Imagine if the Domino development certification involved writing a CRUD application with some set parameters or controls that you had to use. Much more difficult to grade, but much more fun for us to attend and much more telling about an individual's knowledge.

Making drag and drop work in XPages - Part 4: Ajax and REST

2013-07-30T15:42:00.000+02:00

There is a demo site accompanying this series of posts on drag and drop. The text is mostly the same, but you get working examples on the demo site.

Sending sorted list back to server

Once we have a working drag and drop control, it is time to think about sending the sorted list back to server. In order to do that, we need:

few lines of JavaScript code to read the sorted list and prepare values for sending;
a call to jQuery.ajax() to send the prepared values;
a REST control with PUT method that receives the values and stores them.

I use JSON as content type both for sending request and receiving response from the server. These are simple examples and I could have used plain text as well, but JSON is much more robust and flexible (as you will see when we come to the advanced examples).

Reading list and preparing values

We use jQuery to find our sorted list, read values and prepare them for sending. The code looks like this:


var items = [];
$("ul.target").children().each(function() {
  var item = {manufacturer: $(this).text()};
  items.push(item);
});
var jsonData = JSON.stringify(items);

On line 1, we create an empty array to hold our values.
Then on the line 2, we first find our target list by specifying tag and class (ul.target) and then for each child (that would be our li items) we run a function.
In the function, we create a key/value item by specifying key manufacturer and reading text value of the current li item (line 3). Then we add the item to the items array (line 4).
Lastly, on line 6, we call JSON.stringify() on the array in order to get a valid JSON structure.

Ajax call

We use jQuery.ajax() method to send request with sorted list values to the server and receive response:


$.ajax ({
  url: "dnd.xsp/setfavourites",
  type: "PUT",
  data: jsonData,
  dataType: "json",
  contentType: "application/json; charset=utf-8",
  success: function(){},
  error: function(){}
});

Let's analyze the call:

url: The url of your REST control. In 99.9% of cases, the REST control will be on the same page as your drag and drop controls, in this case dnd.xsp. The remaining part is the name of the PUT method, setfavourites.
type: Type of the REST method, we use PUT (we could have used POST as well, but I think PUT is more appropriate for this kind of updates).
data:Data to be sent in the call - we prepared it earlier and saved in the jsonData variable.
dataType: Type of data that server returns. Our PUT method returns JSON. Note that server must return valid JSON (can be an empty element), otherwise jQuery.ajax() raises error.
contentType: Content type to use when sending request to server. Our PUT method expects JSON.
success: Function to execute if Ajax request complets successfully. During development, you can insert alert() call here to monitor what happens. It is here that calls to Pines notification are made in the first example.
error: Function to call if Ajax request fails. During development, you can insert alert() call here to monitor what happens. It is here that calls to Pines notification are made in the first example.

You'll find detailed information on jQuery.ajax() method and additional parameters on the official site.

REST control

Let's begin by dragging REST Service (Controls > Data Access > REST Service) control to the XPage - I usually insert it towards the end of the page.


<xe:restService id="setfavourites" pathInfo="setfavourites">
  <xe:this.service>
    <xe:customRestService contentType="application/json" requestContentType="application/json" requestVar="rqst">
      <xe:this.doPut>
        <![CDATA[#{javascript:
          if (rqst.length == 0) {
            sessionScope.put("myFavourites", "(empty)");
          } else {
            var favourites:java.util.ArrayList = new java.util.ArrayList;
            for (var i = 0; i < rqst.length; i++) {
              favourites.add(rqst[i].manufacturer);
            }
            sessionScope.put("myFavourites", favourites);
          }
          return {};
        }]]>
      </xe:this.doPut>
    </xe:customRestService>
  </xe:this.service>
</xe:restService>

The first thing to note is the pathInfo attribute. Its value must be the same as the last part (after /) of the url in the jQuery.ajax() call.

The REST Service control comes with some predefined methods, but we use custom REST service so that we can freely specify how it should work. We specify that we want to use JSON both as request and response type and we set requestVar attribute. This attribute will hold the content of the request, i.e. our sorted list values.

Since we want to use PUT method, we write all of the code in the doPut.

The first thing we do in the code is to check the length of the request. If it is empty, we'll put value of (empty) in the sessionScope variable.

If the request is not empty, we first create an empty ArrayList to hold decoded values.

Then, we loop through all elements in the request array. The values in the requestVar attribute are stored in an array of com.ibm.jscript.std.ObjectObject elements, thus the somewhat unusual way of getting its value, as shown on line 11. You'll see its usefullness when we come to the advanced examples.

Do note that the key name (manufacturer) in rqst[i].manufacturer must be the same as the key name we used when creating JSON data: var item = {manufacturer: $(this).text()};.

After we looped through all of the elements, we put the ArrayList in the sessionScope.

Finally, we return en empty JSON element {} since the calling jQuery.ajax() method expects it.

Configuring Domino server

In order to use REST services, you will need to enable Domino Access Services on your Domino server.

Start IBM Domino Administrator and access the server that hosts your XPages application. Click the Configuration tab, then Internet Protocols... > Domino Web Engine:

Scroll to the bottom of the page and find the section Domino Access Services. Under Enables services choose Data. Restart the HTTP task.

Putting it all together

Now we have all elements needed to send sorted list values to the server and use them. But, how do we put everything together?

The REST service is self-sufficient, so simply put it on your XPage.

The client side requires a bit more thought. First of all, it would be beneficial to create a function to hold code for reading list values and creating Ajax calls. You could put this function in a JavaScript library or in the same script block as the code to initialize HTML5 Sortable:


<script type="text/javascript">
  $(function () {
    $(".source, .target").sortable({
      connectWith: ".connected"
    });
  });
  
  function updateValues() {
    var items = [];
    $("ul.target").children().each(function() {
      var item = {manufacturer: $(this).text()};
      items.push(item);
    });
    var jsonData = JSON.stringify(items);      
    $.ajax ({
      url: "dnd.xsp/setfavourites",
      type: "PUT",
      data: jsonData,
      dataType: "json",
      contentType: "application/json; charset=utf-8",
      success: function(){},
      error: function(){}
    });
  };
</script>

The second point to ponder is how often to update the server. There are two alternatives - automatically, each time a drag-drop-sort operation is performed or manually, by clicking a button or link once you finished sorting.

If you want automatic updates, you simply add a function call, like this:


$(function () {
  $(".source, .target").sortable({
    connectWith: ".connected"
  }).bind("sortupdate", function() {
    updateValues();
  });
});

This way, each time you change sort order of the elements, the updateValues() function is called and the server receives new sorted list. Depending on your server, available bandwidth, number of users and other requirements, this could be a totally acceptable way.

On the other hand, if you want to keep server load on minimum and only send values to the server when you are finished dragging, dropping and sorting, you can add a button that calls the updateValues() function:


<xp:button value="Update values" id="updateButton">
  <xp:eventHandler event="onclick" submit="false">
    <xp:this.script>
      <![CDATA[
        updateValues();
      ]]>
    </xp:this.script>
  </xp:eventHandler>
</xp:button>

So, now we know how to create a simple, yet fully functional, drag and drop control. We'll add a bit more features in the advanced examples.

Next time

Next time - advanced examples.

XPages the Scrum way

2013-07-30T08:46:00.000+02:00

A picture of our Scrum working area. Still early in the sprint...

Via +Boris Paukovic

Making drag and drop work in XPages - Part 3: Coding drag-and-drop

2013-07-29T10:09:00.001+02:00

There is a demo site accompanying this series of posts on drag and drop. The text is mostly the same, but you get working examples on the demo site.

Coding drag and drop

The basic examples use HTML5 Sortable as it is easier to use. We'll deal with jQuery UI widgets when we get to more advanced examples.

Implementation of drag and drop is usually done using unordered lists, ul, but you could use almost any other HTML element.

There is a simple example on the demo site, with two unordered lists. You simply grab one of the elements from the left-hand side list and drop it on the placeholder on the right-hand side.

Once the element is on the right-hand side, you can change its position relative to the other elements. You can also grab it and drop it back onto the source list on the left-hand side, thus effectively removing it from the target list.

Let's take a look at the code. The HTML code is nothing special:


<div class="sideBySide">
  <div class="left">
    <ul class="source connected">
      <li>Alfa Romeo</li>
      <li>Audi</li>
      <li>BMW</li>
      <li>Ford</li>
      <li>Jaguar</li>
      <li>Mercedes</li>
      <li>Porsche</li>
      <li>Tesla</li>
      <li>Volkswagen</li>
      <li>Volvo</li>
    </ul>
  </div>
  <div class="right">
    <ul class="target connected">
    </ul>
  </div>
</div>

We can ignore div tags as they are only used to position lists. We then have two unordered lists, one with some items, and the other empty. The important thing to note here are the classes attached to these lists: source, target and connected.

JavaScript code required to make this example work is extremely simple:


<script type="text/javascript">
  $(function () {
    $(".source, .target").sortable({
      connectWith: ".connected"
    });
  });
</script>

On line 3 we select all HTML elements with class source and target (i.e. our unordered lists) and attach sortable() to them which enables drag, drop and sort functionality.

On line 4, we also pass connectWith: ".connected" option to the sortable() which enables dragging and dropping between elements of class connected .

Finally, this is the CSS used to style lists in the above example:


ul.source, ul.target {
  min-height: 50px;
  margin: 0px 25px 10px 0px;
  padding: 2px;
  border-width: 1px;
  border-style: solid;
  -webkit-border-radius: 3px;
  -moz-border-radius: 3px;
  border-radius: 3px;
  list-style-type: none;
  list-style-position: inside;
}
ul.source {
  border-color: #f8e0b1;
}
ul.target {
  border-color: #add38d;
}
.source li, .target li {
  margin: 5px;
  padding: 5px;
  -webkit-border-radius: 4px;
  -moz-border-radius: 4px;
  border-radius: 4px;
  text-shadow: 0 1px 0 rgba(255, 255, 255, 0.5);
}
.source li {
  background-color: #fcf8e3;
  border: 1px solid #fbeed5;
  color: #c09853;
}
.target li {
  background-color: #ebf5e6;
  border: 1px solid #d6e9c6;
  color: #468847;
}
.sortable-dragging {
  border-color: #ccc !important;
  background-color: #fafafa !important;
  color: #bbb !important;
}
.sortable-placeholder {
  height: 40px;
}
.source .sortable-placeholder {
  border: 2px dashed #f8e0b1 !important;
  background-color: #fefcf5 !important;
}
.target .sortable-placeholder {
  border: 2px dashed #add38d !important;
  background-color: #fefcf5 !important;
}

You can choose your own class names instead of source and target, simply replace them in the HTML and the JavaScript code.

As you can see, there are no styles defined for the connected class, as it is only used to tell the HTML5 Sortable code which lists to connect.

There are also two class names, sortable-dragging and sortable-placeholder, which are hard-coded in the HTML5 Sortable. These two classes are used for styling items in dragging state and drop placeholders.

Making it XPages friendly

Static lists are great for examples, but I doubt you'll find them useful in your XPages application.

You would probably want to read list items from a view or perhaps some field. The easiest way to implement it is by using the Repeat control, similar to this:


<ul class="source connected">
  <xp:repeat id="repeat1" rows="30" var="manufacturer" removeRepeat="true">
    <xp:this.value>
      <![CDATA[${javascript:
        return ["Alfa Romeo", "Audi", "BMW", "Jaguar", "Mercedes", "Porsche", "Tesla", "Volkswagen", "Volvo"];
      }]]>
    </xp:this.value>
    <xp:text escape="true" id="sourceLI" value="#{manufacturer}" tagName="li"></xp:text>
  </xp:repeat>
</ul>

I am using a static array as data source for the repeat - obviously, you would use your application specific code instead.

Two things to note here:

Use removeRepeat="true" in the repeat control in order to remove div tag that the repeat control generates.
Use tagName="li" in the computed text control to make it output li tag, instead of the usual span.

All of the JavaScript code should be at the end of your XPage, like this:


<xp:view>
  <!-- rest of the code -->
  <script type="text/javascript" src="js/jquery-1.9.1.min.js"></script>
  <script type="text/javascript" src="js/jquery.sortable.min.js"></script>
  <script type="text/javascript">
    $(function () {
      $(".source, .target").sortable({
        connectWith: ".connected"
      });
    });
  </script>
</xp:view>

Next time

Next time - REST and Ajax.

Making drag and drop work in XPages - Part 2: Prerequisites

2013-07-29T10:09:00.000+02:00

There is a demo site accompanying this series of posts on drag and drop. The text is mostly the same, but you get working examples on the demo site.

Prerequisites

jQuery

Regardless of what plugin you choose, you will need jQuery. In case that you are using Twitter Bootstrap, you already have it. Otherwise, go and download it. Unless you plan to do jQuery development, download the compressed (production) version.

The fact that you are using jQuery does not prevent you from using OneUI or any other Dojo-based framework. In fact, my wellness center application is a OneUI-based application.

I usually put my resources in the WebContent folder. If you want to do the same, simply create a new folder under the WebContent folder and put (drag and drop works fine) the downloaded jQuery library there.

Before you use the jQuery library, you need to include it in your XPages. How you do that depends on whether you plan to use it on one (few) pages or in a whole application. To include it in a single page, simply use the script tag:


<script type="text/javascript" src="js/jquery-1.9.1.min.js"></script>

Alternatively, for using it application-wide, add it to your theme file:


<resource>
  <content-type>application/x-javascript</content-type>
  <href>js/jquery-1.9.1.min.js</href>
</resource>

Plug-ins

The plug-ins provide drag and drop, as well as sort functionality. I have tested two different approaches here, which both work very well and can fulfill all imaginable requirements.

HTML5 Sortable The first plug-in is HTML5 Sortable. This is a very small (less than 2 kB minimized) and fast plug-in that utilizes the native HTML5 drag and drop API.
jQuery UI The second plug-in is actually a collection of three different jQuery UI widgets: Draggable, Dropable and Sortable. These widgets provide considerable configurability and extensibility, but come at the price of being significantly larger than the HTML5 Sortable.

Which plug-in you choose will depend on your exact needs - I'll explain the main differences when we come to practical examples.

Installation

First of all, you need to download the plug-ins.

It is easy for HTML5 Sortable: download it, extract it and put the minimized version (jquery.sortable.min.js) in your WebContent/js (or whatever you named it) folder - just like you did with the jQuery library.

It is a bit more work for jQuery UI widgets. jQuery UI is huge and you'll want to choose only the parts that you need. So, head over to the download page. Then select as follows:

Under Version: select 1.10.3 (latest version)
Under UI Core: select Core, Widget and Mouse
Under Interactions: select Draggable, Dropable and Sortable
Under Widgets: deselect all
Under Effects: deselect all
Under Theme: select No Theme

I have chosen not to include CSS styles, as I will provide my own. If you want to choose default jQuery UI styles, make appropriate selection under Theme.

Click the Download button to save your customized library. Extract the files and navigate to the jquery-ui-1.10.3.custom\js folder and then drag and drop the minimized version (jquery-ui-1.10.3.custom.min.js) onto your WebContent/js folder.

Finally, you need to include the plug-ins in your XPage. Presumably, you will use this functionality on a limited number of XPages, so it is best to include them using the script tag, similar to this:


<script type="text/javascript" src="js/jquery-ui-1.10.3.custom.min.js"></script>

Heads up! Depending on how you import libraries (using script tag or via themes) and whether you checked Use runtime optimized JavaScript and CSS resources in the XPage Properties Editor, you might encounter problems running your JavaScript, which (I guess) happen because of namespace collisions. Should this happen to you, try changing the way you import libraries and/or toggling the runtime optimization option.

Extension library

You will also need the Extension Library in order to use the REST Service control. I guess that mostly everyone has the Extension Library installed and configured by now, but in case that you don't, you can download it at the OpenNTF.org site.

Finally, in order to make everything work as it should, you'll have to write some client and server JavaScript code of your own. It is nothing difficult and I'll try to explain as much as possible using practical examples. Hopefully, that will be enough to get you started.

Next time

Next time - coding drag and drop.

Making drag and drop work in XPages - Part 1: Introduction

2013-07-29T10:07:00.000+02:00

There is a demo site accompanying this series of posts on drag and drop. The text is mostly the same, but you get working examples on the demo site.

A bit of introduction

Drag & drop and related sort function have been around since the web became dynamic. I have seen them used to solve some tricky UI problems, but also misused to the point of making me cry.

My first serious encounter with drag & drop was last year when I was working on an application for a wellness center.

As a part of her individual programme, each client would receive customized list of exercises. A trainer would choose from hundreds of available exercises: first sorting them by category (1), then picking an individual exercise and adding it to the list (2) where it could be sorted (3), as shown in this screenshot of the old application written in Delphi.

At that time, my first idea was to use a value picker and a Dojo List Text Box. The value picker would allow me to search and select exercises to be displayed in the List Text Box. However, that approach was rejected because the value picker would output exercises in alphabetic order and there was no possibility to sort the exercises in the List Text Box.

Next in the line was drag and drop, which I felt could solve the problem. But, despite the fact that some XPages resources related to drag and drop existed (like Keith's and Jake's series), my XPages and JavaScript knowledge (as well as the release schedule) at that time made this a no-go.

So, I ended up implementing a half-baked solution using a name picker (because it outputs the values in the order they were selected) and a List Text Box. It was hardly an elegant solution, but it was good enough.

A year later, it was time to revisit this part of the application and finally make it work as it should have been from the beginning. Drag and drop was still my favourite solution, so I wanted to check what had happened since our last encounter.

One new resource that caught my attention was this Marky Roden's post on implementing native HTML5 drag and drop. That article put me in a general direction and after some additional work I ended up with a fully working solution.

This is a screenshot of the new implementation. On the left-hand side, a Dojo Accordion is used to allow the trainer to choose a category and display exercises in that category. Selected exercise is simply drag-and-dropped onto the right-hand side. Once there, the exercises can be sorted or removed.

Result

Instead of writing my own drag and drop function, I decided to use two different, already existing, jQuery plugins. One is small, efficient and fast, albeit a bit restricted. The other one offers all bells and whistles, but is larger and slower. Depending on my needs, I can choose one or the other. As I am increasingly using Twitter Bootstrap in my XPages applications, jQuery represents no additional overhead.

The communication between the drag and drop control and the XPages application on the server is done using Ajax and REST. jQuery has built-in Ajax method and the REST Service from the Extension Library takes care of the server side.

Head over to the demo site to see an example of a finished drag and drop control.

That is an elaborate example that uses jQuery UI Draggable, Dropable and Sortable widgets. It also uses Pines notifications to display status of Ajax calls.

Each time you drop, sort or delete an item, an Ajax call is generated containing the current list values. The REST PUT method on the server reads the values and sets a sessionScope variable. If the PUT call is successful, the Ajax method automatically refreshes a computed field that displays the sessionScope variable. Should the PUT call fail, the Ajax method generates an error notification.

Next time

Next time - prerequisites.

XPages, Bootstrap and links pretending to be buttons

2013-05-06T12:27:00.000+02:00

In a recent post, I wrote about how I used Twitter Bootstrap to style an XPages application and what challenges I faced when using Button core control.
Twitter Bootstrap is very flexible and it allows you to visually turn a link into a button. All you need to do is to apply the same style classes to a link, as you would do with a button. For example, this:

<a class="btn" href="#">Approve</a>

will be rendered as this:

And, just as easily as with the buttons, you can add additional styles or icons:

<a class="btn btn-primary" href="#"><i class="icon-upload-alt"></i> Upload</a>

The result is this button:

The basic styling can be easily applied using the XPages Link core control. The code corresponding to the first example looks like this:

<xp:link text="Approve" id="buttonLink1" styleClass="btn"></xp:link>

When it comes to adding icons, there are two choices. The first is to use an approach similar to the one we used with buttons: use the Computed Field core control to inject the necessary <i> tag:

<xp:link text=" Upload" id="buttonLink2" styleClass="btn btn-primary">
  <xp:text id="uploadIcon" tagName="i" styleClass="icon-upload-alt"></xp:text>
</xp:link>

But, there is also another approach that, depending on your preferences and requirement, you might find easier. The Link core control contains escape property which, when set to false enables us to output the necessary <i> tag as a part of the link's text property:

<xp:link escape="false" id="link1" styleClass="btn btn-danger">
  <xp:this.text>
    <![CDATA[<i class='icon-remove'></i> Delete]]>
  </xp:this.text>
</xp:link>

The second approach might be especially handy if you want to dynamically change the look of the button, as it will require less code.

You can see both approaches in the demo application.

Google+ Comments now turned off on this blog

2013-05-04T16:39:00.002+02:00

For some time I had Google+ comments enabled on this blog. I thought it would be a good idea, having both blog and Google+ conversation collected in one place. However, I failed to understand that only those with Google+ accounts would be allowed to comment.

This mistake has now been fixed - I switched back to the old comments system that allows for anonymous comments as well.

XPages, Twitter Bootstrap and button decorations

2013-05-03T12:52:00.000+02:00

This is a follow-up post to my question at the Stack Overflow.

I am trying to switch completely to Twitter Bootstrap for our XPages applications. I find it easy to work with, with lean and understandable CSS that is easy to modify should the need arise. There are so many different resources on the Internet dedicated to Bootstrap, including the marvelous {wrap}bootstrap with ready-to-use themes.

Using the Boostrap is easy as long as you use static elements, like <div> or <p>. It gets a little tougher when you start using XPages core controls, like Button and Link.

I'll show you on an example. This is the static code you would use when defining a button:

<button class="btn" type="button">Approve</button>

This code results in a button similar to this:

The button can easily be styled by adding additional classes like this:

<button class="btn btn-success btn-large" type="button">Approve</button>

which turns into:

These styles are easily repeatable using the XPages Button core control. You simply specify the necessary class(es), like this:

<xp:button value="Approve" id="button1" styleClass="btn btn-success btn-large">
</xp:button>

Now, the really neat feature is the easy way of adding decorations (icons) to buttons. So for example, if you want to add a tick mark to your button you use the following code:

<button class="btn" type="button"><i class="icon-ok"></i> Approve</button>

resulting in this:

But, when I tried using this technique with the button core control, I realized that I didn't know how to inject the necessary <i> tag inside the generated button code. At first, I tried all possible and impossible ways of calculating the value of the button. But it wouldn't work as the html code would always get escaped.
I then came up with a client-side script that would execute on page load, find the button and inject the necessary tags (see my Stack Overflow question). But this was not an elegant solution and it would be difficult to programatically change the icon.
Thankfully, +Stephan Wissel showed that I could easily include html tags in the XPages markup. So, the solution to the icon problem was as easy as this:

<xp:button value="Approve" id="button1" styleClass="btn">
    <i class="icon-ok"></i>
</xp:button>

But wait, there is more! In XPages you can use property tagName to control what html tag is output when the control is rendered. So, I used the Computed Field core control and told it to output the <i> tag:

<xp:button value="Approve" id="button1" styleClass="btn">
    <xp:text id="buttonIcon1" tagName="i" styleClass="icon-ok"></xp:text>
</xp:button>

Why would you do that? Well, now you can easily change the value of the styleClass property and thus the icon. Which means that you can completely change the look of the button, based on some condition in the application.

This approach is easy to use and completely controllable using the server-side script. I created a small demo application where you can see this technique in action.

Instead of the default Glyphicons, I use Font Awesome for displaying icons. Font Awesome has more symbols, is smaller and easier to use. And it uses exactly the same style names as Bootstrap, so you won't need to change your existing code. Warmly recommended!

And lastly, to demonstrate why I needed this in first place: we are planning an event and I wanted to ask our customers and partners where we should organize it. So I created a simple XPage and provided four alternatives that they can choose from. It is made using Twitter Bootstrap and the techniques described in this blog post. I haven't translated it, but you'll get the idea. Feel free to submit your choices!

Update

I just found another way to display icons using the Link core control. When you set escape="false" you can inject the i tag directly into the text property. Depending on your needs, this could be easier than using computed fields. But, there are caveats. I'll update the demo database and write a blog post about this.

User's login ID, LDAP and Sametime

2013-04-24T23:09:00.002+02:00

Or, when too much RTFM is not a good thing.

Each time we perform a new installation, we try to make it better than the one before. We tighten security, we make it more robust and scalable. We read manuals and check best practices an extra time.

When we were installing Sametime at a new customer some time ago, we did just that - we read the documentation one extra time.

And we noticed that the documentation recommended use of a unique, non-changing, identifier as the user's login ID. For example, the section Specifying a user's login ID states:

To avoid running the name change tool in the future, you can select an LDAP attribute that is not likely to change when users change their name or relocate. Here are examples of stable attributes in some well-known LDAP servers:

IBM Directory Server: ibm-entryUUID
Domino® LDAP: dominounid
Novell Directory Server (NDS): guid
SunOne: nsuniqueid
Active Directory: objectGUID

So, we did exactly as recommended and changed the login ID to dominounid. And everything was fine with the Sametime client (both embedded and standalone), but we could no longer log in using the web client (e.g. via ST Proxy). Attempts to log in using the web client would time out, but the funny thing was that if the user first logged in to the iNotes and then opened the web client, it would work.

So, we contacted the IBM support and started checking all the standard things - SSO configuration in both WAS and Domino, LDAP settings, trusted IPs... And everything was fine. The PMR got escalated all the way until it reached the SWAT (cool name, isn't it?). And then, after going through a massive amount of logs, they found the cause - it was our change of the login ID to dominounid.

The thing is, if you change the login ID, you also need to change the search filter used when resolving a user name to a distinguished name. This can be changed in STSC or directly in stconfig.nsf. So instead of
(&(objectclass=inetOrgPerson)(|(mail=%s)(cn=%s)(uid=%s)))
it should be:
(&(objectclass=inetOrgPerson)(|(mail=%s)(cn=%s)(uid=%s)(dominounid=%s)))

After this change, everything started working as it should. Such a small change, so difficult to find (it took over a month to get this sorted!), so big an impact. I felt quite dumb after this was over: "Did I miss this in the manual?" I double checked, but I couldn't find a reference to the search filter change anywhere.

The moral of this story is thus: reading too much manual is sometimes as bad as not reading it at all. Especially if the manual is missing some very important information. :-)

Connections Business Cards in Notes 9.0

2013-04-23T23:24:00.001+02:00

I really don't know how old this is - I only noticed it yesterday.

When one hovers over a name in Notes 9.0, a Connections business card appears:

We haven't done any integration between our Domino / Connections environments, apart for installing Connections plug-ins for Notes. So I guess that this function is a part of the plug-ins.

Really nice touch, it simply works out of box. No manual configuration of any sort. I wish I could say the same for other aspects of Domino / Connections interaction, such as Embedded Experience and Activity Streams.

Installing Traveler 9.0 on Linux - watch out for this inconsistency

2013-04-23T21:29:00.002+02:00

I don't know if I am the only one to encounter this. I'll share it, maybe it saves some headache.

Prior to Domino 9.0, the default program directory on Linux was:

/opt/ibm/lotus

With version 9.0, it has been changed, as seen in this excerpt from the installation procedure:

Please specify a directory or press Enter to accept the default directory.
Program Files Directory Name [/opt/ibm/domino]

However, it seems that Traveler 9.0 is not completely aware of this change. It will install just fine, but when you try to run it, it will crash with all sorts of nasty errors, such as:


HTTP JVM: IBM Notes Traveler: cannot load allpackages - could not find /local/data/web1/../../../traveler.jar
HTTP JVM: Unable to load /opt/ibm/lotus/notes/90000/linux/travelerca. For more detailed information, please consult error-log-0.xml located in /local/data/web1/domino/workspace/logs
HTTP JVM: java.lang.UnsatisfiedLinkError: com/lotus/sync/dca/access/NativeAccess.jniInitThread()I
HTTP JVM:  at com.lotus.sync.dca.access.NativeAccess.initThread(NativeAccess.java:1133)
HTTP JVM:  at com.lotus.sync.dca.dispatch.DispatchThreadData.ensureJNIThreadInited(DispatchThreadData.java:478)

The second error is actually the clue to the cause and to the solution - for some reason, the Traveler is looking for travelerca (actually libtravelerca.so) in /opt/ibm/lotus - the pre-9.0 installation location.

It is an easy one to fix though, one just needs to make a symbolic link (as root), like this :

# cd /opt/ibm
# ln -s domino lotus
# ll
total 4
drwxr-xr-x 4 root root 4096 Mar 27 00:34 domino
lrwxrwxrwx 1 root root    6 Apr 23 20:56 lotus -> domino

I think this problem only occurs in new Domino 9.0 installations. If you upgrade from 8.5.x, the program directory is /opt/ibm/lotus and this error does not happen.

As a side note, if you are using +Daniel Nashed's start/stop scripts, you'll also need to edit them as they have /opt/ibm/lotus hard-coded in few places.

A Success Story - IBM, Red Hat (and Domino)

2013-01-15T10:15:00.000+01:00

I feel proud to share with you this success story that IBM has recently published. It details the work that we (COMPUTING systems d.o.o.) did on a complete infrastructure overhaul for one of our customers.

While it focuses on IBM System x servers, System Storage and Red Hat Enterprise Virtualization, it does mention Domino / Notes as well.

Sometime this year, when the complete collaboration infrastructure - including Notes / Domino, Sametime and Connections - is in place and operational, I hope that we shall be able to write a success story about it as well.

IBM Notes / Domino 9.0 Beta - initial impressions

2013-01-08T14:39:00.000+01:00

I've spent the last few weeks playing with and testing Notes / Domino 9 Social Edition Beta. In the last post, I highlighted a few issues that I saw as possibly being problematic. I am glad to say that my fears have been unfounded. This is what I have discovered so far:

Domino 64-bit on Linux - works like a charm. I installed one Domino 64-bit server from the scratch and I even upgraded an existing 32-bit 8.5.3 FP3 server. I don't know what official recommendations will be regarding upgrading 32-bit servers, but everything seems to work perfectly. Even though the Domino is 64-bit, you'll still need a few 32-bit libraries (I think they are for GUI installer). The good thing is that installer is very kind and will list all of the missing dependencies.
Symphony - I haven't played with the widget that's supposed to enable Notes applications to use stand-alone Symphony, but judging by the comments elsewhere it doesn't work very well (yet). I have gone through our applications and only one uses the LotusScript Symphony API and is therefore affected by the removal of the embedded Symphony. The rest uses the UNO API and works perfectly with the latest version of the OpenOffice. Some templates will need minor adjustments, but no code change will be necessary. Which is very positive.
XPages Extension Library is integrated in the product and it works flawlessly. I disabled OSGi bundles that I set up previously on the 8.5.3 server and everything just worked. The Domino Designer works properly and recognizes and builds all code written in the previous versions.
Activity Stream and Embedded Experience - this is the most exciting part. In a way, it embeds a part of Connections in the Notes client. You get to see updates from your network and you can interact with some posts directly - I assume that third-party applications will be able to post to the activity stream as well. (I already envision our support application posting tickets or our time tracker posting leave requests to the stream. And users acting on them directly from the stream.)
At the moment, the integration is complicated and hard to get working. I hope that some parts of the integration will be simplified and the rest properly described in the documentation by the time version 9 is released.
Upgrading - I upgraded few servers from 8.5.3 and there is no need to waste words: it just works, without a single problem. Pure excellence!

Those were the initial findings, more testing to come...

Notes / Domino 9.0, Linux 64-bit, Symphony and other stuff

2012-12-18T10:08:00.001+01:00

So, the first ever public beta of a Lotus Notes / Domino release is upon us. Like many others, I rushed to download and install it.

It became obvious immediately that this update will require much more thought, planning and testing than the previous ones. Here are some of the reasons:

Domino 64-bit on Linux - Linux users now have access to a 64-bit version of the Domino server. That is good - no more installing 32-bit libraries, fiddling with memory settings and the like. But, can we do an in-place upgrade of existing 32-bit Domino servers? Are there any restriction to the 64-bit version, which seems to be the case with other OSes?
Supported Linux releases - I tried upgrading one of our Domino servers running on Red Hat 5.7. The installation went smoothly, but the Domino wouldn't run. It requires glibc version 2.7 and RHEL 5.7 ships with glibc 2.5. Which means that I must upgrade to RHEL 6.x that ships with glibc 2.12. Which, in turn, means that when the 9.0 is released there will be quite a lot Linux servers needing upgrade, which may or may not be possible.
Symphony - Symphony is no longer included with the Notes client. As a matter of fact, IBM no longer does any new development of the Symphony, the efforts are instead switched to fully supporting the Apache OpenOffice. We, like many others, have invested in the Symphony: we are using it as the only office suite and we are counting on it in numerous applications for creating reports and outputting (nicely) formatted PDFs. According to a blog post by John Head, there will be a plug-in that will enable applications to seamlessly use the stand-alone version of the Symphony. However, at this moment it it unclear whether this plug-in will be officially supported by IBM and how well it will work. And we'll still need to use the Symphony, which no longer is being developed.
So, for our part, we will bid farewell to the Symphony when Notes / Domino 9.0 is released. We'll switch to either OpenOffice or LibreOffice. And, during this next few months we'll perform a major overhaul of our applications and get rid of the Symphony-related code. Instead of it we plan to use the Apache ODF Toolkit and the Apache POI, which will allow us to create documents on server (adding an additional bonus of being reusable in our XPages applications).
Sametime - at this point it is still unclear how Sametime, Quickr and other products will be affected by upgrade to version 9.0. Judging by the beta forums, the Sametime seems to work. But, the question is whether it is going to be supported and on which platforms: e.g. Sametime Community Server 8.5.2 IFR1 is officially supported on RHEL 5.4, but not on RHEL 6.x that is required for Domino 9.0.
XPages Extension Library - the Extension Library is now included with the Notes / Domino. So far, we used the "stand-alone" version of the Extension Library. The switch to the integrated version will probably be painless, but there is still need to clean the servers and remove the OSGi bundles. As a side note - how and how often is this integrated version updated? Can it be updated independently or must we wait for fix packs for updates?

So, at the moment there are lot of questions. Some of them could be answered immediately if IBM had produced any kind of documentation. I know this is a beta, but I still expect (draft) documentation to be available. In this way, there will be more job for us - but in the end, I am sure, we'll all end up running version 9.0 :-)

Backup Domino to Amazon S3 Cloud Storage

2012-11-30T09:26:00.001+01:00

I can't believe it's been 9 months since the last post!

A lot has happened during that time: there has been a lot of XPages programming, great time time with Scrum, frustration with source code management and administration of some old (Sametime) and some new (Connections) applications. More about thatt some other time.

What I wanted to share with you this time is a way to cheaply and reliably backup your Domino server(s) to cloud.

The Need

We have a few small customers running Domino. They run it on Microsoft Windows servers, use it for mail and a custom application or two. The hardware is often redundant and reliable, but they still want to backup their Domino environment. For, no matter how your hardware is reliable, it won't help when you delete that important e-mail (and it expires from the Trash).

Backup solutions that are able to backup Domino online are expensive and require expensive supporting hardware (dedicated backup servers, tape libraries). Nothing a small customer is ready to purchase.

In the past, we used to write batch files to shut-down server, wait predefined time for a server to stop, than copy files from the Domino data directory to a network share or external disk and then start server again. It worked, but there is a better solution.

The Solution

The solution we are using now involves using Windows PowerShell and Amazon S3 Storage.

PowerShell is an extremely powerful administration tool and makes tasks such as starting and stopping services (such as Domino server) and invoking other applications reliable and easy.

Backup to Amazon S3 has always been an option, but here in Bosnia until recently, the cost of obtaining sufficient upload speeds precluded any practical use of this service.

Thankfully, our ISPs have shown some mercy and, while still snail-paced comparing to the developed countries, the upload speeds are high enough to finish majority of backup uploads during a nightly backup window.

Another important development is the introduction of the Amazon Glacier storage, costing next to nothing. The catch with the Glacier is that your files are not immediately available for download / restore as is the case with the standard S3 storage. Instead, you need to wait some 3 - 5 hours for your data to get ready. The data is still covered by the astonishing 99.999999999% durability guarantee, just like the standard S3 storage.

Amazon has provided seamless integration between S3 and Glacier, so you can simply and completely transparently move data from S3 to Glacier. We usually recommend our customers to keep 1 - 2 weeks of backups in the S3 (for quick and easy restores) and move older backups to Glacier. It is also easy to configure automatic data expiration, so you don't end up paying for obsolete data.

All of our customers fall in the first 1 TB tier, so according to the just reduced prices, they pay only $0.095 per GB per month for standard S3 storage and $0.010 per GB per month for Glacier storage (there is also the Reduced Redundancy Storage at $0.076 per GB per month, which is similar to the standard S3 storage but with "only" 99.99% durability). So a customer with some 100 GB of backup will pay no more than $10 per month. And that is cheap!

I have written a PowerShell script that automates the whole process: it stops the Domino server, copies predefined files from the Domino data directory to a temporary directory, starts the Domino server, zips the files and uploads them to the S3 storage.

By using the PowerShell we can precisely detect when Domino server shuts down so file copy operation can commence immediately afterwards. As soon as files are copied, the Domino server is started again. This means that the downtime is minimal, usually less than 2 minutes.

Once the Domino server is started, the script proceeds with zipping (we use 7-Zip at maximum compression setting) and uploading the data.

The Implementation

If you are interested in this solution, you will need the script, which you can get from the GitHub (I would think that the script is too big for XSnippets, what do you think?). I have also attached the current version to this post.

Apart from the script, you will also need:

PowerShell 2.0 environment. If you are using Windows 7 or Windows Server 2008 R2 you already have it. For older systems head over to Microsoft and download the Windows Management Framework Core for your platform.
7-Zip, which is used for compressing the backup files. The free download is available at 7-Zip.org.
S3Sync, which is used to upload files to Amazon S3 Storage service. S3Sync is available for free download at SprightlySoft's website. No installation is necessary, simply extract it to a suitable location (remember the location, you will need to enter it into the script).
Amazon S3 account, if you don't have one. Simply head to the Amazon AWS site and sign up for the account.There is a Free Usage Tier that will give you 5 GB of free S3 storage, so you can test it for free.

Once you have created your account and logged in into the AWS console, choose to manage S3 storage and create a bucket. The bucket is a place where your files will go. You can create multiple buckets if they are used for different things (you can serve a web site directly from a bucket) or if you want them to have different lifecycle policy.

In the bucket, you can create folders to further organize things.

If you select a bucket and click Actions > Properties you will see properties for that bucket and be able to set, among other things, lifecycle policies:

Click Add rule and you will be able to specify when the data is to be moved to Glacier and if and when it is going to be removed. You can specify the time as either an exact date or as number of days since the creation of an object. The rule can apply to the entire bucket or a single folder:

Lastly, you will need your security credentials for accessing your S3 storage. Click on your name in the upper right corner and choose Security Credentials:

You will be presented with a page where you can manage all sorts of security credentials for accessing AWS. You are interested in Access Keys - copy your Access Key ID and Secret Access Key, you will need them in the script.

You can use AWS console for managing and accessing your files, or you can use dedicated a application like CloudBerry's free Explorer for Amazon S3, that will let you do more things easily.

And that is all that you need to know about the Amazon S3 for this purpose. Refer to the online help (it is good) if you want to know more, or let me know in the comments and I'll try to help.

The script

The script is actually fairly easy. The whole work is done by some ten lines of code. The rest of the code is there to provide meaningful logging - it will let you know how long it took to shut down your Domino server, what files have been copied and how long it took to upload the files. It will also let you know about eventual problems.

Before you start using the script, you will need to set some variables, they are all located at the beginning of the script. Among other things you will need to set Domino server service name, location of the data folder and what files to back-up. You will also need to set your Amazon S3 details. Check also that the default locations for 7-Zip and S3Sync are valid for your environment.

Please make sure to end all Windows directory paths with a backslash (). Also, if you are specifying an S3 folder, it needs to end with a forward slash (/).

Currently, the script compresses the files with 7-Zip algorithm and uses the strongest compression. If that is not suitable for your needs, let me know and I'll let you know what to change to set one of the supported algorithms.

The upload to S3 is secured by SSL, but the 7-Zip file is not encrypted. 7-Zip has support for strong encryption - once again, let me know if you need that and I'll help you fix it.

Running the script

Once you downloaded the script open it in a text editor or PowerShell ISE , which is a PowerShell script editor. (If you plan to spend more time with PowerShell, PowerGUI might be a good alternative. Then you might also find this book interesting: Mastering PowerShell by Tobias Weltner.)

Edit the script variables as described in the file and save the script.

By default, Windows will not allow you to run scripts. To change this, you need to open a PowerShell console (with alleviated privileges (Run as Administrator) if you are on Vista / 7 / 2008) and enter following:

Set-ExecutionPolicy RemoteSigned

This will change the execution policy to allow local scripts. You can check if it applied correctly by issuing command:

Get-ExecutionPolicy

You can now simply navigate to the directory where you saved the script and execute it by entering:

.domino_backup.ps1

You can run the script manually or you can schedule it using Windows Task Manager.

That's it, I hope you find it useful. Let me know if you use the script - how it works, what would you like to improve or what needs to be fixed.

Update: XPages, Directory Assistance and context.getUser()

2012-03-01T13:15:00.001+01:00

A few days ago I wrote about my problems with the Directory Assistance and getting user roles with context.getUser().
Prompted by Mark Leusink's comment, I decided to take a look at our Directory Assistance settings. I started changing settings and... context.getUser().getRoles() started to work!
The setting that made all the difference was "Use exclusively for group authorization or credential authentication".

If you set this to Yes, as it was originally set in our case, context.getUser().getRoles() will not work. Setting this to No makes it work. No other settings have any influence on context.getUser().

So, whether you have access to Directory Assistance settings and / or are allowed to change them will decide what implementation you choose. context.getUser().getRoles() is shorter and more elegant, session.getCurrentDatabase().getACL().getEntry(session.getEffectiveUserName()).getRoles() seems to be safer in the long run. The choice is yours.

Formatting dates in XPages links - the Java way

2012-03-01T12:01:00.001+01:00

As many of you have experienced, date and number formatting of the link text in xp:link does not work. Sure, you can assign a data converter, just as you would for a field or a computed text, but it won't work. Paul Withers and Frédéric Dehédin, among others, have offered nice SSJS solutions to this problem.

When I encountered the same problem, I took a somewhat different approach. I didn't like the idea of creating SSJS libraries for something that is essentially a hack and which can be easily solved with a single line (albeit long) of Java code.

So here is your one-liner that will let you format dates in xp:links:

java.text.DateFormat.getDateInstance().format(yourDateValue);

What happens here is that you use the DateFormat class (you must call it by its full name java.text.DateFormat) which knows how to format dates. Using DateFormat, you first get dateInstance which knows everything about dates and their formatting and then you call format with the date value that you want to format.

Let's first take a look at the date value. This value needs to be of type java.util.Date.

If you already have this value in a field on your xpage, you simply need to call

getComponent("elementID").getValue()

The complete code would look something like this:

java.text.DateFormat.getDateInstance().format(getComponent("elementID").getValue());

You can also read a date value from the Notes back-end, a Notes document for example. Say you have a Notes document as the data source and you call it doc. You'll get your date value by calling:

doc.getItemValueDate("fieldName")

which translates to:

java.text.DateFormat.getDateInstance().format(doc.getItemValueDate("fieldName"));

The method getItemValueDate returns java.util.Date, so you are OK there. If you, however, have a Notes DateTime item, you need to convert it to Java Date first. You do so by calling the method toJavaDate(). For example:

doc.getItemValueDateTime("fieldName").toJavaDate()

getItemValueDateTime returns Notes DateTime and by calling toJavaDate you transform it to Java Date that you can use.

That was the date value. Now, let's look at the format options. Calling:

java.text.DateFormat.getDateInstance().format(doc.getItemValueDate("fieldName"));

will return you a string formatted using the default date format and the default locale for the JVM running your application. Most likely, this is not what you want. Fortunately, you can change this easily. getDateInstance accepts arguments that tell it what date format and what locale you want to use in the form of getDateInstance(int style, Locale aLocale). For example, you could do something like this:

java.text.DateFormat.getDateInstance(java.text.DateFormat.SHORT, Locale.US).format(doc.getItemValueDate("fieldName"));

java.text.DateFormat.getDateInstance(java.text.DateFormat.LONG, Locale.GERMANY).format(doc.getItemValueDate("fieldName"));

As you can see, you can use predefined constants for format and locale. The format constants are: FULL, LONG, MEDIUM and SHORT. Just don't forget you need to call them by the full name, e.g. java.text.DateFormat.FULL.

There are many predefined locales as well, you should check Java API documentation (look under Field Summary) to find out more. If your locale is not predefined, don't despair - it is easy to define it. What you would need to know are the lowercase two-letter ISO-639 code for the language and the uppercase two-letter ISO-3166 code for the country. So, if you want to specify locale for the Italian speaking part of Switzerland, you would use:

new Locale("it", "CH")

which in turn would give something like this:

java.text.DateFormat.getDateInstance(java.text.DateFormat.LONG, new Locale("it", "CH")).format(doc.getItemValueDate("fieldName"));

Of course, you could read language and country from some sort of settings document, making your display fully customizable.

Finally, you can also read the locale from the user's browser by calling:

context.getLocale()

resulting in:

java.text.DateFormat.getDateInstance(java.text.DateFormat.MEDIUM, context.getLocale()).format(doc.getItemValueDate("fieldName"));

And that's all. Let me know if something is unclear. And for your homework you do similar conversion for numbers using either the NumberFormat class or the String.format method.

XPages, Directory Assistance and context.getUser()

2012-02-28T14:42:00.001+01:00

It often happens that some parts of the GUI should be hidden, based on the current users authorizations. For that purpose, we have been using ACL roles and a short line of code to evaluate whether the current user is assigned to a specific role:

context.getUser().getRoles().contains("[RoleName]");

And this had worked very well until we recently introduced Directory Assistance and allowed users from the secondary directory to log in and use the application. For users from the secondary directory, the above procedure did not work - they could never see parts of the GUI requiring a role membership, i.e. the above line of code never evaluated to true. At the same time, users from the primary directory had no problem at all.

I have recently read Tim Tripcony's post on problems with page ACL and Directory Assistance and suspected a similar problem here.

This is what a simple test reveals. If a user from the primary directory logs in, context.getUser() will return a wealth of information:

context.getUser().getRoles() returns all roles assigned to the user and hence checking for a specific role works as expected.

But, this is what happens when a user from a secondary directory logs in:

The context.getUser() is all by empty, no roles are returned and check for a role name will always return false.

And that is not good. I don't know if this is intended or not (feature or bug), but I needed to fix it. The first idea that occurred to me was to define roles for each user in a settings document, but that was too much of a hack for my liking. So I tested different ways to get the information about a user's roles and found that this one works:

session.getCurrentDatabase().getACL().getEntry(session.getEffectiveUserName()).getRoles()

This is how it looks if a user from the secondary directory logs in now:

The assigned roles are correctly recognized and it is easy to check for them by calling the isRoleEnabled() method. It also works correctly with users from the primary directory.

So, if you are using or planning to use Directory Assistance and need to check assigned roles, this is the code that you can use.

Vertically stacking members in Dojo List Text Box

2012-02-09T22:05:00.001+01:00

The extension library provides the Dojo List Text Box (djextListTextBox) control that, connected to a value picker, makes it possible to create a list whose members are easy to remove. Using the oneUI v2.1, the default display is similar to this:

But, what if you instead wanted list members to be stacked vertically? Something like this:

With some CSS, it is actually quite easy to accomplish. First, put your djextListTextBox inside a panel and give it a style class, let's call it verticalStack. Then, edit your existing or create a new style sheet and add following:

.verticalStack span {
 display: block;
 margin-left: 0px !important;
 }
 .verticalStack a span {
 display: inline;
 }

What does it all mean? If you take a look at the HTML code that the Dojo List Text Box control generates, you'll see that each list member is actually a <span> tag, which in turn contains an <a> tag that contains a <span> tag.

So, the CSS says that each <span> in the <div> with class verticalStack should have block display style. <span> tags usually have inline display style, which means that multiple <span> tags are displayed in a row, one after the other. display: block will force them to behave like <p> or <div> tags and display one beneath the other.

Now, there is a <span> within the <a> tag. And it is also affected by the block display. And that makes the close button (little x) to be displayed beneath the rest of the list member. This is taken care of by the .verticalStack a span style, which returns normal, inline, display style to all <span> tags that are within a <a> tag that is within an element with .verticalStack class applied.

Finally, using the original style, each list member has applied left margin of 5 pixels to make separetion between consecutive members.We use margin-left: 0px !important to remove that margin and make our members nicely align. The CSS is hierarchical and there are styles higher in the hierarchy that would take precedence, so we need to use !important.

That's all that is needed. Have in mind that this works if you are using oneUI, but it should point you in the right direction even if you are using other layout. And, by applying some more CSS you can easily make it look even better:

How expensive is view.getColumnValues()?

2012-02-06T08:41:00.001+01:00

I need expert help and advice.

In the application we are building, among other things, we keep record of all payments. One of the resources is an XPage that shows all payments due within certain number of days from 'today'.

This is how we did it: we already have a Domino view that displays all unpaid payments. We don't like creating a Domino view for each new requirement, so we decided to reuse this one. I wrote some Java code to go through all of these payments, find those that fall within the given time period and then extract interesting data (i.e. client name, due amount, due date) into a hash map. This hash map is saved in the view scope and used as data source for a repeat control on the XPage.

We also evaluated a solution with full-text database search, but one of the requirements is that when a new payment is added or a payment is paid, the XPage must display those changes immediately. Which means that delay between payment change and full-text index update is unacceptable. And we didn't think that constantly forcing index update was a good idea either.

Our current solution works well, but I don't like to go through all documents each time I reload the XPage. Now, the view is small and lean (only one sorted column) and it will hardly ever have more than a hundred documents, so the computation is quick. Still, I would like to perform the computation only when the view has changed. One of the solutions that occurred to me is this:

Get view.
Get all column values for the first column (view.getColumnValues(0);).
Calculate the hash value of the column values.
Get previously calculated hash value from the session/application scope and compare the two.
If the hash values are the same, do nothing - the existing hash map with payment details is still valid.
If the hash values are different, perform full computation, update the hash map with payment details and update the hash value.

Is this a sound approach? It should be less intensive than the current solution, but is there a better way to do this? How expensive is the view.getColumnValues() call?

Any comments would be greatly appreciated.