Finland transmits, that...

My experience with airBaltic

2012-01-16T18:44:00.000+02:00

UPD: Please read the entire post. I will not remove the story line written originally, because this is exactly what has happened. However airBaltic contacted me on the phone themselves and told about positive resolution of the case. Please read on.

Original story:
-----
First of all, I would like to assure you that I'm not the best at blaming, meaning I simply don't like doing it publicly. It's probably unfair to only publicly blame an air operator and never praise them. But that's how it works. A happy customer doesn't compile an entire blog post about how cool it was to fly with a certain operator. "If I'm happy, I stay silent" principle. But believe me, if the case I'll tell you here about would resolve positively, I wouldn't hesitate to blog about it.

Here is the case. We planned a 3 days trip to Moscow from Helsinki and back together with my wife. Using skyscanner we've found the cheapest option: fly with airBaltic via Riga. Quick friend survey, all's good, settled. I went online and started my ticket search last Saturday. The cheapest option was to depart on 16:25. Chosen that, prepared to pay. After double-checking dates and times it struck me: nope, no good, departure set to 8:25. Cancelled search, started all over. First question here: is it bad luck or bad system? You choose.
Re-ran my search, all is good, paid 531 euros. But when I printed the travel receipt, this time it REALLY struck me: the return flight date was set to one month later! Another bad luck or system fault? This time I'm inclined to choose the second option. No problems, calling to the Finnish office. "On the weekends office is closed". I decided to call first thing next Monday morning. This is my first mistake and I admit it: should have called to Latvia and pay some euro and a half a minute to change the date.

Calling first thing Monday morning: young lady's voice, I described her the problem. She refused to change the date without an additional fee. I have asked her to connect me to her manager. After a couple of minutes (yeah-yeah, customer is on the first place), manager's voice: teaching and preaching me how I should have used their system. "On every page of multi-page ticket booking process, you can look on your right and check the departure and return flight dates and times." All right, thanks! But look, attempted I to explain the system fault: "Unless you really travel back in one month,there is no way to choose that different month without extra movements. No does the system suggest you the best return flights from in a month period!" This was simply noise for her and she continued teaching me how to use the system. I asked her to give me her manager / director. Guess what was the answer: "This is not possible". "Why?". "I'm sorry, but this is not possible." Being a customer, I'm pretty sure, I can talk to almost any worker of the company, who stays in the customer relations line. This time it is your fault, airBaltic.

"I would like to change the return flight date back to what it should be". But the lady tought me another time: "This is only possible if you pay 150 euros. You should have called us on Saturday and explained the problem." Which I did! And the Finnish office was closed. Is this really my problem now? I doubt it. Because, it is YOUR REPRESENTATIVE, airBaltic! What if I wouldn't have an opportunity to call abroad (yes, Riga is abroad to me) and pay for an international call? And if it didn't work, make sure to take my call on Monday morning seriously, attend to it and make an exception or a good men deal. What on Earth does this rule "if you called after two days, it cannot be changed without a fee" policy mean? Do you want to keep a customer or loose it? What do you loose by changing the month standing away date? Afraid not to find any cusomer during an entire month?

"And if I cancel the entire trip, what sum can be refunded?" "You get 76 euros back". Excellent.

Without further ramblings, I would like to publicly thank airBaltic for 531 euros worth "stay at home and don't travel with us" service. It has really taught me not to use your services. Ever.
Everything seems to be mortal in this wolrd, and airBaltic's serivce will die as well. But by making this type of "friendly" customer service and policies you only bring the end faster.

Good luck and enjoy 531-76 euros for not taking us where we wanted.
-----

UPDATE to the story: airBaltic continues working on the case, here is what they posted on twitter: @DmitryKan Dmitry, your case is not closed. Please give us a bit more time and colleagues will come back to you.

UPDATE 2: The case has been resolved. I have received a call from airBaltic, where they said that the return flight date was changed without an extra fee. I don't know was it a result of my social media activity since yesterday evening, but airBaltic service was extremely fast and accurate this time. Since all the posts I have done on the Internet about airBaltic link here, the landed people will read these updates as well. Thank you, airBaltic.

axis2: serialization and deserialization of wsdl2java generated objects

2011-11-09T12:15:00.000+02:00

Using axis2's wsdl2java tool and a third-party wsdl I have generated service stub and supporting classes (data holders). Since there was a need to do post-processing of loaded data from a service, there was a need to serialize one of the data holder objects.

Questions that I had and posted on stackoverflow.com were:

1) is there a standard axis2 tool / approach that can be used for the purpose?

2) since the data holder class does not implement Serializable interface what would be the easiest way of serializing the object into xml format with the ability to restore the original object?

Data binding option was used (-d jaxbri) and each field of the class in question is annotated with @XmlElement tag, e.g.:

@XmlElement(name = "ID", required = true)
protected String id;

Here is how I solved it:

1. axis2 generated java classes set (client side) had an object called ObjectFactory. Majority of its methods create JAXBElement objects with values of fields of the class holder
2. I had to implement a serializable wrapper class ASerializable for the class holder, such that it uses the ObjectFactory to create the JAXBElement objects for all the fields.
3. some external code uses the wrapper class to create an serializable object and writes it to the output stream.
4. on the receiving end:

ASerializable aSerializable;
        A a;
        aSerializable= (ASerializable)in.readObject();
        a.setID((String)aSerializable.getID().getValue());

It still looks like extra work for the pre-annotated class serialization, but better than serializing into some text format and manual type checking during deserialization.
Some good intro into serialization with java can be found here.

First international publication

2011-10-01T18:21:00.003+03:00

Celebration!

Had my first international publication shown on the DBLP. Somehow it was something I wanted to achieve as an intermediate goal in the academic career. In a way this gives some visibility to what I have been doing for around 4 years. I mean NLP (Natural Language Processing) and Machine Translation more precisely. Before going international, I've had 5 publications in the Russian scientific journals and conferences.

On the same ICSOFT'11 conference where this publication has been presented in a form of poster, I had an honor to serve as knowledge-based systems track chair. Both presenting my work and leading the session were exciting. I wanted also to say thanks to the ICSOFT's organizing committee for giving me the participant's grant, that made my participation possible. Special thanks to Sergio Brissos.

ICSOFT 2011 Conference

ICSOFT is strictly not an NLP conference. However, it has a knowledge-based track, where rather relevant NLP related topics are listed:

Ontology Engineering
Decision Support Systems
Intelligent Problem Solving
Expert Systems
Reasoning Techniques
Knowledge Acquisition
Knowledge Mining
Machine Learning
Natural Language Processing
Human-Machine Cooperation

Two publications I remembered

From these, ontology engineering articles were strong. One of them (Barbara Furletti, Franco Turini: Mining Influence Rules out of Ontologies, see here) was about mining ontology and reasoning rules from the oldest Italian bank's data. This sounds exceptional to me, when some (even old) commercial data is given away to researchers.

Another, non-directly related to NLP, article I remembered was by Manolya Kavakli et al (Manolya Kavakli, Tarashankar Rudra, Manning Li: An Embodied Conversational Agent for Counselling Aborigines - Mr. Warnanggal.), where one of the challenges is providing health assistance to the Australlian aborigines via a computer based system, not very motivated people, poor, stealing food and other things. Another challenge is dealing with about 500 languages, that these aborigines speak. Here is a potential for interesting NLP problems.

Why would I recommend going to a conference not directly related to your research topic?

As a pre-word, I should mention, that in a way whatever we do in the NLP is materialized in the form of programming code. Therefore our work qualifies to a software engineering conference as well as to an NLP one.

Going to a strictly SW conference can give you the following benefits:
* concrete questions of you work in the light of software development practices. Some NLP researchers may think it is not very important to make their SW configurable, re-usable or performant. In the end of the day, this matters a lot, especially if you plan to implement you work into industrial level solution

* if you do a poster presentation, people can give you good insights into the quality of your poster and what can be improved. There were two extreme cases on the conference: one with the entire article text being pasted into the poster and another one with a couple of boxes and an arrow between them. The audience has reacted in an expected way: the first poster did not draw almost any attention, while the second had gathered the majority of the audience.

* you can pause and reflect a little bit: are you doing something valuable? Do you like what you do?

A couple of words about Spain, where ICSOFT'11 happened. +45 is something I have experienced for the first time; visiting royal palace Alcázar of Seville was extremely interesting and of course partying with conference peers over Spanish wine and tapas made the event memorable.

Enjoy you research life and publish your work as soon as possible.

Оценка системы машинного перевода

2011-08-27T14:18:00.017+03:00

Есть система машинного перевода с русского на английский.
Нужно сделать ручную оценку работы системы.

Целевая аудитория: все, кто хочет сделать машинные переводчики лучше (примеры: translate.google.com, translate.ru) и люди, интересующиеся прикладной лингвистикой (Natural Language Processing). Умение программировать НЕ требуется.

Связь: dmitry.kan[+AT+]gmail.com, twitter: DmitryKan

Задача: получить у меня пакет предложений (объём: сколько возьмётесь).

В пакете: предложения на русском языке и их переводы экспертом на английский язык.

Прогнать предложения на русском через систему. Просмотреть вручную их переводы на английский язык.

Составить список слов, которые не были найдены (это просто: не найденные слова будут выведены на русском). Послать мне список слов, я добавлю их в систему машинного перевода.

На выходе от вас три группы предложений из пакета:
1. хорошо перевелись
2. приемлемо перевелись (понятно по английской фразе, что было в русской)
3. плохо перевелись (непонятно по английской фразе, что было в русской)

Работа волонтёрская. Начальный бонус: статья со мной в соавторстве на конференции либо в журнале, если вам это интересно. Если нет -- всяческий пиар вам.

Дальше: если сработаемся, предложу Вам работу в лингвистических проектах (умение программировать обязательно).

Ссылки для интересующихся
[1] http://www.slideshare.net/dmitrykan/icsoft-2011-51cr
[2] http://www.slideshare.net/dmitrykan/automatic-build-of-semantic-translational-dictionary
[3] http://ufal.mff.cuni.cz/umc/

interested in rule based machine translation (rbmt)? / Интересуетесь машинным переводом на правилах?

2011-08-27T11:57:00.003+03:00

I'm looking for students and activists of rule-based machine translation to help me in the evaluation of my machine translation system from Russian into English. Details in the e-mail: dmitry.kan[+AT+]gmail.com (substitute characters from [ fro ] with @).

Я ищу студентов и активистов машинного перевода на правилах для оценки моей системы машинного перевода с русского на английский. Детали по почте: dmitry.kan[+AT+]gmail.com (замените символы с [ по ] знаком @).

Interested in machine translation between Russian and English?

2011-07-14T10:21:00.003+03:00

Then mark August 15-19 2011 in your calendars. Web of Data'11 has accepted my poster on machine translation with semantic features, the full paper title is:

Semantic Feature Machine Translation System for Information Retrieval

Some details on the work from another poster, accepted to ICSOFT'11 can be checked here:

Poster: Method for an automatic generation of a semantic-level contextual translational dictionary

View more documents from Dmitry Kan

Пример кода, интегрирующего AOT морфологический лемматайзер в C#

2011-07-10T21:33:00.002+03:00

АОТ предлагает свой лемматайзер для русского и английского языка на сайте www.aot.ru. Если Вам нужно интегрировать их COM внутри проекта на C#, читайте ниже.

После установки библиотеки при помощи Setup.exe, загрузите Lemmatizer.dll в C# проект. Скопируйте следующий метод или его тело, например, в main-class:


private static void initAOTMorphoanalyzer()
{
 LEMMATIZERLib.ILemmatizer lemmatizerRu = new LEMMATIZERLib.LemmatizerRussian();
 lemmatizerRu.LoadDictionariesRegistry();
 LEMMATIZERLib.IParadigmCollection piParadigmCollection = lemmatizerRu.CreateParadigmCollectionFromForm("мыла", 0, 0);

 Console.Out.WriteLine(piParadigmCollection.Count);       

    for (int j=0; j < piParadigmCollection.Count; j++)
    {
     object[] args = { j };     

     Type paradigmCollectionType = piParadigmCollection.GetType();

     if (paradigmCollectionType != null)
     {
      object Item = paradigmCollectionType.InvokeMember("Item", BindingFlags.GetProperty, null, piParadigmCollection, args);
      Type itemType = Item.GetType();
      if (itemType != null)
      {
       object Norm = itemType.InvokeMember("Norm", BindingFlags.GetProperty, null, Item, null);
       Console.Out.WriteLine(Norm);
      }
      else
       Console.Out.WriteLine("itemType is null");
     }
     else
      Console.Out.WriteLine("paradigmCollectionType is null");
    }
}

Результат:
2
МЫЛО
МЫТЬ

COM test example for C#: AOT lemmatizer

2011-07-10T21:22:00.005+03:00

I will post this both in English and Russian for more people's benefit.

There is a Russian / English lemmatizer from AOT (www.aot.ru). If you need to use the COM that AOT provides inside C#, read on. Load the lemmatizer.dll inside your C# project. Insert the following method or its body inside your code, for example main class:


private static void initAOTMorphoanalyzer()
{
 LEMMATIZERLib.ILemmatizer lemmatizerRu = new LEMMATIZERLib.LemmatizerRussian();
 lemmatizerRu.LoadDictionariesRegistry();
 LEMMATIZERLib.IParadigmCollection piParadigmCollection = lemmatizerRu.CreateParadigmCollectionFromForm("мыла", 0, 0);

 Console.Out.WriteLine(piParadigmCollection.Count);       

    for (int j=0; j < piParadigmCollection.Count; j++)
    {
     object[] args = { j };     

     Type paradigmCollectionType = piParadigmCollection.GetType();

     if (paradigmCollectionType != null)
     {
      object Item = paradigmCollectionType.InvokeMember("Item", BindingFlags.GetProperty, null, piParadigmCollection, args);
      Type itemType = Item.GetType();
      if (itemType != null)
      {
       object Norm = itemType.InvokeMember("Norm", BindingFlags.GetProperty, null, Item, null);
       Console.Out.WriteLine(Norm);
      }
      else
       Console.Out.WriteLine("itemType is null");
     }
     else
      Console.Out.WriteLine("paradigmCollectionType is null");
    }
}

Output:
2
МЫЛО
МЫТЬ

Amazed by Scala #1: objects and compilation

2011-06-08T21:53:00.003+03:00

7 minutes and here is an object, that can be compiled into java classes:


import scala.actors._
import Actor._

object TopStock {
 val symbols = List( "AAPL", "GOOG", "IBM", "MSFT")
 val receiver = self
 val year = 2008
 
  def main(args: Array[String]) = {
 symbols.foreach { symbol =>
   actor { receiver ! getYearEndClosing(symbol, year) }
 }
  
 val (topStock, highestPrice) = getTopStock(symbols.length)
 printf("Top stock of %d is %s closing at price %f\n", year, topStock, highestPrice)
  }

 def getYearEndClosing(symbol : String, year : Int) = {
   val url = "http://ichart.finance.yahoo.com/table.csv?s="+ 
   symbol + "&a=11&b=01&c=" + year + "&d=11&e=31&f=" + year+
             "&g=m"
   
   val data = io.Source.fromURL(url).mkString
   val price = data.split("\n")(1).split(",")(4).toDouble
   (symbol, price)
 }

 def getTopStock(count : Int) : (String, Double) = {
   (1 to count).foldLeft("", 0.0) { (previousHigh, index) =>
  receiveWithin(10000) {
    case (symbol : String, price : Double) =>
   if (price > previousHigh._2) (symbol, price) else previousHigh
  }
   }
 }  
}

Saved in TopStock.scala. Compiled with

> scalac TopStock.scala

ran with

> scala TopStock

Top stock of 2008 is GOOG closing at price 307,650000

Amazed by Scala

2011-06-08T20:21:00.010+03:00

Top stock of 2008 is GOOG closing at price 307,650000 among (AAPL, GOOG, IBM, MSFT). Amazed by simplicity, clarity and beauty of the following code in Scala from this book.


import scala.actors._
import Actor._

val symbols = List( "AAPL", "GOOG", "IBM", "MSFT")
val receiver = self
val year = 2008

symbols.foreach { symbol =>
  actor { receiver ! getYearEndClosing(symbol, year) }
}
  
val (topStock, highestPrice) = getTopStock(symbols.length)

printf("Top stock of %d is %s closing at price %f\n", year,
           topStock, highestPrice)

def getYearEndClosing(symbol : String, year : Int) = {
 val url = "http://ichart.finance.yahoo.com/table.csv?s="+ 
 symbol + "&a=11&b=01&c=" + year + "&d=11&e=31&f=" + year+
       "&g=m"
  
 val data = io.Source.fromURL(url).mkString
 val price = data.split("\n")(1).split(",")(4).toDouble
 (symbol, price)
}

def getTopStock(count : Int) : (String, Double) = {
 (1 to count).foldLeft("", 0.0) { (previousHigh, index) =>
    receiveWithin(10000) {
  case (symbol : String, price : Double) =>
    if (price > previousHigh._2) (symbol, price) else
             previousHigh
  }
 }
}

StackOverflow

2011-05-25T08:25:00.003+03:00

Do you know the famous resource http://stackoverflow.com by Joel Spolsky and team? Here is my own definition of stackoverflow in Java:


private static void log(String logStatement) {
   log(logStatement);
}

XML in Chrome

2011-05-02T10:56:00.001+03:00

Tired of (manual) XML rendering in Google Chrome?
Try this:
XML Tree

NoSQL, Apache SOLR and Apache Hadoop

2011-04-25T22:54:00.001+03:00

Check out this SlideShare Presentation -- talk for NёrdCamp April 23, 2011:

NoSQL, Apache SOLR and Apache Hadoop

View more presentations from Dmitry Kan.

NerdCamp

2011-04-19T20:04:00.004+03:00

NerdCamp is coming and I'm going to present on Apache technologies, specifically on Apache Solr and a bit on Apache Hadoop. If you are into all this and like creative thinking, want to create a start-up and will come around Saint-Petersburg this weekend, you should definitely come to NerdCamp!

Confirmed Key Participants

Yury Lifshits, Sergey Poduzov, Alexander Shtuchkin,
Vladimir Gorovoy, Nikolay Vyahhi, Vladimir Aluferov, Yakov Sirotkin and myself

The preliminary program has juicy topics among which are:
How the Web will Transform Education by Yury Lifshits,
Geo Information Systems Around Us by Aleksander Klechikov,
Software Development for little ones by Yakov Sirotkin,
Introducing NoSQL: Apache SOLR and Hadoop by Dmitry Kan,
Introducing Cloud Services by Dmitry Petrov,
Nature of Enterpreneurship Craft, Lifestyle or Science by Alexey Baranov,
How to grow places for smart people -- workshop

and more!

Angry Birds -pelaaja: DK

2011-03-25T20:52:00.000+02:00

Angry Birds -pelaaja: DK: "DK on kertakaikkisen kyllästynyt sikailuun! Seuraa hänen edesottamuksiaan Angry Birdsin SM-kisoissa ja tule itsekin mukaan."

concert of components?

2011-03-21T11:48:00.004+02:00

"ZooKeeper: Because coordinating distributed systems is a Zoo"

This post is short and not useful in that it doesn't give you any code snippets or technical recommendations. I would like just to cite the SolrCloud's wiki. The SolrCloud is "the set of Solr features that take Solr's distributed search to the next level, enabling and simplifying the creation and use of Solr clusters." They use Apache ZooKeeper project (a subproject of Hadoop) as a distributed system of keeping cluster of SOLRs state updates. In a distributed system every component can potentially crash, yet the system is expected to provide it's service to its users. If a single SOLR crashes, its replica will take over, but if ZooKeeper crashes, the system will still continue serving the user requests, but no updates of the system are visible to the system (sounds interesting, I know). To improve that, this is what is possible:

"Running multiple zookeeper servers in concert (a zookeeper ensemble) allows for high availability of the zookeeper service. Every zookeeper server needs to know about every other zookeeper server in the ensemble, and a majority of servers are needed to provide service. For example, a zookeeper ensemble of 3 servers allows any one to fail with the remaining 2 constituting a majority to continue providing service. 5 zookeeper servers are needed to allow for the failure of up to 2 servers at a time."

So, if you have a big zoo with variety of animals in it, make sure you have 5 zoo keepers for at least 3 of them take care of your pigs and elephants, when 2 others got stuck somewhere else.

SOLR: speed up batch posting

2011-01-29T14:34:00.011+02:00

If you are familiar with the Apache SOLR and deal with index worth of millions documents, it becomes quite important to be able to (re-)index fast. There are various techniques as to what to tweak for the indexing go faster. Another simple way to speed up your (batch) indexing without changing your SOLR schema is to modify logging.

Apparently, when deployed under Tomcat, SOLR logs each and every update request during POSTing process. Experience shows, that heavy http operation is done faster, when logging is minimal.

SOLR (as of 1.4 at least) has admin GUI which serves as a central information hub for the given SOLR core. Among other useful features, it has a page where one can set up logging levels of different SOLR components. In default SOLR installation you can access the page via http://localhost:8983/solr/admin/logging. By default, the logging levels amount mainly to INFO, which permits logging of all the select/update requests (imagine 1 million of such log entries for batch reindexing).

It would be handy to be able to automatically change the logging levels to, say, WARNING before batch POSTing and back to INFO after that. solr/admin/logging is declared as servlet in web.xml of the corresponding SOLR core:


  
    Logging
    org.apache.solr.servlet.LogLevelSelection

All the components which allow changing their logging levels are listed on the page http://localhost:8983/solr/admin/logging. Using curl we can send a post request to the servlet and set the desired levels. It is reasonable to implement a function, which takes logging level and url of SOLR core as parameters (choose your own favourite language, this is done in Perl):


sub setSolrLogLevel
{
 my ($url, $level) = @_;

    print "setting logging level to $level\n";
    my $res = system("curl --user user:pass -d \"submit=set&root=$level&fi=$level" .
"&fi.alphasense=$level&fi.alphasense.solr=$level&fi.alphasense.solr.query=$level" .
"&fi.alphasense.solr.query.AlphaSenseQParserPlugin=$level&httpclient=$level&httpclient.wire=$level&httpclient.wire.content=$level&httpclient.wire.header=$level&javax=$level&javax.management=$level&javax.management.mbeanserver=$level&org=$level&org.apache=$level&org.apache.catalina=$level&org.apache.catalina.core=$level&org.apache.catalina.core.ContainerBase=$level&org.apache.catalina.core.ContainerBase.%5BCatalina%5D=$level&org.apache.catalina.core.ContainerBase.%5BCatalina%5D.%5Blocalhost%5D=$level&org.apache.catalina.core.ContainerBase.%5BCatalina%5D.%5Blocalhost%5D.%5B%2Fsolrtopic%5D=$level&org.apache.catalina.core.ContainerBase.%5BCatalina%5D.%5Blocalhost%5D.%5B%2Fsolrtopic%5D.%5BLogging%5D=$level&org.apache.catalina.core.ContainerBase.%5BCatalina%5D.%5Blocalhost%5D.%5B%2Fsolrtopic%5D.%5BSolrServer%5D=$level&org.apache.catalina.core.ContainerBase.%5BCatalina%5D.%5Blocalhost%5D.%5B%2Fsolrtopic%5D.%5BSolrUpdate%5D=$level&org.apache.catalina.core.ContainerBase.%5BCatalina%5D.%5Blocalhost%5D.%5B%2Fsolrtopic%5D.%5Bdefault%5D=$level&org.apache.catalina.core.ContainerBase.%5BCatalina%5D.%5Blocalhost%5D.%5B%2Fsolrtopic%5D.%5Bjsp%5D=$level&org.apache.catalina.core.ContainerBase.%5BCatalina%5D.%5Blocalhost%5D.%5B%2Fsolrtopic%5D.%5Bping%5D=$level&org.apache.catalina.session=$level&org.apache.catalina.session.ManagerBase=$level&org.apache.commons=$level&org.apache.commons.digester=$level&org.apache.commons.digester.Digester=$level&org.apache.commons.digester.Digester.sax=$level&org.apache.commons.httpclient=$level&org.apache.commons.httpclient.ChunkedInputStream=$level&org.apache.commons.httpclient.HeaderElement=$level&org.apache.commons.httpclient.HttpClient=$level&org.apache.commons.httpclient.HttpConnection=$level&org.apache.commons.httpclient.HttpMethodBase=$level&org.apache.commons.httpclient.HttpMethodDirector=$level&org.apache.commons.httpclient.HttpParser=$level&org.apache.commons.httpclient.HttpState=$level&org.apache.commons.httpclient.MultiThreadedHttpConnectionManager=$level&org.apache.commons.httpclient.SimpleHttpConnectionManager=$level&org.apache.commons.httpclient.auth=$level&org.apache.commons.httpclient.auth.AuthChallengeProcessor=$level&org.apache.commons.httpclient.cookie=$level&org.apache.commons.httpclient.cookie.CookiePolicy=$level&org.apache.commons.httpclient.cookie.CookieSpec=$level&org.apache.commons.httpclient.methods=$level&org.apache.commons.httpclient.methods.EntityEnclosingMethod=$level&org.apache.commons.httpclient.methods.ExpectContinueMethod=$level&org.apache.commons.httpclient.methods.PostMethod=$level&org.apache.commons.httpclient.params=$level&org.apache.commons.httpclient.params.DefaultHttpParams=$level&org.apache.commons.httpclient.params.HttpMethodParams=$level&org.apache.commons.httpclient.util=$level&org.apache.commons.httpclient.util.EncodingUtil=$level&org.apache.commons.httpclient.util.ExceptionUtil=$level&org.apache.commons.httpclient.util.IdleConnectionHandler=$level&org.apache.jasper=$level&org.apache.jasper.EmbeddedServletOptions=$level&org.apache.jasper.JspCompilationContext=$level&org.apache.jasper.compiler=$level&org.apache.jasper.compiler.Compiler=$level&org.apache.jasper.compiler.JspConfig=$level&org.apache.jasper.compiler.JspRuntimeContext=$level&org.apache.jasper.compiler.TldLocationsCache=$level&org.apache.jasper.servlet=$level&org.apache.jasper.servlet.JspServlet=$level&org.apache.jasper.servlet.JspServletWrapper=$level&org.apache.solr=$level&org.apache.solr.analysis=$level&org.apache.solr.analysis.BaseTokenFilterFactory=$level&org.apache.solr.analysis.BaseTokenizerFactory=$level&org.apache.solr.client=$level&org.apache.solr.client.solrj=$level&org.apache.solr.client.solrj.impl=$level&org.apache.solr.client.solrj.impl.CommonsHttpSolrServer=$level&org.apache.solr.common=$level&org.apache.solr.common.util=$level&org.apache.solr.common.util.ConcurrentLRUCache=$level&org.apache.solr.core=$level&org.apache.solr.core.Config=$level&org.apache.solr.core.CoreContainer=$level&org.apache.solr.core.JmxMonitoredMap=$level&org.apache.solr.core.RequestHandlers=$level&org.apache.solr.core.SolrConfig=$level&org.apache.solr.core.SolrCore=$level&org.apache.solr.core.SolrResourceLoader=$level&org.apache.solr.handler=$level&org.apache.solr.handler.AnalysisRequestHandler=$level&org.apache.solr.handler.XmlUpdateRequestHandler=$level&org.apache.solr.handler.admin=$level&org.apache.solr.handler.admin.LukeRequestHandler=$level&org.apache.solr.handler.admin.SystemInfoHandler=$level&org.apache.solr.handler.component=$level&org.apache.solr.handler.component.QueryElevationComponent=$level&org.apache.solr.handler.component.SearchHandler=$level&org.apache.solr.handler.component.SpellCheckComponent=$level&org.apache.solr.highlight=$level&org.apache.solr.highlight.SolrHighlighter=$level&org.apache.solr.request=$level&org.apache.solr.request.BinaryResponseWriter=$level&org.apache.solr.request.XSLTResponseWriter=$level&org.apache.solr.schema=$level&org.apache.solr.schema.FieldType=$level&org.apache.solr.schema.IndexSchema=$level&org.apache.solr.search=$level&org.apache.solr.search.SolrIndexSearcher=$level&org.apache.solr.servlet=$level&org.apache.solr.servlet.LogLevelSelection=$level&org.apache.solr.servlet.SolrDispatchFilter=$level&org.apache.solr.servlet.SolrRequestParsers=$level&org.apache.solr.servlet.SolrServlet=$level&org.apache.solr.servlet.SolrUpdateServlet=$level&org.apache.solr.spelling=$level&org.apache.solr.spelling.AbstractLuceneSpellChecker=$level&org.apache.solr.spelling.FileBasedSpellChecker=$level&org.apache.solr.spelling.IndexBasedSpellChecker=$level&org.apache.solr.update=$level&org.apache.solr.update.SolrIndexConfig=$level&org.apache.solr.update.UpdateHandler=$level&org.apache.solr.util=$level&org.apache.solr.util.SolrPluginUtils=$level&org.apache.solr.util.plugin=$level&org.apache.solr.util.plugin.AbstractPluginLoader=$level\" $url");
    print "Result code:$res\n";
}

There you go. Call setSolrLogLevel("http://localhost:8983/solr/admin/logging", "WARNING"); before the batch POSTing and setSolrLogLevel("http://localhost:8983/solr/admin/logging", "INFO"); after the batch POSTing has finished.

"OOPness" in Java

2010-11-19T15:54:00.004+02:00

Optimisation in my code has led me to a code refactoring, where several methods with return values became one. Since I could not use "pass by reference" feature for all my parameters (primitives and objects as collections) I had to come up with an inner class for holding all return parameters.

So by not supporting "pass by reference" feature Java makes your code even more OOP-ish. Even though the topic can be well known, this article helps to put things together and make a memory refresh.

Successive replacement in regular expressions (java)

2010-11-03T14:27:00.004+02:00

Actually not sure, how often people out there do the successive replacement in a target text having a regular expression pattern, but Java has rather neat solution for it. I'm publishing it here, because I know, that esp. younger developers can re-invent a wheel here and have longer debugging sessions.

So the task is like this: you have a text T, like "cat-1 dog-1 cat-1 elephant-1 cat-2 dog-2 cat-3".
Suppose we want to change numerals attached to the words "cat" to their word representations: "1" to "one", "2" to "two".

One straightforward way would be to match all "cat-([0-9])+" subsequences and then run replace operation on T.

So the code would look something like this:


String T = "cat-1 dog-1 cat-1 elephant-1 cat-2 dog-2 cat-3";
Pattern catPattern = Pattern.compile("cat-([0-9]+)");
Matcher catMatcher = catPattern.matcher(T);
Map numToWord = new HashMap();
numToWord.add("1", "one");
numToWord.add("2", "two");
numToWord.add("3", "three"); // ...
while (catMatcher.find())
{
    T = T.replaceFirst(catMatcher.group(1), numToWord.get(catMatcher.group(1)));
}

This code produces:

cat-one dog-one cat-1 elephant-1 cat-two dog-2 cat-three

Which is missing one substitution. Ok, let's use replaceAll instead and make sure we touch only cats:


{
    T = T.replaceAll("cat-" + catMatcher.group(1), "cat-" + numToWord.get(catMatcher.group(1)));
}

which produces what we want:

cat-one dog-1 cat-one elephant-1 cat-two dog-2 cat-three

But now what happens inside the loop is logically out of sync with the loop condition: we iterate over matches, but call replaceAll (probably not efficient either, as replaceAll will be attempted even when not needed anymore, for duplicate matches).

Any more elegant and correct solution?

Yes! It is called Matcher.appendReplacement


  Pattern catPattern = Pattern.compile("cat-([0-9]+)");
  Matcher catMatcher = catPattern.matcher(T);
  Map numToWord = new HashMap();
  numToWord.put("1", "one");
  numToWord.put("2", "two");
  numToWord.put("3", "three"); // ...
  
  StringBuffer sb = new StringBuffer();
  
  while (catMatcher.find())
  {
   System.out.println("Match:" + catMatcher.group(1));
   catMatcher.appendReplacement(sb, "cat-" + numToWord.get(catMatcher.group(1)));
  }
  catMatcher.appendTail(sb);

now sb.toString() contains:

cat-one dog-1 cat-one elephant-1 cat-two dog-2 cat-three

If you append System.out.println(sb.toString()); inside the while loop, you will also see, that replacements happen in sync with the while loop's state, so that what is inside the loop and what while loops over are in sync.

B2B: what is WSDL (simple explanation)

2010-08-21T15:43:00.006+03:00

Just for those wondering around what is WSDL file or technology in the area of Business 2 Business (when companies speak to each other automatically over the network): you can think of it as a declaration of the client-server conversation. In this case one B (server) provides some functionality (like taking square roots of big numbers) and another B (client) has a lot of these numbers and needs square roots of them.

So the server declares a method (which is in fact a remote method):

Double squareRoot(Double number)

in its WSDL file. The client side then takes this file and generates the client side code, which handles the protocol of communication and remote method invocation automatically. What client needs is simply to implement a business logic around this invocation, like a web page with a text field for the number or the entire hardware with touch screen display.

In general the input and return types of remote methods can be simple and complex -- in fact as complex and "proprietary" as you need, because the entire WSDL file is nothing but xml as well as the messages being sent over the network.

One of the libraries used in the industry is Axis and Axis2. Using the library you can generate the java code from wsdl for you client (and even generate wsdl from your java code for you server).

Zero-witdth negative lookahead group: example

2010-08-17T15:38:00.004+03:00

Suppose you have to split a comma separated string in java where meaningful parts are separated by continious spaces, say from 2 to infinity. But the problem is that sometimes a meaningfull part (like a person name) contains 2 spaces. You want to exclude such cases and retain the name's parts together. Example:

name1[one_space]surname1[two_spaces]age
surname2[two_spaces]surname2[two_spaces]age

In this example it is easy to solve the problem with zero-width negative lookahead group (?!\\w), because age is always supposed to be numeric. So the final code will be something like this:

String s = current_string.split("\\s{2}(?!\\w)");

Парижские впечатления

2010-07-01T11:40:00.004+03:00

Вернулся из второй поездки в Париж. Вот сейчас, уже по возвращению, я подумал: в Париже все отдыхают. Или это только видимость? Как бы там ни было, отдыхать там легко. Понравилось кататься на public велосипедах!

Dumper and sorting of keys

2010-04-27T13:23:00.004+03:00

When you use Dumper for debugging via comfortable logging data structures by reference, you can additionally apply sorting to the output. Say, you have a hash addressed by $hash_ref reference. If you need to output the hash contents, but have its keys sorted, you can do:


use strict;
use Data::Dumper;

$hash_ref = generate your hash here

$Data::Dumper::Sortkeys = \&my_filter;
print Dumper($hash_ref), "\n";

sub my_filter {
 my ($hash) = @_;
 # return an array ref containing the hash keys to dump
 # in the order that you want them to be dumped
        return [
            sort {$a <=> $b} keys %$hash
        ];
}

Сходимость

2010-04-21T07:49:00.005+03:00

Код программы аппроксимирует решение задачи. Сходимость кода к решению осуществляется посредством связки "изменение кода"-"компиляция/интерпретация"-"запуск". Получается что-то вроде временного ряда: код может не сходиться неделю и сойтись за один день. Если код не сходится, стоит посмотреть на саму концепцию в решении / задачу / компилятор.

Всё это к тому, что полезно преподавать программирование, рассматривающее программу как целостную математическую схему, а не только как набор алгоритмов, приёмов и дизайна.

Finnish is like C++

2010-04-20T10:24:00.001+03:00

, because it is highly typed.