Božský Filip

Connection between MaxEnt and Logistic Regression

2012-01-28T21:54:00.001+01:00

This is worth reading.

McDonald, Ryan. 2007. Generalized Linear Classifiers in NLP. Course presented to the Swedish Graduate School in Language Technology.

I am in a train.

2011-06-22T16:32:00.002+02:00

ACM Transactions on Speech and Language Processing (TSLP)

2011-06-10T12:51:00.002+02:00

ACM Transactions on Speech and Language Processing (TSLP)

It has been just published a special issue on Machine Learning for Robust and Adaptive Spoken Dialogue Systems.

Stavíme tunely v OpenSSH - Root.cz

2011-06-08T16:10:00.002+02:00

Stavíme tunely v OpenSSH - Root.cz

Toto je opravdu zajimave. Zejemna tathle ta citace.

Dynamické tunelování

... Dovolíme si tedy problematiku přeskočit a podívat se rovnou na dynamické tunelování:

$ ssh -D1080 user@server
Tímto jednoduchým příkazem jsme z SSH klienta vyrobili SOCKS proxy server. Všechny požadavky o spojení, které SOCKS server obdrží, protuneluje SSH protokolem a vyřídí na straně SSH serveru. Stačí tedy v aplikaci, jako je webový prohlížeč nebo poštovní klient, nastavit použití SOCKS proxy serveru (verze 5 nebo 4) s adresou localhost:1080. Od té chvíle bude veškerý provoz směrován tunelem. Jak snadné.

Education: The PhD factory : Nature News

2011-04-21T11:23:00.000+02:00

I foresee a gloomy future for most of the postdocs.

Education: The PhD factory : Nature News

Great lecture on probability measures

2011-04-20T19:48:00.003+02:00

This is a very nice and easy to follow lecture on probability measures for beginners. It starts with the definition of measures, sigma algebra, continues with explanations of integrals, and abstract probability space, and ends with stochastic processes such as a Dirichlet process or a Gaussian process.

Foundations of Nonparametric Bayesian Methods

Peter Orbanz

2 videos

Dialogue Systems Group - University of Cambridge

2010-03-18T16:00:00.001+01:00

I have recently developed a web interface to a dialogue system on which I am currently working in Cambridge. You can find it here:
Dialogue Systems Group - University of Cambridge - CamInfo demo

You can find similar demo on Amazon Mechanical Turk; however, in this case you can also make some money. We need volunteers! Check out www.mturk.com Cambridge .

CDR reporting (to MySQL)

2010-03-18T15:55:00.001+01:00

If you want to enable CDR logging using MySQL on Asterisk Now 1.4 CentOS distribution. You will have to install the following packages:
- asterisk14-addons-core
- asterisk14-addons-mysql

Although the MySQL logging is necessary for installed FreePBX, the asterisk14-addons-mysql package was not initially installed.

More details can be found here:
0014931: CDR reporting (to MySQL) is completely broken in AsteriskNOW 1.5.0 - Asterisk.org Issue Tracker

Information Theory, Inference, and Learning Algorithms

2009-12-14T10:49:00.002+01:00

If you are looking for a very good book about machine learning look at this free on-line
Information Theory, Inference, and Learning Algorithms book.

Temporal-Difference Learning Policy Evaluation in Python

2009-04-02T18:33:00.009+02:00

In the code bellow, is an example of policy evaluation for very simple task. Example is taken from the book: "Reinforcement Learning: An Introduction, Surto and Barto". 

#!/usr/local/bin/python

"""
This is an example of policy evaluation for a random walk policy.

Example 6.2: Random Walk from the book:
"Reinforcement Learning: An Introduction, Surto and Barto"

The policy is evaluated by dynamic programing and TD(0).

In this example, the policy can start in five states 1, 2, 3, 4, 5 and end in
two states 0 and 6. The allowed transitions between the states are as follwes:

0 <-> 1 <-> 2 <-> 3 <-> 4 <-> 5 <-> 6

The reward for ending in the state 6 is 1.
The reward for ending in the state 0 is 0.

In any state, except the final states, you can take two actions: 'left' and 'right'.
In the final states the policy and episodes end.

Because this example implements the random walk policy then both actions can be
taken with the probability 0.5 .
    """

import random


# Instead of letters, I use number for the states. The states 0 and 6 are the final states.
# The states 1,...,5 are the states A,...,E

states = range(0,7)
finalStates = [0,6]
reward = [ 1 if s == 6 else 0 for s in states]  

def policy(s):
  """Random walk policy: ations are 'go left' nand 'go right'."""

  return random.choice(['left', 'right'])

def execute_policy(s, a):
  """Change state based on the taken action."""

  if a == 'left':
      return s - 1
  else:
      return s + 1

def TD_0(V_star, alpha, gamma,  numOfEpisodes = 10000):
  """Use Temporal-Difference Learning to learn V^*."""

  for episode in range(numOfEpisodes):
      # select random start state
      s = random.randint(min(states)+1, max(states)-1)
    
      endOfEpisode = False
      while not endOfEpisode:
          if s in finalStates:
              # evaluate the value of the final state
              V_star[s] = V_star[s] + alpha*(reward[s] - V_star[s])
        
              # because we are in the final state then end the episode
              endOfEpisode = True
              continue
          else:
              # get an action for this state from the policy
              a = policy(s)
            
              # execute policy => take an action
              s_prime = execute_policy(s, a)

              # evaluate the action
              V_star[s] = V_star[s] + alpha*(reward[s] + gamma*V_star[s_prime] - V_star[s])
            
              s = s_prime

def V(s, d = 0):
  """Value function computed by dynamic programing."""

  if d > 20:
      return 0
    
  if s in finalStates:
      return reward[s]
    
  return 0.5*(V(s-1, d+1) + V(s+1, d+1))

###############################################################################
##
##  Experiments
##
###############################################################################

gamma = 1

print("TD(0): alpha = 0.15")

# init the value function
V_star = [0.5 for s in states]
TD_0(V_star, 0.15, gamma, 100000)
for s, V_s in enumerate(V_star):
  V_s_star = s/6.0
  print("V(%d) = %0.3f   err = %.3f" % (s, V_s, abs(V_s_star - V_s)))

print("TD(0): alpha = 0.05")

# init the value function
V_star = [0.5 for s in states]
TD_0(V_star, 0.05, gamma, 100000)
for s, V_s in enumerate(V_star):
  V_s_star = s/6.0
  print("V(%d) = %0.3f   err = %.3f" % (s, V_s, abs(V_s_star - V_s)))

print "Dynamic programing:"
for s in states:
  V_s_star = s/6.0
  V_s = V(s)
  print("V(%d) = %0.3f   err = %.3f" % (s, V_s, abs(V_s_star - V_s)))

Loebner Prize at InterSpeech 2009

2009-03-30T16:43:00.003+02:00

I have recetly written an article about the Loebner Prize contest. The contest has been organized every year since 1991 and it will be held in conjunction with InterSpeech, in Brighton, UK this year. Check out the whole article!

Difference between the Standard Deviation, Standard Error and Confidence Itervals

2009-03-24T11:58:00.005+01:00

In its simplest:

the standard deviation represents the variability of input values,
the standard error (of the mean) represents variability of computed mean,
the confidence intervals represents where the 'true' mean value might lie.

Computation:

the standard deviation is computed from the variance of your data - input values,
the standard error is computed from the standard deviation,
the confidence intervals are computed from the standard error.

Read this:

Many people confuse the standard deviation (SD) and the standard error of the mean (SE) and are unsure which, if either, to use in presenting data in graphical or tabular form. The SD is an index of the variability of the original data points and should be reported in all studies. The SE reflects the variability of the mean values, as if the study were repeated a large number of times. By itself, the SE is not particularly useful; however, it is used in constructing 95% and 99% confidence intervals (CIs), which indicate a range of values within which the “true” value lies. The CI shows the reader how accurate the estimates of the population values actually are. If graphs are used, error bars equal to plus and minus 2 SEs (which show the 95% CI) should be drawn around mean values. Both statistical significance testing and CIs are useful because they assist the reader in determining the meaning of the findings.

(Can J Psychiatry 1996;41:498–502)

Google Testing Blog: TotT: EXPECT vs. ASSERT

2009-03-16T17:21:00.000+01:00

Google commented on different types of assertations in their testing framework (Google Testing Blog: EXPECT vs. ASSERT)

I have found assertations in my code very useful on many occasions; however, I do not see any need for the EXPECT function. If your code is broken then it is broken and there is no point in continuing and testing conditions which are not likely to be met.

It is like with C++ compiler errors. The most important error is the firtst error. The rest of the erorrs is usually rubish and useless.

Linked list sorted in an ascending order

2009-03-11T15:02:00.010+01:00

Again to practice some of skills I wrote linked list sorted in an ascending order. It is very simple example and it performs only two operation:

inserting an item into the appropriate position in the list
printing the whole list on the screen

The following code is placed in the public domain. Use it as you wish. The code can be downloaded with the solution project file for MS Visual Studio 2008 from here.

// LinkedList.cpp : Defines the entry point for the console application.
//

#include "stdafx.h"
#include

/*
The supplied C code fragment is intended to implement a linked list of
integers stored in ascending order. Each element of the list is a struct
of type Item holding the integer value, a pointer pred to the previous
element, if any, and a pointer succ to the succeeding element, if any. The
variable head points to the ?rst element in the list, and tail points to
the last element. Initially, both head and tail are NULL. The function
insert is intended to insert its argument x into the list. If x is already in
the list, insert should do nothing.
*/

using namespace std;

struct Item {
int value;    // the integer value
Item* succ;   // succeeding value
Item* pred;   // preceding value
};

Item* head = NULL;
Item* tail = NULL;

Item* createItem(int x, Item* succ, Item* pred)
{
Item* item = new Item;
item->value = x;
item->succ = succ;
item->pred = pred;

return item;
}

/**
Insert x into linked list

There are 5 possible cases, any four of these will suffice
- list is initially empty
- x is smaller than any number in the list
- x is larger than any number in the list
- x is already in the list
- the normal case i.e. none of the above

*/

void insert(int x)
{
Item *p = head , *q;
if(!p) {
// add the first item
head = tail = createItem(x,NULL,NULL);
}
else if (x > tail->value){
// add the largest item in the list
tail->succ = createItem(x,NULL,tail);
tail = tail->succ;
}
else{
while (x > p->value)
 p = p->succ;

if (x==p->value)
 // do not add itme because it already exists
 return;

// add the item before p because x < q =" createItem(x,">pred);   // create item, pass the succesor and the predcesor
q->succ->pred = q;               // modify the succesor of q so that it points to q
q->pred->succ = q;               // modify the predcesor of q so that it points to q
}
}

void printList(void)
{
Item* p = head;
int counter = 0;

cout << "------------------------------------------------------------" <<>value << p =" p-">succ;
}
}

int _tmain(int argc, _TCHAR* argv[])
{
cout << "Ascending linked list demonstration." << i =" 0;">

The output of the program is as follows:

Ascending linked list demonstration.
--------------------------------------
Item number:0 Item value:1
--------------------------------------
Item number:0 Item value:1
Item number:1 Item value:10
--------------------------------------
Item number:0 Item value:1
Item number:1 Item value:5
Item number:2 Item value:10
--------------------------------------
Item number:0 Item value:1
Item number:1 Item value:5
Item number:2 Item value:10

Viterbi Algorithm in C++ and using STL

2009-01-21T18:34:00.014+01:00

To practice my C++ and STL skills, I implemented the Viterbi algorithm example from the Wikipedia page: http://en.wikipedia.org/wiki/Viterbi_algorithm. The original algorithm was implemented in Python. I reimplemented the example in C++ and I used STL (mainly vector and map classes). This code is in public-domain. So, use it as you want.

The complete solution for MS Visual C++ 2008 can be found at http://filip.jurcicek.googlepages.com/ViterbiSTL.rar

// ViterbiSTL.cpp : is an C++ and STL implementatiton of the Wikipedia example

// Wikipedia: http://en.wikipedia.org/wiki/Viterbi_algorithm#A_concrete_example

// It as accurate implementation as it was possible

#include "stdafx.h"

#include "string"

#include "vector"

#include "map"

#include "iostream"

using namespace std;

//states = ('Rainy', 'Sunny')

//observations = ('walk', 'shop', 'clean')

//start_probability = {'Rainy': 0.6, 'Sunny': 0.4}

//transition_probability = {

// 'Rainy' : {'Rainy': 0.7, 'Sunny': 0.3},

// 'Sunny' : {'Rainy': 0.4, 'Sunny': 0.6},

// }

//emission_probability = {

// 'Rainy' : {'walk': 0.1, 'shop': 0.4, 'clean': 0.5},

// 'Sunny' : {'walk': 0.6, 'shop': 0.3, 'clean': 0.1},

// }

vector states;

vector observations;

map start_probability;

map> transition_probability;

map> emission_probability;

class Tracking {

public:

double prob;

vector v_path;

double v_prob;

Tracking() {

prob = 0.0;

v_prob = 0.0;

}

Tracking(double p, vector & v_pth, double v_p) {

prob = p;

v_path = v_pth;

v_prob = v_p;

}

};

//////////////////////////////////////////////////////////////////////////

void init_variables(void) {

states.push_back("Rainy");

states.push_back("Sunny");

observations.push_back("walk");

observations.push_back("shop");

observations.push_back("clean");

start_probability["Rainy"] = 0.6;

start_probability["Sunny"] = 0.4;

transition_probability["Rainy"]["Rainy"] = 0.7;

transition_probability["Rainy"]["Sunny"] = 0.3;

transition_probability["Sunny"]["Rainy"] = 0.4;

transition_probability["Sunny"]["Sunny"] = 0.6;

emission_probability["Rainy"]["walk"] = 0.1;

emission_probability["Rainy"]["shop"] = 0.4;

emission_probability["Rainy"]["clean"] = 0.5;

emission_probability["Sunny"]["walk"] = 0.6;

emission_probability["Sunny"]["shop"] = 0.3;

emission_probability["Sunny"]["clean"] = 0.1;

}

//////////////////////////////////////////////////////////////////////////

void print_variables(void) {

// print states

cout << "States:" <<>

for(vector::iterator i=states.begin();i!=states.end();i++) {

cout << "S: " << (*i) <<>

}

// print observations

cout << "Observations:" <<>

for(vector::iterator i=observations.begin();i!=observations.end();i++) {

cout << "O: " << (*i) <<>

}

// print start probabilities

cout << "Start probabilities:" <<>

for(map::iterator i=start_probability.begin();i!=start_probability.end();i++) {

cout << "S: " << (*i).first << " P: " << (*i).second <<>

}

// print transition_probability

cout << "Transition probabilities:" <<>

for(map>::iterator i=transition_probability.begin();i!=transition_probability.end();i++) {

for(map::iterator j=(*i).second.begin();j!=(*i).second.end();j++) {

cout << "FS: " << (*i).first << " TS: " << (*j).first << " P: " << (*j).second <<>

}

// print emission probabilities

cout << "Emission probabilities:" <<>

for(int i=0; i

for(int j=0; j

cout << "FS: " <<>

" P: " <<>

}

//this method compute total probability for observation, most likely viterbi path

//and probability of such path

void forward_viterbi(vector obs, vector states, map start_p,

map> trans_p,

map> emit_p) {

map T;

for(vector::iterator state=states.begin(); state!=states.end();state++) {

vector v_pth;

v_pth.push_back(*state);

T[*state] = Tracking(start_p[*state], v_pth, start_p[*state]);

}

for(vector::iterator output=obs.begin(); output!=obs.end();output++) {

map U;

for(vector::iterator next_state=states.begin(); next_state!=states.end(); next_state++) {

Tracking next_tracker;

for(vector::iterator source_state=states.begin(); source_state!=states.end(); source_state++) {

Tracking source_tracker = T[*source_state];

double p = emit_p[*source_state][*output]*trans_p[*source_state][*next_state];

source_tracker.prob *= p;

source_tracker.v_prob *= p;

next_tracker.prob += source_tracker.prob;

if(source_tracker.v_prob > next_tracker.v_prob) {

next_tracker.v_path = source_tracker.v_path;

next_tracker.v_path.push_back(*next_state);

next_tracker.v_prob = source_tracker.v_prob;

}

U[*next_state] = next_tracker;

}

T = U;

}

// apply sum/max to the final states

Tracking final_tracker;

for(vector::iterator state=states.begin(); state!=states.end(); state++) {

Tracking tracker = T[*state];

final_tracker.prob += tracker.prob;

if(tracker.v_prob > final_tracker.v_prob) {

final_tracker.v_path = tracker.v_path;

final_tracker.v_prob = tracker.v_prob;

}

cout << "Total probability of the observation sequence: " <<>

cout << "Probability of the Viterbi path: " <<>

cout << "The Viterbi path: " <<>

for(vector::iterator state=final_tracker.v_path.begin(); state!=final_tracker.v_path.end(); state++) {

cout << "VState: " << *state <<>

}

int _tmain(int argc, _TCHAR* argv[])

{

cout << "Viterbi STL example" <<>

init_variables();

print_variables();

forward_viterbi(observations,

states,

start_probability,

transition_probability,

emission_probability);

cout << "End" <<>

string end;

cin >> end;

return 0;

}

BBC NEWS | Magazine | Three little words so hard to say

2008-11-06T16:26:00.002+01:00

A very nice article about knowledge and the ability to admit that you don't know. I cite a piece of the text from the article about the work of politicians.
BBC NEWS | Magazine | Three little words so hard to say: "'I, in the early days, was trying to persuade a minister this was a really important public policy. He said, 'Do you know what? Repeat victimisation's got too many syllables. Can't you think of something with fewer syllables? Could we call it repeat offending?''"

No Mas FARC - Praga

2008-03-21T09:32:00.001+01:00

Abych potpořil aktivity proti FARC v Kolumbii, tak jsem na YouTube nahral toto video. http://youtube.com/bozskyfilip

Video bylo vytvořeno pro akci NO MAS FARC - PRAGA 4. 2. 2008

Další informace o akci jsou na http://www.colombiasoyyo.org/