HOANG Cong Duy Vu's research logs

Microsoft Cognitive Services

2016-04-01T03:41:00.000-07:00

Link: https://www.microsoft.com/cognitive-services/en-us/apis

Intro: Various AI-related services through online APIs. More importantly, all are free for use.

Deep Learning Frameworks

2015-11-11T14:42:00.000-08:00

There are a lot of deep learning frameworks out there, depending on your usage purpose or the familiarity of your programming languages or working tasks, here I only summarize ones that I am familiar with:

1) TensorFlow by Google (released on 10 Nov 2015): http://tensorflow.org/

Comment:

2) VELES by Samsung (released on 11 Nov 2015): https://velesnet.ml/

Comment:

3) cnn (lightweight and very fast neural network library in C++, also in Python, works both on Windows and Linux machines): https://github.com/kaishengyao/cnn

Comment: cnn has been proven to be much faster than Theano both with and without GPU. Also, it offers the advantage for software production of neural network models since it has been developing in C++ and more importantly, it supports both Windows and Linux platforms.

4) to be updated

Wiki Parallel Data Extractor

2015-08-30T01:45:00.000-07:00

Link: https://github.com/clab/wikipedia-parallel-titles
Intro: Tools for extracting parallel corpora from article titles across languages in Wikipedia

Tay Nung dictionary

2015-07-25T07:21:00.000-07:00

Link: https://sites.google.com/site/tndict/home

Intro: Thanks to some guys on facebook of VNese NLP group, I just know about this. One of interesting problems is to preserve and develop the local regional languages (e.g. Tay Nung in the above link) in parallel with official Vietnamese language.

P.S.: I will be back one day about this issue.

Japanese-English Parallel Datasets

2015-07-23T23:21:00.002-07:00

Link: http://phontron.com/japanese-translation-data.php

Hansard corpus

2015-07-21T15:22:00.002-07:00

Link: http://www.hansard-corpus.org/

Intro: This Hansard corpus (or collection of texts) contains nearly every speech given in the British Parliament from 1803-2005, and it allows you to search these speeches (including semantically-based searches) in ways that are not possible with any other resource.

easyloggingcpp - light-weight logging library for C++

2015-07-19T18:50:00.001-07:00

Link: https://github.com/easylogging/easyloggingpp

Intro: Single header only C++ logging library. It is extremely light-weight, robust, fast performing, thread and type safe and consists of many built-in features. It provides ability to write logs in your own customized format. It also provide support for logging your classes, third-party libraries, STL and third-party containers etc.

OLAC - Open Language Archives Community

2015-07-17T03:31:00.006-07:00

Link: http://www.language-archives.org/

Intro: OLAC, the Open Language Archives Community, is an international partnership of institutions and individuals who are creating a worldwide virtual library of language resources by: (i) developing consensus on best current practice for the digital archiving of language resources, and (ii) developing a network of interoperating repositories and services for housing and accessing such resources.

Visualgdb

2015-07-17T00:21:00.001-07:00

Link: http://visualgdb.com/tutorials/linux/import/

Intro: This tool will help how to import a Linux project from a Linux machine to Visual Studio to build and debug it remotely.

This tool is probably comfortable for a Windows-based and Visual Studio fan who wants to compile a project remotely on Linux.

In terms of point of view of a Linux coder, it's not a good way. You may learn how to use gdb with CLI programming instead.

IR book by Bruce Croft

2015-07-08T17:56:00.002-07:00

Title: Search Engines Information Retrieval in Practice by Prof. Bruce Croft
Link: http://ciir.cs.umass.edu/irbook/

Intro: This book provides an overview of the important issues in information retrieval, and how those issues affect the design and implementation of search engines. Not every topic is covered at the same level of detail. The focus is on some of the most important alternatives to implementing search engine components and the information retrieval models underlying them. The target audience for the book is advanced undergraduates in computer science, although it is also a useful introduction for graduate students. (from the link)

NAACL 2015 papers

2015-07-08T17:45:00.001-07:00

Link: http://naacl.org/naacl-hlt-2015/papers.html

Here is my subjective list of remarkable papers relating to MT research:

*** Neural Machine Translation

Paul Baltescu and Phil Blunsom. "Pragmatic Neural Language Modelling in Machine Translation"

Adrià de Gispert, Gonzalo Iglesias, Bill Byrne. "Fast and Accurate Preordering for SMT using Neural Networks"

*** Continuous Models for Statistical Machine Translation

Frédéric Blain, Fethi Bougares, Amir Hazem, Loïc Barrault, Holger Schwenk. "Continuous Adaptation to User Feedback for Statistical Machine Translation"

Kai Zhao, Hany Hassan, Michael Auli. "Learning Translation Models from Monolingual Continuous Representations"

*** Multi-language Translation

Raj Dabre, Fabien Cromieres, Sadao Kurohashi, Pushpak Bhattacharyya. "Leveraging Small Multilingual Corpora for SMT Using Many Pivot Languages"

*** Video to Text Translation

Subhashini Venugopalan, Huijuan Xu, Jeff Donahue, Marcus Rohrbach, Raymond Mooney, Kate Saenko. "Translating Videos to Natural Language Using Deep Recurrent Neural Networks"

*** Others

Jonathan H. Clark, Chris Dyer, Alon Lavie. "Locally Non-Linear Learning for Statistical Machine Translation via Discretization and Structured Regularization"

Graham Neubig, Philip Arthur, Kevin Duh. "Multi-Target Machine Translation with Multi-Synchronous Context-free Grammars"

Aurelien Waite and Bill Byrne. "The Geometry of Statistical Machine Translation"

Other papers are also worth reading:

*** News Processing

Areej Alhothali and Jesse Hoey. "Good News or Bad News: Using Affect Control Theory to Analyze Readers' Reaction Towards News Articles"

Python wrapper for online translators

2015-07-07T23:35:00.002-07:00

If you want to use Google Translate and Microsoft Bing Translate for free, you may consider the following Python-based wrappers:

+ Code: Google Translate; Bing Translate

+ Samples:

# Google Translate
import googletrans
gs = googletrans.Googletrans()

languages = gs.get_languages()
print(languages['en'])

print(gs.translate('hello', 'de'))
print(gs.translate('hello', 'zh'))
print(gs.translate('hello', 'vi'))

print(gs.detect('some English words'))

#Bing Translate

from mstranslator import Translator

translator =
Translator('cdvhoang', 'HlUUMftdkETWa8E9/jzD4l1CzC8sOhRSJxH+kk0MDBg=')

print(translator.translate('hello', lang_from='en', lang_to='vi'))

*** Please note that I don't encourage to use the wrapper for Google Translate because you should respect and pay for using its service (simply it's now commercialized ^_^).

ACL 2015 papers

2015-07-04T22:39:00.001-07:00

Link: http://acl2015.org/accepted_papers.html

Here is my subjective list of remarkable papers relating to MT research:

*** Conventional Statistical Machine Translation

A CONTEXT-AWARE TOPIC MODEL FOR STATISTICAL MACHINE TRANSLATION
Jinsong Su, Deyi Xiong, Yang Liu, Xianpei Han, Hongyu Lin and Junfeng Yao

NON-LINEAR LEARNING FOR STATISTICAL MACHINE TRANSLATION
Shujian Huang, Huadong Chen, Xinyu Dai and Jiajun Chen

MULTI-TASK LEARNING FOR MULTIPLE LANGUAGE TRANSLATION
Daxiang Dong, Hua Wu, Wei He, Dianhai Yu and Haifeng Wang

WHAT’S IN A DOMAIN? ANALYZING GENRE AND TOPIC DIFFERENCES IN STATISTICAL MACHINE TRANSLATION
Marlies van der Wees, Arianna Bisazza, Wouter Weerkamp and Christof Monz

*** Neural Machine Translation

ADDRESSING THE RARE WORD PROBLEM IN NEURAL MACHINE TRANSLATION
Thang Luong, Ilya Sutskever, Quoc Le, Oriol Vinyals and Wojciech Zaremba

ENCODING SOURCE LANGUAGE WITH CONVOLUTIONAL NEURAL NETWORK FOR MACHINE TRANSLATION
Fandong Meng, Zhengdong Lu, Mingxuan Wang, Hang Li, Wenbin Jiang and Qun Liu

IMPROVED NEURAL NETWORK FEATURES, ARCHITECTURE AND LEARNING FOR STATISTICAL MACHINE TRANSLATION
Hendra Setiawan, Zhongqiang Huang, Jacob Devlin, Thomas Lamar and Rabih Zbib

NON-PROJECTIVE DEPENDENCY-BASED PRE-REORDERING WITH RECURRENT NEURAL NETWORK FOR MACHINE TRANSLATION
Antonio Valerio Miceli Barone

ON USING VERY LARGE TARGET VOCABULARY FOR NEURAL MACHINE TRANSLATION
Sebastien Jean, Kyunghyun Cho, Roland Memisevic and Yoshua Bengio

CONTEXT-DEPENDENT TRANSLATION SELECTION USING CONVOLUTIONAL NEURAL NETWORK
Baotian Hu, Zhaopeng Tu, Zhengdong Lu and Hang Li

*** Machine Translation Evaluation and Quality Estimation

ONLINE MULTITASK LEARNING FOR MACHINE TRANSLATION QUALITY ESTIMATION
José G. C. de Souza, Matteo Negri, Marco Turchi and Elisa Ricci

PAIRWISE NEURAL MACHINE TRANSLATION EVALUATION
Francisco Guzmán, Shafiq Joty, Lluís Màrquez and Preslav Nakov

EVALUATING MACHINE TRANSLATION SYSTEMS WITH SECOND LANGUAGE PROFICIENCY TESTS
Takuya Matsuzaki, Akira Fujita, Naoya Todo and Noriko H. Arai

Some notes:

*** According to my observation, there are some research trends depending on data characteristics:
- very big data
- heterogeneous data
- multi-lingual data

*** And of course, deep learning research is still very hot.

Torch vs. Theano vs. Caffe

2015-06-25T18:00:00.001-07:00

Link: http://fastml.com/torch-vs-theano/

Here is my summary:

- Torch and Theano are better to be used for research purpose on deep learning (DL) whereas Caffe is more scaled for DL application development.

- Torch and Theano are competitive in terms of speech and performance via different benchmarks. Hence, choosing one of them depends the ease of use from users.

(to be updated)

Jekyll

2015-05-25T00:51:00.001-07:00

Links:
http://karpathy.github.io/2014/07/01/switching-to-jekyll/
http://jekyllrb.com/docs/home/
Intro: to transform your plain text into static websites and blogs.

Andrej Karpathy's blog

2015-05-24T23:01:00.001-07:00

Link 1: http://karpathy.github.io/ (Neural Network's basics)
Link 2: http://karpathy.github.io/2015/05/21/rnn-effectiveness/ (Recurrent NN's view)
Intro: A very useful blog from a very good PhD student of Stanford Uni.

Brat

2015-05-24T22:54:00.001-07:00

Link: http://brat.nlplab.org/introduction.html

Intro: brat is a web-based tool for text annotation; that is, for adding notes to existing text documents. brat is designed in particular for structured annotation, where the notes are not free-form text but have a fixed form that can be automatically processed and interpreted by a computer.

Wikipedia and LM

2015-05-11T21:01:00.004-07:00

Link: http://trulymadlywordly.blogspot.sg/2011/03/creating-text-corpus-from-wikipedia.html
Intro: How to leverage Wikipedia repository to create huge LM data.

C++11

2015-04-19T17:25:00.000-07:00

http://www.stroustrup.com/what-is-2009.pdf

http://stackoverflow.com/questions/8851670/relevant-boost-features-vs-c11

http://blog.smartbear.com/c-plus-plus/the-biggest-changes-in-c11-and-why-you-should-care/

https://marcoarena.wordpress.com/2012/02/05/modern-cpp-cookbooks/

It seems that I really don't need Boost library ^^.

Rapid conversion table

2015-04-16T20:08:00.002-07:00

Link: http://www.rapidtables.com/

git basics guide

2015-03-12T22:55:00.001-07:00

Link: http://rogerdudler.github.io/git-guide/

Vowpal Wabbit

2015-03-05T19:19:00.001-08:00

Link: https://github.com/moses-smt/vowpal_wabbit

Intro: The Vowpal Wabbit (VW) project is a fast out-of-core learning system sponsored by Microsoft Research and (previously) Yahoo! Research. Support is available through the mailing list.

There are two ways to have a fast learning algorithm: (a) start with a slow algorithm and speed it up, or (b) build an intrinsically fast learning algorithm. This project is about approach (b), and it's reached a state where it may be useful to others as a platform for research and experimentation.

Training Google 1T web corpus with IRSTLM

2015-03-01T19:03:00.000-08:00

Thanks to http://www44.atwiki.jp/keisks/pages/50.html , here is the way to train enormous LM with Google 1T web corpus using IRSTLM:

build-sublm.pl --size 3 --ngrams "gunzip -c 3gms/*.gz" --sublm LM.000 --witten-bell
merge-sublm.pl --size 3 --sublm LM -lm g_3grams_LM.gz
compile-lm g_3grams_LM.gz g_3grams_LM.blm
(if you get the error: "lt-compile-lm: lmtable.h:247: virtual double lmtable::setlogOOVpenalty(int): Assertion `dub > dict->size()' failed.")
compile-lm -dub=100000000 g_3grams_LM g_3grams_LM.blm (make the -dub option bigger)

I will validate it soon.

Simple batch programming on Windows

2015-02-27T03:16:00.003-08:00

Some simple tricks for batch programming on Windows environment:

1) For loop (refer: http://ss64.com/nt/for_l.html)
Syntax
FOR /L %%parameter IN (start,step,end) DO command

Key
start : The first number
step : The amount by which to increment the sequence
end : The last number

command : The command to carry out, including any
command-line parameters.

%%parameter : A replaceable parameter:
in a batch file use %%G (on the command line %G)

Example
FOR /L %%G IN (0,1,195) DO (
echo %%G & ping 1.1.1.1 -n 1 -w 1000 >nul

)

2) ...

SEAlang - Southeast Asian Languages Library

2015-02-26T00:24:00.000-08:00

Link: http://sealang.net/library/

Intro: The SEAlang Library was established in 2005. It provides language reference materials for Southeast Asia; initially focused on the non-roman script languages used throughout the mainland, and now concentrating on the languages of insular SEA.

* Take note of the Vietnamese corpus (including dictionary, bitexts, ...)