Natural Language Processing World

Wolfram Mathematica 8 comes with a Free-Form Linguistic input

noreply@blogger.com (Pedro Paulo Balage) — Thu, 12 May 2011 07:32:00 +0000

Wolfram Mathematica, one of the best tools for scientific computing and mathematics learning, announced an impressive feature in its new software version. According with Mathematica's website (http://www.wolfram.com/mathematica/new-in-8/free-form-linguistic-input/), the version 8 comes with a free-from linguistic input where you can use natural language to define the queries.

According the website with the new tool is possible

learn Mathematica syntax by entering a query in English and then viewing the free-form queries as precise Mathematica commands.
Easily specify plot frames, styles, gridlines, and more in natural language.
Perform many specialist operations in fields as diverse as image processing and finance using free-form instructions.
Access Wolfram|Alpha's vast collection of data and integrate it with your Mathematica session.
Transparently refer to the results of previous computations or session variables in free-form commands.

The computational knowledge engine Wolfram Alpha (http://www.wolframalpha.com/) makes use of this new version of Mathematica software and also allows the user to input natural language queries. The strategic of Wolfram labs and many other companies seems to fuse the free-form natural language input with computational tools like web search, mathematics software and knowledge retrieval.

Some example of commands supported by the tool are:

In[5]:=

Out[5]=

In[10]:=

Out[10]=

In[9]:=

Out[9]=

n[3]:=

Out[3]=

In[4]:=

Out[4]=

In[5]:=

Out[5]=

Memory-Based Shallow Parser for Python

noreply@blogger.com (Pedro Paulo Balage) — Sat, 26 Feb 2011 19:12:00 +0000

From the author's website:

MBSP is a text analysis system based on the TiMBL and MBT memory based learning applications developed at CLiPS and ILK. It provides tools for Tokenization and Sentence Splitting, Part of Speech Tagging, Chunking, Lemmatization, Relation Finding and Prepositional Phrase Attachment.

http://www.clips.ua.ac.be/pages/MBSP

MBSP stands for "Memory-Based Shallow Parser" and provides a nice python interface use the software functionalities. In fact, it is a service that once started it is very fast to provide new parsings. An example for the system is outputted bellow.

>>> print MBSP.parse('I ate pizza with a fork.')

I/PRP/I-NP/O/NP-SBJ-1/O/i 
ate/VBD/I-VP/O/VP-1/A1/eat 
pizza/NN/I-NP/O/NP-OBJ-1/O/pizza 
with/IN/I-PP/B-PNP/O/P1/with 
a/DT/I-NP/I-PNP/O/P1/a 
fork/NN/I-NP/I-PNP/O/P1/fork 
././O/O/O/O/.

SENNA: A Fast Semantic Role Labeling (SRL) Tool

noreply@blogger.com (Pedro Paulo Balage) — Sat, 26 Feb 2011 18:49:00 +0000

Last days I played with a very fast Semantic Role Labeling (SRL) tool. The software is quite new and fully developed in pure C to provide efficiency.

The software outputs a host of Natural Language Processing (NLP) predictions: part-of-speech (POS) tags, chunking (CHK), name entity recognition (NER) and semantic role labeling (SRL).part-of-speech (POS) tags, chunking (CHK), name entity recognition (NER) and semantic role labeling (SRL).

http://ml.nec-labs.com/senna/

A friend was using ASSERT (http://cemantix.org/assert.html) to provide the SRL tags, but the software broke every time. Also, because ASSERT is a pipeline of other solutions, the runtime was enormous huge. Solution: change to SENNA.

I did some tests with the software and I read a report about it. The software is very robust and offer an easy tool for those who need to use SRL in their project. The software is distributed with the binaries (Linux32, Linux64, Mac, Windows) or it can be compiled at your machine. My friend's experiment runtime changed from 4 days to 4 hours using SENNA.

The performance in each task is much closer to the best systems. Bellow, the system output.

Google announced a new product yesterday.
   Google       NNP     S-NP     S-ORG           -      S-A0
 announced      VBD     S-VP         O   announced       S-V
         a      DT      B-NP         O           -      B-A1
       new      JJ      I-NP         O           -      I-A1
   product      NN      E-NP         O           -      E-A1
 yesterday      NN      S-NP         O           -  S-AM-TMP
         .       .         O         O           -         O

Experiments with Statistical Language Generation

noreply@blogger.com (Pedro Paulo Balage) — Tue, 18 Jan 2011 13:47:00 +0000

Some time ago I tried to imagine how amazingly our brain is able to generate language. This is a very hard problem and it isn't totally understood by the science. As I wish to do some practical experiments I used what I had in hands: statistics.

Simple, there is some methods to generate text in the natural language toolkit for python (NLTK). I studied them and I created some curious examples. Here, I post my examples.

First, let's create a code for learning a N-gram language model where each gram is a word present in the text. I use two corpora, the Bible Genesis in English and Reuters Trade. Both are available in the NLTK. Bellow the code.

# Import the corpus and functions used from nltk library
from nltk.corpus import reuters
from nltk.corpus import genesis
from nltk.probability import LidstoneProbDist
from nltk.model import NgramModel

# Tokens contains the words for Genesis and Reuters Trade
tokens = list(genesis.words('english-kjv.txt'))
tokens.extend(list(reuters.words(categories = 'trade')))

# estimator for smoothing the N-gram model
estimator = lambda fdist, bins: LidstoneProbDist(fdist, 0.2)

# N-gram language model with 3-grams
model = NgramModel(3, tokens,estimator)

# Apply the language model to generate 50 words in sequence
text_words = model.generate(50)

# Concatenate all words generated in a string separating them by a space.
text = ' '.join([word for word in text_words])

# print the text
print text

In the line 12 I use a Lidstone's smoothing technique for the n-gram language model. For more information about the use of a smoothing techniques in n-gram language modelling consult the Wikipedia or a good statistical natural language processing book. I am generating 50 words in the example using 3-gram model. This means that the model will pick the previous 2 words and check in the text for words that came after and pick one of them respecting the estimation. For each 2 words, the model will select the third, and so do it for all 50 words.

Some examples of two texts generated for the 3-gram language model were (see that each time the algorithm runs it generate a random another example):

In the fiscal year , with tabret , and menservants , and sat in the presence of the heaven ; and darkness was upon all that he may waive any retaliation if it accelerated too fast and the one affected by the hand of Es for I also here looked

In the speech , the brother of Japheth the elder , even with Isaac ' s steel exports , the statistics department said first quarter . There were indications of improvement in Britain ' s deputy minister of Economy and Communication . Speaking in a coffin in Egypt , Jacob

You can see that the algorithm mixed the texts, some times speaks about heaven, Jacob and Egypt and other times about improvements in Britain, retaliation, minister of Economy and Communication. The most important is that you can, in a high level, say that the text respects the English grammar and that the sequence of words are not totally random. This is the result of the n-gram model. For much more n-grams do you have in your model, more accurate will be the text. For example, let's generate for 5 grams:

# N-gram language model with 5-grams
model = NgramModel(5, tokens,estimator)

In the beginning God created the heaven and the earth . And when Joseph saw Benjamin with them , he said . Such an approach would finance the trade deficit but allow for its gradual resolution over time . YEUTTER SAYS U . S ., Company spokesmen told Reuters .

In this example, the text generated starts exactly equal the bible, but in the middle it changes to some random text from Reuters. In my examples I am not moving the start point, so the random text will always start with "In the beginning God ..." (for 5 grams). What we can verify is that the text is much more coherent (in a high level again) than the previous 3-gram modelling. It keeps the basic English syntax even if the meaning (semantic) is not coherent.

Ok, if the text with more n-grams has better coherence, why we do not use a higher n-gram for modelling? Well, first because time/space constrains. The more n-gram we pick, the more time processing and memory we require (it grows experientially). In this example I am using a text with near 200.000 words, but imagine you using it with a more than 1 billion words corpus, which is usual for language modelling. Second because the probability itself. The number of different words that we can find after the same previous 3 words is much bigger than the ones we can find after the previous 6 or 7 words. So, in the case of a high n-gram, the probability function do not performs well because the insufficient amount of data. Again, more data more time, it is a complex trade-off. What is usual in computational linguistics is a 3-5 n-gram models, this is what you will find in the papers.

Just for fun, let's see how crazy the random words are chosen by a 2-gram language model.

# N-gram language model with 2-grams
model = NgramModel(2, tokens,estimator)

In the rods of the land of her pitcher from 27 of an agreement aims to impose permanent quotas on Japan hopes of the latest monthly figure excluding Dutch - import quotas on its currency and 24 . Another possible to rebuild our goal set it said " We cannot

For complementing my example, I will now demonstrate that you can choose even letters for your language model. The code is posted bellow.

from nltk.corpus import reuters
from nltk.corpus import genesis
from nltk.probability import LidstoneProbDist
from nltk.model import NgramModel

# Tokens contains the words for English Genesis and Reuters Trade
tokens = list(genesis.words('english-kjv.txt'))
tokens.extend(list(reuters.words(categories = 'trade')))

# Generate a list of letters for each word in tokens separated by a space
letters = [letter for letter in ' '.join([word for word in tokens])]

# estimator for smoothing the N-gram model
estimator = lambda fdist, bins: LidstoneProbDist(fdist, 0.2)

# N-gram language model with 4-grams
model = NgramModel(4, letters,estimator)

# Apply the language model to generate 200 letters in sequence
text = model.generate(200)
text_output = ''.join([letter for letter in text])
print text_output

I chose 4-grams, i.e., pick 3 previous letters and select the fourth. The text generated for 200 letters is bellow.

In to On to serity seriod on the 62 bid to Wash , and he U . " the rawing up , sinese ther it he sharah fedeji Fulforce againted import becassed the Asensurplus in July alway to Tariffections in 1 . w

Of, course the readability is worse than picking words, since some words do not even exist. The punctuation signals also do not work well and the syntax, well, it is just some letters. You can increase or decrease the n-gram factor to see more results.

The most amazing think in these examples is that: yes we could generate text, we could in some examples generate texts that people will say made by other people (some crazy people maybe). And you see, what are we using? Just a corpus and statistics! Natural Language Processing is amazing!

Spell Checker

noreply@blogger.com (Pedro Paulo Balage) — Sun, 16 Jan 2011 19:23:00 +0000

One of the most important task in Natural Language Processing is Spell Checking. This post will explain how to deal with this. I am going to use python language as example.

First of all, let's not re-create the things. There is a good tutorial by Peter Norvig about spelling Corrector in the link: http://norvig.com/spell-correct.html. Let's use the same code in our example.

import re, collections

def words(text): return re.findall('[a-z]+', text.lower()) 

def train(features):
    model = collections.defaultdict(lambda: 1)
    for f in features:
        model[f] += 1
    return model

NWORDS = train(words(file('big.txt').read()))

alphabet = 'abcdefghijklmnopqrstuvwxyz'

def edits1(word):
   splits     = [(word[:i], word[i:]) for i in range(len(word) + 1)]
   deletes    = [a + b[1:] for a, b in splits if b]
   transposes = [a + b[1] + b[0] + b[2:] for a, b in splits if len(b)>1]
   replaces   = [a + c + b[1:] for a, b in splits for c in alphabet if b]
   inserts    = [a + c + b     for a, b in splits for c in alphabet]
   return set(deletes + transposes + replaces + inserts)

def known_edits2(word):
    return set(e2 for e1 in edits1(word) for e2 in edits1(e1) if e2 in NWORDS)

def known(words): return set(w for w in words if w in NWORDS)

def correct(word):
    candidates = known([word]) or known(edits1(word)) or known_edits2(word) or [word]
    return max(candidates, key=NWORDS.get)

Spelling Corrector code from: http://norvig.com/spell-correct.html

To use this code you have to download the file big.txt in the same directory you are working. As Peter Norvig explain: "the file is a concatenation of several public domain books from Project Gutenberg and lists of most frequent words from Wiktionary and the British National Corpus". So, we expect to have the coverage for the most common words.

To use this code you have to call the function correct(word). In the example above I am using a python interpreter to show the results.

>>> correct('see')
'see'
>>> correct('tee')
'the'
>>> correct('twelfe')
'twelve'
>>> correct('speling')
'spelling'
>>> correct('korrecter')
'corrector'

You see that the function always return a suggestion. If the word is present in the file, so the suggestion is the word itself. If not, the algorithm looks for the edit distance between your word and the words in the file. It is more probable that you mistype by one or two letters and so, the words closer to it are the most probable to be correct. The algorithm looks for the words with edit distance 1 and second with edit distance 2. If it could not found any valid word in the both results, so it returns the input word unchanged.

One modification I suggest is to use a real spell checker dictionary instead the corpus. You can, for example, download the linux aspell dictionary in the follow command line:

aspell --lang en dump repl > big.txt

The aspell program can be found in the major linux distributions, including the Ubuntu linux.
The file generated ('big.txt') has now 138622 words. This file can represent all English language formalized by a dictionary and the variations found for each morpheme (e.g. volunteer,
volunteering, volunteerism, volunteered, volunteer's, volunteers). The new set of test has now the following result:

>>> correct('see')
'see'
>>> correct('tee')
'tee'
>>> correct('twelfe')
'twelve'
>>> correct('speling')
'spelling'
>>> correct('korrecter')
'correcter'

The use of this new file brings you some particular aspects. For example the words tee and correcter sound strange, but they are totally valid in English (tee, correcter).

So, it is a matter of know the purpose of your spell corrector and use the appropriate file to generate the valid words.

For a full explanation for the Norvig's code you can read his page (http://norvig.com/spell-correct.html.). There you can also find the same spell checker code in other languages than Python.

Twitter Sentiment Analysis

noreply@blogger.com (Pedro Paulo Balage) — Sat, 08 Jan 2011 19:05:00 +0000

I have so much work in the last months, that is why I am far from posts since them. During this time I did some interesting projects that I will start posting here.

As I am conducting researches in Sentiment Analysis and I found a very interesting database for twitter sentiment analysis. I am not very particular interesting in twitter, but since there is many researches for sentiment analysis in this tool I found useful to promote it here.

The survey is conducted mainly by the people on http://twittersentiment.appspot.com/, a great app for twitter sentiment analysis. Many others can be found in the survey doc:

https://spreadsheets0.google.com/ccc?key=tVfLhBxao70Fk9bAtebp_xQ

Comic Book Markup Language

noreply@blogger.com (Pedro Paulo Balage) — Thu, 21 Oct 2010 10:15:00 +0000

Yes, it exist. I read about this in the corpora list.

From: http://www.cbml.org/ :

What is CBML?

CBML, or Comic Book Markup Language, is a TEI-based XML vocabulary (with DTD and schema representations) designed to accommodate the XML encoding of comic books and graphic novels.

Understanding a Celebrity with His Salient Events

noreply@blogger.com (Pedro Paulo Balage) — Sun, 29 Aug 2010 15:25:00 +0000

How the NLP is amazing! I was reading the today NLP News and I got the follow paper published in the Springer Lecture Notes in Computer Science, 2010. I would never imagine such application for NLP, but it is, and works!

Internet has become a resourceful platform for people to collect information. Specially, it becomes one of the main ways to understand a celebrity. However, the huge volume of information makes troubles for people to get what they really want. How to filter out needless information through numerous data and form a brief review of a celebrity become necessary for people to understand the person. In this paper, we propose a novel solution for understanding a celebrity by summarizing his most salient historical events, and a framework is outlined. The framework contains three main components: attention tracking, event mining from News, and event summarization. First, with the comparison of users’ attention and media attention on a celebrity, News corpus is proved to be able to represent the users’ attention. Second, keywords are extracted from the News according to different time periods for choosing summary sentences. Third, a final event description of the celebrity will be given. Finally, we will show the user interface of our system. Our experimental results show that the proposed solution can effectively process the news corpus and provide us with accurate description of the celebrity.

Link: http://www.springerlink.com/content/4x333830nq8w80p7/

The NLP field each day has been showed more and more important. A lot of NLP application should yet be discovered! I love study NLP!

The Top Three hottest new majors for a career in technology

noreply@blogger.com (Pedro Paulo Balage) — Thu, 26 Aug 2010 15:45:00 +0000

From: Microsft Jobs blog:

Posted Monday, August 23 2010 by The JobsBloggers

Have you ever wondered what fields of study are hot right now in the world of technology? Or maybe you’re starting to think about declaring your major and you’re looking for some real world guidance?

It is worth thinking beyond a traditional Computer Science degree or even an Electrical Engineering & Computer Science (EECS) program. Microsoft is hiring people with unique backgrounds, some that are new with the inception of the Cloud, web services and the amazing scale at which the industry is operating (Exabyte anyone?).

The following is my list of the Top Three hottest academic areas for a future career in tech:

Data Mining/Machine Learning/AI/Natural Language Processing
All of these fields help us sift through and organize huge amounts of information or data. When you apply your knowledge in these areas to a challenging problem in the online space, you know that you are working at a scale that is just immense. It’s much easier said than done. If you have a passion for this area and have a technical background there are a multitude of open positions that might hold a long-term career for you. With the move to the cloud and the sheer amount of information on the web, this area of expertise will continue to be in great demand. Microsoft has a great need for both people interested in the research space and the applied space which is very refreshing.

Business Intelligence/Competitive Intelligence
The ability to see trends, make sense of data to a business audience and help to understand your customers requires a special person. Someone with a mix of engineering, BI/CI experience and a business mindset can take this field to the next level. You will help increase any employer’s bottom line and be able to provide organized data that is extremely valuable to any business. You can help drive business decisions and help your internal audience understand what the data is telling or showing you.

Analytics/Statistics – specifically Web Analytics, A/B Testing and statistical analysis
All of these subjects are offshoots of traditional degrees in CS and mathematics. They all apply to the online world we live in and will also be in great demand as we continue to monetize the web. Retailers, web services, and advertisers will need people in these fields as they try to get the most for their advertising money. As we continue to see the dollar amounts spent for online advertising worldwide, these fields will be hot and we will see online advertising change over time as a result of these positions.

If these fields interest you and you want to find out what some of these jobs really entail, visit our website and search on the terms above to get a more detailed look at the positions. These fields are very HOT and looking long term, the demand will be just that much greater in these areas.

The AGI Summer School 2009

noreply@blogger.com (Pedro Paulo Balage) — Mon, 16 Aug 2010 14:03:00 +0000

An excellent post from Joel Pitt in the OpenCog blog. Excellent videos!

The AGI Summer School 2009: "

In the middle of last year, Xiamen University hosted the first international summer school on Artificial General Intelligence. While several of the core OpenCog developers, and Ben Goertzel were there to teach, it passed by somewhat quietly on our blog as we waited for the videos and presentations to be put together by independent film-maker Raj Dye.

Raj actually completed these early 2010, but due to various myself and others being very busy at the time, we didn’t post them here. However, we continue to get many requests for an easy introduction and a tutorial to OpenCog. Fortunately one of the videos is an introduction to the software framework:

The OpenCog Software Framework – presented by Joel Pitt (me) and Ben Goertzel.

Of course, this is a the high level overview, and the other videos available on the AGI Summer School site focus on more specific aspects of OpenCog, such as:

Automated Program Learning: the MOSES Algorithm presented by Nil Geisweiller.
An introduction to PLN by myself.

…among many others!

Announcing Python NLTK Demos

noreply@blogger.com (Pedro Paulo Balage) — Wed, 11 Aug 2010 13:22:00 +0000

Bellow a post from the StreamHacker.com blog presenting a Demo for some features in the NLTK tool.

Announcing Python NLTK Demos:

"
If you want to see what NLTK can do, but don't want to go thru the effort of installation and learning how to use it, then check out my Python NLTK demos.
It currently demonstrates the following functionality:

part-of-speech tagging with the default NLTK pos tagger
chunking and named entity recognition with the default NLTK chunker
sentiment analysis with a combination of a naive bayes classifier and a maximum entropy classifier, both trained on the movie reviews corpus

If you like it, please share it. If you want to see more, leave a comment below. And if you are interested in a service that could apply these processes to your own data, please fill out this NLTK services survey.

Other Natural Language Processing Demos

Here's a list of similar resources on the web:

A demo of the Stanford Parser with a javascript API: Natural-language Parsing For The Web
A demo of the FreeLing language analysis suite: FreeLing Demo
Emotional identification from text: EmoLib

Do the languages we speak shape the way we think?

noreply@blogger.com (Pedro Paulo Balage) — Sun, 25 Jul 2010 00:06:00 +0000

Lost in Translation

New cognitive research suggests that language profoundly influences the way people see the world; a different sense of blame in Japanese and Spanish

The Tower of Babel' by Pieter Brueghel the Elder, 1563.

Do the languages we speak shape the way we think? Do they merely express thoughts, or do the structures in languages (without our knowledge or consent) shape the very thoughts we wish to express?

Take "Humpty Dumpty sat on a..." Even this snippet of a nursery rhyme reveals how much languages can differ from one another. In English, we have to mark the verb for tense; in this case, we say "sat" rather than "sit." In Indonesian you need not (in fact, you can't) change the verb to mark tense.

In Russian, you would have to mark tense and also gender, changing the verb if Mrs. Dumpty did the sitting. You would also have to decide if the sitting event was completed or not. If our ovoid hero sat on the wall for the entire time he was meant to, it would be a different form of the verb than if, say, he had a great fall.

In Turkish, you would have to include in the verb how you acquired this information. For example, if you saw the chubby fellow on the wall with your own eyes, you'd use one form of the verb, but if you had simply read or heard about it, you'd use a different form.

Do English, Indonesian, Russian and Turkish speakers end up attending to, understanding, and remembering their experiences differently simply because they speak different languages?

These questions touch on all the major controversies in the study of mind, with important implications for politics, law and religion. Yet very little empirical work had been done on these questions until recently. The idea that language might shape thought was for a long time considered untestable at best and more often simply crazy and wrong. Now, a flurry of new cognitive science research is showing that in fact, language does profoundly influence how we see the world.

The question of whether languages shape the way we think goes back centuries; Charlemagne proclaimed that "to have a second language is to have a second soul." But the idea went out of favor with scientists when Noam Chomsky's theories of language gained popularity in the 1960s and '70s. Dr. Chomsky proposed that there is a universal grammar for all human languages—essentially, that languages don't really differ from one another in significant ways. And because languages didn't differ from one another, the theory went, it made no sense to ask whether linguistic differences led to differences in thinking.

The search for linguistic universals yielded interesting data on languages, but after decades of work, not a single proposed universal has withstood scrutiny. Instead, as linguists probed deeper into the world's languages (7,000 or so, only a fraction of them analyzed), innumerable unpredictable differences emerged.

Of course, just because people talk differently doesn't necessarily mean they think differently. In the past decade, cognitive scientists have begun to measure not just how people talk, but also how they think, asking whether our understanding of even such fundamental domains of experience as space, time and causality could be constructed by language.

For example, in Pormpuraaw, a remote Aboriginal community in Australia, the indigenous languages don't use terms like "left" and "right." Instead, everything is talked about in terms of absolute cardinal directions (north, south, east, west), which means you say things like, "There's an ant on your southwest leg." To say hello in Pormpuraaw, one asks, "Where are you going?", and an appropriate response might be, "A long way to the south-southwest. How about you?" If you don't know which way is which, you literally can't get past hello.

About a third of the world's languages (spoken in all kinds of physical environments) rely on absolute directions for space. As a result of this constant linguistic training, speakers of such languages are remarkably good at staying oriented and keeping track of where they are, even in unfamiliar landscapes. They perform navigational feats scientists once thought were beyond human capabilities. This is a big difference, a fundamentally different way of conceptualizing space, trained by language.

Differences in how people think about space don't end there. People rely on their spatial knowledge to build many other more complex or abstract representations including time, number, musical pitch, kinship relations, morality and emotions. So if Pormpuraawans think differently about space, do they also think differently about other things, like time?

To find out, my colleague Alice Gaby and I traveled to Australia and gave Pormpuraawans sets of pictures that showed temporal progressions (for example, pictures of a man at different ages, or a crocodile growing, or a banana being eaten). Their job was to arrange the shuffled photos on the ground to show the correct temporal order. We tested each person in two separate sittings, each time facing in a different cardinal direction. When asked to do this, English speakers arrange time from left to right. Hebrew speakers do it from right to left (because Hebrew is written from right to left).

Pormpuraawans, we found, arranged time from east to west. That is, seated facing south, time went left to right. When facing north, right to left. When facing east, toward the body, and so on. Of course, we never told any of our participants which direction they faced. The Pormpuraawans not only knew that already, but they also spontaneously used this spatial orientation to construct their representations of time. And many other ways to organize time exist in the world's languages. In Mandarin, the future can be below and the past above. In Aymara, spoken in South America, the future is behind and the past in front.

In addition to space and time, languages also shape how we understand causality. For example, English likes to describe events in terms of agents doing things. English speakers tend to say things like "John broke the vase" even for accidents. Speakers of Spanish or Japanese would be more likely to say "the vase broke itself." Such differences between languages have profound consequences for how their speakers understand events, construct notions of causality and agency, what they remember as eyewitnesses and how much they blame and punish others.

In studies conducted by Caitlin Fausey at Stanford, speakers of English, Spanish and Japanese watched videos of two people popping balloons, breaking eggs and spilling drinks either intentionally or accidentally. Later everyone got a surprise memory test: For each event, can you remember who did it? She discovered a striking cross-linguistic difference in eyewitness memory. Spanish and Japanese speakers did not remember the agents of accidental events as well as did English speakers. Mind you, they remembered the agents of intentional events (for which their language would mention the agent) just fine. But for accidental events, when one wouldn't normally mention the agent in Spanish or Japanese, they didn't encode or remember the agent as well.

In another study, English speakers watched the video of Janet Jackson's infamous "wardrobe malfunction" (a wonderful nonagentive coinage introduced into the English language by Justin Timberlake), accompanied by one of two written reports. The reports were identical except in the last sentence where one used the agentive phrase "ripped the costume" while the other said "the costume ripped." Even though everyone watched the same video and witnessed the ripping with their own eyes, language mattered. Not only did people who read "ripped the costume" blame Justin Timberlake more, they also levied a whopping 53% more in fines.

Beyond space, time and causality, patterns in language have been shown to shape many other domains of thought. Russian speakers, who make an extra distinction between light and dark blues in their language, are better able to visually discriminate shades of blue. The Piraha, a tribe in the Amazon in Brazil, whose language eschews number words in favor of terms like few and many, are not able to keep track of exact quantities. And Shakespeare, it turns out, was wrong about roses: Roses by many other names (as told to blindfolded subjects) do not smell as sweet.

Patterns in language offer a window on a culture's dispositions and priorities. For example, English sentence structures focus on agents, and in our criminal-justice system, justice has been done when we've found the transgressor and punished him or her accordingly (rather than finding the victims and restituting appropriately, an alternative approach to justice). So does the language shape cultural values, or does the influence go the other way, or both?

Languages, of course, are human creations, tools we invent and hone to suit our needs. Simply showing that speakers of different languages think differently doesn't tell us whether it's language that shapes thought or the other way around. To demonstrate the causal role of language, what's needed are studies that directly manipulate language and look for effects in cognition.

One of the key advances in recent years has been the demonstration of precisely this causal link. It turns out that if you change how people talk, that changes how they think. If people learn another language, they inadvertently also learn a new way of looking at the world. When bilingual people switch from one language to another, they start thinking differently, too. And if you take away people's ability to use language in what should be a simple nonlinguistic task, their performance can change dramatically, sometimes making them look no smarter than rats or infants. (For example, in recent studies, MIT students were shown dots on a screen and asked to say how many there were. If they were allowed to count normally, they did great. If they simultaneously did a nonlinguistic task—like banging out rhythms—they still did great. But if they did a verbal task when shown the dots—like repeating the words spoken in a news report—their counting fell apart. In other words, they needed their language skills to count.)

All this new research shows us that the languages we speak not only reflect or express our thoughts, but also shape the very thoughts we wish to express. The structures that exist in our languages profoundly shape how we construct reality, and help make us as smart and sophisticated as we are.

Language is a uniquely human gift. When we study language, we are uncovering in part what makes us human, getting a peek at the very nature of human nature. As we uncover how languages and their speakers differ from one another, we discover that human natures too can differ dramatically, depending on the languages we speak. The next steps are to understand the mechanisms through which languages help us construct the incredibly complex knowledge systems we have. Understanding how knowledge is built will allow us to create ideas that go beyond the currently thinkable. This research cuts right to the fundamental questions we all ask about ourselves. How do we come to be the way we are? Why do we think the way we do? An important part of the answer, it turns out, is in the languages we speak.

—Lera Boroditsky is a professor of psychology at Stanford University and editor in chief of Frontiers in Cultural Psychology.

Source: The Wall Street Journal

Tip: Use Natural Language to Search your Windows 7 System

noreply@blogger.com (Pedro Paulo Balage) — Thu, 22 Jul 2010 18:19:00 +0000

Windows Search supports some pretty complex search capabilities. The set of rules that Windows Search follows when interpreting what you type in a search box are referred to as Advanced Query Syntax (AQS). You can filter by file type, use Boolean operators and Boolean properties, specify ranges, and more. Detailed documentation about AQS is available in the Windows Developer Center.

But did you know Windows Search supports natural language? If you don’t fancy Boolean formulations, you may want to try the natural-language approach to searching.

So, instead of typing kind:email from:(Carl OR Ed) received:this week, you can enter email from Carl or Ed received this week. The system looks for key words (like “email”), filters out prepositions (such as “from”), handles conjunctions without making you capitalize them, and assumes the rest of what you type consists of property values that it should try to match.

But first you need to turn on the natural language searching capabilities. To do this, open Windows Explore, choose Organize, and select Folder And Search Options. In the Folder Options dialog, click the Search tab. On the Search tab, select Use Natural Language Search.

Source: Technet

Google Translates “Call Us For Free” To “Skype” In Italian

noreply@blogger.com (Pedro Paulo Balage) — Sun, 27 Jun 2010 03:25:00 +0000

Google Translates “Call Us For Free” To “Skype” In Italian: "Google Translates “Call Us For Free” To “Skype” In ItalianTechCrunch (blog)Update: Google has confirmed that this was a machine translation error and not the result of a crowdsourced effort to change a translation. ..."

Sometimes statistical machine translation do not work properly :)

Syntax found in the brain -- not in Broca's area

noreply@blogger.com (Pedro Paulo Balage) — Fri, 04 Jun 2010 01:58:00 +0000

Syntactic processing and Broca's area have been cozy bedfellows ever since work in the 1970s showed that patients with Broca's aphasia had difficulty comprehending syntactically complex sentences. Despite the fact that further lesion based evidence severely weakened the relationship (e.g., Broca's aphasics are pretty good at grammaticality judgments) subsequent PET and fMRI studies prolonged the marriage by showing that the comprehension of complex sentences activates Broca's area more than simple sentences. These days a look at the literature on Broca's area and sentence processing is full of controversy with proposals ranging from the region supporting syntactic movement processing (Grodzinsky) hierarchical structure and phrase structure building (Friederici), "linearization" (Bornkessel-Schlesewsky), and several more domain-general functions such as cognitive control (Novick), "unification" (Hagoort) and working memory (Caplan; Rogalsky/Hickok).

Meanwhile, evidence has been accumulating that the anterior temporal lobe may house a network that behaves much more like a syntactic computation system in that it seems to be highly correlated with the presence or absence of syntactic information in a sentence (Dronkers, et al. 2004; Friederici, et al., 2000; Humphries, et al. 2005,2006; Mazoyer et al., 1993; Rogalsky & Hickok, 2008; Stowe et al., 1998; Vandenberghe, Nobre, & Price, 2002).

A new study in Brain and Language weighs in on the issue using the psycholinguists' favorite work of literature, Alice in Wonderland. Unlike most psycholinguistic nods to Lewis Carroll (a.k.a. Charles Lutwidge Dodgson), the study by Jonathan Brennan and colleagues did not use Jabberwocky sentences, but instead had subjects listen to text from Alice while chillin' in a giant magnetic donut. The authors calculated word-by-word the amount of syntactic structure that was involved in integrating each word (basically a syntactic tree node counting analysis). These values were then correlated with the fMRI signal.

So what brain region correlated with syntactic structure? You guessed it: the anterior temporal lobe.

Source: Talking Brains Blog

Microsoft Research Conversation Corpus

noreply@blogger.com (Pedro Paulo Balage) — Tue, 01 Jun 2010 17:38:00 +0000

Microsoft Research Conversation Corpus: "The Microsoft Research Conversation Corpus is a collection of approximately 1.3 million conversations gathered from Twitter.com. These conversations were collected using the Twitter Public API in 2009, between July 1 and August 27. Many of the tweets on Twitter are one-way, “broadcast” tweets, but this corpus is focused on two-way interactions between Twitter users, public conversations that could be used for research, on dialog modeling. Conversations in the corpus range from 2 to 243 posts in length, though the majority of interactions are short; those of length 2 account for 69 percent of the data. A more complete description of this effort can be found in the paper Ritter, Alan, Colin Cherry, and Bill Dolan “Unsupervised Modeling of Twitter Conversations,” NAACL 2010. We would appreciate your citing this work if you publish research exploiting this corpus. Given the conversational nature of Twitter, the data contain content that may be offensive, harmful, inaccurate, inappropriate, or unsuitable for minors. Microsoft did not create, does not endorse, and is not responsible for the content of the data. By using the data, you agree to supervise usage by minors whom you permit to use the data. Under no circumstances will Microsoft be liable in any way for the content of the data."

The Semantic Web as a Linguistic Resource: Opportunities for Natural Language Generation

noreply@blogger.com (Pedro Paulo Balage) — Sat, 15 May 2010 15:47:00 +0000

The Semantic Web as a Linguistic Resource: Opportunities for Natural Language Generation

Research and Development in Intelligent Systems XXII
Proceedings of AI-2005, the Twenty-fifth SGAI International Conference on Innovative Techniques and Applications of Artificial Intelligence, Cambridge, UK, December 2005

10.1007/978-1-84628-226-3_7

Max Bramer, Frans Coenen and Tony Allen

Chris Mellish¹ and Xiantang Sun¹

(1)	Department of Computing Science, University of Aberdeen, Aberdeen, AB24 3UE, UK

Abstract

This paper argues that, because the documents of the semantic web are created by human beings, they are actually much more like natural language documents than theory would have us believe. We present evidence that natural language words are used extensively and in complex ways in current ontologies. This leads to a number of dangers for the semantic web, but also opens up interesting new challenges for natural language processing. This is illustrated by our own work using natural language generation to present parts of ontologies.

PDF (1.2 MB)

Semantic network methods to disambiguate natural language meaning

noreply@blogger.com (Pedro Paulo Balage) — Fri, 07 May 2010 15:17:00 +0000

Semantic network methods to disambiguate natural language meaning: "A computer implemented data processor system automatically disambiguates a contextual meaning of natural language symbols to enable precise meanings to be stored for later retrieval from a natural language database, so that natural language database design is automatic, to enable flexible and efficient natural language interfaces to computers, household appliances and hand-held devices."

What's in a Tweet?

noreply@blogger.com (Pedro Paulo Balage) — Wed, 28 Apr 2010 18:06:00 +0000

Researchers at the Palo Alto Research Center (PARC) are developing new ways to deal with the torrent of information flowing from social media sites like Twitter. They have developed a Twitter "topic browser" that extracts meaning from the posts in a user's timeline. This could help users scan through thousands of tweets quickly, and the underlying technology could also offer novel ways of mining Twitter for information or for creating targeted advertising.

The researchers' idea was to provide a way for users to deal with a large number of Twitter messages quickly. They found that many users wanted to be able to quickly catch up on what's been going on, without having to go through every single tweet in their timeline.

Ed Chi, area manager and principal scientist for the Augmented Social Cognition Research Group at PARC, says that the information coming through Twitter resembles a stream--users will dip into it from time to time, but they don't want to consume it all at once. His group's work is called the "Eddi Project" in reference to the idea of eddies in a stream.

The researchers developed two main ways of filtering Twitter content. The first, presented recently at the ACM Conference on Human Factors in Computing Systems in Atlanta, is a recommendation system that ranks which posts in a Twitter stream a user is likely to find most interesting, based on factors such as the contents of posts as well as his interactions with other Twitter users. The second tool, the Twitter topic browser, summarizes the contents of a user's timeline so that the user can quickly survey what information has come through Twitter without having to read through every post.

Read the full story in http://www.technologyreview.com/web/25189/page1/

The Biological Nature of Human Language

noreply@blogger.com (Pedro Paulo Balage) — Wed, 28 Apr 2010 17:50:00 +0000

The Biological Nature of Human Language
Anna Maria Di Sciullo, Massimo Piattelli-Palmarini, Kenneth Wexler, Robert C. Berwick et al.
Biolinguistics 4.1: 004–034, 2010

Biolinguistics aims to shed light on the specifically biological nature of human language, focusing on five foundational questions: (1) What are the properties of the language phenotype? (2) How does language ability grow and mature in individuals? (3) How is language put to use? (4) How is language implemented in the brain? (5) What evolutionary processes led to the emergence of language? These foundational questions are used here to frame a discussion of important issues in the study of language, exploring whether our linguistic capacity is the result of direct selective pressure or due to developmental or biophysical constraints, and assessing whether the

neural/computational components entering into language are unique to human language or shared with other cognitive systems, leading to a discussion of advances in theoretical linguistics, psycholinguistics, comparative

animal behavior and psychology, genetics/genomics, disciplines that can now place these longstanding questions in a new light, while raising challenges for future research.

More readings on this subject try the BIOLINGUISTICS Magazine:

http://www.biolinguistics.eu/index.php/biolinguistics/

CICLing 2010 Conference Recordings Available

noreply@blogger.com (Pedro Paulo Balage) — Sat, 17 Apr 2010 23:55:00 +0000

There some recordings available for the 11th International Conference on Intelligent Text Processing and Computational Linguistics ( CICLing 2010).

http://profs.info.uaic.ro/~cicling2010/

The recordings are not so good :(

JHU Summer School 2009 available on video

noreply@blogger.com (Pedro Paulo Balage) — Thu, 15 Apr 2010 14:32:00 +0000

I received an e-mail from NAACL with the following message. I took a look and there is interesting topics in NLP with good speakers.

JHU has just recently released videos for their 2009 two week summer school on NLP and Human Language Technology.

http://videolectures.net/clspss09_baltimore/

You can find quite a bit of information about the 2009 summer school here, which may help orient you to the videos :

http://www.clsp.jhu.edu/workshops/ws09/

Stanford software is gaining the sophistication to comprehend what humans write

noreply@blogger.com (Pedro Paulo Balage) — Sat, 20 Feb 2010 21:33:00 +0000

Bellow a very good news from Manning about how Stanford is working with NLP issues and what we can expected for future NLP capabilities.

----

Computer science and linguistics professor says software is steadily gaining the sophistication needed to comprehend what humans write and to help us sort through the chatter of information overload.

BY DAVID ORENSTEIN

For people who despair that there is too much information online, Chris Manning has a response: Technology is not the problem. In fact, technology may understand what you're trying to say.

At the annual conference of the American Association for the Advancement of Science (AAAS) in San Diego, the Stanford associate professor of computer science and linguistics will talk about enabling computers to process human language well enough to use the information it conveys.

"The problem of the age is information overload," said Manning, who'll speak Friday, Feb. 19, at 4:10 p.m. in Room 2 of the San Diego Convention Center. "The fundamental challenge I'm going to talk about is how we can get computers to actually understand at least a reasonable amount of what they read."

As computers make more sense of what's online, they will deliver more relevant search results and will help summarize, structure and act on information that individuals care about, much like a personal assistant.

A smartphone email program that understands the difference between "We need the Q4 figures" and "We found the Q4 figures" could prove invaluable to a busy executive.

Computers also could help researchers extract key facts from a sea of articles to create and update databases. In fact, Manning already has developed software that mines biology research papers for basic data.

State of the art

Manning readily acknowledges that the field of natural language understanding has a long way to go to catch up with popular imagination.

"The state of the art is still highly incomplete," he says. "We're just not at the level of what we see in science fiction movies. But human language technology has been making enormous advances."

In his AAAS talk, Manning will describe work on three emerging technologies at Stanford's Natural Language Processing (NLP) Group.

Working with linguistics Associate Professor Dan Jurafsky, Manning has been developing a fundamental set of tools to help computers do what a pupil does in grammar school: Parse sentences. As with humans, computers begin to understand sentences by recognizing parts of speech and how the sentence is structured.

The underlying technology is a branch of artificial intelligence called probabilistic machine learning. Essentially, computers are programmed to read a large number of sentences and then analyze their structure and elements, compiling statistics about verbs and nouns, and keeping track of what the subject of the sentence is doing.

Based on those statistics, for instance, a computer might conclude that "horse" is likely to be the subject of a sentence and that "hay" is something that horses might eat. Technical demonstrations of Manning and Jurafsky's language parsing software are available on the NLP Group website.

Building on that level of understanding, Manning's group has created software to sort out ambiguities in language by taking whole sentences into account when deciding what each word means. For instance, "make up" can have at least three meanings: to reconcile after a spat, to concoct a story, or to apply cosmetics. The technical solution, called "joint inference," is to look for other words in the sentence that are statistically shown to be relevant. If the word "argument" is there, the computer will lean toward "to reconcile."

Finally, Manning will talk about a technology called robust textual inference, which can read a passage of text and determine whether a conclusion about it is supported. That reading comprehension task is important because it's similar to what people sometimes expect search engines to do; they'll type in a conclusion ("hotels with free Wi-Fi") and hope the engine leads them to text that supports it ("Free Wireless High-Speed Internet access in all rooms").

With tremendous volumes of information appearing online every day in social networks, Manning says the need to train computers to understand human language, rather than meticulously structured data, is only increasing. The next research frontier may therefore be getting a computer to understand "C U soon, QT."

David Orenstein is associate director of communications at the Stanford School of Engineering.

Source: Stanford News

Cognition’s Semantic Technology Contributes to Microsoft’s Bing

noreply@blogger.com (Pedro Paulo Balage) — Fri, 19 Feb 2010 13:13:00 +0000

Bellow I present a news showing that Microsoft Bing will license the Cognition platform (from Cognition.com) to enhance his web search. A good video about the Cognition platform can be found in http://www.cognition.com/info/videodemo.html

-------

LOS ANGELES, CA – February 17, 2010 — Cognition Technologies, the creator of the most advanced and complete semantic Natural Language Processing (NLP) technology on the market, today announced that Microsoft Corp. has licensed some of its proprietary semantic technologies and will be using them to enhance Bing and other applications within Microsoft. Specifically, Microsoft will incorporate Cognition’s comprehensive and robust Semantic Map of the English language.

The non-exclusive licensing arrangement enables Microsoft to embed elements of Cognition’s semantic technologies into any Microsoft application which would benefit from an “understanding” of the English language. Initially, it will be used to enhance the user experience in Bing, Microsoft’s online decision engine.

“Cognition’s comprehensive Semantic Map will help us continue to improve the search experiences we can offer to consumers,” said Ron Kaplan, chief scientist in the Powerset division of Bing at Microsoft. “After several months of evaluation working closely with the technical team at Cognition, we believe that Cognition’s semantic technologies can help us provide better results for Bing customers.”

Dr. Kathleen Dahlgren, Cognition’s founder and CTO, added, “Cognition’s Semantic Map will contribute conceptual reasoning and precise question-answering that will mesh well with Microsoft’s existing search capabilities.”

The scope of Cognition’s Semantic Map is more than double the size of any other computational linguistic dictionary for English, and includes more than ten million semantic connections that are comprised of semantic contexts, meaning representations, taxonomy and word meaning distinctions. The Map encompasses over 540,000 word senses (word and phrase meanings); 75,000 concept classes (or synonym classes of word meanings); 8,000 nodes in the technology’s ontology or classification scheme; and 510,000 word stems (roots of words) for the English language. Cognition’s lexical resources encode a wealth of semantic, morphological and syntactic information about the words contained within documents and their relationships to each other. These resources were created, codified and reviewed by lexicographers and linguists over a span of more than 25 years.

“We are obviously very pleased that Microsoft has recognized the significant value of Cognition’s semantic technologies,” said Scott Jarus, Cognition’s CEO. “Microsoft joins a list of companies in the legal litigation support, publishing and life sciences industries who have also recognized Cognition’s ability to bring meaning and understanding to vast amounts of information.”

More information about Cognition’s Semantic NLP™ technology is available on its Website at www.cognition.com.

About Cognition:

Cognition Technologies, based in Los Angeles, has developed a revolutionary Semantic Natural Language Processing (NLP) technology which adds word and phrase meaning and “understanding” to computer applications, enabling them to be more human-like in their processing of information. Cognition's Semantic Map, the underlying technology developed over the past 24 years, is the largest and most extensive in existence. Applications and technologies which utilize Cognition's Semantic NLP™ technology are positioned to take full advantage of Web 3.0 (the Semantic Web).

Source: http://www.dbusinessnews.com/shownews.php?newsid=201628&type_news=past

Numbers from 1 to 10 in Over 5000 Language

noreply@blogger.com (Pedro Paulo Balage) — Wed, 10 Feb 2010 13:04:00 +0000

The website http://www.zompist.com/numbers.htm shows the numbers from 1 to 10 in over 5000 languages including some historical differences. Bellow the difference between the old English and the today English.

Old English+	án	twá	þrí	féower	fíf	sex	seofon	eahta	nighon	tíen
Middle English+	an	two	three	four	fif	six	seven	eihte	nien	ten
English	one	two	three	four	five	six	seven	eight	nine	ten

More in: http://www.zompist.com/numbers.shtml