Reinforcement Learning

A great talk on "Are Humans Just another Primate"

2012-06-24T18:37:00.001-07:00

Every so often I have discussions with my AI colleagues about whether the best research approach to building human-level intelligence is to build machines that have the linguistic abilities of humans or perhaps the high-level problem-solving skills of humans or whether it is to builds machines that can navigate, physically manipulate their world, and deal with other machines at a basic social

To "know everything"

2010-09-10T06:34:00.000-07:00

Eric Schmidt, CEO of Google, describes the circumstance when "all the information in the world is available at our fingertips" (presumably through always networked mobile devices, etc.) as being a circumstance when "we can literally know everything". It is interesting that common parlance is shifting to call looking something up on the internets as knowing that something. At one level this is

A Visual Representation of a Doctorate

2010-08-16T08:18:00.000-07:00

An amusing representation of what a doctorate means.

Data on Scientists and what they and the public think of them?

2009-07-12T09:05:00.000-07:00

An interesting post from the Pew Research Center for the People & Press. A must read for academics and researchers. Lots of poll results and some commentary.

Is University Science somehow more Pure than Industry Science?

2009-03-22T16:52:00.000-07:00

In this Washington post article the writer (a former stem cell researcher at Harvard) argues that the view that University research/science is "curiosity-driven" is misplaced and that the incentive structure is broken. I agree!Here are two relevant quotes from his argument that I find real."University researchers are in a constant battle for recognition and the rewards associated with success:

On Academic Freedom

2008-11-24T07:30:00.000-08:00

A New York Times opinion writer, Stanley Fish, often writes about questions of interest to university professors. In this article, he writes about the notion of "academic freedom" and what it means and how it is quite different from individual First Amendment rights. He also refers to a new book on this topic. Much of his discussion makes sense to me. In particular, the notion that "... academic

On Reviewing Reinforcement Learning Papers

2008-10-21T12:03:00.001-07:00

*I had drafted this earlier this summer but never got around to finishing and posting this. Am doing so now*I just finished reviewing for a conference and in looking over my reviews was reminded of the following comment by a senior colleague to me at ICML this year. He has been program chair for several conferences and said that he thought that RL reviewers were the hardest on their own community

General Results versus Illuminating Examples

2008-10-14T16:49:00.001-07:00

I recently came across this description of economists attributed by Paul Krugman to Robert Solow. "There are two kinds of economists: those who look for general results and those who look for illuminating examples." This dichotomy struck me as rather interesting and upon reflection, it was pretty clear to me that my own AI-research methodology is overwhelmingly towards the general results side. I

Krugman and Foundation

2008-10-13T17:28:00.001-07:00

On a non-RL note: Paul Krugman, one of the writers whose nytimes blog I have enjoyed reading over the past few months (and incidentally the one who just got a Nobel Memorial Prize in Economics) credits the reading of the Foundation series of books by Isaac Asimov for sparking his interest in economics. His notion being that short of inventing the field of psychohistory, economics is the field

On "What it means to be Human"

2008-06-01T18:10:00.000-07:00

A far too brief report of the World Science Festival stellar panelon What it means to be human.(If someone finds a better report or video, please point us to it.)

An interesting conversation about "Conversation"

2008-06-01T08:03:00.000-07:00

Conversation with Ian McEwan & Steven Pinker (audio only) [Warning: has mature content](or, why building conversational systems would be so hard)

Some RL Community Statistics

2008-05-30T11:01:00.000-07:00

In response to Satinder's post about growth of the field, I did a little experiment. Google scholar allows for date searches. (Citeseer has similar information, but I couldn't figure out a way to work with it.) I searched for the phrase "reinforcement learning" and counted the number of hits in each year. Google scholar also creates a "key author" list for each search, so I included that

Another amazing BMI robot arm

2008-05-30T09:59:00.000-07:00

An online Wired article describes a Robot arm built by Dean Kamen (of the Segway fame). I think it is great that folks like Dean want to innovate in this space. Great for AI and robotics, not to mention all the folks this could end up helping. (BTW, does RL have a role to play in building brain machine interfaces?)Interview of Dean by Walter Mossberg, and another video

AI in Second Life (post 1)

2008-05-29T19:35:00.001-07:00

I have had a casual interest in Second Life (SL) for a little while and periodically I am going to collect links to and briefly discuss AI work being done in SL. Here is the first such discussion.This page has a brief description of work by RPI on building an AI avatar that can do rudimentary conversations in SL. The description characterizes the avatar as a "four year old" based on a theory of

Suggest Captions :)

2008-05-29T13:48:00.000-07:00

Matthew Taylor just sent me this funny picture of me at the recent Barbados workshop on RL along with his caption which was "Satinder is ready to wring the last bit of performance out of some poor PSR algorithm. "How would you caption it? :)

What a totally crazy wacky fruit!

2008-05-28T19:16:00.000-07:00

Ok, this has nothing to do with RL or AI. But what a fruit! Someone should bring a bag to the next conference for fun.To quote the nytimes article:"The miracle fruit, Synsepalum dulcificum, is native to West Africa and has been known to Westerners since the 18th century. The cause of the reaction is a protein called miraculin, which binds with the taste buds and acts as a sweetness inducer when

Monkeying around with robots

2008-05-28T17:13:00.000-07:00

This is a cool project and brain-machine interfaces like this will be a great application of machine learning that will help society. This stuff is exciting!

Measuring Reinforcement Learning

2008-05-27T18:32:00.001-07:00

When I started my PhD in Andy Barto's lab in 1988, there were perhaps a handful of folks doing research in the field of modern RL. There was an outpost of ex-students from Andy's lab at GTE Laboratories including folks like Rich Sutton, Chuck Anderson and Judy Franklin. By 1990 or so, there were a few others including Leslie Kaelbling and Peter Dayan. It was a pretty lonely field back then.But

A beginning

2008-05-27T18:22:00.001-07:00

So I am starting a blog in which I (and others to be invited) will initiate conversations with those who wish to comment (or just read) on things pertaining to RL and more generally AI. The intention is to keep this related to research and only occasionally stray into other areas of life.At this point I have no rules for posts or comments other than to keep things friendly and civil. This may be