tag:blogger.com,1999:blog-32312332305191604532024-09-04T11:56:27.462-07:00A blog about reinforcement learning, machine learning, and more generally artificial intelligence, with occasional posts about professorial life.Unknownnoreply@blogger.comBlogger21125tag:blogger.com,1999:blog-3231233230519160453.post-82579796928866246652016-09-05T18:12:00.000-07:002016-09-05T18:13:18.725-07:00

Every so often, a friend asks what is so exciting or different about what has (unfortunately) come to be called Deep RL (other than the obviously exciting new applications). Here are some points I made in my talk at the IJCAI 2016 Deep RL workshop. 1. Renewed focus on Learning Representations in addition to the usual focus on function approximation (this is a subtle but consequential difference

Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3231233230519160453.post-64147076900371136832016-09-04T18:43:00.000-07:002016-09-05T08:19:44.450-07:00

At a recent meeting, I heard a DARPA synthesis from John Launchbury, a worldview so to speak, of AI that (somewhat to my surprise) I found reasonably consistent with my own view of a useful way to partition AI work. Here is my version. Handcrafted AIs. Expert systems fall into this category, but so do logic-based as well as probability-based planning, problem-solving, reasoning, and inference

Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3231233230519160453.post-46753519021988178832012-06-24T18:37:00.001-07:002012-06-26T11:01:00.938-07:00

Every so often I have discussions with my AI colleagues about whether the best research approach to building human-level intelligence is to build machines that have the linguistic abilities of humans or perhaps the high-level problem-solving skills of humans or whether it is to builds machines that can navigate, physically manipulate their world, and deal with other machines at a basic social

Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3231233230519160453.post-78052383284264991512010-09-10T06:34:00.000-07:002010-09-10T06:34:11.984-07:00

Eric Schmidt, CEO of Google, describes the circumstance when "all the information in the world is available at our fingertips" (presumably through always networked mobile devices, etc.) as being a circumstance when "we can literally know everything". It is interesting that common parlance is shifting to call looking something up on the internets as knowing that something. At one level this is

Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3231233230519160453.post-9435365791876243942010-08-16T08:18:00.000-07:002010-08-16T09:11:50.726-07:00

An amusing representation of what a doctorate means.

Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-3231233230519160453.post-76987003459262552042009-07-12T09:05:00.000-07:002009-07-12T09:08:31.130-07:00

An interesting post from the Pew Research Center for the People & Press. A must read for academics and researchers. Lots of poll results and some commentary.

Unknownnoreply@blogger.com3tag:blogger.com,1999:blog-3231233230519160453.post-30851959924885592092009-03-22T16:52:00.000-07:002009-03-22T16:58:47.699-07:00

In this Washington post article the writer (a former stem cell researcher at Harvard) argues that the view that University research/science is "curiosity-driven" is misplaced and that the incentive structure is broken. I agree!Here are two relevant quotes from his argument that I find real."University researchers are in a constant battle for recognition and the rewards associated with success:

Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3231233230519160453.post-77829607513836024692008-11-24T07:30:00.000-08:002008-11-24T07:37:36.831-08:00

A New York Times opinion writer, Stanley Fish, often writes about questions of interest to university professors. In this article, he writes about the notion of "academic freedom" and what it means and how it is quite different from individual First Amendment rights. He also refers to a new book on this topic. Much of his discussion makes sense to me. In particular, the notion that "... academic

Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3231233230519160453.post-31810674783244841202008-10-21T12:03:00.001-07:002008-10-21T12:04:25.779-07:00

*I had drafted this earlier this summer but never got around to finishing and posting this. Am doing so now*I just finished reviewing for a conference and in looking over my reviews was reminded of the following comment by a senior colleague to me at ICML this year. He has been program chair for several conferences and said that he thought that RL reviewers were the hardest on their own community

Unknownnoreply@blogger.com4tag:blogger.com,1999:blog-3231233230519160453.post-21053296328437653682008-10-14T16:49:00.001-07:002008-10-14T19:06:27.730-07:00

I recently came across this description of economists attributed by Paul Krugman to Robert Solow. "There are two kinds of economists: those who look for general results and those who look for illuminating examples." This dichotomy struck me as rather interesting and upon reflection, it was pretty clear to me that my own AI-research methodology is overwhelmingly towards the general results side. I

Unknownnoreply@blogger.com6tag:blogger.com,1999:blog-3231233230519160453.post-16289509449907311272008-10-13T17:28:00.001-07:002008-10-13T17:37:01.956-07:00

On a non-RL note: Paul Krugman, one of the writers whose nytimes blog I have enjoyed reading over the past few months (and incidentally the one who just got a Nobel Memorial Prize in Economics) credits the reading of the Foundation series of books by Isaac Asimov for sparking his interest in economics. His notion being that short of inventing the field of psychohistory, economics is the field

Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3231233230519160453.post-22384340553512523772008-06-01T18:10:00.000-07:002008-06-01T18:14:50.188-07:00

A far too brief report of the World Science Festival stellar panelon What it means to be human.(If someone finds a better report or video, please point us to it.)

Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-3231233230519160453.post-77466107652220849972008-06-01T08:03:00.000-07:002008-06-01T10:00:39.635-07:00

Conversation with Ian McEwan & Steven Pinker (audio only) [Warning: has mature content](or, why building conversational systems would be so hard)

Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3231233230519160453.post-12759139195444626382008-05-30T11:01:00.000-07:002008-05-30T11:06:19.468-07:00

In response to Satinder's post about growth of the field, I did a little experiment. Google scholar allows for date searches. (Citeseer has similar information, but I couldn't figure out a way to work with it.) I searched for the phrase "reinforcement learning" and counted the number of hits in each year. Google scholar also creates a "key author" list for each search, so I included that

Michael Littmanhttp://www.blogger.com/profile/05878303953204580887noreply@blogger.com2tag:blogger.com,1999:blog-3231233230519160453.post-43843463421871283612008-05-30T09:59:00.000-07:002008-05-30T10:07:57.658-07:00

An online Wired article describes a Robot arm built by Dean Kamen (of the Segway fame). I think it is great that folks like Dean want to innovate in this space. Great for AI and robotics, not to mention all the folks this could end up helping. (BTW, does RL have a role to play in building brain machine interfaces?)Interview of Dean by Walter Mossberg, and another video

Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3231233230519160453.post-41976128254727097422008-05-29T19:35:00.001-07:002008-05-30T10:32:19.410-07:00

I have had a casual interest in Second Life (SL) for a little while and periodically I am going to collect links to and briefly discuss AI work being done in SL. Here is the first such discussion.This page has a brief description of work by RPI on building an AI avatar that can do rudimentary conversations in SL. The description characterizes the avatar as a "four year old" based on a theory of

Unknownnoreply@blogger.com4tag:blogger.com,1999:blog-3231233230519160453.post-3547670252500316712008-05-29T13:48:00.000-07:002008-05-29T13:54:07.092-07:00

Matthew Taylor just sent me this funny picture of me at the recent Barbados workshop on RL along with his caption which was "Satinder is ready to wring the last bit of performance out of some poor PSR algorithm. "How would you caption it? :)

Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3231233230519160453.post-11480874461839377952008-05-28T19:16:00.000-07:002008-05-28T19:31:13.277-07:00

Ok, this has nothing to do with RL or AI. But what a fruit! Someone should bring a bag to the next conference for fun.To quote the nytimes article:"The miracle fruit, Synsepalum dulcificum, is native to West Africa and has been known to Westerners since the 18th century. The cause of the reaction is a protein called miraculin, which binds with the taste buds and acts as a sweetness inducer when

Unknownnoreply@blogger.com2tag:blogger.com,1999:blog-3231233230519160453.post-48787801157782341782008-05-28T17:13:00.000-07:002008-05-28T17:30:00.427-07:00

This is a cool project and brain-machine interfaces like this will be a great application of machine learning that will help society. This stuff is exciting!

Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3231233230519160453.post-26727806755290294312008-05-27T18:32:00.001-07:002008-05-30T15:23:53.050-07:00

When I started my PhD in Andy Barto's lab in 1988, there were perhaps a handful of folks doing research in the field of modern RL. There was an outpost of ex-students from Andy's lab at GTE Laboratories including folks like Rich Sutton, Chuck Anderson and Judy Franklin. By 1990 or so, there were a few others including Leslie Kaelbling and Peter Dayan. It was a pretty lonely field back then.But

Unknownnoreply@blogger.com7tag:blogger.com,1999:blog-3231233230519160453.post-69240647051055784572008-05-27T18:22:00.001-07:002008-05-27T18:30:51.981-07:00

So I am starting a blog in which I (and others to be invited) will initiate conversations with those who wish to comment (or just read) on things pertaining to RL and more generally AI. The intention is to keep this related to research and only occasionally stray into other areas of life.At this point I have no rules for posts or comments other than to keep things friendly and civil. This may be

Unknownnoreply@blogger.com0