Exploring vision and machine learning

Affinity propagation

2011-06-30T22:02:00.003-04:00

@article{frey2007clustering,   title={Clustering by passing messages between data points},   author={Frey, B.J. and Dueck, D.},   journal={science},   volume={315},   number={5814},   pages={972},   year={2007},   publisher={American Association for the Advancement of Science} }

Meta: Affinity propagation is one of those algorithms that seem just too good to be true. The entire matlab code was available for everybody to see..about half a page. Yet it claimed to be the fastest and best clustering algorithm yet. When the article first appeared in Science back in 2007, it took a lot of people by surprised. It solved the crux of clustering algorithms--choosing the number of clusters. It also claimed to solve other problems. It was fast. It worked on text, biological data as well as traditional high dimensional data. Most surprisingly, it worked on arbitrary non-metric similarity measures. When I attended the talk by Frey at MIT the next year, the slides consisted mostly of great empirical results that showed it performed better than the best tuned k-means and was an order of magnitude faster. The algorithm consisted of iterative message passing between points where each point voted for their neighbors to be cluster centers and each point provided its 'availability' for being a cluster center.

Resources:

http://www.psi.toronto.edu/index.php?q=affinity%20propagation (project website)
http://pypi.python.org/pypi/milk/ (Python implementation)
http://www.psi.toronto.edu/affinitypropagation/webapp/ (webapp!)

Machine learning and computer vision resources

2011-06-21T22:39:00.009-04:00

This is a partial list of great books and resources for machine learning. I intend to update this list frequently, so bookmark it!

Machine learning

Gaussian process for machine learning (free book)
Information theory, inference and learning algorithms (free book)
Elements of statistical learning (free book)
Algorithms for graphical models (free book)
Pattern classification (book)
Pattern recognition and machine learning (book)
Machine learning (book)
Neural networks for pattern recognition (book)
Artificial intelligence: A modern approach (book)
Probabilistic graphical models (book)

Computer vision

Computer vision: Algorithms and applications (free book)
Computer vision models (free book)

Optimization

Convex optimization (free book)

2010-04-04T23:05:00.004-04:00

@article{colby2003space,
title={{Space and attention in parietal cortex}},
author={Colby, C.L. and Goldberg, M.E.},
year={2003},
publisher={Annual Reviews}
}

In this paper, the authors discuss the representation of space and location in the brain. Most surprisingly, it appears that the brain encodes space independent of modality. This makes sense in scenarious such as grasping and reaching where position estimates from vision are combined with tactile sensation. In general, location estimations from sensory modalities such as sound, touch, vision are all combined in a single representation.

There exists two popular views of spatial representation in the brain. The traditional view is that there exists a single reference frame (world-view) that guides action and behavior. However psychophysics and behavioral studies suggest that space is encoded in multiple frames of reference such as eye-centered, head centered, world centered etc (Andersen et al. '97, Caminiti et al. '96, Stein '92 etc.). Lesions in the parietal cortex leads to loss of spatial ability that affects one or more of these reference frames. Areas in the parietal cortex seem to be central in representing space in the brain. The parietal cortex is strongly linked to areas in the prefrontal cortex, premotor cortex and eye-fields.

What and where in the human brain, Ungerleider and Haxby

2010-03-25T04:20:00.006-04:00

Bibtex:
@article{ungerleider1994and,
title={{What and where in the human brain}},
author={Ungerleider, L.G. and Haxby, J.V.},
journal={Current Opinion in Neurobiology},
volume={4},
number={2},
pages={157--165},
year={1994},
publisher={Elsevier}
}

Visual area in the primate/human cortex are organized into two hierarchical pathways known as the ventral stream and the dorsal stream. Both pathways receive input from the striate cortex. In this paper, the authors present evidence from literature pointing out that these pathways specialize in 'what' and 'where' processing. Ventral stream is believed to be responsible for 'object' vision while dorsal stream is responsible for 'spatial' vision.

The ventral stream comprises area V1,V2,V4 and IT among other regions in the occipitotemporal regions. V1 cells function as spatio temporal filters. V2 cells respond to illusory contours. V4 cells respond to outlier stimulus and to complex shapes. IT cells have been found to respond to specific objects. Lesions in the ventral stream lead to loss of visual discrimination while preserving visuo-spatial ability.

The dorsal stream comprises areas V1,V3,MT,MST among other regions in the parietal cortex. V1 cells respond to specific directions of motion. MT cells are selective to patterns of motion . Within MST, cells respond to global patterns such as expansion, contraction etc. Lesions in the parietal cortex lead to loss of visuo-spatial ability while preserving visual discrimination. In contrast to ventral stream that remains modality specific (visual inputs only), the dorsal stream receives input from polysensory areas (Hyvarien J, Brain Research, 1981, pp 287--303).

This seems to suggest that these two information are processed independently and perhaps concurrently in the brain. Studies based on regional cerebral blood flow and PET (Haxby et al. "Dissociation of spatial and object visual processing pathways in the human extrastriate cortex, PNAS 91) have shown that regions in the occipitotemporal cortex were activated during a recognition task where as regions in the occipitoparietal cortex were activated during a spatial location. Computational justifications for the separation is also mentioned. In particular, Young (Object analysis of the topological organization of the visual cortex, Nature '92) used MDS on the connectional topology of the striate cortex to show the segregation into ventral and dorsal like pathways.

PET studies examined the effect on these areas ((Haxby et al. "Dissociation of spatial and object visual processing pathways in the human extrastriate cortex, PNAS 91,Corbetta et al.,"Selected and Divided attention during visual discriminations of shape, color and speed: Functional anatomy by PET" Journal of Neuroscience, 91). Studies showed that when attenting to stimulus attributes such as color and shape, same areas that respond to these stimuli get activated. Similarly when attending to location, regions in the parietal get activated.

Conclusion:
Similar to monkeys, human cortex possesses separate cortical areas for processing identity and location information. Also, similar to the monkey, attentional strongly modulates regions in the extrastriate region.

Readings:
1. Mishkin M, Ungerleider LG, Macko Ka. Object vision and spatial vision: two cortical pathways. Trends in Neurosciences. 1983;6:414-417. Available at: http://linkinghub.elsevier.com/retrieve/pii/016622368390190X.

In contrast to the previous post that studies human brain, this paper by Mishkin et. al studies the monkey brain. The paper presents further evidence of the ventral/dorsal dichotomy in the visual area. Instead of 'what' and 'where' dichotomy alternate segregation such as 'what' and 'action' have also been proposed (Goodale and Milner, A Neurological dissciation between perceiving objects and grasping them, Nature 91).

Development of localized oriented receptive fields by learning a translation-invariant code for natural images

2009-11-22T20:17:00.001-05:00

R. P. N. Rao and D. H. Ballard, Computational neural system, Vol 9, 219-234

Learning translation invariant image-patch dictionaries has been a holy-grail for vision scientists. While Fukushima provided the early breakthrough, recent approaches have taken a more formal approach to explain
translation invariance. In this paper, Rao and Ballard attempt to separate the content (what) and location (where) information from an image patch. They treat translation as a linear operator! that can be learned from a set of training images. Thus both translation and image appearance can be derived using a set of basis functions. Further, they show that the translation bases are invariant to image identity and that the identity bases are independent of translation.
The vector of image pixels I can be expressed as,
I = Ur + Jx + n
U = identity basis
J = Jacobian dI/dx approximated using [D_i I], where each matrix D_i provides a single column of the Jacobian

dWork, where Work+dWork = holy-grail

The identity and translation basis are regularized using L2 norm. This does not ensure sparseness. dWork = L1/L0 regularization
The bases are learnt using an iterative gradient descent algorithm with lots of free parameters. Further, the coefficients x,r are obtained using iterative schemes for each image.

Related work:

Fukushima 85: Provides a hebbian like learning rule to learn translation invariant features
Ranzato et al. 08, In their variant of the HMAX type network, they learn translation invariant patches through and EM like learning algorithm, very similar in spirit.

Aude Oliva

2007-11-05T01:29:00.000-05:00

Nov 1:
BCS special talk: Visual Scene Understanding: from Agnosia to Photographic memory

Aude Oliva from CVCL talked about her previous and future work (Was this a tenure talk?). She presented her work on

scene gist representation and statistics of natural images
scene descriptors and categorization
role of memory in scene understanding - there was an interesting result about the capacity of humans to remember object categories and their shapes. She described the results of an experiment where people were able to recall 2400 object categories with more than 90% precision. Moreover,

A lot of her work has been with Antonio torralba, one of the most prolific researchers I've known. He's also a profilic matlab guru!

Adaptive visual models for finding people

2007-10-29T09:40:00.000-04:00

Oct 26, 2007
Devaramanan from TTI/UCI spoke about adaptive models for finding people.
Relevant papers

* Tracking people
<http://ttic.uchicago.edu/%7Eramanan/papers/tracker_journal_draft.pdf>
* Face tracking <http://ttic.uchicago.edu/%7Eramanan/papers/faces.pdf>
* Image parsing <http://ttic.uchicago.edu/%7Eramanan/papers/parse.pdf>

He spoke about detection in three scales
1. high resolution - where individual limbs and body parts can be
tracked. (see image parsing paper)-Had lots of pretty pictures! The
vista and leapord of object localization papers :).
2. med resolution-faces can be tracked and labeled (see face tracking)

1. High resolution: An object is modeled as parts with spatial constraints among parts. The appearance is modeled separately using edge based similarity. An invariant detector finds the approximate position of the limbs. Message passing among parts is used to enforce consistent detection among parts. However, edges are agnostic to negative spaces (gaps between legs appear as legs), and are easily confused by background edges. Using the initial detection, a color model of the individual parts is built and the parts are redetected. With successive iterations, the model adapts to individual object instance (buidling upon skin and dress color) to achieve good localization. He showed pretty
pictures or horses, scenes from the movie lolita, an ice skater with impressive localization of body parts.

2. Med resolution: This work focused on localizing and tracking face images across a huge database of video (All seasons of friends to be exact! I would love to annotate the whole video..That's rewarding research:)). The work builds upon a viola jones detector and adapts the model using torso and hair color to achieve better detection rate. The priniciple is the same: start with an
invariant detector learnt before hand and adapt it using object instances in the given image.

He also talked about the work he's doing with Pedro and on the pascal challenge. Based on HoG based part detector. He also mentioned that they are working on a hierarchical version of the model. They appeared to have done very well on the Pascal challenge this year.

ps: his abstract for the talk
Abstract:
In this talk I will describe a family of algorithms for object detection, with a focus on finding people in images and video. Object detection is hard because objects can vary in appearance due to 3D pose and illumination (among other factors). The visual appearance of people is further complicated by articulated pose and clothing. One approach is to build models that are invariant to such changes; another is to build models that adapt. Part-based models that geometrically deform `on-the-fly' are an example of an adaptive approach. I will describe approaches that also adapt photometrically.

In the context of finding people in video, such approaches simultaneously learn the color of a person's clothes while tracking him/her. In the context of object detection in images, such approaches simultaneously segment and detect an object. Some issues need to be addressed - what is the computational algorithm for efficiently adapting `on-the-fly'? How does one train such an algorithm? I will discuss a few schemes.

Extensive experimental results will the shown throughout the talk, including large-scale detection and tracking on terabytes of video data and several benchmark object detection datasets, yielding state-of-the-art results.

Recursive gaussian filters!

2007-10-25T23:27:00.000-04:00

Goal: To generate a very deep gaussian derivative scale space representation (sigma >30) for a large image.
Problem:

It is not hard to compute the derivatives. The problem is to compute the convolution with large gaussian filters (support ~ 8*sigma) . Even with separable filters, convolution is still linear in the filter size.

Holy grail:

Convolution operations that are independent of the filter size.
Viola and Jones came close with their box filter approximations to the gaussian derivative filters. Using the integral image and the fact that the box filters have sparse derivatives, they are able to achieve convolution speed that doesn't depend on the filter size.
Now wouldn't it be nice if you could do the same with gaussian filters. I'd give up a night with Liv Tyler if I could do that

Quest:

My quest led me to pryamid implementations (Mutch and Lowe, Adelson and Simoncelli's pryamids, Wavelet decompositions), hybrid pyramid-scale space reprsentation (Lindeberg et al, Lowe).

Solution: Recursvie gaussian filters.
Relevant papers:

It blew my mind after I read it. It was like walking into petra (or the "valley of the cresent moon") and finding the holy grail. I had always wondered why people never talked about IIR filters for images. It is hard to imagine a causal sequence of inputs in single dimension, but if you look at each dimension indivdually, it is old-school signal processing again.

Slow feature analysis: Laurenz Wiskott in CBCL!

2007-10-23T18:07:00.001-04:00

Laurenz Wiskott visited CBCL and talked at the Brain machines seminar in BCS. He presented his work on slow feature analysis. He talked about the emergence of complex cell behavior based on deriving temporally stable ("slow") features from natural images. He also talked about his recent work on extending slow feature analysis to spiking neurons.

A good starting point to understand this work is his 2002 paper in neural computation,
Slow Feature Analysis: Unsupervised Learning of Invariances,Neural Computation. 2002;14:715-770

Relevant links: