Straw lab

Position - data-driven modeling of spatial cognition in Drosophila and zebrafish

2017-08-21T00:00:00+00:00

Applications are invited from highly motivated researchers for a PhD or postdoctoral position (TV-L E13) available immediately in the Department of Animal Physiology, Neurobiology and Behavior, in the group of Prof. Dr. Andrew Straw, at University of Freiburg, Germany. Initial appointment is one year, with flexible start date and opportunity for renewal.

The project makes use of recently developed virtual reality technology to study spatial cognition in Drosophila and zebrafish. The primary goal is to develop biologically relevant models of neural processing and behavior during visual tasks involving spatial cognition and awareness. Highly-automated data collection systems will be used to collect large datasets in which animals perform tasks designed to elicit spatial learning and to quantify behavioral contributions from visual reflexes and higher-order processes. The work entails adapting the technology to perform the required experiments, designing and running the experiments, performing data analysis, and writing of scientific papers on the results. The work will quantify the contributions of specific circuits, algorithms, and molecules important for visual and spatial cognition behaviors. Some teaching of data science is required.

Applicants with strong mathematical, engineering or computer science backgrounds are encouraged to apply.

Our lab collaborates with the following groups at the University of Freiburg:

Jun-Prof. Dr. Joschka Boedecker on machine-learning based approaches to vision and spatial cognition making use of techniques such as inverse reinforcement learning and imitation learning.
Prof. Dr. Wolfgang Driever on neural circuits and behavior in zebrafish.
Dr. Matthias Wittlinger on spatial cognition in insects.
Prof. Dr. Dierk Reiff on Drosophila visual circuits
Prof. Dr. Thomas Brox on computer vision, video analysis and deep learning.

Andrew Straw’s lab is situated within the Institute of Biology I (Zoology) at the University of Freiburg and is affiliated with the Bernstein Center Freiburg. Substantial computational resources are available.

Interested candidates should send a research statement, along with a CV including publications, to Prof. Dr. Andrew Straw (straw@bio.uni-freiburg.de). The names and contact details of two or more references should be provided.

FreemoVR published

2017-08-21T00:00:00+00:00

Please visit the FreemoVR page for more information, including press info.

Animal Tracking and VR Bootcamp, 23-27 Oct 2017

2017-08-16T00:00:00+00:00

Animal Tracking and VR Bootcamp

23-27 Oct 2017

Hauptstr. 1, University of Freiburg, 79104 Freiburg Germany

Contact: Andrew Straw

The workshop is full to capacity. No further applications can be accepted. Please contact Andrew Straw to place your name on the waiting list in case of cancellation or to register your interest in a possible future workshop.

The goal of the workshop is to train participants in the theory and practice of:

realtime image processing
multi-camera 3D realtime tracking
perspective correct virtual reality for freely moving animals

An emphasis on understanding the principles and using open source tools will be made. Ultimately, participants should come away with knowledge of how to use available software and hardware to run VR experiments in their own lab.

Instructors:

Guest lecturers:

Who should attend:

The course is aimed at graduate students, postdocs and PIs.
Some participants will already have some experience in animal tracking, virtual reality, or both, but we also welcome motivated beginners.

A large lab space equipped with software and hardware from the lab of Andrew Straw at the Institute of Biology I, University of Freiburg will be available during the workshop. Nicholas Del Grosso (LMU Munich) will be co-instructor for the week. In the lab of Anton Sirota, Del Grosso led the construction of a free-moving rodent VR system and he has substantial experience teaching programming. Together with the Drosophila and zebrafish experience of the Straw Lab and the rodent experience of Del Grosso, the workshop should be of substantial interest to a wide range of behavioral- and neuro-scientists. (During the workshop itself, experiments will primarily be on Drosophila, with the possibility of some fish experiments depending on interest.)

There is a 300 EUR fee, which will be used for costs associated with the co-instructor and additional guest lecturers. (If the registration fee is prohibitive, we may be able to waive it. Please ask.) Participants need to pay for their own hotel, food and travel. According to our research, we expect hotel costs to be about 80 Euros per night.

The workshop will be small - we will limit to a maximum of 10 participants. If successful, we may run the workshop again next year.

Looking forward to seeing you in Freiburg in late October!

Postdoc - Enhancers and Brains

2016-07-14T00:00:00+00:00

Postdoc opening: enhancers and brains

Applications are invited from highly motivated researchers for a postdoctoral position (TV-L E13) available immediately in the Department of Animal Physiology, Neurobiology and Behavior, in the group of Andrew Straw, at University of Freiburg, Germany. Initial appointment is one year, with flexible start date and opportunity for renewal.

The project makes use of two genomic-scale enhancer expression datasets from Drosophila to make testable predictions relating enhancers to brain connectivity, functional properties, developmental specification, and evolutionary history. The work will build from a recent lab publication (Panser et al., 2016, “Automatic segmentation of Drosophila neural compartments using GAL4 expression data reveals novel visual pathways” Current Biology and strawlab.org/braincode). Candidates must have a strong background in computational analysis of biological datasets. Experience with enhancer biology, neural connectivity, and Drosophila genetics is also desirable. Applicants should be committed to applying rigorous computational tools to discover strong correlations between enhancer expression patterns and collaborating with experimental biologists to move from ideas to evidence.

Moving to the University of Freiburg, Germany

2016-01-20T00:00:00+00:00

I have just accepted a position as professor in the Department of Animal Physiology, Neurobiology and Behavior at the University of Freiburg, Germany. While I will miss my colleagues in Vienna, I am excited about the opportunities in Freiburg and look forward to the next steps.

Neuron catalog software released

2015-03-02T00:00:00+00:00

In preparation for the “Harnessing Community Resources for Drosophila Neuroscience” workshop organized by Cahir O’Kane and David Osumi-Sutherland at the 2015 GSA Annual Drosophila Research Conference that will take place in a few days, we are releasing software for small groups (e.g. a lab) to collaborate on circuit mapping.

The software is called neuron-catalog and here are some online resources:

Home page: strawlab.org/neuron-catalog
Download: Release page
Project page on github: github.com/strawlab/neuron-catalog
Demonstration: neuron-catalog demonstration
Documentation: Read The Docs
Online forum: gitter.im/strawlab/neuron-catalog

Fly figure and background responses from a single pathway?

2014-11-13T00:00:00+00:00

Drosophila vision is an exciting field right now – the latest genetic technologies are enabling rapid progress in understanding vision from biophysics to behavior. Today, Current Biology published our paper showing that a single motion detecting circuit may be sufficient to mediate multiple behavioral responses. This is surprising because in the fly, two important behaviors – object fixation and wide-field stabilization – are widely thought to result from circuitry with elements dedicated to the particular tasks. Contrary to this view, we show that turns toward objects can result from the same motion processing pathway already thought to underlie wide-field stabilization. Consequently, we think this work creates a new “null hypothesis” – subsequent work will need to show behavioral responses to visual objects not predicted by this model to argue for the involvement of additional circuitry. This work also has interesting parallels to the circuitry and behavior of mammalian eye movement.

Our paper is both experimental and theoretical in nature. We performed experiments on genetically modified flies and showed that intact motion-processing T4 and T5 cells are necessary for stripe fixation at high gain and in figure ground discrimination tasks. With this experimental result, we implemented both a simple, phenomenological model and a physiologically more realistic model that predict stripe fixation in which the visual system consists of a single, motion-sensitive pathway. Remarkably, these simple models have no explicit object detection or small-field selectivity. We show that a particular configuration of one of our models is formally equivalent to the classical model published by Poggio and Reichardt in 1973. The models also make non-obvious predictions of stripe-fixation behavior. One such prediction is fixation in front of a moving background, and when we tested this prediction experimentally, we found qualitative agreement between both models and behavioral results on tethered flies.

We hope that by including the full source code of our models - including an online version of the phenomenological model - others can easily reproduce our work and, for example, predict fly physiological and behavioral responses to arbitrary visual stimuli.

The full reference is:

Fenk LM*, Poehlmann A*, Straw AD (* equal contribution) (2014) Asymmetric processing of visual motion for simultaneous object and background responses. Current Biology. doi:10.1016/j.cub.2014.10.042

Download and read more about the models from strawlab.org/asymmetric-motion/.

lab portrait, summer 2014

2014-07-22T00:00:00+00:00

the Fly Mind Altering Device

2014-05-25T00:00:00+00:00

Our FlyMAD system is online! This is a system for targeting freely walking flies (Drosophila) with lasers. By steering the optical path of the laser using large diameter mirrors, it also enables simultaneous imaging of the target region. This enables through-the-mirrors (TTM) targeting of specific body parts. Together, all this allows rapid thermo- and opto- genetic manipulation of the fly nervous system.

Read the paper, visit the website, watch the video, and join the email list.

The full reference is:

Bath DE✱, Stowers JR✱, Hörmann D, Poehlmann A, Dickson BJ, Straw AD (✱ equal contribution) (2014) FlyMAD: Rapid thermogenetic control of neuronal activity in freely-walking Drosophila. Nature Methods. doi 10.1038/nmeth.2973

Karin Panser wins first prize in Huygens Image Contest 2013

2014-05-23T00:00:00+00:00

Karin Panser from the lab was the winner in the SVI ‘Huygens Image Contest’ 2013 with a beautiful picture of the fly eye. Her image was New Scientist’s picture of the day and the Lab Times also covered it.

Well-deserved congratulations to Karin!

lab portrait, autumn 2013

2013-09-25T00:00:00+00:00

Andrew Straw, Karin Panser, Lisa Fenk, Sayanne Soselisa, Dorthea Hörmann, John Stowers, Etienne Campione, Andreas Poehlmann, Katje Hellekes

lab portrait, spring 2013

2013-04-19T00:00:00+00:00

Andreas Poehlmann, Andrew Straw, Karin Panser, John Stowers, Lisa Fenk, Nemanja Saric, Katja Hellekes

lab portrait, spring 2012

2012-03-12T00:00:00+00:00

Most of the lab at the Vienna City Hall after the awards ceremony for the WWTF Cognitive Sciences grant.

Back row: Andreas Pöhlmann, Lisa Fenk, Karin Panser. Front row: Andrew Straw (with post-doc in training), John Stowers, Alexander Prochaska, Mareike Forthmann. (Unfortunately Lucile was out of town.)

lab portrait, autumn 2011

2011-11-29T00:00:00+00:00

From left to right: Andrew Straw, Mareike Forthmann and Karin Panser.

high-level visual behavior grant funded by the WWTF

2011-11-10T00:00:00+00:00

We just received good news from the Wiener Wissenschafts-, Forschungs- und Technologiefonds (WWTF, in English this translates to the Viennese Science, Research, and Technology Fund). Our grant propsal “Algorithms, neural circuitry, and genetics of high-level visual behavior in the fly” was selected for funding.

Together with the ERC Starting Grant FlyVisualCircuits we received a few months ago, the lab is off to a great start.

augmented reality - computing the OpenGL projection matrix from intrinsic camera parameters

2011-11-05T00:00:00+00:00

Summary

Here I describe how the Hartley-Zisserman (HZ) pinhole camera model differs from the OpenGL display pipeline and how to build an OpenGL projection matrix directly from the intrinsic camera parameter matrix of HZ. What is particular about this exposition is that I do this by calculating the matrix elements directly from the camera parameters rather than calling the glProjection() function. This allows for a more general camera model including pixels with skew. I also include the algebraic derivation (as a sympy script) so you can follow my logic exactly.

Update (30 Jan 2013): The various implementations of the coordinate transform pipeline used in this post are available here.

Update (26 Dec 2014): I added a few more recent references at the end.

Summary
Note about image coordinates
The OpenGL projection matrix from HZ intrinsic parameters
Comparison with the OpenCV camera calibration
Experimental verification
- Output of CPU implementations
- Output of OpenGL implementation
Download these files
Remaining work: compensating for camera distortion not described by the pinhole model
References

Note about image coordinates

In both OpenGL window coordinates and HZ image coordinate systems, (0,0) is the lower left corner with X and Y increasing right and up, respectively. In a normal image file, the (0,0) pixel is in the upper left corner. We have code paths to deal with this in one of two ways: first, we can draw our images upside down, so that all the pixel-based coordinate systems are the same. This is the code path used when “window_coords=’y up’”. Second, we can keep the images right side up and modify the projection matrix so that OpenGL will generate window coordinates that compensate for the flipped image coordinates. In this “window_coords=’y down’” path, the generated OpenGL Y window coordinates are (height-y).

The OpenGL projection matrix from HZ intrinsic parameters

Enough of the preliminaries. We calculate the OpenGL Projection matrix when window_coords==’y up’ to be:

[2*K00/width,  -2*K01/width,   (width - 2*K02 + 2*x0)/width,                            0]
[          0, -2*K11/height, (height - 2*K12 + 2*y0)/height,                            0]
[          0,             0, (-zfar - znear)/(zfar - znear), -2*zfar*znear/(zfar - znear)]
[          0,             0,                             -1,                            0]

With window_coords==’y down’, we have:

[2*K00/width, -2*K01/width,    (width - 2*K02 + 2*x0)/width,                            0]
[          0, 2*K11/height, (-height + 2*K12 + 2*y0)/height,                            0]
[          0,            0,  (-zfar - znear)/(zfar - znear), -2*zfar*znear/(zfar - znear)]
[          0,            0,                              -1,                            0]

Where Knm is the (n,m) entry of the 3x3 HZ instrinsic camera calibration matrix K. (K is upper triangular and scaled such that the lower-right entry is one.) Width and height are the size of the camera image, in pixels, and x0 and y0 are the camera image origin and are normally zero. Znear and zfar are the standard OpenGL near and far clipping planes, respectively.

This is the cut-and-paste output of our sympy script projection_math.py. The approach is that we enter the operations of OpenGL vertex transformation pipeline and the HZ projection model into sympy, a computer algebra system (CAS). We have sympy solve for the OpenGL projection matrix so that the resulting pixel coordinate is the same for both the HZ camera model and the OpenGL pipeline. See the script for the implementation details. Of course this could be done by hand but is tedious and prone to mistakes.

Comparison with the OpenCV camera calibration

Although I have not directly used OpenCV for camera calibration, their parameterization of the pinhole camera is a subset of the full HZ model. In particular, their matrix A corresponds exactly to the HZ matrix K with pixel skew fixed at zero. Consequently, this page can be directly used with OpenCV camera calibrations by setting K01 to zero.

Experimental verification

To verify that this computation of the OpenGL projection matrix accurately captures the HZ camera model, we have calculated the projection of vertices into image coordinates three ways:

A CPU-based implementation of the HZ camera model. This performs matrix multiplication of the eye coordinates by the intrinsic parameter matrix K.
A CPU-based emulation of the OpenGL pipeline. This performs matrix multiplication of the eye coordinates by the OpenGL projection matrix to produce clip coordinates, transforming those to normalized device coordinates, and then finally using the glViewport parameters to establish the window coordinates.
Direct calls to the OpenGL pipeline, presumably running on your GPU. In this case, we directly load our OpenGL projection matrix by calling glLoadMatrixf() and use OpenGL to perform all vertex transformation.

The first two examples are in the calib_test_numpy.py example and their outputs are overlaid. The third (OpenGL) example is in calib_test_pyglet.py. Each of these three examples gives the same results, suggesting that our formulation is correct. All calculations were done with Python scripts, but the concepts and results should be easily adaptable to any language.

Here is a brief conceptual walkthrough of these programs. Each program starts by loading an image acquired by a (crudely) calibrated camera. This image (luminance.png, included in the zip download below) is of a roughly cylindrical object 1 meter in diameter and 1 meter high. Part of the cylinder is occluded and only the front surface is illuminated. The camera calibration is contained within the file cameramatrix.txt. This calibration is decomposed into the intrinsic camera parameters and the rotation matrix, and the camera translation vector. A simple mathematical model of the cylinder is used to generate world coordinates of many vertices. Each of these world coordinates is transformed to camera coordinates – also called eye coordinates in OpenGL. This is done using the extrinsic camera parameters specifying the camera’s pose and is done either as a matrix multiplication with the extrinsic parameter matrix or by loading them into the OpenGL modelview matrix. From there, each of these vertices in eye coordinate is transformed to window coordinates using the methods described above.

Output of CPU implementations

The blue crosses are the vertices after the HZ transformation. The red points are the vertices after the simulated OpenGL pipeline. Calculations done with numpy and scipy, and plotting done with matplotlib.

Output of OpenGL implementation

The green points are the certices after the OpenGL pipeline. OpenGL called through pyglet.

Download these files

In addition to the individual scripts linked above, all the files are included in this zip file.

Remaining work: compensating for camera distortion not described by the pinhole model

You may want to implement the non-linear warping distortions (radial distortion, tangential distortion) that are often used to extend the pinhole model to be more realistic. A GLSL-based shader implementation of warping and de-warping may be the subject of a future post, but is not described here.

References

A nice page on the OpenGL projection matrix by Song Ho Ahn
A good description of the pinhole camera model by Yannick Morvan
Another description of the pinhole camera model mostly compatible with the terminology here at Wikipedia.
The OpenGL specifications
Carlo Nicolini’s blog post and GitHub repository with similar math and C++ code.
A C++ implementation of the OpenGL matrix math by James Gregson, also on GitHub.
Another description of using calibrated cameras with OpenGL.

lab portrait, summer 2011

2011-08-19T00:00:00+00:00

From left to right: Andrew Straw, Natalia Balcu, Karin Panser and Linda Ma.

upcoming conferences

2011-08-14T00:00:00+00:00

The end of August and September are going to be busy for me, with plans to attend five conferences. I look forward to catching up with colleagues I’ve known a while and meeting new ones. Contact me if you want to meet up.

EuroScipy2011 25-28 August, Paris
JEDI conference 2-4 September, Leysin, Switzerland
EMBO conference 10-13 September, Vienna
Champalimaud Neuroscience Symposium 18-21 September, Lisbon
European Drosophila Research Conference 21-24 September, Lisbon

the Grand Unified Fly (GUF) model

2011-03-23T00:00:00+00:00

Note: This page contains the most important information from the original location at the Dickinson Lab Caltech website (http://dickinson.caltech.edu/Research/Grand_Unified_Fly), which is now offline.

Block diagram of “Grand Unified Fly” and a model fly situated in a tunnel environment under closed-loop visual control.

Overview

We are developing a computer model of Drosophila to investigate the aerodynamics, control, neural processing, and sensory inputs of fly flight. The components of the model are individually useful as research tools but also perform as parts in an integrated whole. For more information, please read the papers above.

Our code consists of the following elements:

Drosophila eye map » Viewing directions (eye map) of each Drosophila ommatidium in the compound eye
fsee » Fly-eye view and neural processing simulation code
fmech » Insect flight mechanics simulator.

References

William B. Dickson, Andrew D. Straw, Michael H. Dickinson (2008) Integrative Model of Drosophila Flight. AIAA Journal 46 (9). doi:10.2514/1.29862 PDF

Epstein M., Waydo, S., Fuller, S.B., Dickson, W., Straw, A., Dickinson, M.H. & Murray, R.M. (2007) Bioligically Inspired Feedback Design for Drosophila Flight. Proceedings of the 2007 American Controls Conference. PDF

Dickson, W.B., Straw, A.D., Poelma, C., & Dickinson, M.H. (2006) An Integrative Model of Insect Flight Control. Proceedings of the 44th AIAA Aerospace Sciences Meeting and Exhibit AIAA-2006-0034 PDF