Fishy Math

Sept 2015: International Training course on Fishery Stock Assessment and Ecosystem Modeling

2015-09-16T23:55:00.002-07:00

Owen Hamel, Aaron Berger and Eli Holmes will be teaching the International Training course on "Fishery Stock Assessment and Ecosystem Modeling" during September 16 - 22, 2015, in Hyderabad, India. Organized by International Training Centre for Operational Oceanography (ITCOocean) and ESSO-INCOIS, Hyderabad, India. This is part on an ongoing technical cooperation between Ministry of Earth Sciences (MoES), India and NOAA to enhance predictive capabilities for fisheries in India.
http://www.incois.gov.in/portal/ITCOocean/fsaem.jsp

Winter 2015 Online Course: Applied Time Series Analysis in Fisheries and Environmental Sciences

2015-01-10T11:25:00.003-08:00

Fish 507: Applied Time Series Analysis in Fisheries and Environmental Sciences
Winter 2015
Fisheries Dept, University of Washington

Instructors: Eric Ward, Eli Holmes, Mark Scheuerell
email: eli.holmes@noaa.gov, mark.scheuerell@noaa.gov, eric.ward@noaa.gov

Reviews current applications of univariate and multivariate time series models for biological and environmental data, emphasizing the estimation, inference, and forecasting aspects of time-series models. Explores effects of covariates and anthropogenic drivers for species that are exploited and/or of conservation concern. We taught a similar course 2 years ago. This time we are emphasizing how to fit these models in a Bayesian context with JAGS along with a MLE context with MARSS. We are recording the lectures and you can follow along with the course at: https://catalyst.uw.edu/workspace/fish203/35553/243766

New paper on MAR modeling of community dynamics

2014-01-24T16:07:00.002-08:00

Quantifying effects of abiotic and biotic drivers on community dynamics with multivariate autoregressive (MAR) models

Stephanie E. Hampton, Elizabeth E. Holmes, Lindsay P. Scheef, Mark D. Scheuerell, Stephen L. Katz, Daniel E. Pendleton, and Eric J. Ward

Long-term ecological data sets present opportunities for identifying drivers of community dynamics and quantifying their effects through time series analysis. Multivariate autoregressive (MAR) models are well known in many other disciplines, such as econometrics, but widespread adoption of MAR methods in ecology and natural resource management has been much slower despite some widely cited ecological examples. Here we review previous ecological applications of MAR models and highlight their ability to identify abiotic and biotic drivers of population dynamics, as well as community-level stability metrics, from long-term empirical observations. Thus far, MAR models have been used mainly with data from freshwater plankton communities; we examine the obstacles that may be hindering adoption in other systems and suggest practical modifications that will improve MAR models for broader application. Many of these modifications are already well known in other fields in which MAR models are common, although they are frequently described under different names. In an effort to make MAR models more accessible to ecologists, we include a worked example using recently developed R packages (MAR1 and MARSS), freely available and open-access software.

Time Series (MARSS) Course offered in March in Stockholm

2014-01-16T10:21:00.003-08:00

Mark and Eli are teaching a week-long multivariate time-series analysis course in Stockholm in March. Course Announcement

MARSS 3.6 up on CRAN. Significant speed increases for large models

2013-11-29T11:00:00.003-08:00

MARSS 3.6 has been uploaded to CRAN. I fixed some inefficiencies that were causing DFA models with many time-series (n>100) and R="diagonal and unequal" to be very, very slow. My tests show 10x faster fits for n=100 and R="diagonal and unequal" for DFA models.

http://cran.r-project.org/web/packages/MARSS/index.html

New paper out by Jim and Eric on using Delta-GLMMs to analyze fisheries survey data

2013-10-08T16:39:00.000-07:00

Thorson, J.T. and E.J. Ward. 2013. Accounting for space-time interactions in index
standardization models. Fisheries Research,147:426:433

Scientific survey data are used to estimate abundance trends for fish populations worldwide, and are frequently analyzed using delta-generalized linear mixed models (delta-GLMMs). Delta-GLMMs incorporate information about both the probability of catch being non-zero (catch probability) and the expected value for non-zero catches (catch rates). Delta-GLMMs generally incorporate year as a main effect, and frequently account for spatial strata and/or covariates. Many existing delta-GLMMs do not account for random or systematic differences in catch probability or rates in particular combinations of spatial strata and year (i.e., space–time interactions), and do not recognize potential correlation in random space–time interactions between catch probability and catch rates. We therefore develop a Bayesian delta-GLMM that estimates correlations between catch probability and rates, and compare it with either (a) ignoring year–strata interactions, (b) modeling year–strata interactions as fixed effects, or (c) estimating year–strata interactions in catch probability or rates as independent random effects. These four models are fitted to bottom trawl survey data for 28 species off the U.S. West Coast. The posterior median of the correlation is positive for the majority (18) of species, including all five for which the posterior distribution has little overlap with zero. However, estimating this correlation has little impact on resulting abundance indices or credible intervals. We therefore conclude that the correlated random model will have a little impact on index standardization of the West Coast bottom trawl dataset. However, we propose that the correlated model can quickly identify correlations between occupancy probability and density, and provide our code to allow researchers to quickly identify whether such a correlation is likely to be significantly different from zero for their chosen data set.

Building R packages with RStudio and embedding R in your documents and reports

2013-07-05T15:51:00.001-07:00

Building R packages with RStudio plus Embedding R in documents short-course on-line:

http://www.iugo-cafe.org/chinook/view_node.php?id=2962

Topics:

how (and why) to make an R package using RStudio
installing packages from github, git or a url to your tar.gz file
Using Sweave and RStudio to do 'reproducible research/programming'.
Using OpenOffice + R to do the same, if you don't like LaTeX
Creating web-apps that run your R code (a few links to demos)

Week 10: Applied Time-Series Analysis for Fisheries and Environmental Data

2013-03-12T15:54:00.000-07:00

Class material: webpage

Week 10: Dynamic linear models
This week, we give a brief introduction to an important class of MARSS models: dynamic linear models. These are multivariate linear regression models where the regression parameters (slope and intercepts) are treated as a AR process and thus are allowed to time evolve. We also review some of the diagnostics for MARSS models fits.

Lab 10 In the lab, you'll go through an simple example of a univariate dynamic linear model with time-varying slope and intercept.
Lecture 10 You can find the pdf of lecture on the class webpage along with the link to watch a recording of the lecture.

Week 9: Applied Time-Series Analysis for Fisheries and Environmental Data

2013-03-05T14:14:00.001-08:00

Class material: webpage

Week 9: Bayesian hierarchical multivariate state-space models
This week, we discuss fitting non-linear MARSS models and MARSS models with non-Gaussian errors using Bayesian methods. This is a very brief introduction and many shows some examples of how one sets up a MARSS model in JAGS and shows you what the posteriors of some models look like.
* Posteriors for MARSS models
* Intro to JAGS (the Gibbs sampler we will be using)
* many examples of fitting non-linear and non-Gaussian MARSS models

Lab 9 The main lab is to go through the JAGS examples shown in the lecture.
Lecture 9 You can find the pdf of lecture on the class webpage along with the link to watch a recording of the lecture.

Week 8: Applied Time-Series Analysis for Fisheries and Environmental Data

2013-02-26T18:41:00.001-08:00

Class material: webpage

Week 8: Estimating interactions (the B matrix)
This week, we discuss issues related to the estimation of the B matrix in the context of using it to represent species interactions in a community dynamics models.
* univariate discrete time Gompertz model
* multivariate discrete time Gompertz model
* including covariates
* spurious density dependence resulting from ignoring observation error
* uncertainty in B elements resulting from estimating observation variance
* different methods for estimating confidence intervals: bootstrapping, hessian approximation, profile likelihood
* diagnostics

Lab 8
The main lab is to go through case study 7 in the MARSS User Guide and the corresponding code.
Lecture 8
You can find the pdf of lecture on the class webpage.

Week 7: Applied Time-Series Analysis for Fisheries and Environmental Data

2013-02-26T12:31:00.000-08:00

Class material: webpage

Week 7: Dynamic factor analysis

This week, we use MARSS to do dynamic factor analysis (DFA), which allows us to look for a set of common underlying trends among a relatively large set of time series (Harvey, 1989, sec. 8.5). This is conceptually different than what we have been doing in the previous weeks. Here we are trying to explain temporal variation in a set of n observed time series using linear combinations of a set of m hidden random walks, where m << n. You can think of this as PCA for time-series data. Zuur et al. (2003) show a number of examples of DFA applied to catch data and densities of zoobenthos.

Lab 7
The main lab is to go through the dynamic factor analysis chapter in the MARSS User Guide and the corresponding code.

Lecture 7
You can find the ppt of lecture on the class webpage. Technical difficulties prevented recording of the lecture.

Week 6: Applied Time-Series Analysis for Fisheries and Environmental Data

2013-02-13T12:03:00.005-08:00

Class material: webpage

Week 6: Introduction to including covariates in multivariate time-series model
This week we introduce the inclusion of covariates using the framework of a multivariate autoregressive model written in state-space form. You will understand the lecture better if you read the chapter on covariates in the MARSS User Guide first. Much of the lecture is about how to include covariates in different mathematically equivalent ways. You'll want to translate the R code in the lecture into the mathematical formulas (matrix form) to see how covariates are entering the mathematical model.

Lab topic:
The main lab is to go through the covariate chapter and examples in the MARSS User Guide. Then we have some salmon data to play with to try different ways of including cycles (in this case driven by cohort strength) into an analysis.

Lecture 6
Click the big arrow to start. You can also find the ppt of lecture t on the class webpage.

Week 5: Applied Time-Series Analysis for Fisheries and Environmental Data

2013-02-06T15:17:00.003-08:00

Class material: webpage

Week 5: Introduction to multivariate autoregressive state-space models
Lecture topics:

Review of dealing with obs error with ARIMA (from last week)
Multivariate state space models
How these are expressed mathematically
Analysis of multi-site data using this framework
Parameter estimation: Kalman filter, Newton methods and EM algorithm

Lab topic:
The main lab is to go through case study 2 in the MARSS User Guide. I have a Fish 507 specific version of the case study code on the course website with questions to answer as you go through. Case study 3 and 8 are optional but going through them will help solidify your understanding of multivariate state-space models. Do go through the ARMA code as it discusses some important points about the effects of data transformation (in this case differences) on the time-series model that is appropriate for the data.

Lecture 5
Click the big arrow to start the show. You can also find just a pdf of lecture 5 on the class webpage.

Week 4: Applied Time-Series Analysis for Fisheries and Environmental Data

2013-01-31T10:33:00.002-08:00

Class material: webpage

Week 4: Introduction to univariate autoregressive state-space models
Topics:

State-space models
Process versus observation error
Model Selection

Lecture 4
Click the big arrow to start the show. You can also find just the ppt of lecture 3 on the class webpage.

Week 3: Applied Time-Series Analysis for Fisheries and Environmental Data

2013-01-22T17:59:00.003-08:00

Class material: webpage

Week 3: Estimation, model selection, and forecasting for time series models
Topics:

Summarizing ARIMA models
Estimation
Model Selection
Prediction & forecasting
Evaluating forecasts
Functions: arima(), lm(), Arima()

Lecture 3
This is our second attempt at recording a lecture. Still much to be learned but we are getting better. Click the big arrow to start the show. You can also find just the ppt of lecture 3 on the class webpage.

Week 2: Applied Time-Series Analysis for Fisheries and Environmental Data

2013-01-15T15:27:00.002-08:00

Class material: webpage

Week 2: Correlation, stationarity & stationary time-series models
The lecture introduces the ACF, PACF, and basic properties of AR, MA and ARMA models. The computer code section shows R code to analyze simulated time-series data so that participants get a feel for ACF and PACF and get a feel for AR and MA processes. The participants then move to analyzing some real time-series data using the 30+ year time-series of Lake Washington plankton.

Lecture 2
This is our first attempt at recording a lecture. Ahem, there is clearly much to be learned to improve the process...Click the big arrow to start the show. You can also find just the ppt of lecture 2 on the class webpage.

Winter stats reading group starting up: Hierarchical Modeling and Analysis for Spatial Data

2012-11-30T13:21:00.001-08:00

The NWFSC/SAFS stats reading group is reading "Hierarchical Modeling and Analysis for Spatial Data" by Banerjee et al. this quarter. Fridays 3pm at SAFS 229 during Winter Qtr 2013. Open to interested statistical ecologists. Contact Eli.

New paper on spatial-temporal time series modeling

2012-11-12T16:49:00.003-08:00

New paper just out by Eric Ward using Bayesian state-space time-series models.

"Applying time series models with spatial correlation to identify the scale of variation in habitat metrics related to threatened coho salmon (Oncorhynchus kisutch) in the Pacific Northwest"
Eric J. Ward, George R. Pess, Kara Anlauf-Dunn, and Chris E. Jordan
Canadian Journal of Fisheries and Aquatic Science (link to paper)

Abstract: Trend analyses are common in the analysis of fisheries data, yet the majority of them ignore either observation error or spatial correlation. In this analysis, we applied a novel hierarchical Bayesian state-space time series model with spatial correlation to a 12-year data set of habitat variables related to coho salmon (Oncorhynchus kisutch) in coastal Oregon, USA. This model allowed us to estimate the degree of spatial correlation separately for each habitat variable and the importance of observation error relative to environmental stochasticity. This framework allows us to identify variables that would benefit from additional sampling and variables where sampling could be reduced. Of the eight variables included in our analysis, we found three metrics related to habitat quality correlated at large spatial scales (gradient, fine sediment, shade cover). Variables with higher observation error (pools, active channel width, fine sediment) could be made more precise with more repeat visits. Our spatio-temporal model is flexible and extendable to virtually any spatially explicit monitoring data set, even with large amounts of missing data and no repeated observations. Potential extensions include fisheries catch data, abiotic indicators, invasive species, or species of conservation concern.

Time-series analysis course winter 2012

2012-08-01T15:16:00.003-07:00

Fish 50X: Applied Time Series Analysis in Fisheries and Environmental Sciences
Winter 2012
Fisheries Dept, University of Washington

Instructors: Eric Ward, Eli Holmes, Mark Scheuerell
email: eli.holmes@noaa.gov, mark.scheuerell@noaa.gov, eric.ward@noaa.gov

Reviews current applications of univariate and multivariate time series models for biological and environmental data, emphasizing the estimation, inference, and forecasting aspects of time-series models. Explores effects of covariates and anthropogenic drivers for species that are exploited and/or of conservation concern. Recommended: FISH 552 or prior experience with R (e.g. FISH 560), QSCI 482 or basic statistics, and at least 1 course in population dynamics (FISH 454 or 458).

R Journal article on the MARSS package

2012-08-01T15:12:00.004-07:00

Holmes, E. E., Ward, E. J. and K. Wills. 2012. MARSS: Multivariate autoregressive state-space models for analyzing time-series data. R Journal 4: 11-19. http://journal.r-project.org/archive/2012-1/RJournal_2012-1_Holmes~et~al.pdf

MARSS 3.1 released on CRAN

2012-08-01T15:11:00.002-07:00

MARSS 3.1 is now up on CRAN. This allows for time-varying constraints and covariates. See the updated User Guide (on CRAN). The major changes are internal and allow for us easily write customized functions for different MARSS forms (like AR-p processes and DFA models). 3.1 is considerably slower than 2.x, however this should be fixed in 3.2 or 3.3 when the Kalman filter in the KFAS package can be hooked back up to MARSS (temporarily disabled). http://cran.r-project.org/web/packages/MARSS

MARSS 3.0 posted for testing

2012-06-23T17:36:00.000-07:00

Hi MARSS users,

A version of MARSS 3.0 is now up for testing. It should be backwards compatible with any MARSS 2.x code you have unless you use control$diffuse or control$kf.x0. diffuse now goes into your model list and kf.x0 is called tinitx and goes in the model list too. control$kf.x0="x00" is now model$tinitx=0 and control$kf.x0="x10" is model$tinitx=1.

Big changes are

Time-varying parameters are allowed. See the Quick_Start.pdf to get a brief intro to that feature but it should be pretty self-explanatory.
Covariates can be added in the standard way. Again see the Quick_Start.pdf for quick intro. See chapter in User Guide on estimating species interactions for an example.
There is a "form" argument in the MARSS() call that allows one to specify special types of models. Default is "marxss" which covers MARSS + covariates. The only other form now is "dfa" for Dynamic Factor Analysis. Check out the DFA chapter in the User Guide for an intro to the form="dfa" which allows you to do a standard DFA by just passing in m (number of states), data and covariates (if wanted). The dfa form is basic now. Later we will specialize its output to give loadings etc.
The AR-p models work now with method="kem" which is much, much faster than method="BFGS". See chapter in User Guide on AR-p models.
You can enter things like B=diag(list("1+2*c+3*b",0,0,"2+3c"),2,2) in your list matrices and MARSS will know what to do, i.e. it will estimate B.c and B.b and it knows that B(1,1)=1+2c+3b . I haven't seen people want to do this..., but you can.
The print call takes a argument called "what". Use ?print.marssMLE to see how to use it. It'll make it easier to print things from your marssMLE objects (what you get back from a MARSS call).

Here is are the tar.gz and .zip files for the 3.0 version. You'll find links to the User Guide and Quick_Start guide here too.

http://fishbox.iugo-cafe.org/user/e2holmes/MARSS%20Dev%20Site

Plan is to upload to CRAN about July 1 assuming no big issues arise. Right now all the prior examples in the User Guide 2.8 and the man files work as before. I've included new examples with covariates using the new covariate code in the chapter on estimating spp interactions and I've added covariates to the DFA chapter. I've included a little more code in the AR-1 chapter on estimating those models.

Feel free to try it out. The more real-world testing it gets before being uploaded to CRAN the better.

Cheers,

Eli

Article from the group on analysis of marine plankton community structure

2012-03-22T17:58:00.000-07:00

Scheef, L.P., D.E. Pendleton, S.E. Hampton, S.L. Katz, E.E. Holmes, M.D. Scheuerell, and D.G. Johns. 2012. Assessing marine plankton community structure from long-term monitoring data with multivariate autoregressive (MAR) models: a comparison of fixed station vs. spatiallydistributed sampling data. Limnology & Oceanography: Methods 10: 54-64.

ABSTRACT: We examined how marine plankton interaction networks, as inferred by multivariate autoregressive (MAR) analysis of time-series, differ based on data collected at a fixed sampling location (L4 station in the Western English Channel) and four similar time-series prepared by averaging Continuous Plankton Recorder (CPR) datapoints in the region surrounding the fixed station. None of the plankton community structures suggested by the MAR models generated from the CPR datasets were well correlated with the MAR model for L4, but of the four CPR models, the one most closely resembling the L4 model was that for the CPR region nearest to L4. We infer that observation error and spatial variation in plankton community dynamics influenced the model performance for the CPR datasets. A modified MAR framework in which observation error and spatial variation are explicitly incorporated could allow the analysis to better handle the diverse time-series data collected in marine environments.

Time-series analysis workshop Sat Aug 5, Portland, OR

2012-03-22T17:43:00.001-07:00

Eli, Eric and Mark will offer their 1-day workshop on multivariate time-series analysis using the MARSS package again at the annual ESA meeting (http://www.esa.org/portland). Workshop is scheduled for the Sat before the meeting. We are working on new case studies involving incorporation of covariates into analyses. MARSS 3.0 will be done by then. This is a major update that allows all parameters to incorporate time-varying covariates.

New article by Eric in Conservation Letters

2011-11-09T12:45:00.000-08:00

Integrating diet and movement data to identify hot spots of predation risk and areas of conservation concern for endangered species, Eric J. Ward, Phillip S. Levin, Monique M. Lance, Steven J. Jeffries, Alejandro Acevedo-Gutiérrez
Effective management of threatened and endangered species requires an understanding of how species of conservation concern are distributed spatially, as well as the spatial distribution of risks to the population, such as predation or human impacts (fishing, pollution, loss of habitat). Identifying high risk areas is particularly important when designing reserves or protected areas. Our novel approach incorporates data on distribution, movement, and diet of a generalist marine predator (harbor seals) to identify and map ‘hot-spots’ of predation risk for an endangered prey species (rockfish). Areas with high concentrations of seals (including some current marine reserves) are also estimated hot spots for rockfish predation. While marine reserve planning currently targets areas with good habitat and low human disturbance, our modeling suggests that future terrestrial and marine reserve design may be made more effective by incorporating other components of the food web that either directly or indirectly interact with target species.

http://onlinelibrary.wiley.com/doi/10.1111/j.1755-263X.2011.00210.x/abstract;jsessionid=7EB218B3908AAED4EC8A2E308F6C9046.d01t02