This is a chapter for a new book entitled Computational Actuarial Science with R to be published by Chapman and Hall/CRC, and edited by Arthur Charpentier.

Download (PDF, 600KB)

The new version of the hts package (v3.01) has a vignette.

Download (PDF, 327KB)

Souhaib Ben Taieb (1) and Rob J Hyndman (2)
(1) Machine Learning Group, Department of Computer Science, Université Libre de Bruxelles
(2) Depart­ment of Eco­no­met­rics & Busi­ness Stat­ist­ics, Mon­ash Uni­ver­sity, Clayton, Vic­toria, Aus­tralia

Abstract : We describe and analyse the approach used by Team TinTin (Souhaib Ben Taieb and Rob J Hyndman) in the Load Forecasting track of the Kaggle Global Energy Forecasting Competition 2012. The competition involved a hierarchical load forecasting problem for a US utility with 20 geographical zones. The available data consisted of the hourly loads for the 20 zones and hourly temperatures from 11 weather stations, for four and a half years. For each zone, the hourly electricity load for nine different weeks needed to be predicted without having the location of zones or stations. We used separate models for each hourly period, with component-wise gradient boosting to estimate each model using univariate penalised regression splines as base learners. The models allow for the electricity demand to change with time-of-year, day-of-week, time-of-day, and on public holidays, with the main predictors being current and past temperatures as well as past demand. Team TinTin ranked fifth out of 105 participating teams.

Keywords: Short-term load forecasting; multi-step forecasting; additive models; gradient boosting; machine learning; Kaggle competition

Download working paper

Abstract:
For 25 years I have been an intrepid statistical consultant, tackling the wild frontiers of real data, real problems and real time constraints. I have faced problems ranging from linguistics to river beds, from making paper plates to selling pies at the MCG, from tax office audits to surveys about the colour purple. University education helps prepare you to be a statistical consultant in the same way that Google maps helps prepare you to cross the Simpson Desert. You have some idea of the main features, but when you get there, nothing looks familiar.

I will describe some of my adventures, and explain how to bluff your way through ignorance, work with completely inadequate tools, and deal with smelly clients. I will tell you the story of the client who wouldn’t give me the data, the client who wouldn’t tell me the problem, and the client who wanted all meetings held at random locations for security reasons.

Along the way we will learn about the skills that statisticians need to survive in the wild.

^aDepartment of Econometrics & Business Statistics, Monash University, Clayton, Victoria, Australia.
^bThe Australian Demographic & Social Research Institute, Australian National University, Canberra, ACT, Australia.

Demography, 50(1), 261–283.

Revised version: 20 April 2012.

Abstract:
When independence is assumed, forecasts of mortality for subpopulations are almost always divergent in the long term. We propose a method for coherent forecasting of mortality rates for two or more subpopulations, based on functional principal components models of simple and interpretable functions of rates. The product-ratio functional forecasting method models and forecasts the geometric mean of subpopulation rates and the ratio of subpopulation rates to product rates. Coherence is imposed by constraining the forecast ratio function through stationary time series models. The method is applied to sex-specific data for Sweden and state-specific data for Australia. Based on out-of-sample forecasts, the coherent forecasts are at least as accurate in overall terms as comparable independent forecasts, and forecast accuracy is homogenised across subpopulations.

Keywords: Mortality forecasting, coherent forecasts, functional data, Lee-Carter method, life expectancy, mortality, age pattern of mortality, sex-ratio.

Download working paper

Online paper

Slides and video on my blog.

Download (PDF, 100KB)

1. Université Libre de Bruxelles
2. Monash University

Abstract:

We propose a new forecasting strategy, called rectify, that seeks to combine the best properties of both the recursive and direct forecasting strategies. The rationale behind the rectify strategy is to begin with biased recursive forecasts and adjust them so they are unbiased and have smaller error. We use linear and nonlinear simulated time series to investigate the performance of the rectify strategy and compare the results with those from the recursive and the direct strategies. We also carry out some experiments using real world time series from the M3 and the NN5 forecasting competitions. We find that the rectify strategy is always better than, or at least has comparable performance to, the best of the recursive and the direct strategies. This finding makes the rectify strategy very attractive as it avoids making a choice between the recursive and the direct strategies which can be a difficult task in real-world applications.

Download paper

Journal of Biometrics and Biostatistics (2012), S7-018.

Abstract

Background: Few case-control studies of time dependent environmental exposures and respiratory outcomes have been performed. Small sample sizes pose modeling challenges for estimating interactions. In contrast, case cross-over studies are well suited where control selection and responses are low, time consuming and costly.

Objective: To demonstrate the feasibility and validity of a case crossover study of children admitted to hospital for asthma to examine interacting effects of time varying environmental exposures. 

Methods: The Melbourne Air Pollen Children and Adolescent Health (MAPCAH) study recruited incident asthma admissions of children and adolescents aged 2–17 years to a tertiary hospital. A case was defined by date of admission, and eligible cases served as their own controls. We used bi-directional sampling design for control selection. At time of admission, participants underwent skin prick tests and nasal/throat swabs (NTS) to test for respiratory viruses. Questionnaires collected data on asthma management, family history and environmental characteristics. Daily concentrations of ambient pollen, air pollution and weather variables were also available.

Results: 644 children were recruited. More than half (63%) were male with mean age 5.2 (SD 3.3) years. Non-participants were slightly younger at admission (mean age 4.3, SD 2.8, p<0.001), although the absolute differences were small. Participants and non-participants were well balanced on gender. The most common reason for refusal to participate in the study was “causing further distress to child by skin prick testing”. Of those recruited, 46% were sensitized to any pollen, 14% were sensitized to fungi, and 22% tested positive to egg or peanut allergens. 68% of children had positive NTS for human rhinovirus (HRV) at admission and 22% were still positive nine weeks later. Parental history of asthma and hay fever was common.  Children who skin-tested positive to any pollen were slightly older (mean 6.4 years, SD 3.6, p<0.001).

Gender and age distributions were similar to the overall admissions to the tertiary hospital as well as in Victoria. Our study slightly under-represented winter admissions (p<0.001), and was over-represented in summer(p<0.002). More admissions occurred during the grass pollen season in our study than in general asthma hospital admissions across Victoria (36% versus 22%, p<0.001).

Conclusions: The case cross-over method is a highly feasible design for a reasonably sized hospital-based study of children with asthma. MAPCAH has robust internal validity and strong generalizability. Collection of data on respiratory viruses and pollen exposure at the time of admission on children with asthma provides important information that will have clinical and public health impacts.

Key words: case crossover design, asthma, respiratory viral infections, internal validity

Online article

Abstract: Many applications require a large number of time series to be forecast completely automatically. For example, manufacturing companies often require weekly forecasts of demand for thousands of products at dozens of locations in order to plan distribution and maintain suitable inventory stocks. In population forecasting, there are often a few hundred time series to be forecast, representing various components that make up the population dynamics. In these circumstances, it is not feasible for time series models to be developed for each series by an experienced statistician. Instead, an automatic forecasting algorithm is required.

I will look at some algorithms recently developed for automatically forecasting various types of time series, including approaches for handling functional time series, hierarchical time series, and time series with multiple seasonality.

Slides:

Downloads

• Long version (for ASC2012 and Lancaster Uni)
• Short version (for COMPSTAT12)