<?xml version="1.0" encoding="UTF-8"?>
<rss  xmlns:atom="http://www.w3.org/2005/Atom" 
      xmlns:media="http://search.yahoo.com/mrss/" 
      xmlns:content="http://purl.org/rss/1.0/modules/content/" 
      xmlns:dc="http://purl.org/dc/elements/1.1/" 
      version="2.0">
<channel>
<title>Hyndsight</title>
<link>https://robjhyndman.com/hyndsight/</link>
<atom:link href="https://robjhyndman.com/hyndsight/index.xml" rel="self" type="application/rss+xml"/>
<description>Blog posts by Professor Rob J Hyndman</description>
<generator>quarto-1.10.18</generator>
<lastBuildDate>Mon, 24 Aug 2026 14:00:00 GMT</lastBuildDate>
<item>
  <title>Evaluating time series foundation model claims</title>
  <link>https://robjhyndman.com/hyndsight/foundation_models.html</link>
  <description><![CDATA[ 




<div>
  <div class="description">
    <em>I was emailed by a journalist writing an article about time series foundation models.</em> <em>Here is my response.</em> <em>While the questions were about a specific new transformer model, the comments are more widely applicable, so I’ve edited my answer to be more generic.</em>
  </div>
</div><p></p>


<p>As background, I would point you to my online forecasting textbook where we have a chapter on foundation models&nbsp;<span class="citation" data-cites="fpppy">(Hyndman et al., 2026)</span>. Another excellent reference on this topic is Christoph Bergmeir’s paper “LLMs and Foundational Models: Not (Yet) as Good as Hoped”&nbsp;<span class="citation" data-cites="Bergmeir2024llms">(Bergmeir, 2024)</span>.</p>
<p><strong>1.&nbsp;How significant a change do time series foundation models represent for the practice of forecasting?</strong></p>
<p>They’re a real shift in how forecasts get <em>produced</em>, but not yet a demonstrated shift in how <em>accurate</em> forecasts are in general. The genuine change is architectural and operational: instead of building, tuning and maintaining a separate model for every series, an organisation can point a single pre-trained model at a new dataset and get a forecast with no dataset-specific training or model selection. That lowers the barrier to entry for forecasting at scale, particularly for teams without in-house statistical expertise.</p>
<p>What it hasn’t yet done is replace the case for careful statistical modelling. The strongest independent critique of the field is <span class="citation" data-cites="Bergmeir2024llms">Bergmeir (2024)</span> who found that when you check the reported empirical results, the performance is often nowhere near as good as claimed. Often the statistical benchmarks are implemented poorly, or missing altogether, or highly selective. So these foundation models are an important development, but they are not close to being the panacea that many of the companies behind them are claiming.</p>
<p><strong>2.&nbsp;How should organisations compare foundation models with well-tuned statistical or task-specific approaches, particularly considering accuracy and cost?</strong></p>
<p>First, make sure the statistical baseline is actually well implemented. This is the single biggest problem <span class="citation" data-cites="Bergmeir2024llms">Bergmeir (2024)</span> identifies in the foundation-model literature: papers routinely compare against under-tuned or de-ensembled versions of classical methods, then claim victory. So I would be asking: what exactly were the statistical benchmarks, how were they tuned, and were they given the same advantages (ensembling, proper cross-validation) as in the studies that established them as strong benchmarks in the first place?</p>
<p>Second, evaluate on your own data with proper rolling-origin backtesting, not on a vendor’s leaderboard, or even a public community leaderboard. Public benchmark scores are increasingly unreliable as measures of genuine out-of-sample skill because the data used in comparisons was also used in training. This “leakage” is a growing problem – see <span class="citation" data-cites="meyer2025rethinking">Meyer et al. (2025)</span> who found that of 401 datasets used across 22 published time-series foundation models, only 6% had never appeared in any model’s pre-training or fine-tuning corpus. Also, historical series that are not in the training data can be correlated with series that are in the training data due to being subject to the same environmental shocks (e.g., COVID). So even if there is not direct leakage, there can be indirect leakage induced by correlations. Therefore, these comparisons on historical data are not a great guide to what the models will do on new data.</p>
<p>Third, weigh total cost, not just error metrics. Well-implemented statistical models (ETS, ARIMA, Theta, and simple combinations of these) are cheap, transparent, fast, and easy to explain. Foundation models bring inference cost, latency, hosting/vendor dependency, and limited interpretability. Even if the accuracy gain over a well-tuned statistical model is real, the operational cost may not be worth it.</p>
<p><strong>3.&nbsp;What do current benchmarks fail to tell us about how these models will perform in real production settings?</strong></p>
<ol type="1">
<li>Public leaderboards report an aggregate score across many datasets and domains at once, which can hide wide variance in performance on the specific kind of series a given business actually needs to forecast.</li>
<li>As noted above, they carry a real contamination risk which may render the benchmark unreliable.</li>
<li>Benchmarks rarely report the things that determine whether a model is usable in production: inference latency at the volumes a business needs, hosting and total compute cost, robustness to structural breaks and regime changes, how the model handles exogenous variables and irregular/missing data, and how failures are detected and explained when they occur. A model that tops a MASE leaderboard by a small margin but takes an order of magnitude longer to run, or performs poorly on genuinely new data, may be the wrong choice operationally even if it’s the “best” model on paper.</li>
</ol>




<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-bibliography"><h2 class="anchored quarto-appendix-heading">References</h2><div id="refs" class="references csl-bib-body hanging-indent" data-entry-spacing="0">
<div id="ref-Bergmeir2024llms" class="csl-entry">
Bergmeir, C. (2024). <span>LLMs</span> and foundational models: Not (yet) as good as hoped. <em>Foresight: The International Journal of Applied Forecasting</em>, <em>73</em>, 33–38. <a href="https://cbergmeir.com/papers/Bergmeir2024LLMs.pdf">https://cbergmeir.com/papers/Bergmeir2024LLMs.pdf</a>
</div>
<div id="ref-fpppy" class="csl-entry">
Hyndman, R. J., Athanasopoulos, G., Garza, A., Challu, C., Mergenthaler, M., &amp; Olivares, K. G. (2026). Ch 15. Foundation forecasting models. In <em>Forecasting: Principles and practice, the pythonic way</em>. OTexts. <a href="http://OTexts.com/fpppy/15-foundation-models.html">http://OTexts.com/fpppy/15-foundation-models.html</a>
</div>
<div id="ref-meyer2025rethinking" class="csl-entry">
Meyer, M., Kaltenpoth, S., Zalipski, K., &amp; Müller, O. (2025). <em>Rethinking evaluation in the era of time series foundation models:(un) known information leakage challenges</em>. <a href="https://arxiv.org/abs/2510.13654v3">https://arxiv.org/abs/2510.13654v3</a>
</div>
</div></section></div> ]]></description>
  <category>forecasting</category>
  <guid>https://robjhyndman.com/hyndsight/foundation_models.html</guid>
  <pubDate>Mon, 24 Aug 2026 14:00:00 GMT</pubDate>
  <media:content url="https://otexts.com/fpp3/fpp_files/figure-html/cv1-1.png" medium="image" type="image/png"/>
</item>
<item>
  <title>Researcher portals</title>
  <link>https://robjhyndman.com/hyndsight/portals/</link>
  <description><![CDATA[ 




<div>
  <div class="description">
    I first wrote this post in 26 November 2011. Since then, some services have been discontinued, and new ones started. So I’ve updated it to reflect current offerings.
  </div>
</div><p></p>


<p>A researcher portal is a website that attempts to list all the publications of a given researcher, and calculate citation statistics, etc. Every researcher wants their work read and cited, so these websites can be useful tools for getting your work noticed. Conversely, they can be a good way to find new work by researchers in your field.</p>
<section id="google-scholar-citations" class="level4">
<h4 class="anchored" data-anchor-id="google-scholar-citations">Google Scholar Citations</h4>
<p><a href="http://scholar.google.com">Google Scholar</a> itself is an incredible resource covering journal articles, working papers, books, and almost everything else a scholar might cite. Google Scholar Citations uses Google Scholar data to provide information about all the outputs from a researcher. Some examples are <a href="http://scholar.google.com.au/citations?user=SEOgduoAAAAJ&amp;hl=en">Andrew Gelman</a> and <a href="https://scholar.google.com.au/citations?user=_b0uchgAAAAJ&amp;hl=en">Di Cook</a>. <a href="http://scholar.google.com.au/citations?user=vamErfkAAAAJ&amp;hl=en">My profile</a> is also there. It provides a way of listing your publications, tracking citations to your publications, computing citation metrics, etc. There is very little work in setting up a profile. When I did it, Google had automatically identified all my publications, and it even listed one paper I had forgotten I had written!</p>
<p>To see the profile of an existing researcher, just search for their name on Google Scholar. If they have made their profile public, it will appear at the top of the search results.</p>
<p>To set up your profile, go to <a href="http://scholar.google.com/citations">scholar.google.com/citations</a>. You may have to do some editing of the results to merge versions of the same publication, or to correct some errors in the database. I wish Google would use the information provided by these edits to correct their Google Scholar database.</p>
</section>
<section id="semantic-scholar" class="level4">
<h4 class="anchored" data-anchor-id="semantic-scholar">Semantic Scholar</h4>
<p>An alternative to Google Scholar is <a href="http://semanticscholar.org">Semantic Scholar</a>. The coverage is not quite as broad as Google Scholar – it misses working papers unless they are on arXiv. On the other hand, there are some useful tools, including AI-generated summaries that are quite good. It is also easy to follow cited papers, and citing papers, along the citation tree. Related papers are also identified, so it is possible to explore further afield than just direct citations. You can follow individual authors, and set up alerts for new citations of specific papers. Citations are classified as “highly influential”, “background”, “methods”, “results”, etc., so you can easily filter papers that have had a big impact, those that apply new methods, etc. One interesting difference is that Semantic Scholar automatically creates personal author pages for each author, without needing each researcher to set them up. Mine is <a href="https://www.semanticscholar.org/author/Rob-J-Hyndman/145211352">here</a>. My h-index on Semantic Scholar is 75, compared to 92 on Google Scholar, reflecting the broader coverage of Google Scholar.</p>
</section>
<section id="all-the-rest" class="level4">
<h4 class="anchored" data-anchor-id="all-the-rest">All the rest</h4>
<p>There are several other sites attempting to provide similar services, but none of them come close to Google Scholar or Semantic Scholar in useful features and usability. I’ve set up profiles on all of them, just to see how they work.</p>
<ul>
<li><p><a href="https://www.webofscience.com/wos/author/record/A-2268-2008">Web of Science</a> (previously called Researcher ID): lists 234 of my publications, and gives my h-index as 61. This tends to miss books, R packages, working papers and book chapters, but has most of my journal articles. Links to online versions of the papers are provided. The “Clarivate Highly Cited” Awards are based on these citations.</p></li>
<li><p><a href="http://www.researchgate.net/profile/Rob_Hyndman/">ResearchGate</a>: lists 364 of my publications (so there must be some double counting). I used to get a lot of unsolicited mail from this site, but that seems to have stopped. In any case, it does nothing that the other sites don’t do better.</p></li>
<li><p><a href="http://monash.academia.edu/robjhyndman">Academia.edu</a>: wants me to sign up for a premium account to do anything more than the basic. I had to block all the emails I was receiving from this site. Not interested.</p></li>
</ul>
</section>
<section id="recommendations" class="level4">
<h4 class="anchored" data-anchor-id="recommendations">Recommendations</h4>
<ol type="1">
<li><p>Ignore the emails from ResearcherID, ResearchGate, Academia.edu, etc. Just hit delete and block them.</p></li>
<li><p>Set up a page on <a href="http://scholar.google.com/citations">Google Scholar Citations</a>. It’s not much work and makes your work more visible. It also allows you to track citations which are useful if you apply for promotion.</p></li>
<li><p>Try <a href="https://semanticscholar.org">Semantic Scholar</a>. You might find it helpful.</p></li>
<li><p>Check that your <a href="https://semanticscholar.org">Semantic Scholar</a> author page is accurate. Spend 10 minutes editing your profile so it gives a little more information about you.</p></li>
</ol>


</section>

 ]]></description>
  <category>computing</category>
  <category>journals</category>
  <category>organization</category>
  <category>productivity</category>
  <category>references</category>
  <guid>https://robjhyndman.com/hyndsight/portals/</guid>
  <pubDate>Sun, 02 Aug 2026 14:00:00 GMT</pubDate>
</item>
<item>
  <title>Mathematical genealogy</title>
  <link>https://robjhyndman.com/hyndsight/mathematical-genealogy/</link>
  <description><![CDATA[ 






<p>Having a PhD student is like having a child. I have had many such “children” graduate, and have another two “on the way”. (See <a href="../../research-team">here</a> for my offspring.)</p>
<p>Going in the other direction, here is my family tree, compiled from the <a href="https://www.mathgenealogy.org">Mathematics Genealogy Project</a> (MGP). Each person advised the one below them. Hover over a portrait (or tap it) to see who they were.</p>
<p>Many people have two or more advisors, so there are multiple “lines of ascent” (as there are with regular family trees). I am showing two of many possible ancestry lines here, which happened to reconnect in the 1600s.</p>
<style>
.gen {
  --gen-fig: 56px;
  --gen-rail: 27px;      /* centre of the portrait, minus the 2px rail width */
  --gen-gap: 2rem;
  --gen-line: rgba(128,128,128,.45);
  --gen-line: color-mix(in srgb, currentColor 30%, transparent);
  --gen-pop-bg: var(--bs-body-bg, #fff);
  --gen-pop-fg: var(--bs-body-color, inherit);
  margin: 2rem 0 1rem;
  font-size: .95rem;
}

/* ---- rows ---------------------------------------------------------- */
.gen ol { list-style: none; margin: 0; padding: 0; }
.gen-chain { position: relative; }
.gen-chain::before {                       /* the vertical rail */
  content: ""; position: absolute; z-index: 0;
  left: var(--gen-rail); top: 0; bottom: 0; width: 2px;
  background: var(--gen-line);
}
.gen-row {
  position: relative; z-index: 1;
  display: flex; align-items: center; gap: .7rem;
  padding: 3px 0;
}
.gen-row:hover, .gen-row:focus-within { z-index: 2; }
.gen-row--spacer { height: calc(var(--gen-fig) + 6px); }
.gen-node {
  display: flex; align-items: center; gap: .7rem;
  border: 0; background: none; padding: 0; margin: 0;
  font: inherit; color: inherit; text-align: left; cursor: help;
}
.gen-node:focus-visible { outline: 2px solid currentColor; outline-offset: 3px; border-radius: 4px; }

.gen-fig {
  position: relative; flex: 0 0 auto;
  width: var(--gen-fig); height: var(--gen-fig);
  border-radius: 50%; overflow: hidden;
  background: var(--gen-line);
  display: grid; place-items: center;
}
.gen-fig::after {                          /* initials, shown if the photo is missing */
  content: attr(data-initials);
  font-size: .8rem; font-weight: 600; letter-spacing: .02em; opacity: .8;
}
.gen-fig img {
  position: absolute; inset: 0; z-index: 1;
  width: 100%; height: 100%; object-fit: cover; object-position: top center;
  margin: 0;
}
.gen-name { font-weight: 600; line-height: 1.2; }
.gen-meta { display: block; font-size: .78rem; font-style: italic; opacity: .75; }

/* ---- pop-ups ------------------------------------------------------- */
.gen-pop {
  position: absolute; z-index: 30; top: -4px; left: calc(var(--gen-fig) + 1rem);
  width: min(24rem, 46vw);
  opacity: 0; visibility: hidden; transform: translateY(-4px);
  transition: opacity .12s ease, transform .12s ease;
  background: var(--gen-pop-bg); color: var(--gen-pop-fg);
  border: 1px solid var(--gen-line); border-radius: 6px;
  padding: .7rem .85rem; font-size: .82rem; line-height: 1.45;
  box-shadow: 0 8px 24px rgba(0,0,0,.18);
}
.gen-pop p { margin: 0 0 .45rem; }
.gen-pop p:last-child { margin-bottom: 0; }
.gen-pop .gen-caveat { opacity: .8; font-size: .95em; }
.gen-pop .gen-inset {
  float: right; width: 64px; margin: 0 0 .4rem .6rem; border-radius: 3px;
}
.gen-pop-head {
  --gen-fig: 34px;
  display: flex; align-items: center; gap: .6rem;
  margin: -.7rem -.85rem .6rem; padding: .6rem .85rem;
  border-bottom: 1px solid var(--gen-line);
}
.gen-pop-head .gen-name { font-size: .95rem; }
.gen-pop-head .gen-meta { font-size: .72rem; }
.gen-branch--right .gen-pop { left: auto; right: calc(var(--gen-fig) + 1rem); }
.gen-row:hover .gen-pop,
.gen-row:focus-within .gen-pop { opacity: 1; visibility: visible; transform: none; }

/* ---- fork and merge ------------------------------------------------ */
.gen-branches { display: grid; grid-template-columns: 1fr 1fr; gap: var(--gen-gap); }
.gen-join { display: grid; grid-template-columns: 1fr 1fr; gap: var(--gen-gap); height: 1.4rem; }
.gen-join > span { position: relative; }
.gen-join > span::before {                 /* the two legs */
  content: ""; position: absolute; left: var(--gen-rail); top: 0; bottom: 0;
  width: 2px; background: var(--gen-line);
}
.gen-join--fork > span:first-child::after,
.gen-join--merge > span:first-child::after {   /* the crossbar */
  content: ""; position: absolute; left: var(--gen-rail);
  width: calc(100% + var(--gen-gap)); height: 2px; background: var(--gen-line);
}
.gen-join--fork  > span:first-child::after { top: 0; }
.gen-join--merge > span:first-child::after { bottom: 0; }
.gen-join--merge > span:last-child::before { bottom: 2px; }

.gen-label {
  font-size: .75rem; text-transform: uppercase; letter-spacing: .06em;
  opacity: .6; margin: 0 0 .35rem calc(var(--gen-fig) + .7rem);
}
.gen-caption { font-size: .8rem; opacity: .75; margin: .6rem 0 0; }

/* ---- narrow screens ------------------------------------------------ */
@media (max-width: 700px) {
  .gen-branches { grid-template-columns: 1fr; gap: 0; }
  .gen-join, .gen-join--fork, .gen-join--merge { display: none; }
  .gen-branch + .gen-branch { margin-top: 1.5rem; }
  .gen-pop {
    position: static; width: auto; display: none; opacity: 1; visibility: visible;
    transform: none; margin: .4rem 0 .4rem calc(var(--gen-fig) + .7rem);
    box-shadow: none;
  }
  .gen-row { flex-wrap: wrap; }
  .gen-row:hover .gen-pop, .gen-row:focus-within .gen-pop { display: block; }
  .gen-branch--right .gen-pop { right: auto; }
  .gen-pop-head { display: none; }
  .gen-row--spacer { display: none; }
}
@media (prefers-reduced-motion: reduce) {
  .gen-pop { transition: none; }
}
</style>

<div class="gen">

<!-- ============ shared root ============ -->
<ol class="gen-chain">
  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="JT"><img src="https://robjhyndman.com/genealogy/thomasius.jpg" alt="Jakob Thomasius"></span>
      <span class="gen-name">Jakob Thomasius<span class="gen-meta">Universität Leipzig, 1643</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="JT"><img src="https://robjhyndman.com/genealogy/thomasius.jpg" alt="Jakob Thomasius"></span><span class="gen-name">Jakob Thomasius<span class="gen-meta">Universität Leipzig, 1643</span></span></span>
      <p>B. 1622, Leipzig. D. 1684, Leipzig.</p>
      <p>Philosopher and philologist at Leipzig, and the teacher both branches share. He supervised Leibniz's 1666 philosophy degree and Otto Mencke's 1665 thesis, which is where the two lines below separate.</p>
      <p><a href="https://en.wikipedia.org/wiki/Jakob_Thomasius">Wikipedia</a></p>
    </span>
  </li>
</ol>

<div class="gen-join gen-join--fork"><span></span><span></span></div>

<!-- ============ the two branches ============ -->
<div class="gen-branches">

<!-- ---------- left branch: through Adrian Raftery ---------- -->
<div class="gen-branch gen-branch--left">
<ol class="gen-chain">

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="GL"><img src="https://robjhyndman.com/genealogy/leibniz.jpg" alt="Gottfried Leibniz"></span>
      <span class="gen-name">Gottfried Leibniz<span class="gen-meta">Dr. jur. Universität Altdorf, 1667</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="GL"><img src="https://robjhyndman.com/genealogy/leibniz.jpg" alt="Gottfried Leibniz"></span><span class="gen-name">Gottfried Leibniz<span class="gen-meta">Dr. jur. Universität Altdorf, 1667</span></span></span>
      <img class="gen-inset" src="https://robjhyndman.com/genealogy/weigel.jpg" alt="Erhard Weigel">
      <p>B. 1646, Leipzig. D. 1716, Hanover.</p>
      <p>German philosopher, mathematician and logician, best known for inventing calculus independently of Newton.</p>
      <p class="gen-caveat">His other advisors were Erhard Weigel at Jena (pictured, professor of mathematics there from 1653), Bartholomäus Schwendendörffer at Altdorf, and Christiaan Huygens in Paris.</p>
      <p><a href="https://en.wikipedia.org/wiki/Gottfried_Wilhelm_Leibniz">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="NM"><img src="https://robjhyndman.com/genealogy/malebranche.jpg" alt="Nicolas Malebranche"></span>
      <span class="gen-name">Nicolas Malebranche<span class="gen-meta">Oratoire de France, Paris</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="NM"><img src="https://robjhyndman.com/genealogy/malebranche.jpg" alt="Nicolas Malebranche"></span><span class="gen-name">Nicolas Malebranche<span class="gen-meta">Oratoire de France, Paris</span></span></span>
      <p>B. 1638, Paris. D. 1715, Paris.</p>
      <p>Oratorian priest and Cartesian philosopher, author of <em>De la recherche de la vérité</em> (1674). He came to mathematics late, after reading Descartes, and his Paris circle carried the new calculus into France.</p>
      <p class="gen-caveat">The MGP records his link to Leibniz to mark intellectual descent, not an actual supervision, so treat this edge as a courtesy.</p>
      <p><a href="https://en.wikipedia.org/wiki/Nicolas_Malebranche">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="JB"><img src="https://robjhyndman.com/genealogy/jabernoulli.jpg" alt="Jacob Bernoulli"></span>
      <span class="gen-name">Jacob Bernoulli<span class="gen-meta">Dr. hab. Universität Basel, 1684</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="JB"><img src="https://robjhyndman.com/genealogy/jabernoulli.jpg" alt="Jacob Bernoulli"></span><span class="gen-name">Jacob Bernoulli<span class="gen-meta">Dr. hab. Universität Basel, 1684</span></span></span>
      <p>B. 1654, Basel. D. 1705, Basel.</p>
      <p>One of the famous Bernoulli family. His contributions include the law of large numbers in probability theory and work on the convergence of infinite series.</p>
      <p><a href="https://en.wikipedia.org/wiki/Jacob_Bernoulli">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="JB"><img src="https://robjhyndman.com/genealogy/jobernoulli.jpg" alt="Johann Bernoulli"></span>
      <span class="gen-name">Johann Bernoulli<span class="gen-meta">Universität Basel, 1690, 1694</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="JB"><img src="https://robjhyndman.com/genealogy/jobernoulli.jpg" alt="Johann Bernoulli"></span><span class="gen-name">Johann Bernoulli<span class="gen-meta">Universität Basel, 1690, 1694</span></span></span>
      <p>B. 1667, Basel. D. 1748, Basel.</p>
      <p>Jacob's brother, elected a fellow of the academies of Paris, Berlin, London, St Petersburg and Bologna. Known as the "Archimedes of his age", he solved the catenary problem (alongside Leibniz and Huygens), and the rule for limits of functions commonly attributed to his student de l'Hôpital is his.</p>
      <p><a href="https://en.wikipedia.org/wiki/Johann_Bernoulli">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="LE"><img src="https://robjhyndman.com/genealogy/euler.jpg" alt="Leonhard Euler"></span>
      <span class="gen-name">Leonhard Euler<span class="gen-meta">Universität Basel, 1726</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="LE"><img src="https://robjhyndman.com/genealogy/euler.jpg" alt="Leonhard Euler"></span><span class="gen-name">Leonhard Euler<span class="gen-meta">Universität Basel, 1726</span></span></span>
      <p>B. 1707, Basel. D. 1783, St Petersburg.</p>
      <p>One of the most prolific mathematicians in history, with over 800 papers covering every branch of mathematics known in his day, an influential series of calculus textbooks, and books on a dozen other fields.</p>
      <p><a href="https://en.wikipedia.org/wiki/Leonhard_Euler">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="JL"><img src="https://robjhyndman.com/genealogy/lagrange.jpg" alt="Joseph Lagrange"></span>
      <span class="gen-name">Joseph Lagrange<span class="gen-meta">Università di Torino, 1754</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="JL"><img src="https://robjhyndman.com/genealogy/lagrange.jpg" alt="Joseph Lagrange"></span><span class="gen-name">Joseph Lagrange<span class="gen-meta">Università di Torino, 1754</span></span></span>
      <p>B. 1736, Turin. D. 1813, Paris.</p>
      <p>Developed the theory of differential equations and produced many new results in number theory. His earlier work on permutations of the roots of polynomial equations is now seen as a forerunner of group theory, and he is remembered in the Lagrange multiplier.</p>
      <p class="gen-caveat">Lagrange was never Euler's student in any formal sense. The MGP lists the relationship as "epistolary correspondence" on the calculus of variations.</p>
      <p><a href="https://en.wikipedia.org/wiki/Joseph-Louis_Lagrange">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="SP"><img src="https://robjhyndman.com/genealogy/poisson.jpg" alt="Siméon Poisson"></span>
      <span class="gen-name">Siméon Poisson<span class="gen-meta">École Polytechnique, 1800</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="SP"><img src="https://robjhyndman.com/genealogy/poisson.jpg" alt="Siméon Poisson"></span><span class="gen-name">Siméon Poisson<span class="gen-meta">École Polytechnique, 1800</span></span></span>
      <p>B. 1781, Pithiviers. D. 1840, Sceaux.</p>
      <p>"Life is only good for two things: to do mathematics and to teach it." Best known for his work on probability, including the Poisson distribution. Laplace was his second advisor.</p>
      <p><a href="https://en.wikipedia.org/wiki/Siméon_Denis_Poisson">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="MC"><img src="https://robjhyndman.com/genealogy/chasles.jpg" alt="Michel Chasles"></span>
      <span class="gen-name">Michel Chasles<span class="gen-meta">École Polytechnique, 1814</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="MC"><img src="https://robjhyndman.com/genealogy/chasles.jpg" alt="Michel Chasles"></span><span class="gen-name">Michel Chasles<span class="gen-meta">École Polytechnique, 1814</span></span></span>
      <p>B. 1793, Épernon. D. 1880, Paris.</p>
      <p>Worked on projective and enumerative geometry, and wrote a history of geometric method that shaped how the subject was taught. The Chasles relation for directed segments is his.</p>
      <p class="gen-caveat">Late in life he spent a fortune on thousands of forged letters purportedly from Pascal, Newton and Galileo, and defended their authenticity for years before the forger was tried and convicted.</p>
      <p><a href="https://en.wikipedia.org/wiki/Michel_Chasles">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="GD"><img src="https://robjhyndman.com/genealogy/darboux.jpg" alt="Gaston Darboux"></span>
      <span class="gen-name">Gaston Darboux<span class="gen-meta">Université de Paris, 1866</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="GD"><img src="https://robjhyndman.com/genealogy/darboux.jpg" alt="Gaston Darboux"></span><span class="gen-name">Gaston Darboux<span class="gen-meta">Université de Paris, 1866</span></span></span>
      <p>B. 1842, Nîmes. D. 1917, Paris.</p>
      <p>Geometer at the Sorbonne who reshaped the differential geometry of surfaces. Anyone who has taught introductory analysis knows him through Darboux sums and the Darboux integral.</p>
      <p><a href="https://en.wikipedia.org/wiki/Jean_Gaston_Darboux">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="ÉG"><img src="https://robjhyndman.com/genealogy/goursat.jpg" alt="Édouard Goursat"></span>
      <span class="gen-name">Édouard Goursat<span class="gen-meta">Université de Paris, 1881</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="ÉG"><img src="https://robjhyndman.com/genealogy/goursat.jpg" alt="Édouard Goursat"></span><span class="gen-name">Édouard Goursat<span class="gen-meta">Université de Paris, 1881</span></span></span>
      <p>B. 1858, Lanzac. D. 1936, Paris.</p>
      <p>Sharpened Cauchy's integral theorem by dropping the continuity assumption on the derivative, giving the Cauchy–Goursat theorem. His three-volume <em>Cours d'analyse mathématique</em> trained a generation of French analysts.</p>
      <p><a href="https://en.wikipedia.org/wiki/Édouard_Goursat">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="GD"><img src="https://robjhyndman.com/genealogy/darmois.png" alt="Georges Darmois"></span>
      <span class="gen-name">Georges Darmois<span class="gen-meta">Université de Paris, 1921</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="GD"><img src="https://robjhyndman.com/genealogy/darmois.png" alt="Georges Darmois"></span><span class="gen-name">Georges Darmois<span class="gen-meta">Université de Paris, 1921</span></span></span>
      <p>B. 1888, Éply. D. 1960, Paris.</p>
      <p>Started in differential geometry, then turned to statistics, joining the Institut de Statistique de l'Université de Paris (founded by Émile Borel) in 1925 and later becoming its second director. The first statistician in my line: the Darmois–Koopman–Pitman theorem characterises the exponential family, and the Darmois–Skitovich theorem characterises the normal distribution.</p>
      <p><a href="https://en.wikipedia.org/wiki/Georges_Darmois">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="DD"><img src="https://robjhyndman.com/genealogy/dugue.jpg" alt="Daniel Dugué"></span>
      <span class="gen-name">Daniel Dugué<span class="gen-meta">Université de Paris, 1937</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="DD"><img src="https://robjhyndman.com/genealogy/dugue.jpg" alt="Daniel Dugué"></span><span class="gen-name">Daniel Dugué<span class="gen-meta">Université de Paris, 1937</span></span></span>
      <p>B. 1912. D. 1987.</p>
      <p>French probabilist who worked on estimation theory and characteristic functions, and who succeeded Darmois in directing the Paris statistics institute.</p>
      <p><a href="https://en.wikipedia.org/wiki/Daniel_Dugué">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="PD"><img src="https://robjhyndman.com/genealogy/deheuvels.jpg" alt="Paul Deheuvels"></span>
      <span class="gen-name">Paul Deheuvels<span class="gen-meta">Université Paris VI, 1974</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="PD"><img src="https://robjhyndman.com/genealogy/deheuvels.jpg" alt="Paul Deheuvels"></span><span class="gen-name">Paul Deheuvels<span class="gen-meta">Université Paris VI, 1974</span></span></span>
      <p>B. 1948. D. 2026</p>
      <p>Works on empirical processes, extreme value theory and dependence. His 1979 paper on the empirical dependence function, building on related work by Rüschendorf a few years earlier, gave the empirical copula used in dependence modelling today its lasting form. Elected to the Académie des Sciences.</p>
      <p><a href="https://en.wikipedia.org/wiki/Paul_Deheuvels">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="AR"><img src="https://robjhyndman.com/genealogy/raftery.jpg" alt="Adrian Raftery"></span>
      <span class="gen-name">Adrian Raftery<span class="gen-meta">Université Paris VI, 1980</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="AR"><img src="https://robjhyndman.com/genealogy/raftery.jpg" alt="Adrian Raftery"></span><span class="gen-name">Adrian Raftery<span class="gen-meta">Université Paris VI, 1980</span></span></span>
      <p>B. 1955, Dublin.</p>
      <p>Bayesian model averaging, model-based clustering, and the probabilistic population projections the United Nations now uses. Professor of Statistics and Sociology at the University of Washington.</p>
      <p><a href="https://en.wikipedia.org/wiki/Adrian_Raftery">Wikipedia</a></p>
    </span>
  </li>

</ol>
</div>

<!-- ---------- right branch: through Peter Guttorp ---------- -->
<div class="gen-branch gen-branch--right">
<ol class="gen-chain">

  

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="OM"><img src="https://robjhyndman.com/genealogy/mencke.jpg" alt="Otto Mencke"></span>
      <span class="gen-name">Otto Mencke<span class="gen-meta">Universität Leipzig, 1665</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="OM"><img src="https://robjhyndman.com/genealogy/mencke.jpg" alt="Otto Mencke"></span><span class="gen-name">Otto Mencke<span class="gen-meta">Universität Leipzig, 1665</span></span></span>
      <p>B. 1644, Oldenburg. D. 1707, Leipzig.</p>
      <p>Professor of moral philosophy at Leipzig, and founder in 1682 of <em>Acta Eruditorum</em>, the first learned journal in Germany. Leibniz published much of his calculus there.</p>
      <p><a href="https://en.wikipedia.org/wiki/Otto_Mencke">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="JW"><img src="https://robjhyndman.com/genealogy/wichmannshausen.jpg" alt="Johann Christoph Wichmannshausen"></span>
      <span class="gen-name">J. C. Wichmannshausen<span class="gen-meta">M.A. Universität Leipzig, 1685</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="JW"><img src="https://robjhyndman.com/genealogy/wichmannshausen.jpg" alt="Johann Christoph Wichmannshausen"></span><span class="gen-name">J. C. Wichmannshausen<span class="gen-meta">M.A. Universität Leipzig, 1685</span></span></span>
      <p>B. 1663, Ilsenburg. D. 1727, Wittenberg.</p>
      <p>Orientalist and professor at Wittenberg, and Mencke's son-in-law. His dissertation was on divorce under natural law, which makes him the least mathematical of my ancestors.</p>
      <p><a href="https://en.wikipedia.org/wiki/Johann_Christoph_Wichmannshausen">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="CH"><img src="https://robjhyndman.com/genealogy/hausen.jpg" alt="Christian August Hausen"></span>
      <span class="gen-name">Christian August Hausen<span class="gen-meta">Universität Halle-Wittenberg, 1713</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="CH"><img src="https://robjhyndman.com/genealogy/hausen.jpg" alt="Christian August Hausen"></span><span class="gen-name">Christian August Hausen<span class="gen-meta">Universität Halle-Wittenberg, 1713</span></span></span>
      <p>B. 1693, Dresden. D. 1743, Leipzig.</p>
      <p>Professor of mathematics at Leipzig. He built one of the early electrostatic friction machines and studied the sparks it produced.</p>
      <p><a href="https://en.wikipedia.org/wiki/Christian_August_Hausen">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="AK"><img src="https://robjhyndman.com/genealogy/kastner.jpg" alt="Abraham Gotthelf Kästner"></span>
      <span class="gen-name">A. G. Kästner<span class="gen-meta">Universität Leipzig, 1739</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="AK"><img src="https://robjhyndman.com/genealogy/kastner.jpg" alt="Abraham Gotthelf Kästner"></span><span class="gen-name">A. G. Kästner<span class="gen-meta">Universität Leipzig, 1739</span></span></span>
      <p>B. 1719, Leipzig. D. 1800, Göttingen.</p>
      <p>Professor at Göttingen, prolific textbook writer, historian of mathematics, and a well-known writer of epigrams. Gauss studied at Göttingen during Kästner's time there, though he reportedly found Kästner's lectures too elementary to attend.</p>
      <p><a href="https://en.wikipedia.org/wiki/Abraham_Gotthelf_Kästner">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="JM"><img src="https://robjhyndman.com/genealogy/mayer.jpg" alt="Johann Tobias Mayer"></span>
      <span class="gen-name">Johann Tobias Mayer<span class="gen-meta">Universität Göttingen, 1773</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="JM"><img src="https://robjhyndman.com/genealogy/mayer.jpg" alt="Johann Tobias Mayer"></span><span class="gen-name">Johann Tobias Mayer<span class="gen-meta">Universität Göttingen, 1773</span></span></span>
      <p>B. 1752, Göttingen. D. 1830, Göttingen.</p>
      <p>Physicist and mathematician at Göttingen, and son of the astronomer Tobias Mayer. He wrote on geometry and on the design of angle-measuring instruments.</p>
      <p><a href="https://en.wikipedia.org/wiki/Johann_Tobias_Mayer">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="ED"><img src="https://robjhyndman.com/genealogy/dirksen.jpg" alt="Enno Heeren Dirksen"></span>
      <span class="gen-name">Enno Heeren Dirksen<span class="gen-meta">Universität Göttingen, 1820</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="ED"><img src="https://robjhyndman.com/genealogy/dirksen.jpg" alt="Enno Heeren Dirksen"></span><span class="gen-name">Enno Heeren Dirksen<span class="gen-meta">Universität Göttingen, 1820</span></span></span>
      <p>B. 1788, Eilsum. D. 1850, Paris.</p>
      <p>Professor in Berlin, where his students included Jacobi, Göpel and Heine. His own work was on the calculus of variations and transcendental analysis.</p>
      <p><a href="https://en.wikipedia.org/wiki/Enno_Dirksen">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="CJ"><img src="https://robjhyndman.com/genealogy/jacobi.jpg" alt="Carl Gustav Jacob Jacobi"></span>
      <span class="gen-name">Carl Gustav Jacob Jacobi<span class="gen-meta">Universität Berlin, 1825</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="CJ"><img src="https://robjhyndman.com/genealogy/jacobi.jpg" alt="Carl Gustav Jacob Jacobi"></span><span class="gen-name">Carl Gustav Jacob Jacobi<span class="gen-meta">Universität Berlin, 1825</span></span></span>
      <p>B. 1804, Potsdam. D. 1851, Berlin.</p>
      <p>Founded the theory of elliptic functions alongside Abel, and gave us the Jacobian determinant and the Hamilton–Jacobi equation. His advice to students was to invert.</p>
      <p><a href="https://en.wikipedia.org/wiki/Carl_Gustav_Jacob_Jacobi">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="WS"><img src="https://robjhyndman.com/genealogy/scheibner.jpg" alt="Wilhelm Scheibner"></span>
      <span class="gen-name">Wilhelm Scheibner<span class="gen-meta">Universität Halle-Wittenberg, 1848</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="WS"><img src="https://robjhyndman.com/genealogy/scheibner.jpg" alt="Wilhelm Scheibner"></span><span class="gen-name">Wilhelm Scheibner<span class="gen-meta">Universität Halle-Wittenberg, 1848</span></span></span>
      <p>B. 1826, Ölsnitz. D. 1908, Leipzig.</p>
      <p>Professor at Leipzig, working on analysis and number theory.</p>
      <p class="gen-caveat">Story's other Leipzig advisor was Carl Neumann, who also studied under Jacobi (through Hesse and Richelot), so both routes rejoin here.</p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="WS"><img src="https://robjhyndman.com/genealogy/Story.jpg" alt="William Story"></span>
      <span class="gen-name">William Story<span class="gen-meta">Universität Leipzig, 1875</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="WS"><img src="https://robjhyndman.com/genealogy/Story.jpg" alt="William Story"></span><span class="gen-name">William Story<span class="gen-meta">Universität Leipzig, 1875</span></span></span>
      <p>B. 1850, Boston. D. 1930, Worcester.</p>
      <p>Worked on algebraic problems and was an important player in the development of American mathematics. He helped found the <em>American Journal of Mathematics</em>.</p>
      <p><a href="https://en.wikipedia.org/wiki/William_Edward_Story">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="SL"><img src="https://robjhyndman.com/genealogy/lefschetz.jpg" alt="Solomon Lefschetz"></span>
      <span class="gen-name">Solomon Lefschetz<span class="gen-meta">Clark University, 1911</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="SL"><img src="https://robjhyndman.com/genealogy/lefschetz.jpg" alt="Solomon Lefschetz"></span><span class="gen-name">Solomon Lefschetz<span class="gen-meta">Clark University, 1911</span></span></span>
      <p>B. 1884, Moscow. D. 1972, Princeton.</p>
      <p>Russian-born mathematician who became the main source of the algebraic aspects of topology.</p>
      <p><a href="https://en.wikipedia.org/wiki/Solomon_Lefschetz">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="JT"><img src="https://robjhyndman.com/genealogy/tukey.jpg" alt="John Tukey"></span>
      <span class="gen-name">John Tukey<span class="gen-meta">Princeton University, 1939</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="JT"><img src="https://robjhyndman.com/genealogy/tukey.jpg" alt="John Tukey"></span><span class="gen-name">John Tukey<span class="gen-meta">Princeton University, 1939</span></span></span>
      <p>B. 1915, New Bedford. D. 2000, New Brunswick.</p>
      <p>The most innovative statistician of the 20th century. He invented the box plot and the stem-and-leaf plot, co-invented the fast Fourier transform, and contributed heavily to jackknife estimation and spectral density estimation. He is also credited with the words "software" and "bit".</p>
      <p><a href="https://en.wikipedia.org/wiki/John_Tukey">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="DB"><img src="https://robjhyndman.com/genealogy/brillinger.jpg" alt="David Brillinger"></span>
      <span class="gen-name">David Brillinger<span class="gen-meta">Princeton University, 1961</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="DB"><img src="https://robjhyndman.com/genealogy/brillinger.jpg" alt="David Brillinger"></span><span class="gen-name">David Brillinger<span class="gen-meta">Princeton University, 1961</span></span></span>
      <p>B. 1937, Toronto.</p>
      <p>Prolific researcher with over 200 papers, best known for his work on stochastic processes and time series, especially spectral analysis and earthquakes. His 1975 book <em>Time Series: Data Analysis and Theory</em> has been particularly influential. At the University of California, Berkeley.</p>
      <p><a href="https://en.wikipedia.org/wiki/David_R._Brillinger">Wikipedia</a></p>
    </span>
  </li>

  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="PG"><img src="https://robjhyndman.com/genealogy/guttorp.jpg" alt="Peter Guttorp"></span>
      <span class="gen-name">Peter Guttorp<span class="gen-meta">University of California, Berkeley, 1980</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="PG"><img src="https://robjhyndman.com/genealogy/guttorp.jpg" alt="Peter Guttorp"></span><span class="gen-name">Peter Guttorp<span class="gen-meta">University of California, Berkeley, 1980</span></span></span>
      <p>B. 1949.</p>
      <p>Uses stochastic models in hydrology, atmospheric science, geophysics, environmental science and haematology. At the University of Washington.</p>
      <p><a href="https://en.wikipedia.org/wiki/Peter_Guttorp">Wikipedia</a></p>
    </span>
  </li>

</ol>
</div>

</div><!-- /.gen-branches -->

<div class="gen-join gen-join--merge"><span></span><span></span></div>

<!-- ============ back to one line ============ -->
<ol class="gen-chain">
  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="GG"><img src="https://robjhyndman.com/img/GaryGrunwald.jpg" alt="Gary Grunwald"></span>
      <span class="gen-name">Gary Grunwald<span class="gen-meta">University of Washington, 1987</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="GG"><img src="https://robjhyndman.com/img/GaryGrunwald.jpg" alt="Gary Grunwald"></span><span class="gen-name">Gary Grunwald<span class="gen-meta">University of Washington, 1987</span></span></span>
      <p>B. 1954.</p>
      <p>Gary spent the first part of his career on time series analysis, particularly non-Gaussian time series. Then he worked at the University of Colorado on nutrition, physiology, and cardiovascular health. Guttorp and Raftery jointly supervised his thesis, which is why the tree above has two branches. <a href="https://coloradosph.cuanschutz.edu/resources/directory/directory-profile/Grunwald-Gary-UCD13990">Website</a></p>
    </span>
  </li>
  <li class="gen-row">
    <button type="button" class="gen-node">
      <span class="gen-fig" data-initials="RH"><img src="https://robjhyndman.com/img/population.jpg" alt="Rob J Hyndman"></span>
      <span class="gen-name">Rob J Hyndman<span class="gen-meta">University of Melbourne, 1993</span></span>
    </button>
    <span class="gen-pop">
      <span class="gen-pop-head"><span class="gen-fig" data-initials="RH"><img src="https://robjhyndman.com/img/population.jpg" alt="Rob J Hyndman"></span><span class="gen-name">Rob J Hyndman<span class="gen-meta">University of Melbourne, 1993</span></span></span>
      <p>B. 1967, Melbourne.</p>
      <p>Peter Brockwell was also my advisor, and he was supervised by Joe Moyal, but then the line stops --- the MGP does not record who Moyal's advisor was.</p>
      <p><a href="https://en.wikipedia.org/wiki/Rob_J._Hyndman">Wikipedia</a></p>
    </span>
  </li>
</ol>


</div>

<script>
document.querySelectorAll('.gen .gen-fig img').forEach(function (img) {
  img.addEventListener('error', function () { img.remove(); });
});
</script>



 ]]></description>
  <category>genealogy</category>
  <category>mathematics</category>
  <guid>https://robjhyndman.com/hyndsight/mathematical-genealogy/</guid>
  <pubDate>Wed, 29 Jul 2026 14:00:00 GMT</pubDate>
</item>
<item>
  <title>Monthly, quarterly and annual growth rates</title>
  <link>https://robjhyndman.com/hyndsight/growth_rates.html</link>
  <description><![CDATA[ 




<div>
  <div class="description">
    I received the following question by email:
  </div>
</div><p></p>


<blockquote class="blockquote">
<p>I was wondering if mathematical relationships exist between monthly, quarterly, and annual growth rates (percentage changes). For time series data …<br> 1) Is there a mathematical formula to know by how many months the Month-to-Month changes lead turns in this-month-to-the-same-month-one-year-ago changes (annual changes)?<br> 2) Is there a mathematical formula to know by how many quarters the Quarter-to-Quarter changes lead turns in this-quarter-to-the-same-quarter-one-year-ago changes (annual changes)?<br> Also, regarding simple moving averages/totals:<br> 3) Is there an exact or approximate formula to know by how many months a 3-month simple moving average (or moving total) leads a turn in a 12-month simple moving average (or moving total)?<br> As an example: Inflation is typically measured with Y/Y percentage changes in the CPI. So, it would be helpful to be able to identify when the 12-month rate of change will likely reach a top or bottom, using the 1-month or 3-months rates of change.</p>
</blockquote>
<section id="monthly-growth-vs-year-over-year-growth" class="level2">
<h2 class="anchored" data-anchor-id="monthly-growth-vs-year-over-year-growth">Monthly growth vs year-over-year growth</h2>
<p>Suppose we have monthly data <img src="https://latex.codecogs.com/png.latex?y_1,y_2,%5Cdots">. Then the monthly growth rates are given by <img src="https://latex.codecogs.com/png.latex?g_t%20=%20%5Cfrac%7By_t%20-%20y_%7Bt%20-%201%7D%7D%7By_%7Bt%20-%201%7D%7D">, and the year-over-year growth rates are given by <img src="https://latex.codecogs.com/png.latex?G_t%20=%20%5Cfrac%7By_t%20-%20y_%7Bt%20-%2012%7D%7D%7By_%7Bt%20-%2012%7D%7D">. A little algebra gives <span id="eq-product-growth"><img src="https://latex.codecogs.com/png.latex?%0A%20%201%20+%20G_t%20=%20%5Cprod_%7Bj%20=%201%7D%5E%7B12%7D%20(1%20+%20g_%7Bt%20-%20j%20+%201%7D).%0A%5Ctag%7B1%7D"></span> So the year-over-year rate is the compounded product of the last 12 monthly rates. This gives a recurrence relationship: <img src="https://latex.codecogs.com/png.latex?%0A%20%20%5Cfrac%7B1%20+%20G_t%7D%7B1%20+%20G_%7Bt%20-%201%7D%7D%20=%20%5Cfrac%7B1%20+%20g_t%7D%7B1%20+%20g_%7Bt%20-%2012%7D%7D.%0A"> Therefore <img src="https://latex.codecogs.com/png.latex?%0A%20%20G_t%20%3E%20G_%7Bt%20-%201%7D%20%5Ciff%20g_t%20%3E%20g_%7Bt%20-%2012%7D.%0A"> In other words, the year-over-year rate rises when the current monthly change exceeds the change 12 months ago, and falls when the reverse holds.</p>
<p>For example, annual inflation peaks when the monthly rates entering the 12-month window fall below the rates dropping out of it. Because year-over-year growth is effectively a 12-month trailing window of monthly growth, month-to-month growth tends to lead year-over-year growth by about half the window length: the <em>centre lag</em> of an equally-weighted <img src="https://latex.codecogs.com/png.latex?k">-period trailing window is <img src="https://latex.codecogs.com/png.latex?(k%20-%201)/2"> periods, or 5.5 months when <img src="https://latex.codecogs.com/png.latex?k%20=%2012">.</p>
<p>This centre-lag rule works best when the underlying series changes slowly within the 12-month window, as is typical of series like inflation, and is less reliable for more volatile or noisy series.</p>
</section>
<section id="quarterly-growth-vs-year-over-year-growth" class="level2">
<h2 class="anchored" data-anchor-id="quarterly-growth-vs-year-over-year-growth">Quarterly growth vs year-over-year growth</h2>
<p>The quarterly version is exactly analogous: just replace 12 by 4. Since four-quarter growth is a trailing 4-quarter window of quarterly growth, quarter-to-quarter growth leads year-over-year growth by about 1.5 quarters, under the same condition as above.</p>
</section>
<section id="log-growth-rates" class="level2">
<h2 class="anchored" data-anchor-id="log-growth-rates">Log growth rates</h2>
<p>Taking logarithms turns the multiplicative relationship above into a linear one, which is more convenient for compounding over several periods, for forecasting, and for series such as price indices, where growth is conventionally expressed in log form.</p>
<p>Define monthly log growth <img src="https://latex.codecogs.com/png.latex?%5Cell_t%20=%20%5Cln(1%20+%20g_t)"> and year-over-year log growth <img src="https://latex.codecogs.com/png.latex?L_t%20=%20%5Cln(1%20+%20G_t)">, using <img src="https://latex.codecogs.com/png.latex?g_t"> and <img src="https://latex.codecogs.com/png.latex?G_t"> from above. Taking logs of Equation&nbsp;1 turns the product into a sum: <img src="https://latex.codecogs.com/png.latex?%0A%20%20L_t%20=%20%5Csum_%7Bj%20=%200%7D%5E%7B11%7D%20%5Cell_%7Bt%20-%20j%7D.%0A"> So <img src="https://latex.codecogs.com/png.latex?L_t"> rises when <img src="https://latex.codecogs.com/png.latex?%5Cell_t%20%3E%20%5Cell_%7Bt%20-%2012%7D">: the same turning condition as before, now linear rather than multiplicative.</p>
<p>This also gives a forecasting formula: <img src="https://latex.codecogs.com/png.latex?%0A%20%20L_%7Bt%20+%20h%7D%20=%20L_t%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20+%20%5Csum_%7Bs%20=%201%7D%5E%7Bh%7D%20%5Cleft(%5Cell_%7Bt%20+%20s%7D%20-%20%5Cell_%7Bt%20+%20s%20-%2012%7D%5Cright).%0A"> So given a model for future monthly growth, you can forecast the future year-over-year rate in log terms.</p>
<p>The same idea works over any horizon, comparing the most recent 3-month block to the corresponding block a year earlier: <img src="https://latex.codecogs.com/png.latex?%0A%20%20L_t%0A%20%20-%20L_%7Bt%20-%203%7D%20=%20%5Csum_%7Bj%20=%200%7D%5E%7B2%7D%20%5Cell_%7Bt%20-%20j%7D%0A%20%20-%20%5Csum_%7Bj%20=%200%7D%5E%7B2%7D%20%5Cell_%7Bt%20-%2012%20-%20j%7D.%0A"> So the year-over-year rate has risen over the past three months if the most recent 3-month growth exceeds the same 3-month block a year earlier.</p>
<p>This is exactly how CPI inflation is usually reported: <img src="https://latex.codecogs.com/png.latex?y_t"> is the price index, <img src="https://latex.codecogs.com/png.latex?%5Cell_t"> the monthly inflation rate, and <img src="https://latex.codecogs.com/png.latex?L_t"> the standard year-over-year rate quoted in the news.</p>
</section>
<section id="month-moving-averages-vs-12-month-moving-averages" class="level2">
<h2 class="anchored" data-anchor-id="month-moving-averages-vs-12-month-moving-averages">3-month moving averages vs 12-month moving averages</h2>
<p>A <em>trailing</em> <a href="https://otexts.com/fpp3/moving-averages.html">simple moving average</a> of <img src="https://latex.codecogs.com/png.latex?%5C%7By_1,%20y_2,%20%5Cdots%5C%7D"> is defined as: <img src="https://latex.codecogs.com/png.latex?%0A%20%20%5Chat%7BT%7D_%7Bk,t%7D%20=%20%5Cfrac%7B1%7D%7Bk%7D%20%5Csum_%7Bj%20=%200%7D%5E%7Bk%20-%201%7D%20y_%7Bt%20-%20j%7D.%0A"> Then <img src="https://latex.codecogs.com/png.latex?%0A%20%20%5Chat%7BT%7D_%7Bk,t%7D%20-%20%5Chat%7BT%7D_%7Bk,t%20-%201%7D%20=%20%5Cfrac%7B1%7D%7Bk%7D%20(y_%7Bt%7D%20-%20y_%7Bt%20-%20k%7D).%0A"> So <img src="https://latex.codecogs.com/png.latex?%5Chat%7BT%7D_%7Bk,t%7D"> rises when the newest observation entering the window, <img src="https://latex.codecogs.com/png.latex?y_%7Bt%7D">, exceeds the oldest observation leaving the window, <img src="https://latex.codecogs.com/png.latex?y_%7Bt%20-%20k%7D">, and falls when the reverse holds.</p>
<p>As with the growth-rate case, the practical lead between two moving averages of different lengths can be approximated from the centre of each trailing window. A trailing <img src="https://latex.codecogs.com/png.latex?k">-period average is centred about <img src="https://latex.codecogs.com/png.latex?(k%20-%201)/2"> periods in the past, so a 3-month average is centred about 1 month in the past, while a 12-month average is centred about 5.5 months in the past. So a 3-month moving average tends to lead a 12-month moving average by about <img src="https://latex.codecogs.com/png.latex?5.5%20-%201%20=%204.5"> months, for the same reason that monthly growth leads year-over-year growth above. As before, this holds as long as the series moves slowly relative to the 12-month window.</p>


</section>

 ]]></description>
  <category>time series</category>
  <guid>https://robjhyndman.com/hyndsight/growth_rates.html</guid>
  <pubDate>Mon, 27 Jul 2026 14:00:00 GMT</pubDate>
</item>
<item>
  <title>Women in Forecasting</title>
  <link>https://robjhyndman.com/hyndsight/wif2026.html</link>
  <description><![CDATA[ 




<div>
  <div class="description">
    The second edition of the Women in Forecasting (WIF) Conference is taking place on 27 June, right before the International Symposium on Forecasting in Montreal Canada.
  </div>
</div><p></p>


<p><a href="https://wif.nixtla.io/">Women in Forecasting</a> was created to showcase the important contributions women are making to time series forecasting, a field where women remain underrepresented. Beyond the talks, the conference is also an opportunity to build community, foster professional connections, and support mentorship for those entering the field and those advancing in their careers.</p>
<p>I’m delighted to see four of my own former PhD students are <a href="https://wif.nixtla.io/#speakers">invited speakers</a> at the conference, along with Mariana Menchero who was a great help to me in solving numerous technical issues related to the <a href="https://OTexts.com/fpppy">Python edition of FPP</a>.</p>
<p><strong>Date:</strong> Saturday, June 27</p>
<p><strong>Venue:</strong> Université du Québec à Montréal (The same venue as ISF 2026)</p>
<p>Registration is required for both the in-person venue and on the Zoom.</p>
<p>Register for the WIF Conference: <a href="https://wif.nixtla.io/" class="uri">https://wif.nixtla.io/</a></p>



 ]]></description>
  <category>forecasting</category>
  <guid>https://robjhyndman.com/hyndsight/wif2026.html</guid>
  <pubDate>Sun, 14 Jun 2026 14:00:00 GMT</pubDate>
  <media:content url="https://robjhyndman.com/img/wif2026.jpg" medium="image" type="image/jpeg"/>
</item>
<item>
  <title>Forecasting: Principles and Practice, the Pythonic Way</title>
  <link>https://robjhyndman.com/hyndsight/fpppy.html</link>
  <description><![CDATA[ 




<div>
  <div class="description">
    The Python edition of “Forecasting: Principles and Practice” is now <a href="https://otexts.com/fpppy/print-version.html">available in print</a>.
  </div>
</div><p></p>


<p><a href="https://otexts.com/fpppy/print-version.html"><img src="https://otexts.com/fpppy/figs/fpppy_front_cover.jpg" alt="FPPPY front cover" style="width: 300px; float: right; margin-left: 20px; margin-bottom: 20px;"></a></p>
<p>About a year ago, we did a soft launch of the online edition of <a href="https://OTexts.com/fpppy/">“Forecasting: Principles and Practice, the Pythonic Way” (FPPPY)</a>. In this latest edition of the FPP book, George Athanasopoulos and I are joined by four additional co-authors: Azul Garza, Cristian Challu, Max Mergenthaler and Kin Olivares. <a href="https://otexts.com/fpppy/authors.html">Our new co-authors</a> are Python and forecasting experts, and have all been involved in developing the open-source <a href="https://github.com/Nixtla/">Nixtla software</a> that is used throughout the book.</p>
<p>Since the online edition was launched, we have made numerous small updates and improvements to the book, ironing out errors, improving the presentation, and simplifying the Python code where possible. Now it is <a href="https://otexts.com/fpppy/print-version.html">available in print</a> as well!</p>
<p>The first 13 chapters of the Python edition closely follow the corresponding <a href="https://otexts.com/fpp3/">R fpp3 edition</a>, but with code examples in Python rather than R. We have also included two new chapters: <a href="https://otexts.com/fpppy/14-neural-networks.html">Neural networks</a> and <a href="https://otexts.com/fpppy/15-foundation-models.html">Foundation forecasting models</a>, which are dedicated to recent techniques and developments in neural networks applied to forecasting.</p>
<p>As with previous editions, all code and examples are fully reproducible, and exercises are provided for teaching or self-study. The <a href="https://otexts.com/fpppy/">online edition</a> will remain free, and will be updated as needed. The <a href="https://otexts.com/fpppy/print-version.html">print edition</a> is available for purchase, and we hope it will be a useful resource for students, practitioners, and anyone interested in learning about forecasting with Python.</p>



 ]]></description>
  <category>forecasting</category>
  <category>Python</category>
  <category>teaching</category>
  <guid>https://robjhyndman.com/hyndsight/fpppy.html</guid>
  <pubDate>Thu, 04 Jun 2026 14:00:00 GMT</pubDate>
  <media:content url="https://otexts.com/fpppy/figs/fpppy_front_cover.jpg" medium="image" type="image/jpeg"/>
</item>
<item>
  <title>Podcasts</title>
  <link>https://robjhyndman.com/hyndsight/podcasts/</link>
  <description><![CDATA[ <p>I’ve been interviewed for several podcasts over the last few years. It’s always fun to talk about my work, and I hope there are enough differences between them to make it interesting for listeners. Here is a full list of them.</p>
<p><strong>Updated:</strong> 23 April 2026</p>
<div class="quarto-listing-container-table">
<div class="cell">
<div class="cell-output-display">
<table class="quarto-listing-table table table" style="margin-left: auto; margin-right: auto;">
<thead><tr>
<th style="text-align:left;"> Date </th>
   <th style="text-align:left;"> Podcast </th>
   <th style="text-align:left;"> Episode </th>
  </tr></thead>
<tbody>
<tr>
<td style="text-align:left;width: 25%; "> 23 April 2026 </td>
   <td style="text-align:left;width: 30%; font-weight: bold;"> Value Driven Data Science </td>
   <td style="text-align:left;width: 45%; "> <a href="https://valuedrivendatascience.com/102">How giving away your work for free can build your authority as a data scientist</a> </td>
  </tr>
<tr>
<td style="text-align:left;width: 25%; "> 16 April 2026 </td>
   <td style="text-align:left;width: 30%; font-weight: bold;"> Value Driven Data Science </td>
   <td style="text-align:left;width: 45%; "> <a href="https://valuedrivendatascience.com/101">Why traditional statistics still matters in the age of AI</a> </td>
  </tr>
<tr>
<td style="text-align:left;width: 25%; "> 26 May 2023 </td>
   <td style="text-align:left;width: 30%; font-weight: bold;"> Forecasting Impact </td>
   <td style="text-align:left;width: 45%; "> <a href="https://forecastingimpact.buzzsprout.com/1641538/12809499-forecasting-software-panel">Forecasting software panel</a> </td>
  </tr>
<tr>
<td style="text-align:left;width: 25%; "> 14 March 2022 </td>
   <td style="text-align:left;width: 30%; font-weight: bold;"> Faculty.net </td>
   <td style="text-align:left;width: 45%; "> <a href="https://faculti.net/forecasting-in-social-settings/">Forecasting in social settings</a> </td>
  </tr>
<tr>
<td style="text-align:left;width: 25%; "> 17 November 2021 </td>
   <td style="text-align:left;width: 30%; font-weight: bold;"> The Random Sample </td>
   <td style="text-align:left;width: 45%; "> <a href="https://www.therandomsample.com.au/podcast/software-as-a-first-class-research-output/">Software as a first class research output</a> </td>
  </tr>
<tr>
<td style="text-align:left;width: 25%; "> 24 May 2021 </td>
   <td style="text-align:left;width: 30%; font-weight: bold;"> Data Skeptic </td>
   <td style="text-align:left;width: 45%; "> <a href="https://podcastaddict.com/data-skeptic/episode/123511469">Forecasting principles and practice</a> </td>
  </tr>
<tr>
<td style="text-align:left;width: 25%; "> 12 April 2021 </td>
   <td style="text-align:left;width: 30%; font-weight: bold;"> Seriously Social </td>
   <td style="text-align:left;width: 45%; "> <a href="https://seriouslysocial.org.au/podcasts/forecasting-the-future-the-science-of-prediction/">Forecasting the future: the science of prediction</a> </td>
  </tr>
<tr>
<td style="text-align:left;width: 25%; "> 6 February 2021 </td>
   <td style="text-align:left;width: 30%; font-weight: bold;"> Forecasting Impact </td>
   <td style="text-align:left;width: 45%; "> <a href="https://forecastingimpact.buzzsprout.com/1641538/7708129-rob-hyndman">Rob Hyndman</a> </td>
  </tr>
<tr>
<td style="text-align:left;width: 25%; "> 19 July 2020 </td>
   <td style="text-align:left;width: 30%; font-weight: bold;"> The Curious Quant </td>
   <td style="text-align:left;width: 45%; "> <a href="https://share.transistor.fm/s/dc2dfd76">Forecasting COVID, time series, and why causality doesnt matter as much as you think</a> </td>
  </tr>
<tr>
<td style="text-align:left;width: 25%; "> 27 May 2020 </td>
   <td style="text-align:left;width: 30%; font-weight: bold;"> The Random Sample </td>
   <td style="text-align:left;width: 45%; "> <a href="https://www.therandomsample.com.au/podcast/forecasting-the-future-the-future-of-forecasting/">Forecasting the future &amp; the future of forecasting</a> </td>
  </tr>
<tr>
<td style="text-align:left;width: 25%; "> 9 October 2019 </td>
   <td style="text-align:left;width: 30%; font-weight: bold;"> Thought Capital </td>
   <td style="text-align:left;width: 45%; "> <a href="https://impact.monash.edu/podcasts/forecasts-are-always-wrong-but-we-need-them-anyway/">Forecasts are always wrong (but we need them anyway)</a> </td>
  </tr>
</tbody>
</table>
</div>
</div>
</div>
<script>
document.addEventListener("DOMContentLoaded", function () {
  document.querySelectorAll(".quarto-listing-container-table .quarto-listing-table tbody tr").forEach(function (row) {
    row.addEventListener("click", function (e) {
      if (e.target.closest("a")) return;
      const link = row.querySelector("a");
      if (link) {
        window.location.href = link.href;
      }
    });
  });
});
</script>


 ]]></description>
  <category>forecasting</category>
  <category>podcast</category>
  <guid>https://robjhyndman.com/hyndsight/podcasts/</guid>
  <pubDate>Wed, 22 Apr 2026 14:00:00 GMT</pubDate>
  <media:content url="https://robjhyndman.com/img/podcasts.png" medium="image" type="image/png" height="37" width="144"/>
</item>
<item>
  <title>Why traditional statistics still matters in the age of AI</title>
  <link>https://robjhyndman.com/hyndsight/stats_ai.html</link>
  <description><![CDATA[ 






<p>I was interviewed on this topic recently by <a href="https://www.genevievehayes.com/">Dr Genevieve Hayes</a> for her podcast <a href="https://valuedrivendatascience.com">Value Driven Data Science</a>.</p>
<p>Here’s the blurb (not written by me):</p>
<blockquote class="blockquote">
<p>Data scientists today are under pressure to adopt the latest tools - machine learning, LLMs, generative AI. But in the rush to embrace what’s new, many are leaving some of the most powerful analytical tools sitting on the shelf. Tools that handle something modern AI largely can’t: uncertainty. <br><br> In this episode, Prof.&nbsp;Rob Hyndman joins Dr.&nbsp;Genevieve Hayes to make the case for why rigorous statistical thinking remains indispensable in the age of AI, and what data scientists are giving up when they abandon it.</p>
</blockquote>
<p><a href="https://valuedrivendatascience.com/101"><strong>Check it out</strong></a> if you’re interested. Other podcasts I’ve done are <a href="../hyndsight/podcasts/index.html">listed here</a>.</p>



 ]]></description>
  <category>forecasting</category>
  <category>statistics</category>
  <category>AI</category>
  <guid>https://robjhyndman.com/hyndsight/stats_ai.html</guid>
  <pubDate>Wed, 15 Apr 2026 14:00:00 GMT</pubDate>
  <media:content url="https://robjhyndman.com/img/vdds.png" medium="image" type="image/png" height="144" width="144"/>
</item>
<item>
  <title>Rolling scaled forecast accuracy</title>
  <link>https://robjhyndman.com/hyndsight/rolling_mase.html</link>
  <description><![CDATA[ 




<div>
  <div class="description">
    When we compute a MASE or RMSSE using a rolling origin, should the scaling factor be recalculated every time?
  </div>
</div><p></p>


<p>I’ve been asked this a couple of times, so perhaps it is worth a blog post.</p>
<p>For a simple training/test split, the Mean Absolute Scaled Error (MASE) <span class="citation" data-cites="HK06">(Hyndman &amp; Koehler, 2006)</span> is defined as <img src="https://latex.codecogs.com/png.latex?%0A%5Ctext%7BMASE%7D%20=%20%5Cfrac%7B%5Cfrac%7B1%7D%7BH%7D%5Csum_%7Bt=T+1%7D%5E%7BT+H%7D%20%7Cy_t%20-%20%5Chat%7By%7D_%7Bt%7CT%7D%7C%7D%7B%5Cfrac1%7BT-m%7D%5Csum_%7Bt=m+1%7D%5ET%20%7Cy_t%20-%20y_%7Bt-m%7D%7C%7D%0A"> and the Root Mean Squared Scaled Error (RMSSE) is defined as <img src="https://latex.codecogs.com/png.latex?%0A%5Ctext%7BRMSSE%7D%20=%20%5Csqrt%7B%5Cfrac%7B%5Cfrac%7B1%7D%7BH%7D%5Csum_%7Bt=T+1%7D%5E%7BT+H%7D%20(y_t%20-%20%5Chat%7By%7D_%7Bt%7CT%7D)%5E2%7D%7B%5Cfrac%7B1%7D%7BT-m%7D%5Csum_%7Bt=m+1%7D%5ET%20(y_t%20-%20y_%7Bt-m%7D)%5E2%7D%7D.%0A"> In both cases, <img src="https://latex.codecogs.com/png.latex?m=1"> for non-seasonal data, where <img src="https://latex.codecogs.com/png.latex?m"> is the seasonal period for seasonal data, and the sum in the numerator is over the <em>test</em> set, while the sum in the denominator is over the <em>training</em> set. The notation <img src="https://latex.codecogs.com/png.latex?%5Chat%7By%7D_%7Bt%7CT%7D"> means the forecast of <img src="https://latex.codecogs.com/png.latex?y_%7Bt%7D"> given data <img src="https://latex.codecogs.com/png.latex?y_1,%5Cdots,y_T">.</p>
<p>These measures are discussed in my <a href="https://otexts.com/fpp3/accuracy.html#scaled-errors">forecasting textbook with George Athanasopoulos</a>. The denominator is a scaling factor, introduced so that you can compare MASE or RMSSE values across series of different units. For example, are the forecasts of widget sales more accurate than the forecasts of electricity demand? If all your forecasts are in the same units, then you don’t need to remove the scale, and it is simpler to just use MAE or RMSE (i.e., only the numerators of the above equations).</p>
<p>Now, the question is, when we are doing <a href="https://otexts.com/fpp3/tscv.html">time series cross-validation</a>, and computing forecast accuracy over a series of training/test sets with a rolling origin, does it make sense to compute a different scaling factor each time? For example, a cross-validated MASE for <img src="https://latex.codecogs.com/png.latex?h">-step forecasts is <img src="https://latex.codecogs.com/png.latex?%0A%5Ctext%7BMASE%7D_h%20=%20%5Cfrac%7B1%7D%7BT-I-h+1%7D%5Csum_%7Bt=I%7D%5E%7BT-h%7D%20%5Cfrac%7B%7Cy_%7Bt+h%7D%20-%20%5Chat%7By%7D_%7Bt+h%7Ct%7D%7C%7D%7B%7B%5Cfrac1%7Bt-m%7D%5Csum_%7Bs=m+1%7D%5E%7Bt%7D%20%7Cy_s%20-%20y_%7Bs-m%7D%7C%7D%7D,%0A"> where the first <img src="https://latex.codecogs.com/png.latex?I"> observations form the smallest training set, and subsequent training sets increase one observation at a time.</p>
<p>An alternative approach would be to compute the scaling factor across all available data, rather than calculate it separately for each training set. Then MASE would become <img src="https://latex.codecogs.com/png.latex?%0A%5Ctext%7BMASE%7D_h%20=%20%5Cfrac%7B%5Cfrac%7B1%7D%7BT-I-h+1%7D%5Csum_%7Bt=I%7D%5E%7BT-h%7D%20%7Cy_%7Bt+h%7D%20-%20%5Chat%7By%7D_%7Bt+h%7Ct%7D%7C%7D%7B%5Cfrac1%7BT-m%7D%5Csum_%7Bt=m+1%7D%5E%7BT%7D%20%7Cy_t%20-%20y_%7Bt-m%7D%7C%7D%0A"></p>
<p>Let’s think about the advantages of this alternative:</p>
<ol type="1">
<li>It is (slightly) faster. But the denominators are very fast to compute, so this really doesn’t make much difference.</li>
<li>It removes a source of variation from the calculation. The scaled errors will be more variable when the denominator changes with each test set, and that makes it harder to see the difference between forecasting methods.</li>
<li>It uses more data in computing the scaling factor, which reduces the uncertainty in the estimate of the accuracy measure. This is potentially important if <img src="https://latex.codecogs.com/png.latex?I%20%5Cll%20T">, so that some training sets are relatively small compared to the available data.</li>
</ol>
<p>As for disadvantages:</p>
<ol type="1">
<li>The measures are no longer true measures of forecast accuracy because the calculation potentially involves future observations. On the face of it, this seems important, but in reality it isn’t. The future observations aren’t affecting the forecasts, only the scaling factor, so there is no leakage involved.</li>
<li>One of my correspondents suggested that it changed the interpretation. I think the interpretability of these scaled measures is over-rated. They can only be interpreted as a ratio of out-of-sample accuracy to in-sample accuracy, and the two are not necessarily even over the same horizons. How the scaling factor is calculated doesn’t really affect the interpretation, because there is not much value in interpretation either way.</li>
</ol>
<p>So I suggest that computing the scaling factor across all available training data when calculating cross-validated MASE and RMSSE values is a good idea.</p>
<p>That raises the question as to why shouldn’t we use all available data in the denominator when doing a simple training/test split? In that context, the only advantage that is relevant is #3 above, and unless the test set is particularly large, it shouldn’t make much difference. Nevertheless, I can’t see any real problem in using all available data in computing the scaling factor.</p>




<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-bibliography"><h2 class="anchored quarto-appendix-heading">References</h2><div id="refs" class="references csl-bib-body hanging-indent" data-entry-spacing="0">
<div id="ref-HK06" class="csl-entry">
Hyndman, R. J., &amp; Koehler, A. B. (2006). Another look at measures of forecast accuracy. <em>International J Forecasting</em>, <em>22</em>(4), 679–688. <a href="https://robjhyndman.com/publications/another-look-at-measures-of-forecast-accuracy/">https://robjhyndman.com/publications/another-look-at-measures-of-forecast-accuracy/</a>
</div>
</div></section></div> ]]></description>
  <category>forecasting</category>
  <guid>https://robjhyndman.com/hyndsight/rolling_mase.html</guid>
  <pubDate>Mon, 19 Jan 2026 13:00:00 GMT</pubDate>
  <media:content url="https://otexts.com/fpp3/fpp_files/figure-html/cv1-1.png" medium="image" type="image/png"/>
</item>
<item>
  <title>forecast package v9</title>
  <link>https://robjhyndman.com/hyndsight/forecast9.html</link>
  <description><![CDATA[ <p>It’s been nearly nine years since the <a href="../hyndsight/forecast8/">last major release of the forecast package</a>, as we have been focusing on developing <a href="https://fable.tidyverts.org/"><code>fable</code></a> and <a href="https://tidyverts.org">related packages</a> as a tidyverse-based alternative for time series forecasting in R.</p>
<p>However, version 9.0 of the <code>forecast</code> package has now been released on CRAN. Thanks to Maximilian Muecke for helping with this release, and for modernising some of the old code.</p>
<p>Here are the main new features and changes.</p>
<section id="new-forecasting-models" class="level2"><h2 class="anchored" data-anchor-id="new-forecasting-models">New forecasting models</h2>
<p>The usual workflow with the forecast package is to fit a model to a time series, and then forecast from the model. For example</p>
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb1" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">USAccDeaths</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">ets</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">forecast</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">USAccDeaths</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">auto.arima</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">forecast</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</div>
<p>But there have always been a few functions that produce forecasts directly from a time series, without the need to explicitly fit a model first. For example,</p>
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb2" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">USAccDeaths</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">snaive</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</div>
<p>produces seasonal naive forecasts.</p>
<p>But now you can obtain the same forecasts by first fitting a model and then forecasting from it:</p>
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb3" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">USAccDeaths</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">rw_model</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>lag <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">12</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">forecast</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</div>
<p>Similarly, you can now fit and forecast from mean models, cubic spline models, theta models, and Croston’s method using the new <code>mean_model()</code>, <code>spline_model()</code>, <code>theta_model()</code>, and <code>croston_model()</code> functions. The existing <code>naive()</code>, <code>snaive()</code>, <code>rwf</code>, <code>meanf()</code>, <code>splinef()</code>, <code>thetaf()</code>, and <code>croston()</code> functions are still available for backward compatibility, but are now merely wrappers around the modelling and <code>forecast</code> functions.</p>
<p>This should make it easier to set up forecasting workflows involving many different models. For example, suppose you wanted to compute forecasts from five different models applied to the <code>USAccDeaths</code> time series.</p>
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb4" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;"><a href="https://rdrr.io/r/base/library.html">library</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;"><a href="https://purrr.tidyverse.org/">purrr</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model_fns</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/list.html">list</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">ets</span>, <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">auto.arima</span>, <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">rw_model</span>, <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">rw_model</span>, <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">mean_model</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">args</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/vector.html">vector</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"list"</span>, <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/length.html">length</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model_fns</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">args</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">[[</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">3</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">]</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">]</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/list.html">list</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>lag <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">12</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">args</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">[[</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">4</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">]</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">]</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/list.html">list</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>lag <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">12</span>, drift <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">TRUE</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">models</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://purrr.tidyverse.org/reference/map2.html">map2</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model_fns</span>, <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">args</span>, <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">~</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/do.call.html">do.call</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">.x</span>, <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/c.html">c</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/list.html">list</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>y<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">USAccDeaths</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span>, <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">.y</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/names.html">names</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">models</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://purrr.tidyverse.org/reference/map.html">map</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">models</span>, <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">u</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">u</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">method</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">forecasts</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://purrr.tidyverse.org/reference/map.html">map</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">models</span>, <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">forecast</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</div>
<p>Note that <code>rw_model</code> is included twice, with different arguments, allowing the calculation of seasonal naive, and seasonal naive with drift, forecasts.</p>
<p>Forecasting with multiple models is easy to do using the <a href="https://fable.tidyverts.org/"><code>fable</code> package</a> (and was one of the key considerations in its design). Now it is also possible to do it using the <code>forecast</code> package, with this more consistent interface.</p>
</section><section id="missing-values-in-ets-models" class="level2"><h2 class="anchored" data-anchor-id="missing-values-in-ets-models">Missing values in ETS models</h2>
<p>I wrote the first version of the ETS code (more than 25 years ago) for a consulting project where there were no missing values, so I didn’t write the code to handle them. But obviously missing values occur a lot in many applications, and I’ve now belatedly added support for them. If there are gaps in a time series, <code>ets()</code> can be applied, and it computes the likelihood and fitted values correctly using all available data. (Previously, it would fit a model to the largest contiguous segment of the data.)</p>
<p>This change has also been made to the <code>ETS()</code> function in the <a href="https://fable.tidyverts.org/"><code>fable</code> package</a>.</p>
</section><section id="bootstrapped-prediction-intervals" class="level2"><h2 class="anchored" data-anchor-id="bootstrapped-prediction-intervals">Bootstrapped prediction intervals</h2>
<p>The default prediction intervals in the forecast package are based on the (usually reasonable) assumption that forecast errors are Gaussian. Bootstrapped prediction intervals allow for non-Gaussian forecast errors. These have long been available using ETS, ARIMA, and other models; they are now also available using ARFIMA and TBATS models.</p>
</section><section id="other-changes" class="level2"><h2 class="anchored" data-anchor-id="other-changes">Other changes</h2>
<ul>
<li>Added parallelization for <code>nnetar()</code>.</li>
<li>More consistent handling of the <code>biasadj</code> argument across models.</li>
<li>More consistent interface for simulated and bootstrapped prediction intervals for all models.</li>
<li>Bug fixes and performance improvements.</li>
<li>Documentation improvements.</li>
</ul></section><section id="future-of-the-forecast-package" class="level2"><h2 class="anchored" data-anchor-id="future-of-the-forecast-package">Future of the forecast package</h2>
<p>I would still encourage people to use the <a href="https://fable.tidyverts.org/"><code>fable</code> package</a> for new projects, as it provides a more modern and consistent interface for time series forecasting in R. However, the <code>forecast</code> package will continue to be maintained, with bug fixes and minor improvements as needed.</p>


</section> ]]></description>
  <category>forecasting</category>
  <category>R</category>
  <category>time series</category>
  <guid>https://robjhyndman.com/hyndsight/forecast9.html</guid>
  <pubDate>Sun, 11 Jan 2026 13:00:00 GMT</pubDate>
  <media:content url="https://pkg.robjhyndman.com/forecast/logo.png" medium="image" type="image/png"/>
</item>
<item>
  <title>WAPE: Weighted Absolute Percentage Error</title>
  <link>https://robjhyndman.com/hyndsight/wape.html</link>
  <description><![CDATA[ 




<div>
  <div class="description">
    I was recently asked for my view on the so-called “Weighted Absolute Percentage Error” (WAPE).
  </div>
</div><p></p>


<p>The WAPE was introduced by <span class="citation" data-cites="wmape">Kolassa &amp; Schütz (2007)</span> who called it the MAD/Mean ratio. It is defined as <img src="https://latex.codecogs.com/png.latex?%0A%5Ctext%7BWAPE%7D%20=%20%5Cfrac%7B%5Csum_%7Bt%7D%20%7Cy_t%20-%20%5Chat%7By%7D_t%7C%7D%7B%5Csum_%7Bt%7D%20%7Cy_t%7C%7D%0A"> where</p>
<ul>
<li><img src="https://latex.codecogs.com/png.latex?y_t"> is the actual value at time <img src="https://latex.codecogs.com/png.latex?t">,</li>
<li><img src="https://latex.codecogs.com/png.latex?%5Chat%7By%7D_t"> is the forecast value at time <img src="https://latex.codecogs.com/png.latex?t"></li>
</ul>
<p>You can think of it as a weighted percentage error by writing it as <img src="https://latex.codecogs.com/png.latex?%0A%5Ctext%7BWAPE%7D%20=%20%5Csum_%7Bt%7D%20w_t%20%5Cfrac%7B%7Cy_t%20-%20%5Chat%7By%7D_t%7C%7D%7B%7Cy_t%7C%7D%0A"> where the weights are given by <img src="https://latex.codecogs.com/png.latex?w_t%20=%20%5Cfrac%7B%7Cy_t%7C%7D%7B%5Csum_%7Bt%7D%20%7Cy_t%7C%20%7D"></p>
<p>It can also be considered a relative MAE where the comparison method has all forecasts equal to zero. In fact, <img src="https://latex.codecogs.com/png.latex?1-%5Ctext%7BWAPE%7D"> would give the skill score relative to the forecast of zero for all <img src="https://latex.codecogs.com/png.latex?t">.</p>
<p>Or you could think of it as like a MASE <span class="citation" data-cites="HK06">(Hyndman &amp; Koehler, 2006)</span> but with scaling based on the sum of absolute values on the test set rather than the sum of absolute differences on the training set.</p>
<p>This has some obvious advantages over the MAPE:</p>
<ol type="1">
<li>The MAPE is undefined when <em>any</em> actual value in the test set is zero. The WAPE is defined even when some actuals are zero. It is only undefined when <em>all</em> the actuals used in the denominator are zero.</li>
<li>Optimising the MAPE does not lead to a sensible point forecasts <span class="citation" data-cites="gneiting2011">(Gneiting, 2011)</span>, but optimising the WAPE will lead to the median forecast.</li>
</ol>
<p>However, I think there are a couple of problems that do not seem to have been widely recognized.</p>
<ol type="1">
<li>The resulting estimate is only consistent when the time series is stationary. So it should not be used with data that has trends, or seasonality, or heteroscedasticity.</li>
<li>It is quite possible to have all actuals in the test set equal to zero, especially with intermittent demand time series and small test sets. Then the denominator is zero, and the WAPE is undefined.</li>
</ol>
<p>For these reasons, I think the Mean Absolute Scaled Error (MASE) <span class="citation" data-cites="HK06">(Hyndman &amp; Koehler, 2006)</span> is a better choice than the WAPE. The MASE is defined as <img src="https://latex.codecogs.com/png.latex?%0A%5Ctext%7BMASE%7D%20=%20%5Cfrac%7B%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7Bt%7D%20%7Cy_t%20-%20%5Chat%7By%7D_t%7C%7D%7B%5Cfrac%7B1%7D%7BT-m%7D%5Csum_%7Bt=m+1%7D%5ET%20%7Cy_t%20-%20y_%7Bt-m%7D%7C%7D%0A"> where <img src="https://latex.codecogs.com/png.latex?m=1"> for non-seasonal data, and <img src="https://latex.codecogs.com/png.latex?m"> is the seasonal period for seasonal data, and the sum in the numerator is over the <em>test</em> set (of size <img src="https://latex.codecogs.com/png.latex?n">), and the sum in the denominator is over the <em>training</em> set (of size <img src="https://latex.codecogs.com/png.latex?T">). Because the denominator is defined on the <em>training</em> data, not the <em>test</em> data, it avoids the above problems with the WAPE:</p>
<ul>
<li>It is a consistent estimator provided the series is <em>difference stationary</em>, which is a much weaker condition than stationarity.</li>
<li>The training data is also usually much longer than the test data, so it is much less likely to contain only zeros. In fact, if the training data did contain only zeros, then the obvious forecasts would all be zeros too.</li>
</ul>
<p>I don’t want to suggest there are no problems with the MASE. <span class="citation" data-cites="wmape">Kolassa &amp; Schütz (2007)</span> point out one potential drawback of MASE — when there are structural breaks or outliers in the training data. Also, like the WAPE, optimising the MASE will lead to the median forecast, which is probably not what you want, especially when you have intermittent demand time series.</p>
<p>These days, if I want a scale-free accuracy measure, I prefer the Root Mean Squared Scaled Error (RMSSE): <img src="https://latex.codecogs.com/png.latex?%0A%5Ctext%7BRMSSE%7D%20=%20%5Csqrt%7B%5Cfrac%7B%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7Bt%7D%20(y_t%20-%20%5Chat%7By%7D_t)%5E2%7D%7B%5Cfrac%7B1%7D%7BT-m%7D%5Csum_%7Bt=m+1%7D%5ET%20(y_t%20-%20y_%7Bt-m%7D)%5E2%7D%7D,%0A"> where again the sum in the numerator is over the <em>test</em> set, and the sum in the denominator is over the <em>training</em> set. This has all the advantages (and most disadvantages) of the MASE, but optimising it leads to the mean forecast rather than the median. It also aligns better with how models are estimated. Almost all models are estimated by minimising the sum of squared errors, so it makes sense to evaluate them using squared errors as well.</p>
<p>Point forecast reconciliation also works on means rather than medians, and is optimised using least squares, so it is more natural to evaluate using a squared error measure.</p>
<section id="update-on-consistency-and-stationarity" class="level2">
<h2 class="anchored" data-anchor-id="update-on-consistency-and-stationarity">Update on consistency and stationarity</h2>
<p>The comments below suggest that I need to explain the consistency issue in more detail. Imagine you are forecasting a time series that is a simple linear trend with some noise. So <img src="https://latex.codecogs.com/png.latex?y_t%20=%20a%20+%20bt%20+%20%5Cvarepsilon_t">, where <img src="https://latex.codecogs.com/png.latex?%5Cvarepsilon_t"> is Gaussian white noise with mean 0 and variance <img src="https://latex.codecogs.com/png.latex?%5Csigma%5E2">. As a good forecaster, you would fit a linear trend to this series, so your forecasts would be <img src="https://latex.codecogs.com/png.latex?%5Chat%7By%7D_t%20=%20%5Chat%7Ba%7D%20+%20%5Chat%7Bb%7Dt">. To keep it simple, let’s ignore the estimation error in this model and set <img src="https://latex.codecogs.com/png.latex?%5Chat%7Ba%7D%20=%20a"> and <img src="https://latex.codecogs.com/png.latex?%5Chat%7Bb%7D%20=%20b">.</p>
<p>Now let’s generate forecasts for this series over a test set of <img src="https://latex.codecogs.com/png.latex?n"> observations (either the next <img src="https://latex.codecogs.com/png.latex?n"> time periods, or over a cross-validated set). If you use the RMSE to evaluate these forecasts, you obtain <img src="https://latex.codecogs.com/png.latex?%0A%5Ctext%7BRMSE%7D%20=%20%5Csqrt%7B%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7Bt%7D%20(y_%7Bt%7D%20-%20%5Chat%7By%7D_%7Bt%7D)%5E2%7D%20=%20%5Csqrt%7B%5Cfrac%7B1%7D%7Bn%7D%20%5Csum_%7Bt%7D%20%5Cvarepsilon_%7Bt%7D%5E2%7D,%0A"> which is an estimate of the variance of the noise. As more observations are included in the calculation, the RMSE will converge to <img src="https://latex.codecogs.com/png.latex?%5Csigma">. That makes it “consistent” in a statistical sense. Practically, it also means that the expected RMSE does not depend on which observations you use to evaluate the forecasts — the RMSE values will be similar regardless of the test set used in the calculation. That is, if you evaluate the forecasts over the next week, the RMSE will be similar to the RMSE calculated on a cross-validated set.</p>
<p>Suppose, instead, we use the RMSSE to evaluate the forecasts. The RMSSE is equal to the RMSE divided by <img src="https://latex.codecogs.com/png.latex?%5Csqrt%7B%5Cfrac%7B1%7D%7BT-1%7D%5Csum_%7Bt=2%7D%5ET%20(y_%7Bt%7D%20-%20y_%7Bt-1%7D)%5E2%7D%20=%20%5Csqrt%7B%5Cfrac%7B1%7D%7BT-1%7D%5Csum_%7Bt=2%7D%5ET%20(b%20+%20%5Cvarepsilon_t%20-%20%5Cvarepsilon_%7Bt-1%7D)%5E2%7D"> So the RMSSE is still consistent (it converges to <img src="https://latex.codecogs.com/png.latex?%5Csigma/%5Csqrt%7Bb%5E2%20+%202%5Csigma%5E2%7D">), and the expected RMSSE does not depend on which observations you use to evaluate the forecasts.</p>
<p>Now, suppose we use the WAPE to evaluate these forecasts. Then we obtain <img src="https://latex.codecogs.com/png.latex?%0A%5Ctext%7BWAPE%7D%20=%20%5Cfrac%7B%5Csum_%7Bt%7D%20%7Cy_t%20-%20%5Chat%7By%7D_t%7C%7D%7B%5Csum_%7Bt%7D%20%7Cy_t%7C%7D%20=%20%5Cfrac%7B%5Cfrac1n%20%5Csum_%7Bt%7D%20%7C%5Cvarepsilon_t%7C%7D%7B%5Cfrac1n%20%5Csum_%7Bt%7D%20%7Ca%20+%20bt%20+%20%5Cvarepsilon_t%7C%7D,%0A"> where both sums are over the test set. As <img src="https://latex.codecogs.com/png.latex?n"> increases, the numerator will converge to the mean absolute value of the noise, approximately <img src="https://latex.codecogs.com/png.latex?0.798%20%5Csigma">. But the denominator won’t converge at all as it depends on the trend in the series. So the WAPE is not a consistent estimator of anything.</p>
<p>To see the effect of this, imagine the time series is daily sales, and you produce forecasts each week for the next 7 observations, and evaluate them using the WAPE. Over time, your WAPE values get smaller and smaller, because the denominator is growing larger and larger as the trend accumulates. So you have the illusion of improving accuracy, but your forecasts are really no more accurate than they were at the start.</p>
<p>While I have used a linear trend as the example here, similar problems can occur whenever the underlying time series is not stationary.</p>



</section>

<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-bibliography"><h2 class="anchored quarto-appendix-heading">References</h2><div id="refs" class="references csl-bib-body hanging-indent" data-entry-spacing="0">
<div id="ref-gneiting2011" class="csl-entry">
Gneiting, T. (2011). Making and evaluating point forecasts. <em>Journal of the American Statistical Association</em>, <em>106</em>(494), 746–762.
</div>
<div id="ref-HK06" class="csl-entry">
Hyndman, R. J., &amp; Koehler, A. B. (2006). Another look at measures of forecast accuracy. <em>International J Forecasting</em>, <em>22</em>(4), 679–688. <a href="https://robjhyndman.com/publications/another-look-at-measures-of-forecast-accuracy/">https://robjhyndman.com/publications/another-look-at-measures-of-forecast-accuracy/</a>
</div>
<div id="ref-wmape" class="csl-entry">
Kolassa, S., &amp; Schütz, W. (2007). Advantages of the MAD/mean ratio over the MAPE. <em>Foresight: The International Journal of Applied Forecasting</em>, <em>6</em>, 40–43.
</div>
</div></section></div> ]]></description>
  <category>forecasting</category>
  <guid>https://robjhyndman.com/hyndsight/wape.html</guid>
  <pubDate>Thu, 07 Aug 2025 14:00:00 GMT</pubDate>
  <media:content url="https://robjhyndman.com/img/wape.png" medium="image" type="image/png" height="89" width="144"/>
</item>
<item>
  <title>fpp3 package update</title>
  <link>https://robjhyndman.com/hyndsight/fpp3-update.html</link>
  <description><![CDATA[ <div>
  <div class="description">
    The <a href="https://pkg.robjhyndman.com/fpp3/"><code>fpp3</code> package</a> has had its first major update since the book was published.
  </div>
</div>
<p></p>


<p>The <a href="https://pkg.robjhyndman.com/fpp3/"><code>fpp3</code> package</a> is a companion to the book <a href="https://OTexts.com/fpp3/">Forecasting: Principles and Practice</a> (3rd edition, Hyndman &amp; Athanasopoulos, OTexts). When you load the package, it loads the data and functions needed for the examples in the book.</p>
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb1" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;"><a href="https://rdrr.io/r/base/library.html">library</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;"><a href="https://pkg.robjhyndman.com/fpp3/">fpp3</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
<div class="cell-output cell-output-stderr">
<pre><code>── Attaching packages ──────────────────────────────────────────── fpp3 1.0.0 ──</code></pre>
</div>
<div class="cell-output cell-output-stderr">
<pre><code>✔ tibble      3.2.1     ✔ tsibble     1.1.4
✔ dplyr       1.1.4     ✔ tsibbledata 0.4.1
✔ tidyr       1.3.1     ✔ feasts      0.3.2
✔ lubridate   1.9.3     ✔ fable       0.3.4
✔ ggplot2     3.5.1     ✔ fabletools  0.4.2</code></pre>
</div>
<div class="cell-output cell-output-stderr">
<pre><code>── Conflicts ───────────────────────────────────────────────── fpp3_conflicts ──
✖ lubridate::date()    masks base::date()
✖ dplyr::filter()      masks stats::filter()
✖ tsibble::intersect() masks base::intersect()
✖ tsibble::interval()  masks lubridate::interval()
✖ dplyr::lag()         masks stats::lag()
✖ tsibble::setdiff()   masks base::setdiff()
✖ tsibble::union()     masks base::union()</code></pre>
</div>
</div>
<p>This provides a shorthand way of attaching the packages needed for most time series forecasting tasks, using packages from the <a href="https://tidyverts.org">tidyverts</a>, just as the <a href="https://www.tidyverse.org/packages/"><code>tidyverse</code> package</a> does for data manipulation and visualization.</p>
<p>In the most recent update, we have added many new data sets that can be used as examples by instructors, or for users to practice their forecasting skills.</p>
<p>Each of these new data sets were previously used in an exam for the forecasting subjects taught by George Athanasopoulos and me at Monash University. Three of our exams are available online for those who are interested to see how we assess forecasting students.</p>
<ul>
<li><a href="https://af.numbat.space/exams/exam_2021.pdf">2021</a></li>
<li><a href="https://af.numbat.space/exams/exam_2022.pdf">2022</a></li>
<li><a href="https://af.numbat.space/exams/exam_2023.pdf">2023</a></li>
</ul>
<p>Each year, we take a new data set, and write the exam around how to analyse the time series. We strongly believe in assessing students on their forecasting skills with real data, rather than using artificial data, or asking technical questions that are not relevant to real-world forecasting. As a result, our exams are quite different from anything we have seen elsewhere.</p>
<p>We also think it is best to use “fresh and local”<sup>1</sup> data sets. So the data are almost always from Australia, and included data up to about a month before the exam was written.</p>
<p>In the latest update to the <a href="https://pkg.robjhyndman.com/fpp3/"><code>fpp3</code> package</a>, we have expanded these data sets to cover more series than we did in the exams, and in most cases updated them to include more recent observations.</p>
<p>The new data sets are:</p>
<ul>
<li>
<a href="https://pkg.robjhyndman.com/fpp3/reference/aus_births.html"><code>aus_births</code></a>: Australian births data</li>
<li>
<a href="https://pkg.robjhyndman.com/fpp3/reference/aus_fertility.html"><code>aus_fertility</code></a>: Australian fertility rates</li>
<li>
<a href="https://pkg.robjhyndman.com/fpp3/reference/aus_inbound.html"><code>aus_inbound</code></a>: Monthly short term (&lt;1 year) visitor arrivals to Australia</li>
<li>
<a href="https://pkg.robjhyndman.com/fpp3/reference/aus_migration.html"><code>aus_migration</code></a>: Australian migration data</li>
<li>
<a href="https://pkg.robjhyndman.com/fpp3/reference/aus_mortality.html"><code>aus_mortality</code></a>: Australian mortality data</li>
<li>
<a href="https://pkg.robjhyndman.com/fpp3/reference/aus_outbound.html"><code>aus_outbound</code></a>: Monthly short term (&lt;1 year) resident departures in Australia</li>
<li>
<a href="https://pkg.robjhyndman.com/fpp3/reference/aus_tobacco.html"><code>aus_tobacco</code></a>: Australian cigarette and tobacco expenditure</li>
<li>
<a href="https://pkg.robjhyndman.com/fpp3/reference/aus_vehicle_sales.html"><code>aus_vehicle_sales</code></a>: Australian vehicle sales</li>
<li>
<a href="https://pkg.robjhyndman.com/fpp3/reference/melb_walkers.html"><code>melb_walkers</code></a>: Average daily total pedestrian count in Melbourne</li>
<li>
<a href="https://pkg.robjhyndman.com/fpp3/reference/nsw_offences.html"><code>nsw_offences</code></a>: Monthly offences in NSW</li>
<li>
<a href="https://pkg.robjhyndman.com/fpp3/reference/ny_childcare.html"><code>ny_childcare</code></a>: New York childcare data</li>
<li>
<a href="https://pkg.robjhyndman.com/fpp3/reference/otexts_views.html"><code>otexts_views</code></a>: OTexts page views</li>
</ul>
<p>Thanks to Nuwani Palihawadana and Shanika Wickramasuriya for most of the work on this latest update, completed at the <a href="https://numbat.space/posts/hackathon_2024/">NUMBAT hackathon</a> last month.</p>
<p><img src="https://numbat.space/posts/hackathon_2024/project1.jpg" class="img-fluid"></p>


<div id="quarto-appendix" class="default"><section id="footnotes" class="footnotes footnotes-end-of-document"><h2 class="anchored quarto-appendix-heading">Footnotes</h2>
<ol>
<li id="fn1"><p>Thanks to <a href="https://dicook.org">Di Cook</a> for this phrase.↩︎</p></li>
</ol></section></div> ]]></description>
  <category>forecasting</category>
  <category>R</category>
  <category>teaching</category>
  <guid>https://robjhyndman.com/hyndsight/fpp3-update.html</guid>
  <pubDate>Tue, 25 Jun 2024 14:00:00 GMT</pubDate>
  <media:content url="https://pkg.robjhyndman.com/fpp3/logo.png" medium="image" type="image/png"/>
</item>
<item>
  <title>AIC calculations</title>
  <link>https://robjhyndman.com/hyndsight/lm_aic.html</link>
  <description><![CDATA[ <div>
  <div class="description">
    The AIC returned by <code>TSLM()</code> is different from that returned by <code><a href="https://rdrr.io/r/stats/lm.html">lm()</a></code>. Why?
  </div>
</div>
<p></p>


<p>I get this question a lot, so I thought it might help to explain some issues with AIC calculation.</p>
<p>First, the equation for the AIC is given by <img src="https://latex.codecogs.com/png.latex?%0A%5Ctext%7BAIC%7D%20=%202k%20-%202%5Clog(L),%0A"> where <img src="https://latex.codecogs.com/png.latex?L"> is the likelihood of the model and <img src="https://latex.codecogs.com/png.latex?k"> is the number of parameters that are estimated (including the error variance). For a linear regression model with iid <img src="https://latex.codecogs.com/png.latex?N(0,%5Csigma%5E2)"> errors, fitted to <img src="https://latex.codecogs.com/png.latex?n"> observations, the log-likelihood can be written as <img src="https://latex.codecogs.com/png.latex?%0A%5Clog(L)%20=%20-%5Cfrac%7Bn%7D%7B2%7D%5Clog(2%5Cpi)%20-%20%5Cfrac%7Bn%7D%7B2%7D%5Clog(%5Csigma%5E2)%20-%20%5Cfrac%7B1%7D%7B2%5Csigma%5E2%7D%5Csum_%7Bi=1%7D%5En%20%5Chat%7Be%7D_i%5E2%0A"> where <img src="https://latex.codecogs.com/png.latex?%5Chat%7Be%7D_i"> is the residual for the <img src="https://latex.codecogs.com/png.latex?i">th observation. The AIC is then <img src="https://latex.codecogs.com/png.latex?%0A%5Ctext%7BAIC%7D%20=%202k%20+%20n%5Clog(2%5Cpi)%20+%20n%5Clog(%5Csigma%5E2)%20%20+%20%5Cfrac%7B1%7D%7B%5Csigma%5E2%7D%5Csum_%7Bi=1%7D%5En%20%5Chat%7Be%7D_i%5E2.%0A"> Since we don’t know <img src="https://latex.codecogs.com/png.latex?%5Csigma%5E2">, we estimate it using the mean squared error (the maximum likelihood estimator), giving <img src="https://latex.codecogs.com/png.latex?%5Cbegin%7Balign*%7D%0A%5Ctext%7BAIC%7D%20&amp;%20=%202k%20+n%5Clog(2%5Cpi)%20+%20n%5Clog(%5Ctext%7BMSE%7D)%20+%20n%20%5C%5C%0A&amp;%20=%202k%20+%20n%5Clog(%5Ctext%7BMSE%7D)%20+%20C%0A%5Cend%7Balign*%7D"> where <img src="https://latex.codecogs.com/png.latex?C%20=%20n%20+%20n%5Clog(2%5Cpi)"> is a constant that depends only on the sample size and not on the model. This constant is often ignored. Thus, different software implementations can lead to different AIC values for the same model, since they may include or exclude the constant <img src="https://latex.codecogs.com/png.latex?C">.</p>
<p>Now, let’s look at what R returns in a simple case using the <code><a href="https://rdrr.io/r/stats/lm.html">lm()</a></code> function.</p>
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb1" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/Random.html">set.seed</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">2023</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;"><a href="https://rdrr.io/r/base/library.html">library</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;"><a href="https://pkg.robjhyndman.com/fpp3/">fpp3</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">df</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">tibble</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span></span>
<span>  time <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/seq.html">seq</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span>,</span>
<span>  x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/stats/Normal.html">rnorm</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span>,</span>
<span>  y <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">x</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/stats/Normal.html">rnorm</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/stats/lm.html">lm</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">y</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">~</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">x</span>, data <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">df</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/stats/AIC.html">AIC</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
<div class="cell-output cell-output-stdout">
<pre><code>[1] 275.6267</code></pre>
</div>
</div>
<p>We can check how this is calculated by computing it ourselves.</p>
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb3" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">mse</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/mean.html">mean</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/stats/residuals.html">residuals</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">^</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">n</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/length.html">length</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/stats/residuals.html">residuals</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">k</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/length.html">length</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">coefficients</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span></span>
<span><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># With constant</span></span>
<span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">k</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">n</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/Log.html">log</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">mse</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">n</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/Log.html">log</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">pi</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">n</span></span></code></pre></div></div>
<div class="cell-output cell-output-stdout">
<pre><code>[1] 275.6267</code></pre>
</div>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb5" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Without constant</span></span>
<span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">k</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">n</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/Log.html">log</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">mse</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
<div class="cell-output cell-output-stdout">
<pre><code>[1] -8.161047</code></pre>
</div>
</div>
<p>Clearly, <code><a href="https://rdrr.io/r/stats/AIC.html">AIC()</a></code> applied to the output from <code><a href="https://rdrr.io/r/stats/lm.html">lm()</a></code> is using the version with the constant.</p>
<p>Now compare that with what we obtain using the <code>TSLM()</code> function from the fable package.</p>
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb7" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">df</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span>  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">as_tsibble</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>index <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">time</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span>  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">model</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">TSLM</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">y</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">~</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">x</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span>  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">glance</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span>  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">pull</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">AIC</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
<div class="cell-output cell-output-stdout">
<pre><code>[1] -8.161047</code></pre>
</div>
</div>
<p>This is the AIC without the constant.</p>
<p>The situation is even more confusing with ARIMA models, and some other model classes, because some functions use approximations to the likelihood, rather than the exact likelihood.</p>
<p>Thus, AIC values can be compared across models fitted using the same functions, but not necessarily when models have been fitted using different functions.</p>



 ]]></description>
  <category>AIC</category>
  <guid>https://robjhyndman.com/hyndsight/lm_aic.html</guid>
  <pubDate>Tue, 31 Oct 2023 13:00:00 GMT</pubDate>
  <media:content url="https://robjhyndman.com/img/aic_lm.png" medium="image" type="image/png" height="69" width="144"/>
</item>
<item>
  <title>P-values for prediction intervals</title>
  <link>https://robjhyndman.com/hyndsight/forecasting-pvalues.html</link>
  <description><![CDATA[ 




<div>
  <div class="description">
    Does it make any sense to compute p-values for prediction intervals?
  </div>
</div><p></p>


<p>I received this email today:</p>
<blockquote class="blockquote">
<p>My team recently used some techniques found in your writings to perform forecasts … Our work has been well received by reviewers, but one commenter asked two questions that I was hoping you may be able to provide insight on.</p>
</blockquote>
<blockquote class="blockquote">
<p>First, they wanted to know if we could provide P-values for our prediction intervals. In our work, we said, “Observed rates were deemed significantly different from expected rates when they did not fall within the 95% PI.” This same language has been used by others published in the same journal. I am curious to hear your thoughts on giving P-values for these PIs and what the appropriate method for doing so would be (if any).</p>
</blockquote>
<blockquote class="blockquote">
<p>Second, they asked about making a correction for multiple comparisons. … I believe we could apply a Bonferroni correction to the PIs, but that feels too liberal. Moreover, I am curious if this is even called for given our statement of what is deemed significant and the fact that our prediction interval construction relies on a non-parametric method.</p>
</blockquote>
<p>Here is my reply:</p>
<ol type="1">
<li><p>I don’t think this makes any sense. A p-value is the probability of obtaining observations at least as extreme as those observed given a null hypothesis. What’s the hypothesis here? In forecasting, we don’t usually have a hypothesis. Instead, we fit a model to the data, and make predictions based on the model. I guess you could make the null hypothesis “The future observations come from the forecast distributions”, and then the p-value for each future time period would be the probability of the tails beyond the observations. But it is well-known that the estimated prediction intervals are almost always too narrow due to them not taking into account all sources of variance. So the size of this test would not be well-calibrated. I think you’re better off pushing back rather than trying to meet the request.</p></li>
<li><p>A Bonferroni correction assumes independence between the intervals, and that is not true for PIs from a forecasting model. The future forecast errors are all correlated (with the strength of the correlation depending on the model and the DGP). Usually we just say that these are pointwise PI, and so we expect 5% of observations to fall outside the 95% prediction intervals. It is possible to generate uniform PI, which contain 95% of all future sample paths, but this is a little tricky due to the correlations between horizons. It could be done via simulation – simulate a 1000 future sample paths and compute the envelope that contains 950 of them.</p></li>
</ol>
<p>It sounds like the reviewers are only familiar with inferential statistics, and not with predictive modelling. You could point them to Shmueli’s excellent <a href="https://doi.org/10.1214/10-STS330">2010 paper “To explain or predict”</a>, highlighting the differences between the two paradigms.</p>



 ]]></description>
  <category>forecasting</category>
  <category>time series</category>
  <guid>https://robjhyndman.com/hyndsight/forecasting-pvalues.html</guid>
  <pubDate>Tue, 17 Oct 2023 13:00:00 GMT</pubDate>
  <media:content url="https://robjhyndman.com/img/PI.png" medium="image" type="image/png" height="71" width="144"/>
</item>
<item>
  <title>How NASA didn’t discover the hole in the ozone layer</title>
  <link>https://robjhyndman.com/hyndsight/ozone-hole-anomaly.html</link>
  <description><![CDATA[ <div>
  <div class="description">
    I am writing a new textbook on anomaly detection. It probably won’t be finished for at least a year, but here is an excerpt.
  </div>
</div>
<p></p>


<hr>
<p>There is a widespread myth that NASA did not discover the hole in the ozone layer above the Antarctic because they had been throwing away anomalous data that would have revealed it. This is not true, but the real story is also instructive <span class="citation" data-cites="Pukelsheim1990 Christie2001 christie2004data">(Pukelsheim 1990; Christie 2001, 2004)</span>.</p>
<p>NASA had been collecting satellite data on Antarctic ozone levels using a Total Ozone Mapping Spectrometer (TOMS) since 1979, while British scientists had collected ozone data using ground sensors at the Halley Research Station, on the edge of the Brunt Ice Shelf in Antarctica, since 1957. Figure&nbsp;1 shows average daily values from the NASA measurements in blue, and from the British observations in orange. There is a clear downward trend in the British data, especially from the late 1970s, which is confirmed with the NASA data. So why wasn’t the “ozone hole” discovered until 1985?</p>
<div class="cell">
<div class="cell-output-display">
<div id="fig-halley" class="quarto-float quarto-figure quarto-figure-center anchored">
<figure class="quarto-float quarto-float-fig figure"><div aria-describedby="fig-halley-caption-0ceaefa1-69ba-4598-a22c-09a6ac19f8ca">
<img src="https://robjhyndman.com/hyndsight/ozone-hole-anomaly_files/figure-html/fig-halley-1.png" class="img-fluid figure-img" style="width:100.0%">
</div>
<figcaption class="quarto-float-caption-bottom quarto-float-caption quarto-float-fig" id="fig-halley-caption-0ceaefa1-69ba-4598-a22c-09a6ac19f8ca">
Figure&nbsp;1: Observations of Antarctic ozone levels since 1957, measured in Dobson units (DU). Observations are mean daily values from October each year. Ground observations are from Halley Research Station, while satellite observations were obtained using a Total Ozone Mapping Spectrometer (TOMS). The satellite data were obtained from <a href="https://ozonewatch.gsfc.nasa.gov/facts/history_SH.html">Leslie R Lait (NASA)</a>, while the Halley ground observations were obtained from <a href="https://www.antarctica.ac.uk/met/jds/ozone/">Jonathan Shanklin (British Antarctic Survey)</a>. The horizontal line shows the threshold of 180 DU, used by NASA to determine when the ozone level was unusually low.
</figcaption></figure>
</div>
</div>
</div>
<p>The British scientists had noticed the low ozone values as early as 1981, but it took a few years for the scientists to be convinced that the low values were real and not due to instrument problems, and then there were the usual publication delays. Eventually, the results were published in <span class="citation" data-cites="Farman1985">Farman et al. (1985)</span>.</p>
<p>Meanwhile, NASA was flagging observations as anomalous when they were below 180 DU (shown as a horizontal line in Figure&nbsp;1). As is clear from the figure, this is much lower than any of the plotted points before the early 1980s. However, the 180 threshold was used for the <em>daily</em> measurements, which are much more variable than the monthly averages that are plotted. Occasionally daily observations did fall below 180, and so it was a reasonable threshold for the purpose of identifying instrument problems.</p>
<p>In fact, NASA had checked the unusually low TOMS values obtained before 1985 by comparing them against other available data. But the other data available to them showed ozone values of about 300 DU, so it was assumed that the satellite sensor was malfunctioning. The British Halley data were not available to them, and only after the publication of <span class="citation" data-cites="Farman1985">Farman et al. (1985)</span> did the NASA scientists realise that the TOMS results were accurate.</p>
<p>In 1986, NASA scientists were able to confirm the British finding, also demonstrating that the ozone hole was widespread across the Antarctic <span class="citation" data-cites="stolarski1986nimbus">(Stolarski et al. 1986)</span>.</p>
<p>This example reveals some lessons about anomaly detection:</p>
<ul>
<li>The NASA threshold of 180 was based on daily data, and was designed to identify instrument problems, not genuine systematic changes in ozone levels. The implicit assumption was that ozone levels varied seasonally, but that otherwise the distribution of observations was stable. All anomaly detection involves some implicit assumptions like this, and it is well to be aware of them.</li>
<li>Sometimes what we think are anomalies are not really anomalies, but the result of incorrect assumptions.</li>
<li>Often smoothing or averaging data will help to reveal issues that are not so obvious from the original data. This reduces the variation in the data, and allows more systematic variation to be uncovered.</li>
<li>Always plot the data. In this case, a graph such as Figure&nbsp;1 would have revealed the problem in the late 1970s, but it seems no-one was producing plots like this.</li>
</ul>




<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-bibliography"><h2 class="anchored quarto-appendix-heading">References</h2><div id="refs" class="references csl-bib-body hanging-indent">
<div id="ref-Christie2001" class="csl-entry">
Christie, M. 2001. <em>The Ozone Layer: A Philosophy of Science Perspective</em>. Cambridge University Press.
</div>
<div id="ref-christie2004data" class="csl-entry">
Christie, M. 2004. <span>“Data Collection and the Ozone Hole: Too Much of a Good Thing?”</span> <em>History of Meteorology</em> 1: 99–105.
</div>
<div id="ref-Farman1985" class="csl-entry">
Farman, J C, B G Gardiner, and J D Shanklin. 1985. <span>“Large Losses of Total Ozone in <span>Antarctica</span> Reveal Seasonal <span>ClO<img src="https://latex.codecogs.com/png.latex?_x">/NO<img src="https://latex.codecogs.com/png.latex?_x"></span> Interaction.”</span> <em>Nature</em> 315 (6016): 207–10.
</div>
<div id="ref-Pukelsheim1990" class="csl-entry">
Pukelsheim, F. 1990. <span>“Robustness of Statistical Gossip and the <span>Antarctic</span> Ozone Hole.”</span> <em>The IMS Bulletin</em> 19 (4): 540–45.
</div>
<div id="ref-stolarski1986nimbus" class="csl-entry">
Stolarski, R S, A J Krueger, M R Schoeberl, R D McPeters, P A Newman, and J C Alpert. 1986. <span>“Nimbus 7 Satellite Measurements of the Springtime <span>Antarctic</span> Ozone Decrease.”</span> <em>Nature</em> 322 (6082): 808–11.
</div>
</div></section></div> ]]></description>
  <category>anomalies</category>
  <guid>https://robjhyndman.com/hyndsight/ozone-hole-anomaly.html</guid>
  <pubDate>Thu, 28 Sep 2023 14:00:00 GMT</pubDate>
  <media:content url="https://robjhyndman.com/img/fig-halley-1.png" medium="image" type="image/png" height="62" width="144"/>
</item>
<item>
  <title>Forecast model selection</title>
  <link>https://robjhyndman.com/hyndsight/model-selection.html</link>
  <description><![CDATA[ 




<div>
  <div class="description">
    When using a training/test split, or time-series cross-validation, are you choosing a specific model or a model class?
  </div>
</div><p></p>


<p>This question arises most time I teach a forecasting workshop, and it was raised again in the following email I received today:</p>
<blockquote class="blockquote">
<p>I have a time series that I have split into training and test datasets with an 80%-20% ratio. I fit a series of different models (ETS, BATS, ARIMA, NN etc) to the training data and generate my forecasts from each model. When evaluating the forecasts against the test set I find the model that gives the best outcome is an ARIMA(1,1,1) that was selected using the auto.arima function. My question is this, should I proceed to fit an ARIMA(1,1,1) to the whole data set, or should I use the auto.arima function again which may give me a slightly different (p,d,q) order as there is now an extra 20% of unseen data available to the forecast model? Any guidance would be greatly received.</p>
</blockquote>
<p>If you only have one class of model to consider (e.g., only ETS or only ARIMA), then it is easy enough to select the model on all available data using the AIC, and use the selected model to forecast the future. But if you are selecting between model classes, then you need to use either a training/test split, or (preferably) a <a href="https://otexts.com/fpp3/tscv.html">time-series cross-validation</a> procedure.</p>
<p>If you use time-series cross-validation, then there would usually be different models selected for each training set, and the cross-validated error is a measure of how well the model class works for your data. In that case, there is no single model for the training data, and you are selecting the <em>model class</em> rather than a specific model. This makes it clear that you should then apply the selected model class to all the data, when forecasting beyond the end of the available data. In other words, if you choose ARIMA over ETS, then you would then fit an ARIMA model to all the data, and use that model to forecast the future.</p>
<p>You can think of a simple training/test split as a special case of time-series cross-validation, where there is a single fold. So the same argument applies. That is, you are selecting the model class that works best for your data, and so you should apply that model class to all the data, when forecasting beyond the end of the available data.</p>
<p>This example also illustrates why it is important to use a time-series cross-validation procedure, rather than a simple training/test split. In this case, the ARIMA model was selected because it happened to work best for the particular training/test split that was used. But if a different split had been used, then a different model might have been selected. So the model selection is not stable. By averaging over multiple folds using a time-series cross-validation procedure, you can get a more stable estimate of the model class that works best for your data.</p>



 ]]></description>
  <category>forecasting</category>
  <category>time series</category>
  <guid>https://robjhyndman.com/hyndsight/model-selection.html</guid>
  <pubDate>Tue, 26 Sep 2023 14:00:00 GMT</pubDate>
  <media:content url="https://otexts.com/fpp3/fpp_files/figure-html/cv1-1.png" medium="image" type="image/png"/>
</item>
<item>
  <title>Videos for Forecasting: principles and practice (3rd ed)</title>
  <link>https://robjhyndman.com/hyndsight/fpp3_videos.html</link>
  <description><![CDATA[ 




<div>
  <div class="description">
    Over the past 6 months, George Athanasopoulos and I have added videos to most sections of the 3rd edition of our textbook <em>Forecasting: principles and practice</em>.
  </div>
</div><p></p>


<p>We have taught from the book many times, but this year we decided to pre-record short videos for each section. Our students often prefer a video explanation than reading the textbook, and we thought other readers might appreciate hearing from us as well.</p>
<p>These videos are embedded in most sections of the book. So far, we’ve covered the sections that we include in our own courses, but we hope to eventually have videos for all sections. Most of these were done in a single take, so they are sometimes a little rough, but hopefully still useful.</p>
<p>You can view the <a href="https://www.youtube.com/playlist?list=PLyCNZ_xXGzpm7W9jLqbIyBAiSO5jDwJeE">entire playlist on YouTube</a>.</p>



 ]]></description>
  <category>forecasting</category>
  <category>time series</category>
  <category>fpp</category>
  <category>otexts</category>
  <category>video</category>
  <guid>https://robjhyndman.com/hyndsight/fpp3_videos.html</guid>
  <pubDate>Thu, 20 Jul 2023 14:00:00 GMT</pubDate>
  <media:content url="https://robjhyndman.com/img/fpp3videos.jpg" medium="image" type="image/jpeg"/>
</item>
<item>
  <title>Degrees of freedom for a Ljung-Box test</title>
  <link>https://robjhyndman.com/hyndsight/ljung_box_df.html</link>
  <description><![CDATA[ <div>
  <div class="description">
    The Ljung-Box test is widely used to test for autocorrelation remaining in the residuals after fitting a model to a time series. In this post, I look at the degrees of freedom used in such tests.
  </div>
</div>
<p></p>

<section id="the-ljung-box-test" class="level2"><h2 class="anchored" data-anchor-id="the-ljung-box-test">The Ljung-Box test</h2>
<p>Suppose an ARMA(<img src="https://latex.codecogs.com/png.latex?p,q">) model is fitted to a time series of length <img src="https://latex.codecogs.com/png.latex?T">, giving a series of residuals <img src="https://latex.codecogs.com/png.latex?e_1,%5Cdots,e_T">, and let the autocorrelations of this residual series be denoted by <img src="https://latex.codecogs.com/png.latex?%0Ar_k%20=%20%5Csum_%7Bt=k+1%7D%5ET%20e_te_%7Bt-k%7D%20%5CBig/%20%5Csum_%7Bt=1%7D%5ET%20e_t%5E2,%20%5Cqquad%20k=1,2,%5Cdots%0A"> The first <img src="https://latex.codecogs.com/png.latex?%5Cell"> autocorrelations are used to construct the statistic <img src="https://latex.codecogs.com/png.latex?%0A%20%20Q%20=%20T(T+2)%20%5Csum_%7Bk=1%7D%5E%5Cell%20(T-k)%5E%7B-1%7Dr_k%5E2.%0A"></p>
<p>This statistic was discussed by <span class="citation" data-cites="BP70">Box and Pierce (1970)</span>, who argued that if <img src="https://latex.codecogs.com/png.latex?T"> is large, and the model parameters correspond to the true data generating process, then <img src="https://latex.codecogs.com/png.latex?Q"> has a <img src="https://latex.codecogs.com/png.latex?%5Cchi%5E2"> distribution with <img src="https://latex.codecogs.com/png.latex?%5Cell"> degrees of freedom. Later, <span class="citation" data-cites="LB78">Ljung and Box (1978)</span> showed that if the model is correct, but with unknown parameters, then <img src="https://latex.codecogs.com/png.latex?Q"> has a <img src="https://latex.codecogs.com/png.latex?%5Cchi%5E2"> distribution with <img src="https://latex.codecogs.com/png.latex?%5Cell-p-q"> degrees of freedom.</p>
</section><section id="extensions-to-other-models" class="level2"><h2 class="anchored" data-anchor-id="extensions-to-other-models">Extensions to other models</h2>
<p>These days, the Ljung-Box test is applied to a lot more models than non-seasonal ARMA models, and it is not clear what the degrees of freedom should be for other models. For example:</p>
<ul>
<li>What if the model includes an intercept term? Should that be included in the degrees of freedom calculation?</li>
<li>What about a seasonal ARIMA model? Do we just count all coefficients?</li>
<li>Or a regression with ARMA errors? Should we include the regression coefficients when computing the degrees of freedom?</li>
<li>Or an ETS model? Do we count just the smoothing parameters, or do we include the states as well, or something else?</li>
</ul>
<p>Not long ago, I had naively assumed that the correct degrees of freedom would be <img src="https://latex.codecogs.com/png.latex?%5Cell-K"> where <img src="https://latex.codecogs.com/png.latex?K"> is the number of parameters estimated. I am in good company because Andrew Harvey in <span class="citation" data-cites="harvey90">Harvey (1990, p259)</span> made exactly the same conjecture. That was what was coded in the <a href="https://pkg.robjhyndman.com/forecast/reference/checkresiduals.html"><code>forecast::checkresiduals()</code></a> function prior to v8.21, and how the test was applied in <span class="citation" data-cites="fpp2">Hyndman and Athanasopoulos (2018)</span> and <span class="citation" data-cites="fpp3">Hyndman and Athanasopoulos (2021)</span> until February 2023. But a recent <a href="https://github.com/robjhyndman/forecast/issues/908">github discussion with Achim Zeilis</a> convinced me that it is incorrect.</p>
<p>Let’s look at a few examples. For each model, we will simulate 5000 series, each of length 250 observations. For each series, we compute the p-value of a Ljung-Box test with <img src="https://latex.codecogs.com/png.latex?%5Cell=10"> and <img src="https://latex.codecogs.com/png.latex?%5Cell-K"> degrees of freedom, for different values of <img src="https://latex.codecogs.com/png.latex?K">. Under the null hypothesis of uncorrelated residuals, the <img src="https://latex.codecogs.com/png.latex?p"> values should have a uniform distribution.</p>
<div class="cell" data-hash="ljung_box_df_cache/html/setup_73aa324bb41a8896d092ef7907d377c6">
<details class="code-fold"><summary>Code</summary><div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb1" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;"><a href="https://rdrr.io/r/base/library.html">library</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;"><a href="https://pkg.robjhyndman.com/forecast/">forecast</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;"><a href="https://rdrr.io/r/base/library.html">library</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;"><a href="https://ggplot2.tidyverse.org">ggplot2</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/Random.html">set.seed</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span></span>
<span><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Function to simulate p-values given a DGP model and</span></span>
<span><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># a function to fit the model to a time series</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">simulate_pvalue</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span>, <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit_fn</span>, <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">l</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span>  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">## simulate series</span></span>
<span>  <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">if</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/NULL.html">is.null</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">xreg</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span>    <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">y</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/stats/simulate.html">simulate</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span>, n <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">250</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span>  <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">else</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span>    <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">y</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/stats/simulate.html">simulate</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span>, xreg<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">xreg</span>, n<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">250</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span>  <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span>  <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">if</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/class.html">inherits</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"ets"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span>    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># If multiplicative errors, fix non-positive values</span></span>
<span>    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">if</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">components</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">[</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">]</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">==</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"M"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span>      <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">y</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">[</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">y</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;=</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">]</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1e-5</span></span>
<span>  <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span>  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">## Fit model</span></span>
<span>  <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">m</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">fit_fn</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">y</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span>  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">## compute p-values for various df</span></span>
<span>  <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">pv</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">purrr</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://purrr.tidyverse.org/reference/map.html">map_vec</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">3</span>, m<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">m</span>,</span>
<span>    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">x</span>, <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">m</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span>      <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/stats/box.test.html">Box.test</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/stats/residuals.html">residuals</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">m</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span>, lag <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">l</span>, fitdf <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">x</span>, type <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Ljung-Box"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">p.value</span></span>
<span>    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span>  <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span>  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/names.html">names</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">pv</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/paste.html">paste</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"K ="</span>, <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">3</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span>  <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;"><a href="https://rdrr.io/r/base/function.html">return</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">pv</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Function to replicate the above function</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">simulate_pvalues</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span>, <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit_fn</span>, <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">nsim</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">5000</span>, <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">l</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span>  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">purrr</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://purrr.tidyverse.org/reference/map_dfr.html">map_dfr</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/seq.html">seq</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">nsim</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span>, <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">x</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span>    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">simulate_pvalue</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span>, <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit_fn</span>, l<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">l</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span>  <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Histograms of p values</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">hist_pvalues</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">pv</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span>  <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">pv</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span>    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">tidyr</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://tidyr.tidyverse.org/reference/pivot_longer.html">pivot_longer</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>cols <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/seq.html">seq</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/nrow.html">NCOL</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">arima_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span>    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://ggplot2.tidyverse.org/reference/ggplot.html">ggplot</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://ggplot2.tidyverse.org/reference/aes.html">aes</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">value</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span>    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://ggplot2.tidyverse.org/reference/geom_histogram.html">geom_histogram</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>bins <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">30</span>, boundary <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span>    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://ggplot2.tidyverse.org/reference/facet_grid.html">facet_grid</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">.</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">~</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">name</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span>    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://ggplot2.tidyverse.org/reference/labs.html">labs</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>title <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"P value distributions"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># A nice table of the size of the test</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">table_pvalues</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">pv</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span>  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">tibble</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://tibble.tidyverse.org/reference/tibble.html">tibble</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>`test size` <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/c.html">c</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.01</span>, <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.05</span>, <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.1</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"><a href="https://magrittr.tidyverse.org/reference/pipe.html">%&gt;%</a></span></span>
<span>    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">dplyr</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://dplyr.tidyverse.org/reference/bind_cols.html">bind_cols</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span></span>
<span>      <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">purrr</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://purrr.tidyverse.org/reference/map_dfr.html">map_df</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">pv</span>, <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">x</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/stats/ecdf.html">ecdf</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">x</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">.</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">`test size`</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span>    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span>    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">knitr</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/pkg/knitr/man/kable.html">kable</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
</details>
</div>
</section><section id="arima-models-with-an-intercept" class="level2"><h2 class="anchored" data-anchor-id="arima-models-with-an-intercept">ARIMA models with an intercept</h2>
<p>We will simulate from an ARIMA(2,0,0) model with a non-zero intercept. For the Ljung-Box test, we will consider <img src="https://latex.codecogs.com/png.latex?0%20%5Cle%20K%20%5Cle%203">. Note that <img src="https://latex.codecogs.com/png.latex?K=0"> was the original proposal by <span class="citation" data-cites="BP70">Box and Pierce (1970)</span>, <img src="https://latex.codecogs.com/png.latex?K=2=p+q"> counts only ARMA coefficients, and <img src="https://latex.codecogs.com/png.latex?K=3"> counts all parameters estimated in the model. The resulting distributions of the p-values are shown below.</p>
<div class="cell" data-hash="ljung_box_df_cache/html/ARIMA_bf5b946ec3736ef80cfade5f8f0a5bbd">
<details class="code-fold"><summary>Code</summary><div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb2" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://pkg.robjhyndman.com/forecast/reference/Arima.html">Arima</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/MathFun.html">sqrt</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">lynx</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span>, order<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/c.html">c</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>,<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>,<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit_fn</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">y</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span>  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://pkg.robjhyndman.com/forecast/reference/Arima.html">Arima</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">y</span>, order <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/c.html">c</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>, <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span>, include.mean <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">TRUE</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">arima_pvalues</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">simulate_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span>, <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit_fn</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">hist_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">arima_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</details><div class="cell-output-display">
<div>
<figure class="figure"><p><img src="https://robjhyndman.com/hyndsight/ljung_box_df_files/figure-html/ARIMA-1.png" class="img-fluid figure-img" width="768"></p>
</figure>
</div>
</div>
<details class="code-fold"><summary>Code</summary><div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb3" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">table_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">arima_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</details><div class="cell-output-display">
<table class="caption-top table table-sm table-striped small">
<thead><tr class="header">
<th style="text-align: right;">test size</th>
<th style="text-align: right;">K = 0</th>
<th style="text-align: right;">K = 1</th>
<th style="text-align: right;">K = 2</th>
<th style="text-align: right;">K = 3</th>
</tr></thead>
<tbody>
<tr class="odd">
<td style="text-align: right;">0.01</td>
<td style="text-align: right;">0.0046</td>
<td style="text-align: right;">0.0072</td>
<td style="text-align: right;">0.0124</td>
<td style="text-align: right;">0.0220</td>
</tr>
<tr class="even">
<td style="text-align: right;">0.05</td>
<td style="text-align: right;">0.0226</td>
<td style="text-align: right;">0.0354</td>
<td style="text-align: right;">0.0534</td>
<td style="text-align: right;">0.0818</td>
</tr>
<tr class="odd">
<td style="text-align: right;">0.10</td>
<td style="text-align: right;">0.0474</td>
<td style="text-align: right;">0.0678</td>
<td style="text-align: right;">0.1016</td>
<td style="text-align: right;">0.1514</td>
</tr>
</tbody>
</table>
</div>
</div>
<p>Clearly the one with <img src="https://latex.codecogs.com/png.latex?K=2"> is better than the alternatives. The table shows the empirical size of the test for different threshold levels. The empirical sizes are closest to the nominal sizes when <img src="https://latex.codecogs.com/png.latex?K=2=p+q">. So we shouldn’t count the intercept when computing the degrees of freedom.</p>
</section><section id="seasonal-arima-model" class="level2"><h2 class="anchored" data-anchor-id="seasonal-arima-model">Seasonal ARIMA model</h2>
<p>Next, we will simulate from an ARIMA(0,1,1)(0,1,1)<img src="https://latex.codecogs.com/png.latex?_%7B12%7D"> model, often called the “airline model” due to its application to the Air passenger series in <span class="citation" data-cites="BJ2016">Box et al. (2016)</span>. In fact, our DGP for the simulations will be a model fitted to the <code>AirPassengers</code> data set. Again, we consider <img src="https://latex.codecogs.com/png.latex?0%5Cle%20K%20%5Cle%203">. There are two parameters to be estimated.</p>
<div class="cell" data-hash="ljung_box_df_cache/html/SARIMA_4f343c5c334a772e904f79bb38dbff4b">
<details class="code-fold"><summary>Code</summary><div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb4" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://pkg.robjhyndman.com/forecast/reference/Arima.html">Arima</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/Log.html">log</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">AirPassengers</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span>, order<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/c.html">c</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>,<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>,<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span>, seasonal<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/c.html">c</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>,<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>,<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit_fn</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">y</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span>  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://pkg.robjhyndman.com/forecast/reference/Arima.html">Arima</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">y</span>, order <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/c.html">c</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>,<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>,<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span>, seasonal<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/c.html">c</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>,<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>,<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">sarima_pvalues</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">simulate_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span>, <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit_fn</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">hist_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">sarima_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</details><div class="cell-output-display">
<div>
<figure class="figure"><p><img src="https://robjhyndman.com/hyndsight/ljung_box_df_files/figure-html/SARIMA-1.png" class="img-fluid figure-img" width="768"></p>
</figure>
</div>
</div>
<details class="code-fold"><summary>Code</summary><div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb5" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">table_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">sarima_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</details><div class="cell-output-display">
<table class="caption-top table table-sm table-striped small">
<thead><tr class="header">
<th style="text-align: right;">test size</th>
<th style="text-align: right;">K = 0</th>
<th style="text-align: right;">K = 1</th>
<th style="text-align: right;">K = 2</th>
<th style="text-align: right;">K = 3</th>
</tr></thead>
<tbody>
<tr class="odd">
<td style="text-align: right;">0.01</td>
<td style="text-align: right;">0.0100</td>
<td style="text-align: right;">0.0170</td>
<td style="text-align: right;">0.0282</td>
<td style="text-align: right;">0.0456</td>
</tr>
<tr class="even">
<td style="text-align: right;">0.05</td>
<td style="text-align: right;">0.0484</td>
<td style="text-align: right;">0.0684</td>
<td style="text-align: right;">0.1018</td>
<td style="text-align: right;">0.1452</td>
</tr>
<tr class="odd">
<td style="text-align: right;">0.10</td>
<td style="text-align: right;">0.0882</td>
<td style="text-align: right;">0.1218</td>
<td style="text-align: right;">0.1774</td>
<td style="text-align: right;">0.2526</td>
</tr>
</tbody>
</table>
</div>
</div>
<p>Interesting. Although there are two parameters here, the tests with <img src="https://latex.codecogs.com/png.latex?K=0"> and <img src="https://latex.codecogs.com/png.latex?K=1"> do better than <img src="https://latex.codecogs.com/png.latex?K=2">. I would have expected <img src="https://latex.codecogs.com/png.latex?K=p+q+P+Q"> to be the right choice, but the test with <img src="https://latex.codecogs.com/png.latex?K=2"> has empirical size about twice the nominal size.</p>
<p>As a guess, perhaps the seasonal parameters aren’t having an effect with <img src="https://latex.codecogs.com/png.latex?%5Cell=10">. We can test what happens for larger <img src="https://latex.codecogs.com/png.latex?%5Cell"> by setting <img src="https://latex.codecogs.com/png.latex?%5Cell=24"> (covering two years), and repeating the exercise.</p>
<div class="cell" data-hash="ljung_box_df_cache/html/SARIMA2_dc188e115e90f0c6d881e8c3058e4d5f">
<details class="code-fold"><summary>Code</summary><div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb6" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">sarima_pvalues</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">simulate_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span>, <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit_fn</span>, l<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">24</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">hist_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">sarima_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</details><div class="cell-output-display">
<div>
<figure class="figure"><p><img src="https://robjhyndman.com/hyndsight/ljung_box_df_files/figure-html/SARIMA2-1.png" class="img-fluid figure-img" width="768"></p>
</figure>
</div>
</div>
<details class="code-fold"><summary>Code</summary><div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb7" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">table_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">sarima_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</details><div class="cell-output-display">
<table class="caption-top table table-sm table-striped small">
<thead><tr class="header">
<th style="text-align: right;">test size</th>
<th style="text-align: right;">K = 0</th>
<th style="text-align: right;">K = 1</th>
<th style="text-align: right;">K = 2</th>
<th style="text-align: right;">K = 3</th>
</tr></thead>
<tbody>
<tr class="odd">
<td style="text-align: right;">0.01</td>
<td style="text-align: right;">0.0134</td>
<td style="text-align: right;">0.0164</td>
<td style="text-align: right;">0.0216</td>
<td style="text-align: right;">0.0278</td>
</tr>
<tr class="even">
<td style="text-align: right;">0.05</td>
<td style="text-align: right;">0.0492</td>
<td style="text-align: right;">0.0614</td>
<td style="text-align: right;">0.0748</td>
<td style="text-align: right;">0.0940</td>
</tr>
<tr class="odd">
<td style="text-align: right;">0.10</td>
<td style="text-align: right;">0.0864</td>
<td style="text-align: right;">0.1066</td>
<td style="text-align: right;">0.1312</td>
<td style="text-align: right;">0.1590</td>
</tr>
</tbody>
</table>
</div>
</div>
<p>I was expecting <img src="https://latex.codecogs.com/png.latex?K=2"> to do best there, but not so. <img src="https://latex.codecogs.com/png.latex?K=1"> is the most uniform, and <img src="https://latex.codecogs.com/png.latex?K=0"> gives empirical sizes closest to the nominal sizes, with the results getting worse as <img src="https://latex.codecogs.com/png.latex?K"> increases. Perhaps always setting <img src="https://latex.codecogs.com/png.latex?K=p+q"> would be a sensible strategy for ARIMA models, even if they contain seasonal components. This needs some theoretical analysis.</p>
</section><section id="regression-with-arma-errors" class="level2"><h2 class="anchored" data-anchor-id="regression-with-arma-errors">Regression with ARMA errors</h2>
<p>We will simulate from a linear trend model with AR(1) errors. Here, <img src="https://latex.codecogs.com/png.latex?K=1"> counts only ARMA coefficients, while <img src="https://latex.codecogs.com/png.latex?K=3"> counts all parameters estimated. The resulting distributions of the p-values are shown below.</p>
<div class="cell" data-hash="ljung_box_df_cache/html/RegARIMA_cbaf48eec0adfe101d66336169d575ee">
<details class="code-fold"><summary>Code</summary><div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb8" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://pkg.robjhyndman.com/forecast/reference/Arima.html">Arima</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/seq.html">seq</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">250</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/stats/arima.sim.html">arima.sim</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/list.html">list</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>ar<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.7</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span>, n<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">250</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span>,</span>
<span>               order <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/c.html">c</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>,<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>,<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span>, xreg <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/seq.html">seq</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">250</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit_fn</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">y</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span>  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://pkg.robjhyndman.com/forecast/reference/Arima.html">Arima</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">y</span>, order <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/c.html">c</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>,<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>,<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span>, include.constant <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">TRUE</span>, xreg <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/seq.html">seq</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">250</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">regarima_pvalues</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">simulate_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span>, <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit_fn</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">hist_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">regarima_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</details><div class="cell-output-display">
<div>
<figure class="figure"><p><img src="https://robjhyndman.com/hyndsight/ljung_box_df_files/figure-html/RegARIMA-1.png" class="img-fluid figure-img" width="768"></p>
</figure>
</div>
</div>
<details class="code-fold"><summary>Code</summary><div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb9" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">table_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">regarima_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</details><div class="cell-output-display">
<table class="caption-top table table-sm table-striped small">
<thead><tr class="header">
<th style="text-align: right;">test size</th>
<th style="text-align: right;">K = 0</th>
<th style="text-align: right;">K = 1</th>
<th style="text-align: right;">K = 2</th>
<th style="text-align: right;">K = 3</th>
</tr></thead>
<tbody>
<tr class="odd">
<td style="text-align: right;">0.01</td>
<td style="text-align: right;">0.0072</td>
<td style="text-align: right;">0.0104</td>
<td style="text-align: right;">0.0176</td>
<td style="text-align: right;">0.0294</td>
</tr>
<tr class="even">
<td style="text-align: right;">0.05</td>
<td style="text-align: right;">0.0310</td>
<td style="text-align: right;">0.0532</td>
<td style="text-align: right;">0.0782</td>
<td style="text-align: right;">0.1200</td>
</tr>
<tr class="odd">
<td style="text-align: right;">0.10</td>
<td style="text-align: right;">0.0696</td>
<td style="text-align: right;">0.0996</td>
<td style="text-align: right;">0.1478</td>
<td style="text-align: right;">0.2150</td>
</tr>
</tbody>
</table>
</div>
</div>
<p>The test with <img src="https://latex.codecogs.com/png.latex?K=1"> looks the most uniform, with the size of the test closest to the nominal values. So only counting ARMA coefficients seems to be correct here.</p>
</section><section id="regression-model" class="level2"><h2 class="anchored" data-anchor-id="regression-model">Regression model</h2>
<p>Next, we will consider a linear trend model with iid errors. That is the same as the previous model, but with a simpler error structure.</p>
<div class="cell" data-hash="ljung_box_df_cache/html/trend_e646b62c4f784041c6d6ca8ffbfb8c90">
<details class="code-fold"><summary>Code</summary><div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb10" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://pkg.robjhyndman.com/forecast/reference/Arima.html">Arima</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/seq.html">seq</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">250</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/stats/Normal.html">rnorm</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">250</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span>,</span>
<span>               order <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/c.html">c</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>,<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>,<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span>, xreg <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/seq.html">seq</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">250</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit_fn</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">y</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span>  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://pkg.robjhyndman.com/forecast/reference/Arima.html">Arima</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">y</span>, include.constant <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">TRUE</span>, xreg <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/seq.html">seq</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">250</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">trend_pvalues</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">simulate_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span>, <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit_fn</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">hist_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">trend_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</details><div class="cell-output-display">
<div>
<figure class="figure"><p><img src="https://robjhyndman.com/hyndsight/ljung_box_df_files/figure-html/trend-1.png" class="img-fluid figure-img" width="768"></p>
</figure>
</div>
</div>
<details class="code-fold"><summary>Code</summary><div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb11" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">table_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">trend_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</details><div class="cell-output-display">
<table class="caption-top table table-sm table-striped small">
<thead><tr class="header">
<th style="text-align: right;">test size</th>
<th style="text-align: right;">K = 0</th>
<th style="text-align: right;">K = 1</th>
<th style="text-align: right;">K = 2</th>
<th style="text-align: right;">K = 3</th>
</tr></thead>
<tbody>
<tr class="odd">
<td style="text-align: right;">0.01</td>
<td style="text-align: right;">0.0148</td>
<td style="text-align: right;">0.0214</td>
<td style="text-align: right;">0.0338</td>
<td style="text-align: right;">0.0554</td>
</tr>
<tr class="even">
<td style="text-align: right;">0.05</td>
<td style="text-align: right;">0.0580</td>
<td style="text-align: right;">0.0844</td>
<td style="text-align: right;">0.1204</td>
<td style="text-align: right;">0.1746</td>
</tr>
<tr class="odd">
<td style="text-align: right;">0.10</td>
<td style="text-align: right;">0.1054</td>
<td style="text-align: right;">0.1478</td>
<td style="text-align: right;">0.2062</td>
<td style="text-align: right;">0.2816</td>
</tr>
</tbody>
</table>
</div>
</div>
<p>The test with <img src="https://latex.codecogs.com/png.latex?K=0"> looks best. If we think of a regression model as a RegARIMA model with ARIMA(0,0,0) errors, this is consistent with the previous results, setting <img src="https://latex.codecogs.com/png.latex?K=p+q">.</p>
</section><section id="etsann-model" class="level2"><h2 class="anchored" data-anchor-id="etsann-model">ETS(A,N,N) model</h2>
<p>Now let’s try an ETS(A,N,N) model, again using 5000 series each of length 250. If we count only the smoothing parameter, <img src="https://latex.codecogs.com/png.latex?K=1">, but if we count all estimated parameters, <img src="https://latex.codecogs.com/png.latex?K=2">. The distributions of p-values are shown below.</p>
<div class="cell" data-hash="ljung_box_df_cache/html/ETSANN_9f1c0d5c8fff6f356206cab28c8540d1">
<details class="code-fold"><summary>Code</summary><div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb12" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://pkg.robjhyndman.com/forecast/reference/ets.html">ets</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">fma</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">::</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;"><a href="https://pkg.robjhyndman.com/fma/reference/strikes.html">strikes</a></span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"ANN"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit_fn</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">y</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span>  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://pkg.robjhyndman.com/forecast/reference/ets.html">ets</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">y</span>, model <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"ANN"</span>, damped <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">FALSE</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">ets_pvalues</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">simulate_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span>, <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit_fn</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">hist_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">ets_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</details><div class="cell-output-display">
<div>
<figure class="figure"><p><img src="https://robjhyndman.com/hyndsight/ljung_box_df_files/figure-html/ETSANN-1.png" class="img-fluid figure-img" width="768"></p>
</figure>
</div>
</div>
<details class="code-fold"><summary>Code</summary><div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb13" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">table_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">ets_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</details><div class="cell-output-display">
<table class="caption-top table table-sm table-striped small">
<thead><tr class="header">
<th style="text-align: right;">test size</th>
<th style="text-align: right;">K = 0</th>
<th style="text-align: right;">K = 1</th>
<th style="text-align: right;">K = 2</th>
<th style="text-align: right;">K = 3</th>
</tr></thead>
<tbody>
<tr class="odd">
<td style="text-align: right;">0.01</td>
<td style="text-align: right;">0.0064</td>
<td style="text-align: right;">0.0112</td>
<td style="text-align: right;">0.0174</td>
<td style="text-align: right;">0.0316</td>
</tr>
<tr class="even">
<td style="text-align: right;">0.05</td>
<td style="text-align: right;">0.0334</td>
<td style="text-align: right;">0.0522</td>
<td style="text-align: right;">0.0778</td>
<td style="text-align: right;">0.1196</td>
</tr>
<tr class="odd">
<td style="text-align: right;">0.10</td>
<td style="text-align: right;">0.0702</td>
<td style="text-align: right;">0.0960</td>
<td style="text-align: right;">0.1442</td>
<td style="text-align: right;">0.2098</td>
</tr>
</tbody>
</table>
</div>
</div>
<p><img src="https://latex.codecogs.com/png.latex?K=1"> looks about right. That makes sense as an ETS(A,N,N) model is equivalent to an ARIMA(0,1,1) model.</p>
</section><section id="etsmnn-model" class="level2"><h2 class="anchored" data-anchor-id="etsmnn-model">ETS(M,N,N) model</h2>
<p>Next, let’s try an ETS(M,N,N) model, which has no ARIMA equivalent, but which has one smoothing parameter and one initial state to estimate.</p>
<div class="cell" data-hash="ljung_box_df_cache/html/ETSMNN_f5812bbdc26dd90d28e49ba2533a74d9">
<details class="code-fold"><summary>Code</summary><div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb14" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://pkg.robjhyndman.com/forecast/reference/ets.html">ets</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">fma</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">::</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;"><a href="https://pkg.robjhyndman.com/fma/reference/strikes.html">strikes</a></span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"MNN"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit_fn</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">y</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span>  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://pkg.robjhyndman.com/forecast/reference/ets.html">ets</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">y</span>, model <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"MNN"</span>, damped <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">FALSE</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">ets_pvalues</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">simulate_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span>, <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit_fn</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">hist_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">ets_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</details><div class="cell-output-display">
<div>
<figure class="figure"><p><img src="https://robjhyndman.com/hyndsight/ljung_box_df_files/figure-html/ETSMNN-1.png" class="img-fluid figure-img" width="768"></p>
</figure>
</div>
</div>
<details class="code-fold"><summary>Code</summary><div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb15" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">table_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">ets_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</details><div class="cell-output-display">
<table class="caption-top table table-sm table-striped small">
<thead><tr class="header">
<th style="text-align: right;">test size</th>
<th style="text-align: right;">K = 0</th>
<th style="text-align: right;">K = 1</th>
<th style="text-align: right;">K = 2</th>
<th style="text-align: right;">K = 3</th>
</tr></thead>
<tbody>
<tr class="odd">
<td style="text-align: right;">0.01</td>
<td style="text-align: right;">0.0054</td>
<td style="text-align: right;">0.0084</td>
<td style="text-align: right;">0.0172</td>
<td style="text-align: right;">0.0308</td>
</tr>
<tr class="even">
<td style="text-align: right;">0.05</td>
<td style="text-align: right;">0.0334</td>
<td style="text-align: right;">0.0548</td>
<td style="text-align: right;">0.0814</td>
<td style="text-align: right;">0.1274</td>
</tr>
<tr class="odd">
<td style="text-align: right;">0.10</td>
<td style="text-align: right;">0.0708</td>
<td style="text-align: right;">0.1062</td>
<td style="text-align: right;">0.1520</td>
<td style="text-align: right;">0.2136</td>
</tr>
</tbody>
</table>
</div>
</div>
<p>Again, <img src="https://latex.codecogs.com/png.latex?K=1"> appears to be the best.</p>
</section><section id="etsaan-model" class="level2"><h2 class="anchored" data-anchor-id="etsaan-model">ETS(A,A,N) model</h2>
<p>An ETS(A,A,N) model is equivalent to an ARIMA(0,2,2) model, so I expect this one to need <img src="https://latex.codecogs.com/png.latex?K=2">.</p>
<div class="cell" data-hash="ljung_box_df_cache/html/ETSAAN_ce0bcb2507c97eab13f6cc5c746220c5">
<details class="code-fold"><summary>Code</summary><div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb16" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://pkg.robjhyndman.com/forecast/reference/ets.html">ets</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">fma</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">::</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;"><a href="https://pkg.robjhyndman.com/fma/reference/strikes.html">strikes</a></span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"AAN"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit_fn</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">y</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span>  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://pkg.robjhyndman.com/forecast/reference/ets.html">ets</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">y</span>, model <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"AAN"</span>, damped <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">FALSE</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">ets_pvalues</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">simulate_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span>, <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit_fn</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">hist_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">ets_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</details><div class="cell-output-display">
<div>
<figure class="figure"><p><img src="https://robjhyndman.com/hyndsight/ljung_box_df_files/figure-html/ETSAAN-1.png" class="img-fluid figure-img" width="768"></p>
</figure>
</div>
</div>
<details class="code-fold"><summary>Code</summary><div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb17" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">table_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">ets_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</details><div class="cell-output-display">
<table class="caption-top table table-sm table-striped small">
<thead><tr class="header">
<th style="text-align: right;">test size</th>
<th style="text-align: right;">K = 0</th>
<th style="text-align: right;">K = 1</th>
<th style="text-align: right;">K = 2</th>
<th style="text-align: right;">K = 3</th>
</tr></thead>
<tbody>
<tr class="odd">
<td style="text-align: right;">0.01</td>
<td style="text-align: right;">0.0034</td>
<td style="text-align: right;">0.0062</td>
<td style="text-align: right;">0.0106</td>
<td style="text-align: right;">0.0172</td>
</tr>
<tr class="even">
<td style="text-align: right;">0.05</td>
<td style="text-align: right;">0.0180</td>
<td style="text-align: right;">0.0308</td>
<td style="text-align: right;">0.0514</td>
<td style="text-align: right;">0.0802</td>
</tr>
<tr class="odd">
<td style="text-align: right;">0.10</td>
<td style="text-align: right;">0.0430</td>
<td style="text-align: right;">0.0658</td>
<td style="text-align: right;">0.1008</td>
<td style="text-align: right;">0.1486</td>
</tr>
</tbody>
</table>
</div>
</div>
<p>This time, my conjecture is correct, and <img src="https://latex.codecogs.com/png.latex?K=2"> works well.</p>
</section><section id="etsaaa-model" class="level2"><h2 class="anchored" data-anchor-id="etsaaa-model">ETS(A,A,A) model</h2>
<p>Finally, we will check a seasonal ETS model</p>
<div class="cell" data-hash="ljung_box_df_cache/html/ETSAAA_1385d6238e693baceddf0a02ce3fa351">
<details class="code-fold"><summary>Code</summary><div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb18" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://pkg.robjhyndman.com/forecast/reference/ets.html">ets</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://rdrr.io/r/base/Log.html">log</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">AirPassengers</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span>, model<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"AAA"</span>, damped<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">FALSE</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit_fn</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">y</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span>  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;"><a href="https://pkg.robjhyndman.com/forecast/reference/ets.html">ets</a></span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">y</span>, model <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"AAA"</span>, damped <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">FALSE</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">ets_pvalues</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">simulate_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">model</span>, <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">fit_fn</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">hist_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">ets_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</details><div class="cell-output-display">
<div>
<figure class="figure"><p><img src="https://robjhyndman.com/hyndsight/ljung_box_df_files/figure-html/ETSAAA-1.png" class="img-fluid figure-img" width="768"></p>
</figure>
</div>
</div>
<details class="code-fold"><summary>Code</summary><div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb19" style="background: #f1f3f5;"><pre class="downlit sourceCode r code-with-copy"><code class="sourceCode R"><span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">table_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">ets_pvalues</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</details><div class="cell-output-display">
<table class="caption-top table table-sm table-striped small">
<thead><tr class="header">
<th style="text-align: right;">test size</th>
<th style="text-align: right;">K = 0</th>
<th style="text-align: right;">K = 1</th>
<th style="text-align: right;">K = 2</th>
<th style="text-align: right;">K = 3</th>
</tr></thead>
<tbody>
<tr class="odd">
<td style="text-align: right;">0.01</td>
<td style="text-align: right;">0.0178</td>
<td style="text-align: right;">0.0276</td>
<td style="text-align: right;">0.0396</td>
<td style="text-align: right;">0.0602</td>
</tr>
<tr class="even">
<td style="text-align: right;">0.05</td>
<td style="text-align: right;">0.0634</td>
<td style="text-align: right;">0.0916</td>
<td style="text-align: right;">0.1360</td>
<td style="text-align: right;">0.1942</td>
</tr>
<tr class="odd">
<td style="text-align: right;">0.10</td>
<td style="text-align: right;">0.1206</td>
<td style="text-align: right;">0.1672</td>
<td style="text-align: right;">0.2244</td>
<td style="text-align: right;">0.3078</td>
</tr>
</tbody>
</table>
</div>
</div>
<p>Here there are 3 smoothing parameters, and 13 initial states to estimate. So I was expecting <img src="https://latex.codecogs.com/png.latex?K=3"> to do best, but it is the worst. Instead, <img src="https://latex.codecogs.com/png.latex?K=0"> is the best. I’m not sure what to make of this result.</p>
</section><section id="conclusions" class="level2"><h2 class="anchored" data-anchor-id="conclusions">Conclusions</h2>
<p>Based only on this empirical evidence:</p>
<ul>
<li>For ARIMA models, use <img src="https://latex.codecogs.com/png.latex?%5Cell-p-q"> degrees of freedom.</li>
<li>For seasonal ARIMA models, it appears that <img src="https://latex.codecogs.com/png.latex?%5Cell-p-q"> also gives the best results.</li>
<li>For regression with ARIMA errors, use <img src="https://latex.codecogs.com/png.latex?%5Cell-p-q"> degrees of freedom.</li>
<li>For OLS regression, use <img src="https://latex.codecogs.com/png.latex?%5Cell"> degrees of freedom.</li>
<li>For non-seasonal ETS models, use <img src="https://latex.codecogs.com/png.latex?K="> the number of smoothing parameters.</li>
<li>For seasonal ETS models, use <img src="https://latex.codecogs.com/png.latex?K=0">.</li>
</ul>
<p>The last two of these appear to be contradictory, and it is not clear why.</p>
<p>It seems like this might be a good project for a PhD student to explore. In particular, can these suggestions based on empirical evidence be supported theoretically? It would also be good to explore other models such as TBATS, ARFIMA, NNETAR, etc.</p>
<p>For now, I might avoid teaching the Ljung-Box test, and just get students to look at the ACF plot of the residuals instead.</p>
</section><section id="other-literature" class="level2"><h2 class="anchored" data-anchor-id="other-literature">Other literature</h2>
<ul>
<li>
<span class="citation" data-cites="Kim2004">Kim et al. (2004)</span> shows that <img src="https://latex.codecogs.com/png.latex?Q%20%5Csim%20%5Cchi%5E2_%5Cell"> for an AR(1) model with ARCH errors.</li>
<li>
<span class="citation" data-cites="McLeod1983">McLeod and Li (1983)</span> consider the equivalent test applied to autocorrelations of squared residuals, and show that <img src="https://latex.codecogs.com/png.latex?Q%5E*%20%5Csim%20%5Cchi%5E2_%5Cell">.</li>
<li>
<span class="citation" data-cites="Mahdi2016">Mahdi (2016)</span> discusses a variation on the LB test for seasonal ARIMA models considering only autocorrelations at the seasonal lags.</li>
<li>Several other portmanteau tests (i.e., based on multiple autocorrelations) are available, and perhaps we should be using them and not the older Ljung-Box test. See <span class="citation" data-cites="Mahdi2021">Mahdi (2021)</span> for some recent developments.</li>
</ul></section><section id="references" class="level2">



</section><div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-bibliography"><h2 class="anchored quarto-appendix-heading">References</h2><div id="refs" class="references csl-bib-body hanging-indent">
<div id="ref-BJ2016" class="csl-entry">
Box, George E P, Gwilym M Jenkins, Gregory C Reinsel, and Greta M Ljung. 2016. <em>Time Series Analysis: Forecasting and Control</em>. 5th ed. John Wiley; Sons.
</div>
<div id="ref-BP70" class="csl-entry">
Box, George E P, and David A Pierce. 1970. <span>“Distribution of Residual Autocorrelations in Autoregressive-Integrated Moving Average Time Series Models.”</span> <em>Journal of the American Statistical Association</em> 65 (332): 1509–26. <a href="https://doi.org/10.2307/2284333">https://doi.org/10.2307/2284333</a>.
</div>
<div id="ref-harvey90" class="csl-entry">
Harvey, Andrew C. 1990. <em>Forecasting, Structural Time Series Models and the Kalman Filter</em>. Cambridge University Press.
</div>
<div id="ref-fpp2" class="csl-entry">
Hyndman, Rob J, and George Athanasopoulos. 2018. <em>Forecasting: Principles and Practice</em>. 2nd ed. OTexts. <a href="https://OTexts.org/fpp2">OTexts.org/fpp2</a>.
</div>
<div id="ref-fpp3" class="csl-entry">
Hyndman, Rob J, and George Athanasopoulos. 2021. <em>Forecasting: Principles and Practice</em>. 3rd ed. OTexts. <a href="https://OTexts.org/fpp3">OTexts.org/fpp3</a>.
</div>
<div id="ref-Kim2004" class="csl-entry">
Kim, Eunhee, Jeongcheol Ha, Youngsook Jeon, and Sangyeol Lee. 2004. <span>“Ljung-Box Test in Unit Root <span>AR-ARCH</span> Model.”</span> <em>Communications for Statistical Applications and Methods</em> 11 (2): 323–27. <a href="https://doi.org/10.5351/ckss.2004.11.2.323">https://doi.org/10.5351/ckss.2004.11.2.323</a>.
</div>
<div id="ref-LB78" class="csl-entry">
Ljung, Greta M, and George E P Box. 1978. <span>“On a Measure of Lack of Fit in Time Series Models.”</span> <em>Biometrika</em> 65 (2): 297–303. <a href="https://doi.org/10.1093/biomet/65.2.297">https://doi.org/10.1093/biomet/65.2.297</a>.
</div>
<div id="ref-Mahdi2016" class="csl-entry">
Mahdi, Esam. 2016. <span>“Portmanteau Test Statistics for Seasonal Serial Correlation in Time Series Models.”</span> <em>SpringerPlus</em> 5 (1): 1485. <a href="https://doi.org/10.1186/s40064-016-3167-4">https://doi.org/10.1186/s40064-016-3167-4</a>.
</div>
<div id="ref-Mahdi2021" class="csl-entry">
Mahdi, Esam. 2021. <span>“New Goodness-of-Fit Tests for Time Series Models.”</span> <a href="http://arxiv.org/abs/2008.08176">http://arxiv.org/abs/2008.08176</a>.
</div>
<div id="ref-McLeod1983" class="csl-entry">
McLeod, A I, and W K Li. 1983. <span>“Diagnostic Checking <span>ARMA</span> Time Series Models Using Squared-Residual Autocorrelations.”</span> <em>Journal of Time Series Analysis</em> 4 (4): 269–73. <a href="https://doi.org/10.1111/j.1467-9892.1983.tb00373.x">https://doi.org/10.1111/j.1467-9892.1983.tb00373.x</a>.
</div>
</div></section></div> ]]></description>
  <category>forecasting</category>
  <category>time series</category>
  <guid>https://robjhyndman.com/hyndsight/ljung_box_df.html</guid>
  <pubDate>Tue, 27 Jun 2023 14:00:00 GMT</pubDate>
  <media:content url="https://robjhyndman.com/img/ETSMNN.png" medium="image" type="image/png" height="54" width="144"/>
</item>
<item>
  <title>Forecasting workshops in New York and Chicago</title>
  <link>https://robjhyndman.com/hyndsight/workshops2023.html</link>
  <description><![CDATA[ 




<div>
  <div class="description">
    I’m giving a 2-day workshop on “Tidy Time Series and Forecasting in R”, first at the New York R Conference in July, and then at the Posit Conference in Chicago in September.
  </div>
</div><p></p>


<p>Places are limited, so please book in early.</p>
<ul>
<li><a href="https://web.archive.org/web/20230612041012/https://rstats.ai/nyr#tidy-time-series-and-forecasting-in-r">New York</a>, 11-12 July 2023. Followed by the <a href="https://web.archive.org/web/20230612041012/https://rstats.ai/nyr">New York R conference</a>. (Register for the workshop as part of the NYR conference registration.)</li>
<li><a href="https://posit-conf-2023.github.io/workshops/workshops/forecasting/">Chicago</a>, 17-18 September 2023. Followed by the <a href="https://posit.co/conference">Posit conference</a>. (Register for the workshop as part of the Posit conference registration.)</li>
</ul>
<p>The workshop introduces the <a href="https://tidyverts.org">tidyverts</a> set of packages. Further details about the workshop are <a href="https://robjhyndman.com/teaching/#tidy-time-series-forecasting-in-r">here</a>.</p>
<iframe width="560" height="315" src="https://www.youtube.com/embed/g4221KXM7hU" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen="">
</iframe>
<p><a href="https://tidyverts.org"><img src="https://robjhyndman.com/img/tidyverts.png" class="img-fluid"></a></p>



 ]]></description>
  <category>forecasting</category>
  <category>R</category>
  <category>conferences</category>
  <category>teaching</category>
  <guid>https://robjhyndman.com/hyndsight/workshops2023.html</guid>
  <pubDate>Wed, 12 Apr 2023 14:00:00 GMT</pubDate>
  <media:content url="https://robjhyndman.com/img/tidyverts.png" medium="image" type="image/png" height="98" width="144"/>
</item>
<item>
  <title>Monash Quarto Templates</title>
  <link>https://robjhyndman.com/hyndsight/quarto_templates.html</link>
  <description><![CDATA[ 




<div>
  <div class="description">
    I’ve created some quarto templates with Monash University branding.
  </div>
</div><p></p>


<section id="monash-letter-template" class="level2">
<h2 class="anchored" data-anchor-id="monash-letter-template"><a href="https://github.com/quarto-monash/letter">Monash Letter Template</a></h2>
<p>This is a Quarto template that assists you in creating a letter on Monash University letterhead.</p>
<table>
<tbody><tr>
<td valign="top">
<pre><code>quarto use template quarto-monash/letter</code></pre>
<p>This will install the extension and create an example qmd file that you can use as a starting place for your letter.</p>
<p>You may also use this format with an existing Quarto project or document. From the quarto project or document directory, run the following command to install this format:</p>
<pre><code>quarto install extension quarto-monash/letter</code></pre>
</td>
<td valign="top">
<a href="https://github.com/quarto-monash/letter"><img src="https://github.com/quarto-monash/letter/raw/main/examples/template.png" class="img-fluid" width="700"></a>
</td>
</tr>
</tbody></table>
</section>
<section id="monash-memo-template" class="level2">
<h2 class="anchored" data-anchor-id="monash-memo-template"><a href="https://github.com/quarto-monash/memo">Monash Memo Template</a></h2>
<p>This is a Quarto template that assists you in creating a memo, with optional Monash University branding.</p>
<table>
<tbody><tr>
<td valign="top">
<pre><code>quarto use template quarto-monash/memo</code></pre>
<p>This will install the extension and create an example qmd file that you can use as a starting place for your memo.</p>
<p>You may also use this format with an existing Quarto project or document. From the quarto project or document directory, run the following command to install this format:</p>
<pre><code>quarto install extension quarto-monash/memo</code></pre>
</td>
<td valign="top">
<a href="https://github.com/quarto-monash/memo"><img src="https://github.com/quarto-monash/memo/raw/main/examples/template.png" class="img-fluid" width="700"></a>
</td>
</tr>
</tbody></table>
</section>
<section id="monash-working-paper-template" class="level2">
<h2 class="anchored" data-anchor-id="monash-working-paper-template"><a href="https://github.com/quarto-monash/workingpaper">Monash Working Paper Template</a></h2>
<p>This is a Quarto template that assists you in creating a working paper for the Department of Econometrics &amp; Business Statistics, Monash University.</p>
<table>
<tbody><tr>
<td valign="top">
<pre><code>quarto use template quarto-monash/workingpaper</code></pre>
<p>This will install the extension and create an example qmd file that you can use as a starting place for your working paper.</p>
<p>You may also use this format with an existing Quarto project or document. From the quarto project or document directory, run the following command to install this format:</p>
<pre><code>quarto install extension quarto-monash/workingpaper</code></pre>
</td>
<td valign="top">
<a href="https://github.com/quarto-monash/workingpaper"><img src="https://github.com/quarto-monash/workingpaper/raw/main/examples/template.png" class="img-fluid" width="900"></a>
</td>
</tr>
</tbody></table>
</section>
<section id="monash-report-template" class="level2">
<h2 class="anchored" data-anchor-id="monash-report-template"><a href="https://github.com/quarto-monash/report">Monash Report Template</a></h2>
<p>This is a Quarto template that assists you in creating a Monash University report.</p>
<table>
<tbody><tr>
<td valign="top">
<pre><code>quarto use template quarto-monash/report</code></pre>
<p>This will install the extension and create an example qmd file that you can use as a starting place for your report.</p>
<p>You may also use this format with an existing Quarto project or document. From the quarto project or document directory, run the following command to install this format:</p>
<pre><code>quarto install extension quarto-monash/report</code></pre>
</td>
<td valign="top">
<a href="https://github.com/quarto-monash/report"><img src="https://github.com/quarto-monash/report/raw/main/examples/template.png" class="img-fluid" width="900"></a>
</td>
</tr>
</tbody></table>
</section>
<section id="monash-thesis-template" class="level2">
<h2 class="anchored" data-anchor-id="monash-thesis-template"><a href="https://github.com/quarto-monash/thesis">Monash Thesis Template</a></h2>
<p>This is a Quarto template for a Monash University thesis.</p>
<table>
<tbody><tr>
<td valign="top">
<pre><code>quarto use template quarto-monash/thesis</code></pre>
<p>This will copy the template to your current directory, which you can use as a starting place for your thesis.</p>
</td>
<td width="300" valign="top" align="right">
<a href="https://github.com/quarto-monash/thesis"><img src="https://github.com/quarto-monash/thesis/raw/main/examples/template.png" class="img-fluid quarto-figure quarto-figure-right" width="300"></a>
</td>
</tr>
</tbody></table>
</section>
<section id="monash-presentation-template" class="level2">
<h2 class="anchored" data-anchor-id="monash-presentation-template"><a href="https://github.com/quarto-monash/presentation">Monash Presentation Template</a></h2>
<p>This is a Quarto template that assists you in creating a presentation using either Beamer (pdf) or RevealJS (html), with Monash University branding.</p>
<table>
<tbody><tr>
<td valign="top">
<pre><code>quarto use template quarto-monash/presentation</code></pre>
<p>This will install the extension and create an example qmd file that you can use as a starting place for your presentation.</p>
<p>You may also use this format with an existing Quarto project or document. From the quarto project or document directory, run the following command to install this format:</p>
<pre><code>quarto install extension quarto-monash/presentation</code></pre>
</td>
<td valign="top" width="300">
<a href="https://github.com/quarto-monash/presentation"><img src="https://github.com/quarto-monash/presentation/raw/main/examples/pdftemplate.png" class="img-fluid" width="1200"></a>
</td>
</tr>
</tbody></table>
</section>
<section id="rmarkdown-templates" class="level2">
<h2 class="anchored" data-anchor-id="rmarkdown-templates">Rmarkdown templates</h2>
<p>These are all based on my Rmarkdown templates which are distributed via the <a href="https://github.com/numbats/monash"><code>monash</code> R package</a>.</p>


</section>

 ]]></description>
  <category>quarto</category>
  <category>rmarkdown</category>
  <category>Monash University</category>
  <category>reproducible research</category>
  <guid>https://robjhyndman.com/hyndsight/quarto_templates.html</guid>
  <pubDate>Sun, 01 Jan 2023 13:00:00 GMT</pubDate>
  <media:content url="https://robjhyndman.com/img/monash_letter_template.png" medium="image" type="image/png" height="203" width="144"/>
</item>
</channel>
</rss>
