Forecasting demand: autocorrelation, seasonality and honest error bands

The Friday the packing room ran out of ribbon

In week 51 of 2024 The Gift Bow ran out of the wide satin ribbon that goes round every Christmas hamper. Not out of hampers, out of ribbon. Someone had ordered "a bit more than last year", which turned out to be about a fifth less than the week actually brought. Two packers spent a Friday evening driving to a haberdashery in Kingston, the hampers went out on time, and the margin on that week did not survive the petrol, the overtime and the emergency courier.

Nothing about that week was unforeseeable. The shop had three years of weekly orders, Christmas arrives on the same date every year, and the Mothering Sunday bump had shown up twice before. What was missing was not data. It was a forecast with an honest error band, and someone willing to plan against the upper end of it rather than the middle. That is this chapter: predicting demand through time, with the seasons built in and the uncertainty stated in orders rather than adjectives.

Where this sits in the series

This is chapter 10 and it closes Part two of the series, the dependent variable techniques (chapters 4 to 10), where one outcome is explained by a set of drivers. The previous chapter, Panel regression and same store sales, followed many units through time. This one follows a single shop through time and takes the time dimension seriously. The next chapter, Simultaneous equations: when marketing causes sales and sales cause marketing, opens Part three, the interrelationship techniques, where the arrow between variables points both ways.

A forecast is regression with the calendar on the right hand side

Most of the mystique around forecasting dissolves once you see it as regression whose explanatory variables are made out of the calendar. Instead of price and ad spend on the right hand side you have a trend (a counter that goes 1, 2, 3 and so on, one step per week), indicators for seasons and holidays, and, if you have them, events such as a corporate campaign or a stockout. The dependent variable is orders per week. Everything from earlier chapters about coefficients, standard errors and residuals still applies.

What changes is the error term. In a cross section the 300th customer tells you nothing about the 301st, so the residuals are independent. In a time series a good week tends to follow a good week. If the shop was 40 orders above the model in week 20, it is probably above the model in week 21 too, because whatever caused the surplus (a press mention, a competitor's stockout, a warm weekend) does not switch off at midnight on Sunday. That is autocorrelation, and it quietly breaks ordinary regression the moment time is involved.

The simplest description of it is the first order autoregressive process, written AR(1):

\varepsilon_t = \rho\,\varepsilon_{t-1} + u_t

Here $\varepsilon_t$ is the model's error in week $t$ , $\rho$ (rho) is the autocorrelation coefficient, a number between minus 1 and 1, $\varepsilon_{t-1}$ is last week's error, and $u_t$ is a fresh, independent shock. With $\rho = 0$ you are back in ordinary regression. With $\rho = 0.34$ , roughly what The Gift Bow's residuals show, about a third of any surprise carries over into the following week.

Why does this matter? Ordinary least squares still gives unbiased coefficients when the errors are autocorrelated, but it lies about their precision. With positive $\rho$ the standard errors come out too small, the t values too large, and a trend that is really noise gets three stars. Worse for forecasting: the forecast ignores what last week's error says about next week, free information thrown away.

Two quick diagnostics. The first is Durbin Watson:

DW = \frac{\sum_{t=2}^{T} (e_t - e_{t-1})^2}{\sum_{t=1}^{T} e_t^2} \approx 2\,(1 - \hat\rho)

where $e_t$ are the fitted residuals, $T$ is the number of weeks and $\hat\rho$ is the estimated autocorrelation. A value near 2 means no autocorrelation. Below about 1.5 means positive autocorrelation, the usual case for sales data. Above 2.5 means negative autocorrelation, which usually means someone has over differenced the series. The Gift Bow's dummy model gives 1.31, matching $\hat\rho \approx 0.34$ .

The second is the autocorrelation function, the ACF: one bar per lag, showing how strongly the residuals correlate with themselves one, two, three weeks back. A tall bar at lag 52 means you have missed a season. A tall lag 1 with the rest decaying quickly means AR(1) errors will do.

Seasons as dummy variables

A dummy variable is a column of zeros with ones in the weeks that belong to a season. Several of them in one regression give the seasonal dummy model:

y_t = \beta_0 + \beta_1\,t + \sum_{k=1}^{K} \gamma_k\,D_{k,t} + \varepsilon_t

In words: $y_t$ is orders in week $t$ . $\beta_0$ is the level of a base week at the start of the sample. $\beta_1$ is the trend, the change in weekly orders from one week to the next. $D_{k,t}$ equals 1 if week $t$ belongs to season $k$ and 0 otherwise, with $K$ seasons in total. $\gamma_k$ (gamma) is how many orders season $k$ adds to, or removes from, a base week. $\varepsilon_t$ is the error, allowed to follow the AR(1) process above.

The temptation is to give every one of the 52 weeks its own dummy. With 156 weeks of history that is 52 coefficients estimated from three observations each: you reproduce the past perfectly and forecast the future badly, because every random wobble becomes a "season". Group weeks into seasons that mean something to the business instead. For a hamper shop: November, early December, the last order week before Christmas, the dead stretch around New Year, January, and the two Mother's Days (Mothering Sunday in the UK in March, Muttertag in Austria in May). Everything else is base, and the base is what the intercept measures. Moving holidays get a dummy that moves with them: the two weeks before Mothering Sunday, wherever those land in a given year.

One more choice: linear or log trend. A linear trend adds a fixed number of orders each week. A log trend, where you regress the logarithm of orders, adds a fixed percentage, and the dummies become percentage effects too, which is right when Christmas triples orders rather than adding a fixed 500. For The Gift Bow the Christmas peak grew year on year, but less than proportionally with the base, and the linear model won on the holdout. It is a testable choice, not a philosophical one.

The Gift Bow: 156 weeks, one model

The data is weekly completed orders from the first week of 2023 to the last week of 2025, 156 weeks in a row. The numbers are illustrative but the shape is what a real gift shop looks like. I fitted on the first 130 weeks and kept the last 26, the second half of 2025 including the whole Christmas run, as a holdout the model never saw. Then I refitted on all 156 weeks to forecast 2026. Here is the coefficient table from the refitted model with AR(1) errors.

Term	Coefficient (orders per week)	Std error	Reading
Intercept (base week, start of 2023)	212	9	a plain week at the start of the sample
Trend per week	0.62	0.09	about 32 extra weekly orders each year
November (weeks 45 to 48)	plus 148	21	gift buyers start early, corporate enquiries land
Early December (weeks 49 and 50)	plus 391	29	the ramp
Last order week (week 51)	plus 523	41	the peak, the ribbon week
Christmas to New Year (weeks 52 and 1)	minus 96	29	nobody orders a hamper on Boxing Day
January (weeks 2 to 5)	minus 61	21	the resolution weeks
Mothering Sunday UK (two weeks before)	plus 117	29	second peak of the year
Muttertag Austria (two weeks before)	plus 64	29	smaller, the Austrian shop is younger
Corporate campaign running (event)	plus 88	26	the weeks a corporate mailing was live

Fit statistics: R² of 0.91, residual standard error of 36 orders per week, Durbin Watson 1.31, estimated $\rho$ of 0.34. The standard errors shown allow for the AR(1) errors. The naive ones were about a quarter smaller and made the Muttertag effect look far more certain than it is.

Reading it line by line. A plain week at the start of 2023 brought 212 orders, and each week since has added 0.62, so a plain week at the end of 2025 sits around 309. November lifts a week by 148 on top of wherever the trend has got to. Early December adds 391 and week 51 adds 523, so a week 51 at the end of 2025 is roughly 309 plus 523, about 830 orders, nearly three times a plain week. Then the floor drops out: the two weeks around New Year run 96 below base and January 61 below. Mothering Sunday is worth 117 extra orders in each of the two weeks before it; Muttertag 64 with a standard error of 29, so the honest statement is "somewhere between 20 and 110, probably about 60". The corporate campaign dummy says a live mailing is worth about 88 orders a week, which is the number the sales team wanted and the number I would not have believed without the standard error next to it.

Seasonal dummy coefficients from The Gift Bow model. Negative bars are weeks that run below a plain week.

Simple models that are hard to beat

Before you fall in love with a regression, meet the benchmarks. Seasonal naive says next week will be what the same week was last year, full stop. Add a growth factor (last year times 1.12, say) and you have a model many businesses would do well to adopt tomorrow. It knows nothing, costs nothing, and is embarrassingly hard to beat.

Exponential smoothing in the Holt Winters flavour keeps three running numbers, a level, a trend and a seasonal pattern, and nudges each towards what just happened, weighting recent weeks more. It adapts when the business changes, which the dummy model does not, and it needs no calendar knowledge, which is also its weakness: it cannot know that Mothering Sunday moved.

ARIMA, in plain words, is three ideas bolted together. AR: regress the series on its own past. I: difference it (this week minus last week) to remove the trend. MA: regress on past errors. Seasonal ARIMA repeats the trick at lag 52. My dummy model with AR(1) errors is essentially a small ARIMA with calendar regressors built by hand, which is why the two land so close on the scoreboard below.

My rule: the model that goes into production has to beat seasonal naive with growth on a holdout, by a margin that would change a decision. If it cannot, it has not earned its complexity, and the memo says so.

Judging a forecast on weeks it has never seen

Fit statistics are about the past. A forecast is judged on the holdout, the weeks you hid from the model, with two measures:

\text{MAE} = \frac{1}{n}\sum_{t=1}^{n}\left|y_t - \hat y_t\right| \qquad \text{MAPE} = \frac{100}{n}\sum_{t=1}^{n}\left|\frac{y_t - \hat y_t}{y_t}\right|

$y_t$ is the actual value in holdout week $t$ , $\hat y_t$ the forecast for it, $n$ the number of holdout weeks. MAE, the mean absolute error, is in orders and is the number the warehouse understands. MAPE, the mean absolute percentage error, is comparable across products and shops but punishes small weeks: a 28 order miss in the dead week after Christmas is a 12 percent error, the same 28 orders in week 51 would be 3 percent. So I plan with MAE and compare with MAPE.

Here is the scoreboard on the 26 week holdout. The last column is the forecast for the whole of Q4 2025 minus what actually happened, as a percentage.

Model	MAE (orders per week)	MAPE	Q4 2025 total, forecast minus actual
Seasonal naive (same week last year)	41	14.2%	minus 7.8%
Seasonal naive times 1.12 growth	33	11.0%	plus 1.9%
Seasonal dummy regression, plain OLS	29	9.6%	minus 3.6%
Seasonal dummy regression with AR(1) errors	26	8.7%	minus 3.1%
Holt Winters exponential smoothing	30	9.9%	plus 3.4%
Seasonal ARIMA, automatically selected	31	10.3%	minus 4.1%

Reading it. Plain seasonal naive underestimates by nearly 8 percent because it does not know the shop is growing. Give it a growth factor and it is suddenly respectable, within 2 percent on the quarter and only 7 orders a week behind the winner. The dummy model with AR(1) errors wins on MAE, 26 orders per week, and the AR(1) correction alone is worth 3 orders a week over plain OLS, a 10 percent reduction in error for zero extra data. All the serious models land within 4 percent on the quarterly total; the differences are in the weeks, which is where the ribbon gets ordered.

Bars are actual weekly orders in the holdout quarter, the line is the seasonal dummy model with AR(1) errors, fitted without seeing these weeks.

The forecast is a staircase, because that is what dummies produce: flat within a season, a step between. The actuals wobble around it. Over these 13 weeks the MAE was 26 orders and the MAPE 6.2 percent, lower than the 26 week figure because the low summer weeks pulled that percentage up. The biggest relative miss is week 52, 208 forecast against 236 actual, 12 percent off on a week that does not matter operationally. The miss that would have mattered, week 51, was 828 against 845, 2 percent. That is the week the ribbon ran out on "a bit more than last year".

The error band is the forecast

A point forecast without a band is a target in a spreadsheet costume. The band is the prediction interval:

\hat y_{T+h} \pm z_{0.90}\,\hat\sigma_h \qquad \hat\sigma_h^2 = \hat\sigma_u^2\,\frac{1 - \rho^{2h}}{1 - \rho^2} + \mathbf{x}_{T+h}^\top \mathbf{V}\,\mathbf{x}_{T+h}

$\hat y_{T+h}$ is the point forecast $h$ weeks after the last observed week $T$ . $z_{0.90}$ is 1.28, the normal quantile that leaves 10 percent in each tail, so the band covers 80 percent. $\hat\sigma_h$ is the forecast standard error at horizon $h$ and has two parts. The first is the accumulated AR(1) error variance: $\hat\sigma_u^2$ is the variance of the fresh shocks, and the fraction grows with $h$ and settles at $\hat\sigma_u^2/(1-\rho^2)$ , the long run variance of the errors. The second is uncertainty about the coefficients: $\mathbf{x}_{T+h}$ is the row of regressors for the forecast week (the trend value and which dummies are on) and $\mathbf{V}$ is the covariance matrix of the estimated coefficients. Week 51 gets a wide band partly because its dummy rests on three weeks of history and has a standard error of 41.

Two things people mix up. A confidence interval says where the average week 51 would land if you could rerun history many times. A prediction interval says where the actual week 51 will land, and it is always wider, because a single week carries its own noise. Planning needs the prediction interval. And 80 percent, not 95: a 95 percent band on a young shop is too wide to inform anything, while 80 percent means one week in five lands outside, a rate people can feel and plan around.

Here is the 2026 Q4 forecast from the model refitted on all 156 weeks. Weeks within one season differ only by the trend, so I show each season once.

Week of 2026	Point forecast	80% low	80% high	Season active
40 to 44 (each week)	335	280	390	base
45 to 48 (each week)	486	415	555	November
49 and 50 (each week)	730	630	830	early December
51	863	740	990	last order week
52	245	180	310	Christmas to New Year
Q4 total	6,190	5,500	6,900

The band on the quarterly total, 5,500 to 6,900, is narrower in relative terms than the weekly bands: plus or minus 11 percent against plus or minus 15 percent in week 51. Weekly errors partly cancel across a quarter. They cancel less than independent errors would, because of the autocorrelation, which is exactly why the $\rho$ in the formula matters: ignore it and the total band comes out about a third too narrow.

Forecast, target, decision

The marketing plan for The Gift Bow says 7,000 orders in Q4 2026. The forecast says 6,190, with an 80 percent band that does not reach 7,000. The two numbers are not in conflict, they are different things. The forecast is what happens if the shop keeps doing what it has been doing. The target is what the owner wants to happen. The gap, about 810 orders, is what the marketing has to earn above trend, and now it has a size, which means it can have a budget and a test plan. The worst thing you can do is let the target quietly become the forecast, because then the warehouse plans for 7,000, finance books the revenue, and in January somebody discovers that hope is not a season.

The decisions in the memo. Packaging costs about £1.80 per hamper to hold and keeps for years, so it is ordered to the upper band: 990 boxes and ribbon lengths for week 51, 830 for each of the two weeks before. Perishables are ordered to the point forecast with a reorder trigger at the end of week 49. A packer manages about 140 hampers a week, so week 51 needs just over six packers at the point forecast and seven at the upper band. Six are rostered, the seventh is an agency shift confirmed by week 49 if November comes in above 500 a week. The ribbon does not run out again, and nobody drives to Kingston.

Running it yourself

Data you need: one row per week with completed orders, ideally revenue too, going back at least two full years and preferably three, because a season seen once is an anecdote and a season seen twice is a hypothesis. From a Solidus shop it is one query:

select date_trunc('week', completed_at)::date as week_start,
       count(*)                              as orders,
       sum(total)                            as revenue
from spree_orders
where state = 'complete'
  and completed_at >= '2023-01-02'
group by 1
order by 1;

Then the seasons, a holdout, plain OLS for the Durbin Watson check, and a refit with AR(1) errors:

import pandas as pd, statsmodels.api as sm

df = pd.read_csv("weekly_orders.csv", parse_dates=["week_start"])
df["t"] = range(1, len(df) + 1)
df["iso_week"] = df.week_start.dt.isocalendar().week.astype(int)
df["nov"] = df.iso_week.between(45, 48).astype(int)
df["early_dec"] = df.iso_week.between(49, 50).astype(int)
df["w51"] = (df.iso_week == 51).astype(int)  # one column per season, base weeks stay zero

X = sm.add_constant(df[["t", "nov", "early_dec", "w51"]])
train, test = df.index < 130, df.index >= 130
ols = sm.OLS(df.orders[train], X[train]).fit()
print("Durbin Watson:", sm.stats.durbin_watson(ols.resid))

ar1 = sm.GLSAR(df.orders[train], X[train], rho=1).iterative_fit(maxiter=5)
pred = ar1.predict(X[test])
print("holdout MAE:", (pred - df.orders[test]).abs().mean())

For the benchmark, shift the series by 52 weeks and multiply by last year's growth. If your regression does not beat that on the holdout, stop and think before adding anything.

How long it takes: a day for clean weekly data and a first model, a week to settle the seasons, the holdout and the bands, a few more days to turn it into a table the warehouse and the marketing lead both accept. Longer if the history contains stockouts, because a sold out week records what you could ship, not what people wanted.

Checks before you trust it. Plot the residuals against time and look for runs above and below zero. Look at the residual ACF at lags 1 and 52. Beat seasonal naive with growth on the same holdout. Check that the fitted quarterly totals agree with what finance reported. Then show the band to the people who will act on it and ask what would have to happen for the actuals to leave it. Their answers are next year's event dummies.

Pitfalls

A dummy for every week. Fifty two dummies on three years of data is memorisation, not modelling. Group weeks into seasons the business recognises, and let the ACF tell you whether you have missed one.

Holidays that move. Mothering Sunday shifts between weeks 10 and 13, Easter wanders across March and April, Black Friday flips between weeks 47 and 48, and some years have 53 weeks. A fixed week dummy for a moving holiday smears the effect across two weeks and halves it. Build the dummy from the calendar, not the week number.

Grading on the fit. An R² of 0.91 on the training data says the model has learned the past. Only the holdout says whether it knows anything about the future, and if you tune the seasons by looking at the holdout error, you have turned it into training data and need a fresh one.

Reporting the naive standard errors. With positive autocorrelation, plain OLS makes every coefficient look sharper than it is. The Muttertag effect had a t value of 3.1 before the correction and 2.2 after. One is a finding, the other a maybe, and the client deserves to know which.

Letting the target become the forecast. The plan says 7,000, so the forecast gets nudged, one assumption at a time, until it says 7,000 too. Then nobody has a forecast. Keep them in separate columns, name the gap, and make the marketing earn it.

How I do this for clients

The deliverable is a forecast the business can plan against, not a model. It arrives as a table of weeks with a point forecast, an 80 percent band and the season each week belongs to, plus a one page decision memo that says what to order, whom to roster, and what would have to happen for the actuals to leave the band. Behind it sits the model, in a repository you own, with the holdout scoreboard so you can see what it beat and by how much.

We start with a free workshop where we look at your history together and agree the seasons, the events and the decisions the forecast has to support. Then I spend two weeks on it before you commit to anything: pulling weekly orders from your shop or ERP, building the calendar dummies, running the benchmarks and the holdout, writing the memo. If the history is too short or too broken to forecast honestly, the memo says that instead, with what to collect and for how long.

The data I need is unglamorous: one row per week with completed orders and revenue for at least two years, the dates of every campaign, stockout and price change you can remember, and the last order dates you published each Christmas. For Solidus shops I take that straight from the database, which is part of my Solidus development work. The modelling and the honest bands are the data science side, and the plain terms are on the pricing page: after the workshop and the two weeks you decide, you own everything I have built, and the numbers come with error margins or they do not come at all. For the wider toolkit, my complete guide to forecasting covers the method families, and the piece on AI powered demand forecasting for e commerce covers what changes when you have thousands of products rather than one shop.

Questions to ask whoever hands you a forecast

What is the 80 percent band on this number, in units, and which week is the widest?
Which weeks were held out when the model was judged, and what was the MAE on them?
Did it beat "same week last year plus growth"? By how much?
What was the Durbin Watson statistic, and what did you do about it?
How are moving holidays handled, and what happens in a 53 week year?
Where is the target written down separately from the forecast, and how big is the gap?
What would have to happen for the actuals to fall outside the band, and who is watching for it?

This series is inspired by Mike Grigsby's Marketing Analytics (Kogan Page). The explanations, examples and numbers here are my own.