Modelling demand and price elasticity with regression

TL;DR

Difficulty: Intermediate

Every pricing argument comes down to one number: how much volume you lose for each 1% of price you add.

  • A log log regression on a year of weekly sales estimates that elasticity directly, with an error range.
  • Revenue peaks where elasticity is minus one, but the price that pays is the contribution maximum, $P^* = MC / (1 + 1/\eta)$.
  • Near the top the contribution curve is flat, and the best price is very sensitive to the elasticity, so read the ladder, not one number.
  • Segments differ: corporate buyers can be inelastic while web gift buyers are elastic.
  • Price is rarely set at random, so the model is the hypothesis and a test against a control group is the proof.

Every pricing argument comes down to one unknown number: how much volume you lose for every 1% of price you add. Here I take a year of weekly sales for one hamper at an invented Cork shop and walk through a demand regression from start to finish, from reading the output to the price that pays and the test that proves it. Chapter 4 of 21 in the Marketing Analytics series.

The hamper that would not tell anyone its price

Priya runs Northgate Hampers, an invented online hamper shop in Cork at northgatehampers.example. Her Afternoon Tea Hamper is the quiet workhorse of the range. It sells all year, and it has sat at €49 for so long that the price feels like a fact of nature. Last spring she wanted to know whether it should be €54.

Her marketing instinct said the opposite: drop it to €45, sell more, win the gift buyer who is comparing five tabs. Her finance instinct said raise it, the margin is thin once the courier has been paid. Both instincts are guesses about the same unknown number: how many hampers she loses for every euro she adds.

The good news is that the shop had already run the experiment, badly and by accident. Over the previous year the price had moved seven times, between €39 in a January clearance and €59 in the run up to Christmas. There had been 11 promotion weeks with a free delivery code, six Christmas weeks, and a competitor whose comparable hamper wandered between €52 and €58. Fifty two weeks of price, units and context: exactly the raw material a regression needs.

This chapter turns it into one number you can defend in a meeting, the price elasticity of demand. Northgate Hampers and every number in it are made up so I can show each step. The method is the one I run for real.

Where this chapter sits in the series

This is chapter 4 of 21 in my Marketing Analytics series, and the first of part two, the dependent variable techniques, which run from here to chapter 10. Part one ended with what is an insight, and why most reports do not contain one, where I argued that an insight has to change a decision. This chapter is the first method that reliably produces one.

The next chapter, polynomial distributed lags, keeps the same regression machinery and adds time, because an advert seen this week still sells hampers next month. Every basic idea used here, from the mean to logarithms, is built from scratch on the Foundations page.

Two families of technique, and the question that separates them

Nearly every method in marketing analytics belongs to one of two families. The sorting question is simple: is there a variable you are trying to explain?

If yes, you are in the dependent variable family. Units sold, revenue, churn, whether a trial converts: something on the left hand side of an equation that you believe is caused, at least in part, by the things on the right. Regression, logistic regression, survival models and their relatives live here, and so does this whole part of the series.

If no, you are in the interrelationship family. There is no target, only structure, such as which customers resemble each other or which products share a basket. Segmentation and factor analysis live there, and the series gets to them in part three.

Within the first family there is a second distinction. Some equations are deterministic: revenue equals price times units, always and exactly, with nothing to estimate. Others are probabilistic: units this week depend on price, promotion and season, plus something nobody measured. That something is the error term. Putting it in the equation is the honest admission that the model is a simplification.

An example: Priya's revenue for a week in which she sold 60 hampers at €49 is 60×49=€2,94060 \times 49 = €2{,}940. There is no error term, because the till does not guess. But "how many hampers will she sell next week at €49" is a probabilistic question. A model might say 60, and the honest version says 60 give or take a typical miss. A demand forecast without an error margin has either hidden the uncertainty or pretended an identity was a model.

Ordinary least squares, the workhorse

The demand model for one hamper, in its plainest linear form, looks like this:

Qt=β0+β1Pt+β2Dt+β3St+β4Ct+εtQ_t = \beta_0 + \beta_1 P_t + \beta_2 D_t + \beta_3 S_t + \beta_4 C_t + \varepsilon_t

Here QtQ_t is the number of hampers sold in week tt and PtP_t is Priya's price that week. DtD_t is a promotion dummy, 11 if a discount or free delivery code was live and 00 otherwise. StS_t is a season dummy, 11 in the six Christmas weeks. CtC_t is the competitor's price. The betas are the unknowns: β0\beta_0 is the baseline, β1\beta_1 is the change in weekly units for one extra euro of price, and so on. εt\varepsilon_t is the error term, everything about week tt the model does not know. The groundwork is in Foundations: regression.

Ordinary least squares (OLS) picks the betas that bring the predicted units as close as possible to the actual units across all 52 weeks, where "close" means the sum of squared misses:

min⁡β0,…,β4  ∑t=1n(Qt−Q^t)2\min_{\beta_0,\ldots,\beta_4} \; \sum_{t=1}^{n} \left( Q_t - \hat{Q}_t \right)^2

Here Q^t\hat{Q}_t is the model's prediction for week tt, nn is the number of weeks, and ∑\sum adds up the squared misses over all of them. Squaring stops positive and negative misses cancelling, and it punishes one huge miss more than several small ones. See Foundations: regression and, for why squares, Foundations: spread.

The method has been around since Adrien Marie Legendre published it in 1805.1 It is still the default because it is simple and transparent, and because under a short list of conditions (errors with a mean of zero, constant spread, no correlation between weeks) OLS is the best linear unbiased estimator, the result known as the Gauss Markov theorem.2 Weekly sales data breaks the last condition, which matters in the code.

Worked example: reading one line of a regression table

What comes back from the software is a table, and reading it is the actual skill. Say a linear fit returns a price coefficient of minus 1.95 with a standard error of 0.29.

Step one, the coefficient. Each extra euro of price costs about two hampers a week, holding promotion, season and the competitor constant.

Step two, the standard error. It is the uncertainty around the estimate, the typical distance between this estimate and the one a different year of the same shop would give.

Step three, the t statistic: the coefficient divided by its standard error, −1.95/0.29=−6.7-1.95 / 0.29 = -6.7. With about 50 weeks, an absolute t above about 2 means the sign is unlikely to be luck. See Foundations: hypothesis tests.

Step four, R squared, how much of the week to week variation in units the model explains:

R2=1−∑t(Qt−Q^t)2∑t(Qt−Qˉ)2R^2 = 1 - \frac{\sum_{t} \left( Q_t - \hat{Q}_t \right)^2}{\sum_{t} \left( Q_t - \bar{Q} \right)^2}

The numerator is what the model leaves unexplained. The denominator is the total variation around the average Qˉ\bar{Q}. The ratio is the share still missing, and one minus it is the share explained. The idea is set out in Foundations: regression and Foundations: the mean.

An R squared of 0.81 means the model accounts for 81% of the ups and downs. It does not mean the model is right. A high R squared with a nonsense coefficient is worse than a low one with a sensible coefficient.

Step five, the residuals, the misses week by week. Plot them against time and against price. Random noise is good. Trends, clusters or a fan shape mean the model is missing something and the standard errors are lying to you.

From a coefficient to an elasticity

A coefficient of minus 1.95 hampers per euro is useful for this hamper and useless for comparing with a €120 software plan or a €190 hotel room. Elasticity fixes that by putting both sides in percentages: the percentage change in quantity for a 1% change in price.3

η=∂Q∂P⋅PQ\eta = \frac{\partial Q}{\partial P} \cdot \frac{P}{Q}

Here η\eta (eta) is the elasticity. ∂Q/∂P\partial Q / \partial P is the slope of demand with respect to price, which in the linear model is simply β1\beta_1. P/QP/Q rescales it from euro and hampers into percentages. Foundations writes the same idea as ε=%ΔQ/%ΔP\varepsilon = \%\Delta Q / \%\Delta P, in Foundations: elasticity. I use η\eta here because ε\varepsilon is already the error term.

Because P/QP/Q changes along a straight demand line, a linear model's elasticity is different at every price. By convention you evaluate it at the averages.

Worked example: the elasticity at the averages

Step one, the inputs: β1=−1.95\beta_1 = -1.95, an average price of €49 and average weekly sales of 60 hampers.

Step two, the rescaling: P/Q=49/60=0.817P/Q = 49 / 60 = 0.817.

Step three, the product: −1.95×0.817=−1.59-1.95 \times 0.817 = -1.59.

A 1% price rise loses about 1.6% of volume. Demand is elastic, more than one in absolute value,3 and that single fact drives the whole pricing discussion that follows. If you want the intuition without the algebra, I wrote it up in price elasticity in plain English. This chapter is where we actually estimate it.

The log log model hands you the elasticity directly

The linear model gives you an elasticity that depends on where you stand. For pricing work I usually reach for a different functional form first, because it makes the elasticity the thing you estimate. Take natural logarithms of quantity and of every price:

ln⁡Qt=α+ηln⁡Pt+γ1Dt+γ2St+θln⁡Ct+εt\ln Q_t = \alpha + \eta \ln P_t + \gamma_1 D_t + \gamma_2 S_t + \theta \ln C_t + \varepsilon_t

Here ln⁡\ln is the natural logarithm and α\alpha is the intercept. η\eta, the coefficient on log price, is now the own price elasticity itself, the same at every price level. θ\theta (theta) is the cross price elasticity, the percentage change in Priya's units for a 1% change in the competitor's price, positive for a substitute. DtD_t and StS_t are the dummies from before, and γ1\gamma_1 and γ2\gamma_2 (gamma) are their effects on log units. See Foundations: logarithms and Foundations: elasticity.

The trick is that a small change in the log of a variable is a percentage change. So a regression of log on log is a regression of percentages on percentages, and the coefficient reads as the percentage change in units for a 1% change in price.4 A dummy in a log model works slightly differently: its exact percentage lift is eγ−1e^{\gamma} - 1, found by exponentiating the coefficient and subtracting one.4

The log log model has a cost, which I come back to under the pitfalls: it assumes constant elasticity, and demand rarely obliges across the whole price range. As a first estimate on a year of shop data it is still hard to beat.

Worked example: 52 weeks of the Afternoon Tea Hamper

Here is the output of the log log model on Priya's year: 52 weekly observations, price between €39 and €59 at seven distinct levels, 11 promotion weeks, six Christmas weeks from mid November, and the competitor's price checked every Monday.

TermCoefficientStd errort statistic
Intercept8.311.924.3
ln price (own)minus 1.610.24minus 6.7
Promotion week (0 or 1)0.280.093.1
Christmas week (0 or 1)0.740.126.2
ln competitor price0.520.311.7

The fit: n = 52 weeks, R squared 0.81, adjusted R squared 0.79, residual standard error 0.14 in log units, which is a typical miss of about 15% (e0.14=1.15e^{0.14} = 1.15).

Reading it line by line.

The intercept is the log of units at a price of €1 with a competitor at €1. It has no business meaning, and I ignore it.

The own price coefficient is the headline: an elasticity of minus 1.61 with a standard error of 0.24. Two standard errors either side gives a plausible range of roughly minus 1.1 to minus 2.1, and the t statistic of −1.61/0.24=−6.7-1.61 / 0.24 = -6.7 says the sign is not in doubt. Whether it is minus 1.2 or minus 2.0 matters a great deal for pricing, and a year of data does not pin it down tighter than that. See Foundations: confidence intervals.

Promotion at 0.28 means a free delivery week lifts units by about 32%, since e0.28=1.32e^{0.28} = 1.32, holding price constant. That is the code's effect over and above the price cut it usually comes with, which is why price belongs in the same model.

Christmas at 0.74 is a lift of about 110%, since e0.74=2.10e^{0.74} = 2.10: demand roughly doubles. Its t of 0.74/0.12=6.20.74 / 0.12 = 6.2 makes it one of the most certain things in the table.

The competitor coefficient needs care. A value of 0.52 says a 10% rise in their price gives Priya about 5% more units, the right sign for a substitute. But the t of 0.52/0.31=1.70.52 / 0.31 = 1.7 means I would not bet the house on the size. The competitor's price barely moved in the year, so there is little information there, and the standard error says so honestly.

Afternoon Tea Hamper weekly units by priceObserved average weekly units at seven price points from €39 to €59 against the fitted log log demand curve, ordinary weeks only. Both fall from close to 90 hampers a week at €39 to about 44 at €59, and the curve tracks the observations closely.Weekly units by price, ordinary weeks0255075100€39€42€45€49€52€55€59Observed averageModelHampers per weekPrice

The horizontal axis is the price Priya charged, the vertical axis is hampers sold per week, and only ordinary weeks are shown (no promotion, no Christmas). One line is the observed average at each price, the other is the model's prediction, which is 60×(P/49)−1.6160 \times (P / 49)^{-1.61}. The model curve falls more steeply at the low end and flattens at the high end. That is what constant elasticity looks like in units: the same percentage loss for each percentage rise means fewer hampers lost in absolute terms as the price climbs.

Revenue peaks where elasticity is minus one

Now the part the meeting actually wants. Revenue is price times units, so the effect of a small price rise on revenue is:

dRdP=Q+PdQdP=Q(1+η)\frac{dR}{dP} = Q + P \frac{dQ}{dP} = Q \left( 1 + \eta \right)

Here RR is revenue, QQ is units, PP is price and η\eta is the elasticity from before. The second step uses PdQdP=QηP \frac{dQ}{dP} = Q \eta, which is just the elasticity formula rearranged. See Foundations: elasticity and Foundations: marginal versus average.

Revenue rises with price while 1+η1 + \eta is positive, that is while demand is inelastic, and falls while it is negative, while demand is elastic.5 So revenue peaks exactly where η=−1\eta = -1. With an elasticity of minus 1.6 that condition says Priya is past the revenue peak: every price rise loses revenue, every price cut gains some. Read carelessly, the model says cut the price.

Read carefully, it says no such thing, for two reasons.

First, a constant elasticity model of minus 1.6 has no revenue peak anywhere. It says cut, and keep cutting, all the way to zero, which is a property of the functional form and not of hampers.

Second, the linear model does have a peak, at P∗=−β0/(2β1)P^{*} = -\beta_0 / (2 \beta_1).

Worked example: two models, two revenue peaks

Step one, the linear fit's baseline for ordinary weeks. With a slope of minus 1.95 and 60 hampers at €49, the intercept is 60+1.95×49=155.660 + 1.95 \times 49 = 155.6.

Step two, the peak: 155.6/(2×1.95)=155.6/3.9=€39.90155.6 / (2 \times 1.95) = 155.6 / 3.9 = €39.90.

So the linear model says revenue peaks near €40, and the log log model says it has no peak at all. Two sensible models, two answers: a revenue peak estimated from a year of data is soft, and revenue is the wrong target anyway.

The price that pays is set by contribution, not revenue

Nobody should maximise revenue. In this example the hamper costs €19 to sell: goods, box, ribbon and courier. Contribution, price minus that cost, times units, is what pays the bills. Under constant elasticity, maximising it gives the markup rule from any economics course:6

P∗=MC1+1/ηP^{*} = \frac{MC}{1 + 1/\eta}

Here P∗P^{*} is the contribution maximising price, MCMC is the marginal cost, the cost of selling one more unit (€19 here), and η\eta is the elasticity, which must be below minus one for the rule to give an answer. It is the same rule as P∗=c×β/(β−1)P^{*} = c \times \beta / (\beta - 1) in stochastic optimisation for budget and pricing, written there with the elasticity as a positive number β=−η\beta = -\eta. See Foundations: elasticity and Foundations: marginal versus average.

Worked example: the markup rule on the hamper

Step one: 1/η=1/(−1.61)=−0.6211/\eta = 1 / (-1.61) = -0.621.

Step two: 1+(−0.621)=0.3791 + (-0.621) = 0.379.

Step three: P∗=19/0.379=€50.15P^{*} = 19 / 0.379 = €50.15, about €50.

Here is the same story as a price ladder, using the model's units for ordinary weeks. The units are shown to one decimal place so that revenue and contribution add up.

PriceModel units per weekRevenueContribution at €19 cost
€3986.6€3,379€1,733
€4276.9€3,230€1,769
€4568.8€3,097€1,789
€4960.0€2,940€1,800
€5254.5€2,835€1,799
€5549.8€2,740€1,793
€5944.5€2,625€1,780

Revenue falls all the way down the ladder, as the elasticity said it would. Contribution barely moves. It tops out at about €1,800 a week between €49 and €52, and even €45 or €55 costs only €7 to €11 a week.

That flatness is the real finding. The shop is near the top of a broad hill, and the question of €49 against €54 is worth about €4 a week (35×51.3=€1,79635 \times 51.3 = €1{,}796 against €1,800), well inside the model's error. The marketing instinct to cut to €45 would have bought about nine more parcels a week for slightly less contribution. The finance instinct to go to €54 is defensible and not urgent.

The rule is also very sensitive near minus one. Feed it the two ends of the plausible range: at minus 2.09 it gives 19/(1−1/2.09)=€3619 / (1 - 1/2.09) = €36, at minus 1.13 it gives 19/(1−1/1.13)=€16519 / (1 - 1/1.13) = €165. A standard error of 0.24 on the elasticity turns into a best price somewhere between €36 and €165, which is why the ladder, with its flat top, is a better guide than the single number. For why €49 and €54 are not the same to a gift buyer even when the maths says they nearly are, pricing psychology covers the part regression cannot see.

Elasticity is not one number: corporate and private buyers differ

The minus 1.6 is an average over everyone who bought the hamper, and averages hide the useful part. Northgate Hampers has three recognisable groups: corporate buyers ordering five or more on an invoice, private gift buyers, and private self buyers. Splitting the weekly data by segment and refitting the same model gives this:

SegmentShare of unitsElasticityStd error
Corporate orders31%minus 0.70.30
Private gift buyers54%minus 1.90.28
Private self buyers15%minus 2.40.55
All buyers100%minus 1.60.24

A quick sanity check: the overall elasticity is the average of the segment elasticities weighted by their share of units, 0.31×0.7+0.54×1.9+0.15×2.4=0.22+1.03+0.36=1.600.31 \times 0.7 + 0.54 \times 1.9 + 0.15 \times 2.4 = 0.22 + 1.03 + 0.36 = 1.60 in absolute value. The split adds up to the whole.

Own price elasticity by segmentAbsolute own price elasticity by buyer segment: corporate orders 0.7, private gift buyers 1.9, private self buyers 2.4, all buyers 1.6.Elasticity by segment, absolute value0.01.32.53.85.00.7Corporate1.9Private gift2.4Private self1.6All buyersElasticity, absolute valueSegment

The horizontal axis lists the buyer segments, the vertical axis is the size of the elasticity with the minus sign dropped, so a taller bar means more price sensitive buyers. Anything below one is inelastic, anything above one is elastic. The takeaway: corporate buyers are the only inelastic group, and the private self buyers are the most sensitive.

Corporate demand is inelastic. A purchasing manager sending 40 hampers to clients is spending a budget approved in September, the sign off is the same at €49 as at €54, and the alternative is not a lower priced hamper but the effort of finding a new supplier. Private self buyers are the opposite, and the wide standard error says there are not many of them to learn from. Gift buyers sit in between and dominate the volume.

Worked example: what the segments do to the best price

For corporate buyers at minus 0.7, the markup rule has no answer: demand is inelastic, so every price rise in the tested range adds contribution. That is a case for a corporate price list above the web price.

For private gift buyers at minus 1.9: 1/η=−0.5261/\eta = -0.526, 1−0.526=0.4741 - 0.526 = 0.474, and 19/0.474=€40.1019 / 0.474 = €40.10.

So on the web, where gift buyers dominate, the rule points well below €49. Marketing's instinct was half right, just for the wrong half of the customers. But the standard error of 0.28 still matters. One standard error either side, at minus 1.62 the best price is about €50, at minus 2.18 it is about €35. The segment model tells you which direction to test, not where to land.

The practical consequence is that one list price is the wrong instrument. A corporate price list with volume tiers above the web price, and a web price tested downwards for the gift buyer, will beat any single number, and the regression is what tells you the gap is real rather than a hunch. Cross price elasticity splits the same way: near zero for corporate buyers, who are not comparing tabs, larger for self buyers, who are. Take that further, letting price move by segment and by time on the model's say so, and you are in the territory of dynamic pricing, which deserves its own playbook.

The test against a control group settles it

Everything above is observational. Priya set the prices in the data, for reasons, and some of those reasons were related to demand, which is the first pitfall below. So the model is a hypothesis with a number attached, and the proof is a field test with a control group.

Worked example: eight weeks at €49 against €54

The setup. Over eight ordinary autumn weeks, with no promotion running, half of the web sessions, chosen at random, saw the hamper at €54 and the other half at €49. Each half had about 4,400 sessions.

The prediction. For private buyers at minus 1.9, a 10.2% price rise (54/49=1.10254 / 49 = 1.102) should cut units by about 17% on the constant elasticity curve, 1−1.102−1.9=0.171 - 1.102^{-1.9} = 0.17.

The result. The control group bought 268 hampers, 61 per thousand sessions. The test group bought 216, 49 per thousand. That is a fall of 12/61=20%12 / 61 = 20\%.

The uncertainty. The standard error of the gap between two rates is 0.061×0.939/4,400+0.049×0.951/4,400=0.0049\sqrt{0.061 \times 0.939 / 4{,}400 + 0.049 \times 0.951 / 4{,}400} = 0.0049. A 95% interval on the gap of 12 per thousand is 12 plus or minus 9.5, so the fall lies somewhere between about 4% and 35%. See Foundations: hypothesis tests and Foundations: sample size.

The money. Revenue per thousand sessions was 61×49=€2,98961 \times 49 = €2{,}989 in control against 49×54=€2,64649 \times 54 = €2{,}646 in test. Contribution was 61×30=€1,83061 \times 30 = €1{,}830 against 49×35=€1,71549 \times 35 = €1{,}715. For the web shop, €49 won, and the model had called the direction. The obvious next test is €45 against €49.

On the corporate side Priya quoted €54 to half of that period's enquiries and €49 to the rest. Eleven of 14 accepted the higher price, 12 of 15 the lower. That is far too few to prove anything, and entirely consistent with an elasticity near minus 0.7.

Test against control is the proof because it removes the confounding at source: same weeks, same weather, same competitor price, same distance to Christmas, only the price differed. The regression said where to look and how big an effect to expect. The test said it was real. If you can only afford one of the two, run the test. If you can afford both, the model makes the test smaller and sharper.

How to run this yourself

You need one row per week for one product: units sold, average realised price, a promotion flag, season flags, and any competitor price you can collect. Building that table is most of the work. In a Solidus shop the orders, line items and variants sit in three tables, and each line item carries its price, its quantity and its own promotion total.7 On PostgreSQL, date_trunc('week', ...) rolls each order up to its week.8

select date_trunc('week', o.completed_at)::date as week,
       sum(li.quantity) as units,
       sum(li.price * li.quantity + li.promo_total) / sum(li.quantity) as price
from spree_line_items li
join spree_orders o on o.id = li.order_id
join spree_variants v on v.id = li.variant_id
where v.sku = 'AFTERNOON-TEA' and o.state = 'complete'
group by 1 order by 1;

The promo_total on a line item is negative when a discount applies, so adding it gives the price the customer actually paid. Add the promotion and Christmas flags from your marketing calendar and the competitor price, then fit the model. In Python with statsmodels:

import numpy as np
import statsmodels.formula.api as smf

df["ln_units"] = np.log(df["units"])
df["ln_price"] = np.log(df["price"])
df["ln_comp"] = np.log(df["competitor_price"])

model = smf.ols("ln_units ~ ln_price + promo + christmas + ln_comp", data=df)
fit = model.fit(cov_type="HAC", cov_kwds={"maxlags": 4})
print(fit.summary())
print("elasticity:", round(fit.params["ln_price"], 2))

The HAC option gives heteroskedasticity and autocorrelation robust standard errors,9 which matters because one week's sales are correlated with the next, the Gauss Markov condition weekly data breaks. R users get the same from lm plus the sandwich package, which includes the Newey West estimator.10

Time: a day to build a clean weekly table, an hour to fit and plot, and about a week of arguing about what the coefficients mean. Before you trust the result, five checks.

  1. Count the distinct price levels. Fewer than five and the elasticity is a guess with decimal places.
  2. Plot residuals against time and against fitted values and look for structure.
  3. Check the correlation between the promotion flag and the season flag. Above about 0.6 they are fighting over the same weeks.
  4. Refit on the first 40 weeks, predict the last 12, and see whether the misses are the size the residual standard error promised.
  5. Read the sign of every coefficient before the size. A positive own price coefficient is not a discovery, it is a data problem.

From the order database to a price decisionThe route from a weekly table to a decision memo: fit the log log model, check the residuals and loop back to add a missing driver if they are not clean, then read the elasticity with its error range, build the price ladder and best price, run a test against a control group, and write the decision memo.Build the weekly tableFit the log log modelResiduals clean?noAdd the missing driver and refityesElasticity with its error rangePrice ladder and best priceTest against a control groupDecision memo

The diagram runs from the weekly table down to the decision memo. The one loop is the residual check: if the misses show a pattern, add the missing driver and refit before reading any coefficient. The test is not optional.

Pitfalls that make an elasticity lie

Price is not set at random. This is the big one, and it has a name, endogeneity. If you raise prices when demand is strong and cut them when it is weak, price and the error term move together. OLS will then report an elasticity that is too close to zero, or even the wrong sign.11 Season dummies absorb some of that. The rest needs price changes made for reasons unrelated to demand, or a randomised test. When a client's history shows every price rise in November, I say so before I fit anything.

Promotions live in the season. If every promotion ran in December, the model cannot separate the code from the Christmas lift, and the standard errors on both will balloon. The cure is boring: run a few promotions in ordinary months.

Too little variation. A price that moved twice in a year gives you two points and a line through them. The standard error will be enormous, but people read the coefficient and skip the rest. If you want to know your elasticity, you have to move the price, deliberately and in both directions.

Extrapolation. Priya's data covered €39 to €59. It says nothing about €29 or €79, and the constant elasticity form will happily produce a number for either. An elasticity is valid over the range of prices you actually charged, and no further.

Constant elasticity is an assumption. The log log model forces the same percentage response everywhere. Real demand often has a kink, a threshold like €50 where behaviour changes. Add a squared log price term or fit a piecewise model, and if the fit improves, quote different elasticities for different ranges.

How I do this for clients

The deliverable is a pricing decision with a number and a risk on it, not a model. It starts with the free workshop: we work out which products have moved in price often enough to say anything, what the cost per unit really is, and what decision the elasticity would change. If the answer is "none", I say so and we stop there.

Then I spend two weeks on it before you commit to anything. I need order lines with realised prices, whatever promotion history exists, and any competitor pricing you have or can start collecting. From that I build the weekly table, fit the linear and log log models, split by segment where the data allows, and design a test against a control group for the product that matters most.

You get three things. The model itself, in a notebook you can rerun. A price ladder like the one above, with revenue and contribution at every level and the error range shown rather than hidden. And a one page decision memo that says what price to test, what the model expects to happen, and what would make me wrong. If you want the elasticity kept alive, it becomes one dashboard panel that answers one question: are we still on the flat part of the hill?

This sits under my Data Science work, and the fees are spelled out on the pricing page. For eCommerce stores there is also the commission model described there: a monthly base rate or a commission on measured growth, whichever is higher, never both.

Questions to ask before you move a price

  • How many distinct price levels does our history contain for this product, and over what range?
  • Is the elasticity quoted with a standard error or confidence interval, and how wide is it?
  • Were the price changes in the data made for reasons unrelated to demand, and if not, how was that handled?
  • Are promotions and season in the model separately, and how correlated are they?
  • Is the recommended price a revenue maximum or a contribution maximum, and what cost per unit was used?
  • Does the elasticity differ by customer segment, and does our price structure reflect that?
  • What result from a test against a control group would make us abandon the recommendation?

The series is inspired by Mike Grigsby's Marketing Analytics.12 The explanations, examples and numbers here are my own.

Footnotes

  1. Jeff Suzuki, Legendre, Adrien-Marie, Biographical Encyclopedia of Astronomers, Springer, 2007 (via MacTutor, University of St Andrews), https://mathshistory.st-andrews.ac.uk/BEA/legendre_bea.pdf

  2. Marco Taboga, Gauss Markov theorem, Lectures on probability theory and mathematical statistics, Statlect, https://statlect.com/fundamentals-of-statistics/Gauss-Markov-theorem

  3. Steven A. Greenlaw, David Shapiro and others, Principles of Economics 3e, section 5.1 Price Elasticity of Demand and Price Elasticity of Supply, OpenStax, 2022, https://openstax.org/books/principles-economics-3e/pages/5-1-price-elasticity-of-demand-and-price-elasticity-of-supply

  4. Clay Ford, Interpreting Log Transformations in a Linear Model, University of Virginia Library StatLab, 2018, https://library.virginia.edu/data/articles/interpreting-log-transformations-in-a-linear-model

  5. Steven A. Greenlaw, David Shapiro and others, Principles of Economics 3e, section 5.3 Elasticity and Pricing, OpenStax, 2022, https://openstax.org/books/principles-economics-3e/pages/5-3-elasticity-and-pricing

  6. Robert S. Pindyck, Lecture Notes on Pricing, 15.013 Industrial Economics for Strategic Decisions, MIT Sloan School of Management, August 2010, https://www.mit.edu/~rpindyck/Courses/Pricing_10.pdf

  7. Solidus contributors, core database schema (spree_orders, spree_line_items, spree_variants), Solidus source code on GitHub, https://github.com/solidusio/solidus/blob/main/core/db/migrate/20160101010000_solidus_one_four.rb

  8. PostgreSQL Global Development Group, Date/Time Functions and Operators, PostgreSQL documentation, https://www.postgresql.org/docs/current/functions-datetime.html

  9. statsmodels developers, RegressionResults.get_robustcov_results, statsmodels documentation, https://www.statsmodels.org/stable/generated/statsmodels.regression.linear_model.RegressionResults.get_robustcov_results.html

  10. Achim Zeileis, Thomas Lumley and others, sandwich: Robust Covariance Matrix Estimators, CRAN, https://cran.r-project.org/package=sandwich

  11. Anubhav Jha, Lecture 7: Demand Systems: Discrete Choice Models (endogeneity problem and bias of the OLS estimator), UG Empirical Industrial Organization, Ashoka University, 2024, https://anubhavpcjha.github.io/files/ug_emp_IO_lec_07_demand_function_estimation.pdf

  12. Mike Grigsby, Marketing Analytics: A Practical Guide to Improving Consumer Insights Using Data Techniques, Kogan Page, 2022, ISBN 9781398608191, https://www.netgalley.co.uk/catalog/book/267990

If you have a year of sales data and a price you have been afraid to touch, bring it to my free workshop. We work out whether the history can support an elasticity at all, which product to model and which decision the number would change. Then I put two weeks of real work in on me before you commit to anything, with honest error margins and one accountable person.