Causal inference in marketing: from correlation to cause with propensity scores and instruments

A campaign ran and sales went up. That proves less than it seems.

A loyalty email goes out. The people who got it spend more than the people who did not. The report says the email worked, and the next budget is built on that sentence.

Maybe it did. But the email rarely goes out at random. Someone chose the customers who looked most promising, and they were always going to spend more. The comparison mixes two things together: what the email did, and who was picked to receive it. Correlation is the mixture. Causation is the first part on its own.

Causal inference is the toolkit for separating the two. When you can run a proper randomised test, do that (the options are in the complete guide to testing methods). This article is for when you cannot, and the data already exists. I use three ideas: potential outcomes, which define what "the effect" means, propensity score matching, which compares like with like, and instrumental variables, which borrow randomness from somewhere unexpected. A causal diagram ties them together.

Throughout I use one invented business. Corrib Pantry is a subscription snack box company in Galway, run by a woman called Niamh. It has 10,000 customers. Nobody real, every number made up so I can show every step. Niamh's question is simple: did the loyalty email cause repeat orders, and if so, how much was that worth in euro?

The effect you want is a comparison you can never see

The cleanest way to define a causal effect comes from the potential outcomes framework, set out by Donald Rubin in the 1970s and built on notation Jerzy Neyman used in 1923.¹²

Every customer has two possible futures. In one they receive the email, in the other they do not. Each future has an outcome, here the money they spend on repeat orders in the next 30 days. The effect of the email on that customer is the difference between the two futures.

The average treatment effect, the ATE, is that difference averaged over all customers:

$$\tau_{\text{ATE}} = E\left[Y_i(1) - Y_i(0)\right] = E\left[Y(1)\right] - E\left[Y(0)\right]$$

Here $\tau_{\text{ATE}}$ is the average treatment effect, $Y_i(1)$ is what customer $i$ would spend if they received the email, $Y_i(0)$ is what the same customer would spend if they did not, and $E[\cdot]$ is the expected value, the long run average over all customers. The treatment is the thing being tested, here the email. Expected values are explained from scratch in Foundations: expected value.

Now the catch. For any one customer you only ever see one of the two futures. Niamh's customer either got the email or did not. The other outcome, the counterfactual, never happens. Paul Holland called this the fundamental problem of causal inference, and no amount of extra data solves it.² The craft is estimating the missing half honestly.

Worked example: two customers, two futures

Take two Corrib Pantry customers. Suppose, in a world where we could see everything, the numbers were these:

• Customer A would spend €40 with the email and €30 without. Effect: $40 - 30 = €10$.
• Customer B would spend €12 with the email and €12 without. Effect: $12 - 12 = €0$.

The average treatment effect over the two is $\frac{10 + 0}{2} = €5$.

In the real world Niamh sees half of this. Customer A got the email, so she sees €40 and never the €30. Customer B did not, so she sees €12. Comparing them directly gives $40 - 12 = €28$, more than five times the true average effect. Customer A was simply the bigger spender to begin with.

The naive difference is the real effect plus a selection bias

That €28 has a precise anatomy, and it explains every method that follows. The difference between the treated group and the untreated group splits into two parts:²

$$\underbrace{E[Y \mid D=1] - E[Y \mid D=0]}_{\text{naive difference}} = \underbrace{E[Y(1) - Y(0) \mid D=1]}_{\text{ATT}} + \underbrace{E[Y(0) \mid D=1] - E[Y(0) \mid D=0]}_{\text{selection bias}}$$

Here $D$ is 1 for a customer who received the email and 0 for one who did not, $Y$ is the spend we actually observe, and the vertical bar $\mid$ means "among those who", so $E[Y \mid D=1]$ is the average spend among emailed customers. The ATT is the average treatment effect on the treated, the effect of the email on the customers who actually got it. The selection bias is how much more the emailed customers would have spent anyway, without any email at all. Conditional averages like these are covered in Foundations: probability.

The naive comparison equals the true effect only when the selection bias is zero, when the two groups would have spent the same without the email. Randomisation guarantees that. A team choosing its favourite customers guarantees the opposite.

Worked example: Niamh's naive comparison

Last spring Niamh's team sent the loyalty email to 4,000 of the 10,000 customers. They picked the ones who seemed engaged. Over the next 30 days:

• The 4,000 emailed customers spent on average €46.40 on repeat orders.
• The 6,000 others spent on average €18.40.

The naive difference is $46.40 - 18.40 = €28.00$ per customer. Multiplied across 4,000 emails, that is a story of $4{,}000 \times 28 = €112{,}000$ in extra revenue, and it was about to justify a €30,000 points and gifts loyalty programme. Keep that number in mind.

A causal diagram shows what to adjust for before you touch the data

Before any arithmetic I draw the problem. A causal diagram, formally a directed acyclic graph (DAG), is a set of boxes for variables and arrows for "this causes that", with no way to follow the arrows round in a circle. Judea Pearl turned these pictures into a full mathematical language.³

It tells you which variables to control for and which to leave alone.⁴

• A confounder causes both the treatment and the outcome. It opens a "backdoor" path that leaks correlation from one to the other. You control for it.
• A mediator sits on the path from treatment to outcome, the way "opened the email" sits between "sent the email" and "ordered". Control for it and you subtract part of the very effect you want.
• A collider is caused by two things. Left alone it blocks a path. Control for it and you open a false one. The textbook example in marketing is analysing only customers who replied to a survey, when replying is caused by both the email and by being a keen customer.

For Corrib Pantry: engagement and buying intent is the confounder, the email the treatment, repeat spend the outcome, and a random send slot the instrument, which I explain later.

Read it from the top. The send slot and the customer's engagement both cause the email to arrive, and the email perhaps causes some extra spend. The last box carries the two remaining arrows. Engagement pushes spend directly, the backdoor path that makes the naive comparison lie. The send slot reaches spend only through the email, which is what makes it useful later.

A pen will do grand for a diagram this size. For bigger ones I use DAGitty, a free browser tool from Radboud University that tells you which variables close every backdoor path.⁵

Worked example: what Niamh should and should not control for

Niamh's data holds four columns besides spend: boxes received so far, whether the customer opened last month's newsletter, whether they opened the loyalty email, and whether they left a review after the email.

• Boxes received and the newsletter open happened before the email and drive both the selection and the spend. Confounders. Control for them.
• Opening the loyalty email happens after it was sent and on the way to an order. A mediator. Leave it out, or the estimate shrinks towards zero for no good reason.
• Leaving a review after the email is caused by the email and by being a happy, high spending customer. A collider. Filtering to reviewers distorts the comparison.

The diagram also shows what Niamh cannot control for: part of the buying intent is invisible. Someone planning a gift order next week looks identical in her data to someone who is not. Hold that thought.

A propensity score compresses everything you know into one number

The first method compares like with like. If emailed and unemailed customers differ in boxes received and newsletter opens, compare only customers who match on those. With two variables you could do it by hand. With twenty you run out of customers who match on all of them.

Paul Rosenbaum and Donald Rubin solved this in 1983.⁶ They showed that you do not need to match on every variable. It is enough to match on one number, the probability of being treated given those variables, which they called the propensity score:

$$e(x) = P(D = 1 \mid X = x)$$

Here $e(x)$ is the propensity score, $D = 1$ means the customer received the email, $X$ is the set of observed characteristics before the email (boxes received, newsletter opens and so on), and $x$ is one particular customer's values of them. In words: among customers who look like this one, what share were emailed? Probability and the vertical bar are explained in Foundations: probability.

Customers with the same score are, on average, balanced on every variable that went into it. Twenty variables become one.

The score is usually estimated with a logistic regression, the standard model for a yes or no outcome:

$$\ln\frac{e(x)}{1 - e(x)} = \beta_0 + \beta_1 x_1 + \beta_2 x_2$$

Here $\ln$ is the natural logarithm, $\frac{e(x)}{1 - e(x)}$ is the odds of being emailed, $x_1$ is the number of boxes received, $x_2$ is 1 if the customer opened last month's newsletter and 0 if not, $\beta_0$ is the intercept, and $\beta_1$ and $\beta_2$ say how much each characteristic raises the log odds of being chosen. Logistic regression is covered in Foundations: regression.

Worked example: scoring three of Niamh's customers

Suppose the fitted model is $\ln\frac{e}{1-e} = -3.0 + 0.45 x_1 + 1.2 x_2$. To turn log odds back into a probability, use $e = \frac{1}{1 + \exp(-z)}$, where $z$ is the right hand side and $\exp(-z)$ means the number 2.718 raised to the power $-z$.

• Customer with two boxes, no newsletter open: $z = -3.0 + 0.45 \times 2 + 1.2 \times 0 = -2.1$, so the score is $\frac{1}{1 + \exp(2.1)} = 0.11$.
• Customer with four boxes who opened the newsletter: $z = -3.0 + 1.8 + 1.2 = 0$, so the score is $\frac{1}{1 + \exp(0)} = 0.50$.
• Customer with six boxes who opened the newsletter: $z = -3.0 + 2.7 + 1.2 = 0.9$, so the score is $\frac{1}{1 + \exp(-0.9)} = 0.71$.

The third customer had a 71% chance of being chosen, the first 11%. Putting them side by side compares two different kinds of customer. To keep the arithmetic on one screen, I group all 10,000 scores into three bands: low (about 0.10), middle (about 0.40) and high (about 0.80).

Matching on the score compares like with like

Once every customer has a score, matching is mechanical. For each emailed customer, find an unemailed customer with the nearest score, and compare their spend. Average the differences:

$$\hat{\tau}_{\text{ATT}} = \frac{1}{N_1} \sum_{i:\, D_i = 1} \left( Y_i - Y_{j(i)} \right)$$

Here $\hat{\tau}_{\text{ATT}}$ is the estimated average effect on the treated (the hat means "estimated"), $N_1$ is the number of emailed customers, the sum runs over every emailed customer $i$, $Y_i$ is that customer's spend, and $Y_{j(i)}$ is the spend of $j(i)$, the unemailed customer matched to them. It is a plain average of differences, see Foundations: the mean.

It estimates the ATT, the effect on the customers who were emailed, which is usually the honest question for a campaign that has already run.

Worked example: Niamh's matched estimate

Here are Niamh's 10,000 customers by score band, with the share emailed and the average 30 day repeat spend in each group:

• Low band, 4,000 customers, score about 0.10: 400 emailed spent €14, 3,600 not emailed spent €10. Difference €4.
• Middle band, 3,000 customers, score about 0.40: 1,200 emailed spent €30, 1,800 not emailed spent €24. Difference €6.
• High band, 3,000 customers, score about 0.80: 2,400 emailed spent €60, 600 not emailed spent €52. Difference €8.

First, check these add up to the naive figures. Emailed: $\frac{400 \times 14 + 1{,}200 \times 30 + 2{,}400 \times 60}{4{,}000} = \frac{185{,}600}{4{,}000} = €46.40$. Not emailed: $\frac{3{,}600 \times 10 + 1{,}800 \times 24 + 600 \times 52}{6{,}000} = \frac{110{,}400}{6{,}000} = €18.40$. They do.

Now match. Every emailed customer is compared with an unemailed customer from the same band, so each contributes their band's difference. Weight by the number of emailed customers in each band:

$$\hat{\tau}_{\text{ATT}} = \frac{400 \times 4 + 1{,}200 \times 6 + 2{,}400 \times 8}{4{,}000} = \frac{28{,}000}{4{,}000} = €7.00$$

Here 400, 1,200 and 2,400 are the emailed customers in each band, which play the role of $N_1$ split by band, €4, €6 and €8 are the band differences $Y_i - Y_{j(i)}$, and 4,000 is $N_1$, all emailed customers. Weighted averages are covered in Foundations: the mean.

The email is worth about €7 per emailed customer, not €28. Go back to the decomposition: naive €28.00 equals ATT €7.00 plus a selection bias of €21.00. Three quarters of the headline was the team's good eye for keen customers, and fair play to them for that, but it is not the email's doing.

A detail a senior reader will spot: the high band has 2,400 emailed customers and only 600 controls, so each control is used about four times. That is matching with replacement. It works, but makes the estimate noisier.

Check the balance, and respect what matching cannot see

Matching is only as good as the balance it produces. I check it, variable by variable, with the standardised mean difference:

$$d = \frac{\bar{x}_{\text{treated}} - \bar{x}_{\text{control}}}{s_{\text{pooled}}}$$

Here $d$ is the standardised difference, $\bar{x}_{\text{treated}}$ and $\bar{x}_{\text{control}}$ are the average of one characteristic in the two groups, and $s_{\text{pooled}}$ is the standard deviation of that characteristic across both groups, so the gap is measured in units of natural spread. Standard deviation is explained in Foundations: spread. A common convention treats a value below 0.1 as a negligible difference.⁷

Worked example: balance on boxes received

Before matching, Niamh's emailed customers had received 6.2 boxes on average and the others 3.4, with a pooled standard deviation of 2.8. So $d = \frac{6.2 - 3.4}{2.8} = 1.00$, a whole standard deviation apart. After matching, the matched controls average 6.1 boxes, so $d = \frac{6.2 - 6.1}{2.8} = 0.04$. Comfortably under 0.1.

Two limits deserve plain words.

The first is overlap. Matching needs unemailed customers at every score. Had the team emailed every customer in the high band, there would be nobody to compare them with. The formal condition is that every score sits strictly between 0 and 1.⁸ When it fails, I drop the unmatched customers and say so.

The second limit is the important one. Matching balances what you measured, and nothing else. It assumes that, once the observed characteristics are accounted for, who got the email was as good as random.⁸ For Niamh that is shaky. Her team also picked customers who had chatted warmly with customer service, some about upcoming gift orders. That intent is in no column, so the €7 may still carry some selection bias. A matched sample looks like a randomised trial only on the variables you can see.

An instrument borrows randomness from somewhere else

When the confounder is hidden, matching runs out of road. The instrumental variable (IV) approach comes from the other side. Instead of measuring every reason customers were chosen, it finds something that changed who got the email for reasons unrelated to the customer.

An instrument $Z$ has to pass three tests:⁹

1. Relevance. It genuinely changes who receives the treatment. This one you can check in the data.
2. Independence. It is as good as randomly assigned, unrelated to the hidden confounder.
3. Exclusion. It affects the outcome only through the treatment, never directly. This one you cannot check in the data. You argue it from how the world works.

Customer location is a tempting instrument, because send times often differ by region. It fails the exclusion test: people in Kilkenny and in Co. Mayo have different incomes, delivery times and weather, all of which move spend without going near the email. A good instrument is usually boring and mechanical.

Corrib Pantry has one. To protect its sender reputation, Niamh's email platform throttled the loyalty campaign by giving every customer a random send slot: early, on day one of the 30 day window, or late, on day 40, after the window had closed. The slot came from a coin flip, so it is independent of engagement and intent. And the slot can only change spend by changing whether the email arrived in time.

Assignment was imperfect, which is normal. Some early slot customers never received the email because they had unsubscribed or the team had left them off. Some late slot customers got it inside the window anyway, because the team sent it by hand to a good few favourites.

The Wald estimator divides two simple differences

With one yes or no instrument, the instrumental variable estimate is a single division, known as the Wald estimator:¹⁰

$$\hat{\beta}_{\text{IV}} = \frac{E[Y \mid Z=1] - E[Y \mid Z=0]}{E[D \mid Z=1] - E[D \mid Z=0]}$$

Here $\hat{\beta}_{\text{IV}}$ is the estimated effect of the email, $Z$ is 1 for the early slot and 0 for the late slot, $Y$ is 30 day repeat spend, and $D$ is 1 if the email arrived inside the window. The top line, called the reduced form, is how much more the early slot group spent. The bottom line, called the first stage, is how much more likely the early slot group was to get the email in time. Because $D$ is 0 or 1, $E[D \mid Z=1]$ is simply the share treated in the early group. Both lines are differences of averages, covered in Foundations: expected value.

The logic is a scaling. The coin moved spend a little and moved delivery by less than 100%. Dividing one by the other gives the effect per customer whose email actually changed.

With several instruments or extra controls, the same idea runs as two stage least squares: regress the treatment on the instrument, then the outcome on the predicted treatment. Done by hand as two separate regressions, the second one's standard errors are wrong. A proper IV routine corrects them.

Worked example: Niamh's instrumental variable estimate

The coin flip split the 10,000 customers into 5,000 early and 5,000 late.

• Early slot: 3,400 of 5,000 got the email in the window, a share of $0.68$. Average 30 day repeat spend €31.00.
• Late slot: 600 of 5,000 got it in the window anyway, a share of $0.12$. Average spend €28.20.

That is the same 4,000 emailed customers as before, $3{,}400 + 600 = 4{,}000$, and the same overall average, $\frac{31.00 + 28.20}{2} = €29.60$, just cut a different way.

The reduced form: $31.00 - 28.20 = €2.80$. The first stage: $0.68 - 0.12 = 0.56$. The estimate:

$$\hat{\beta}_{\text{IV}} = \frac{2.80}{0.56} = €5.00$$

Here €2.80 is the reduced form, $E[Y \mid Z=1] - E[Y \mid Z=0]$, and 0.56 is the first stage, $E[D \mid Z=1] - E[D \mid Z=0]$. See Foundations: expected value.

Before trusting it, I check the instrument is strong. One that barely moves the treatment makes the estimate unstable and biased. The old rule of thumb asked for a first stage F statistic above 10, and current advice treats 10 as too low.¹⁰ Niamh's first stage is anything but weak. The standard error of the 0.56 difference is $\sqrt{\frac{0.68 \times 0.32}{5{,}000} + \frac{0.12 \times 0.88}{5{,}000}} = 0.0080$, so the t statistic is $\frac{0.56}{0.0080} = 70$. With a single instrument the F statistic is the square of that, about 4,900. The standard error of a proportion is explained in Foundations: sampling.

The instrument answers a narrower question than you might think

The €5 is not the effect of the email on all customers, and not on all emailed customers. Guido Imbens and Joshua Angrist showed in 1994, and with Donald Rubin in 1996, that an instrument identifies the effect for one particular group: the compliers, the customers whose treatment was actually switched by the instrument.¹¹⁹ They called it the local average treatment effect, or LATE.

That result needs one more assumption, monotonicity: the instrument pushes everyone the same way, or not at all. Being in the early slot never made anyone less likely to get the email in time. For a send delay that is easy to believe.

Under monotonicity the first stage tells you who the compliers are, at least as a share:

• Always takers, who got the email in the window whatever their slot (the hand picked favourites): 12%, the share treated in the late group.
• Never takers, who did not get it whatever their slot (unsubscribed or left off): $1 - 0.68 = 32\%$.
• Compliers, whose delivery followed the coin: $0.68 - 0.12 = 56\%$.

Worked example: what the €5 is about

Niamh's IV estimate says: for the 56% of customers whose email timing was set by the coin, receiving the email in the window raised 30 day repeat spend by about €5. It says nothing directly about the always takers, the team's favourites. If the favourites respond more strongly, the effect on all emailed customers could sit above €5. If they were going to buy anyway, it could sit below.

So the €7 and the €5 answer two slightly different questions, with different weaknesses. Matching covers all emailed customers but may carry hidden intent. The instrument is protected from hidden intent but covers only compliers, with a wider margin of error, because it uses only the variation the coin created. When two methods that fail differently land within a couple of euro of each other, both far below the naive figure, I believe the broad picture.

Three estimates, one decision

Put the three numbers side by side.

The horizontal axis shows the three methods: the naive comparison, the matched estimate and the instrumental variable estimate. The vertical axis shows the estimated extra repeat spend per customer over 30 days, in euro. The takeaway is the size of the first drop. Most of the naive €28 was selection, the team's skill at picking customers who would have bought anyway. The two methods built to remove it put the real effect at around €5 to €7.

Worked example: what Niamh does with it

The naive story was $4{,}000 \times €28 = €112{,}000$ of extra revenue from one email. The honest range is $4{,}000 \times €5 = €20{,}000$ to $4{,}000 \times €7 = €28{,}000$. Corrib Pantry keeps about 40% of revenue as gross margin, so the email earned roughly $0.40 \times 20{,}000 = €8{,}000$ to $0.40 \times 28{,}000 = €11{,}200$ of gross profit.

The email costs almost nothing to send, so Niamh keeps it. The €30,000 points and gifts programme, justified by the €112,000, would need to roughly triple the email's effect just to pay for itself. She shelves it and does the one thing that ends the argument: she randomises the next loyalty send properly, with a held out control group, sized using the method in statistical testing, sample size and lift.

That is the real use of causal inference in a small business: fewer expensive decisions built on a number that was mostly selection.

How I use this in practice

In my experience the order matters more than the method. I draw the diagram first, because it decides what goes into the propensity model and what stays out. Then I look for anything genuinely random already hiding in the operation, such as a send throttle or a delivery slot lottery. Then I estimate with at least two methods that fail in different ways, and report the range, not the most flattering point.

Whenever the question will come up again, I stop estimating and start testing. A randomised holdout removes selection bias by design instead of by argument. When marketing and sales cause each other, that is a different tool, covered in simultaneous equations.

To go further, two free texts are excellent: Scott Cunningham's online book, with code,² and Miguel Hernán and James Robins's "Causal Inference: What If", free to download.¹²

Footnotes

1. Guido W. Imbens and Donald B. Rubin, Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction, Cambridge University Press, 2015, https://www.cambridge.org/core/books/causal-inference-for-statistics-social-and-biomedical-sciences/71126BE90C58F1A431FE9B2DD07938AB

2. Scott Cunningham, Causal Inference: The Remix (the updated online edition of Causal Inference: The Mixtape), chapter 3, Potential outcomes and randomisation, read November 2026, https://mixtape.scunning.com/03-potential_outcomes_and_randomization

3. Judea Pearl, Causality: Models, Reasoning, and Inference, second edition, Cambridge University Press, 2009, https://www.cambridge.org/core/books/causality/B0046844FAE10CBF274D4ACBDAEB5F5B

4. Scott Cunningham, Causal Inference: The Remix, chapter 4, Directed acyclic graphs, read November 2026, https://mixtape.scunning.com/04-directed_acyclical_graphs

5. Johannes Textor and others, DAGitty: drawing and analysing causal diagrams, Radboud University, read November 2026, https://www.dagitty.net/

6. Paul R. Rosenbaum and Donald B. Rubin, The central role of the propensity score in observational studies for causal effects, Biometrika, volume 70, issue 1, pages 41 to 55, 1983, https://academic.oup.com/biomet/article/70/1/41/240879

7. Peter C. Austin, An introduction to propensity score methods for reducing the effects of confounding in observational studies, Multivariate Behavioral Research, volume 46, issue 3, pages 399 to 424, 2011, https://pmc.ncbi.nlm.nih.gov/articles/PMC3144483/

8. Scott Cunningham, Causal Inference: The Remix, chapter 5, Unconfoundedness, read November 2026, https://mixtape.scunning.com/05-causal_cross_sectional_designs

9. Joshua D. Angrist, Guido W. Imbens and Donald B. Rubin, Identification of causal effects using instrumental variables, NBER Technical Working Paper 136, 1993, published in the Journal of the American Statistical Association, 1996, https://www.nber.org/papers/t0136

10. Scott Cunningham, Causal Inference: The Remix, chapter 7, Instrumental variables, read November 2026, https://mixtape.scunning.com/07-instrumental_variables

11. Guido W. Imbens and Joshua D. Angrist, Identification and estimation of local average treatment effects, Econometrica, volume 62, issue 2, pages 467 to 475, 1994, https://ideas.repec.org/a/ecm/emetrp/v62y1994i2p467-75.html

12. Miguel A. Hernán and James M. Robins, Causal Inference: What If, Chapman and Hall/CRC, 2020, https://miguelhernan.org/whatifbook