Programmatic SEO and content automation: scaling with strategy and structure

Programmatic SEO is a database with a template, and the database is the product

Programmatic Search Engine Optimisation (SEO) means building many pages from one template and one table of data, instead of writing each page by hand. Every row becomes a page: a route on a timetable site, an integration on a software site, a part and the machine it fits on a shop.

None of that is a trick, and Google says so. In its 2023 guidance on AI and automation it wrote that "Automation has long been used to generate helpful content, such as sports scores, weather forecasts, and transcripts."¹ The template is fine. The automation is fine.

What decides whether the result is useful or spam is the table. If each row holds facts that are true of that row and nothing else, every page has something to say. If the only thing that changes from row to row is the keyword, you have built the same page 400 times with a different word in the headline. Google has a name for that, and I come to it in the next section.

The data is the product. The template is only the packaging. I follow one invented business through every step.

Worked example: the removals firm in this article

Shandon Removals is an invented removals firm in Cork. Nobody real, every number made up. Its owner, Aoife, runs four vans and a crew of nine, and most of her work is households leaving Cork for somewhere else in Ireland. Her site lives at www.shandonremovals.example.

Over four years her booking system has recorded 1,900 moves out of Cork, to 140 different destination towns. For each move it holds the destination, the size of the home, the price, the hours on the road, the hours loading and unloading, and a free text note from the crew lead. That table is the most valuable SEO asset she owns, and at the start of this story she does not know it.

What Google's spam policies say about pages at scale

Google publishes its spam policies in one document, and three of them describe exactly how programmatic SEO goes wrong.²

Scaled content abuse arrived in March 2024. Google defines it as "when many pages are generated for the primary purpose of manipulating search rankings and not helping users", typically "large amounts of unoriginal content that provides little to no value to users, no matter how it's created."² The March 2024 announcement explained what changed: the new policy builds on the older rule against automatically generated content, and it applies "no matter whether content is produced through automation, human efforts, or some combination of human and automated processes."³ So a team of writers typing out 400 near identical pages by hand is caught by the same rule as a script.

Doorway abuse is older, and it is the one programmatic sites trip over most. Google's examples include "Having multiple domain names or pages targeted at specific regions or cities that funnel users to one page" and "Creating substantially similar pages that are closer to search results than a clearly defined, browseable hierarchy."² Read that second line twice. A pile of pages each answering a slightly different query, with no structure a person could browse, is a search results page you built yourself.

Thin affiliation covers pages with affiliate links where the product descriptions "are copied directly from the original merchant without any original content or added value." Google is explicit that affiliate sites as such are fine: it lists original reviews, testing, price information and product comparisons as ways good affiliate pages add value.²

The consequence is the same for all three. Sites that break the policies "may rank lower in results or not appear in results at all", found by automated systems and, where needed, a human reviewer and a manual action.² And the damage need not stay on the offending pages. Google says its ranking works page by page, but that it also uses signals and classifiers that look at the site as a whole.⁴ A thousand thin pages can weigh on the 30 good ones next to them.

I went through Google's position on AI writing, and why the detectors people buy do not work, in AI content in search, so I will not repeat it here.

Worked example: the 400 pages Aoife was offered

An agency, the kind that sends proposals with a stock photo of a rocket on the cover, offers Aoife 400 pages for €2,000. One page for every town and village within 200 km of Cork, titled "Removals from Cork to [town]". The sample page has 600 words: a paragraph on how stressful moving is, a paragraph on her fleet, a paragraph on packing tips, and the town name in the headline, the first sentence and the last.

Hold it against the policies, one at a time. It is many pages made mainly to rank, with nothing original on them: scaled content abuse. It is a page per town that funnels everyone into the same quote form, with substantially similar text: doorway abuse, almost word for word Google's own example. And of the 400 towns, Aoife has moved someone to 140. For the other 260 she has no experience at all to put on the page, even if she wanted to.

She says no, and fair play to her. It is the least expensive decision in this article: it costs nothing and saves the domain.

The test: would this page be worth visiting if search did not exist

Google's guidance on helpful content gives a list of warning signs, and two of them read as if they were written for programmatic SEO: "Are you producing lots of content on many different topics in hopes that some of it might perform well in search results?" and "Are you using extensive automation to produce content on many topics?"⁵ The same page sets the standard in plain words: content that is "useful to visitors if they come to your site directly."⁵

Since March 2024 that is not a separate filter: the helpful content system "evolved and became part of" Google's core ranking systems.⁴

That is hard to apply to 140 pages at once, so I turn it into a number I can check on each template. For one page, count the distinct facts it states. Then count how many of them are true of this row and no other:

$$s = \frac{f_{\text{row}}}{f_{\text{total}}}$$

Here $s$ is the specific share of the page, $f_{\text{row}}$ is the number of facts on the page that are true only of this row (this route, this integration, this product), and $f_{\text{total}}$ is the number of facts on the page in total. It is an ordinary ratio of a part to a whole, explained in Foundations: ratios and rates.

This is my own check, not something Google computes. Google has said plainly that it has no preferred word count, so I count facts, not words.⁵ A page with $s$ near zero is the same page as its neighbours with the nouns swapped. A page where most facts belong to the row has a reason to exist.

Worked example: scoring two pages

The agency's Kilkenny page states 14 facts: moving is stressful, Aoife has four vans, she offers packing, and so on. Not one is about Kilkenny, apart from the word itself. So $f_{\text{row}} = 0$, $f_{\text{total}} = 14$, and $s = 0 / 14 = 0$.

Now a page built from Aoife's own table. It states 12 facts: how many households she has moved to Kilkenny (22), the typical price of a move, the range most customers paid, the usual hours on the road, the usual day count, the crew's note about the narrow lanes along the Medieval Mile, which arrival slot works best, that two of the 22 moves needed a parking suspension, and four general facts about her service. Eight facts belong to Kilkenny and four do not, so $f_{\text{row}} = 8$, $f_{\text{total}} = 12$, and $s = 8 / 12 = 0.67$.

Two thirds of that page could not appear anywhere else on her site. That is the difference the spam policies describe, as a number.

Start with the data, not with the keyword list

Most programmatic projects start in a keyword tool: export every "removals Cork to X", build a page per row. That is backwards. Keyword research tells you which rows people care about, never what to say about them. Only data can.

So I start with the question: what table does this business hold that nobody else holds? Good sources, in my experience, in the order I trust them:

1. Your own records: orders, bookings, tickets, test results. Nobody else has them.
2. Your own expertise, written down per row: the crew's notes, the engineer's verdict on each integration, the buyer's opinion of each product.
3. Public data you combine in a way nobody else has, used under its licence and credited.
4. A supplier or manufacturer feed. Useful as a skeleton, but on its own it is exactly what the thin affiliation policy describes, the same text everyone else with that feed already publishes.²

Content copied or lightly reworded from other sites is not a source at all. Google's scraping policy names "Copying content from other sites, modify it only slightly (for example, by substituting synonyms or using automated techniques), and republish it."²

Worked example: what Aoife's table actually supports

Aoife's 1,900 moves are not spread evenly over the 140 destinations. A handful of towns take most of the work and a long tail of towns appear once or twice.

The horizontal axis groups the destination towns by how many moves Aoife has made to each. The vertical axis is how many towns fall in each group. The takeaway is the first bar: more than half of the 140 towns, 78 of them, have seen fewer than five of her moves. Only the last three bars, 17 plus nine plus five, 31 towns in all, have 15 moves or more. The data supports about 31 strong pages, not 140, and certainly not 400.

Set a threshold: a row earns a page only when it has enough to say

A threshold is a rule, decided before any page is built, that says how much data a row needs before it gets a URL. Without one, the long tail of thin rows becomes a long tail of thin pages.

For numbers, the threshold comes from how much a figure would wobble if you had a different handful of moves. That wobble is the standard error of the mean:

$$SE = \frac{s}{\sqrt{n}}$$

Here $SE$ is the standard error of the average price, $s$ is the standard deviation of the individual prices (how far a typical move sits from the average), and $n$ is the number of moves the average is based on. The full derivation, and why a sample average varies at all, is in Foundations: sampling. Roughly 95% of the time, the true average sits within $1.96 \times SE$ of the one you measured, as Foundations: confidence intervals explains.

Worked example: Kilkenny against a town with four moves

Assume, to keep the arithmetic clear, that prices spread by the same amount on every long route, a standard deviation of €310.

Kilkenny has 22 moves at an average of €1,240. So $SE = 310 / \sqrt{22} = 310 / 4.69 = 66$, which is €66. The interval is $1.96 \times 66 = 130$, so €130 either side, and the true average is very likely between €1,110 and €1,370. That is a figure a customer can plan around.

A town with four moves at an average of €1,310 gives $SE = 310 / \sqrt{4} = 310 / 2 = 155$, which is €155. The interval is $1.96 \times 155 = 304$, so €304 either side, from €1,006 to €1,614. Printing "a typical move costs €1,310" on that basis would be telling the customer something Aoife does not know.

At 15 moves, $SE = 310 / \sqrt{15} = 310 / 3.87 = 80$, so €80, and the interval is about €157 either side. Aoife decides that is the least she is willing to print. So the rule is 15 moves or more, and the 31 towns from the chart get pages. The other 109 towns do not get a page of their own. They are named, with a count of moves, on a provincial page, and the quote form works for every town in Ireland either way.

One honest note on the page itself. A confidence interval describes the average, not the spread of individual moves, which is a mistake Foundations warns about. So the page shows the typical price (the median) and the range the middle half of customers paid. The standard error is Aoife's private test for whether the numbers are stable enough to show at all.

The diagram runs top to bottom, one row at a time. The first question is about quantity, the second about substance, and a row must pass both. A row that fails either still appears on its parent page, so nothing a customer needs goes missing. It just does not get a URL of its own.

Build the template from conditional sections, not swapped nouns

A weak template is a paragraph with blanks in it. A strong template is a set of sections, each of which appears only when the row has the data to fill it. Two pages from the same template should differ in shape, not just in nouns.

Worked example: Aoife's route page in Ruby on Rails

Aoife's site runs on Ruby on Rails. Here is the core of a plain Ruby class that decides what a route page can say.

class RoutePage
  MIN_MOVES = 15

  attr_reader :town, :moves

  def initialize(town, moves)
    @town = town
    @moves = moves
  end

  def publishable?
    moves.size >= MIN_MOVES && town.crew_notes.present?
  end

  def median_price
    median(moves.map(&:price))
  end

  def middle_half
    prices = moves.map(&:price).sort
    [percentile(prices, 0.25), percentile(prices, 0.75)]
  end

  def overnight?
    town.km_from_cork > 180
  end

  private

  def median(values)
    sorted = values.sort
    mid = sorted.size / 2
    sorted.size.odd? ? sorted[mid] : (sorted[mid - 1] + sorted[mid]) / 2.0
  end

  def percentile(sorted, share)
    sorted[((sorted.size - 1) * share).round]
  end
end

And the view, in Rails' ERB templates. Every section after the first is conditional:

<h1>Removals from Cork to <%= page.town.name %></h1>

<p>
  Shandon Removals has moved <%= page.moves.size %> households from Cork
  to <%= page.town.name %>. A move there typically costs
  <%= number_to_currency(page.median_price, unit: "€", precision: 0) %>, and half
  of customers paid between
  <%= number_to_currency(page.middle_half.first, unit: "€", precision: 0) %> and
  <%= number_to_currency(page.middle_half.last, unit: "€", precision: 0) %>.
</p>

<% if page.overnight? %>
  <h2>Why this move takes two days</h2>
  <%= render "overnight_explainer", town: page.town %>
<% end %>

<% if page.town.parking_note.present? %>
  <h2>Parking on moving day</h2>
  <p><%= page.town.parking_note %></p>
<% end %>

<h2>What the crew says about <%= page.town.name %></h2>
<p><%= page.town.crew_notes %></p>

Walk it through for two towns. Kilkenny is about 160 km away, so overnight? is false and the two day section does not appear. The crew has a parking note for Kilkenny, so that section does. A town in Co. Donegal, over 300 km away, gets the overnight section and no parking section, because the crew never needed one there. Same template, two pages with different sections and different facts. A town with 12 moves, or with no crew notes yet, fails publishable? and gets no page at all.

Two details matter more than the code. The crew notes come from the crew, one town at a time. And Aoife reads all 31 pages before they go live, which takes an afternoon. Automation builds the pages. A person signs them off.

Give the pages a structure people can browse

Remember the doorway line: "substantially similar pages that are closer to search results than a clearly defined, browseable hierarchy."² The cure is a real hierarchy, the kind a visitor could click through without ever touching a search box.

For Aoife that means three levels. A page for all removals out of Cork. Below it, one page per province, listing every town she has moved people to, with a count of moves and a link where a town has its own page. Below that, the 31 town pages, each linking back up to its province and across to its nearest neighbours. Every town page is reachable in two clicks from the top, and the provincial pages carry the long tail of 109 towns so nothing is hidden.

Inside each page, the same thinking applies at a smaller scale. Clear sections with real headings help readers and help search engines understand which part of the page answers which question. I explained how that works, and what passage ranking does and does not do, in Passage indexing and content fragmentation.

Worked example: one visitor's path

A customer moving to Carlow lands on the Leinster page. Carlow is listed with nine past moves and no page of its own, Kilkenny next to it with 22 moves and a link. She reads the Kilkenny page and asks for a quote for Carlow. The thin row did not need a page. The strong row next to it did the persuading.

Sitemaps and indexing: list only the pages you stand behind

A sitemap is a file that lists the URLs you want a search engine to know about. For a programmatic site it is also a statement: these are the pages I stand behind. The rules that matter here are few. A single sitemap may hold 50,000 URLs or 50MB uncompressed, whichever comes first, and a larger site splits its URLs over several sitemaps listed in a sitemap index file.⁶⁷ Google uses the lastmod date only "if it's consistently and verifiably" accurate, and it should reflect "the last significant update to the page", not a changed copyright line.⁶ And submitting a sitemap "is merely a hint". It does not guarantee anything is crawled or indexed.⁶

Here is one entry of Aoife's sitemap. The lastmod comes from the last time new move data or a new crew note changed the page, not from when the file was generated:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://www.shandonremovals.example/removals/cork-to-kilkenny</loc>
    <lastmod>2026-10-14</lastmod>
  </url>
</urlset>

Only rows that pass publishable? go in. Rows that do not pass never get a URL, so there is nothing to keep out of the index. That is cleaner than publishing thin pages and hiding them with noindex, a rule that tells Google to drop a page from results but only works if Google is allowed to crawl the page and see it.⁸

Crawl budget is not a problem for a site of a few hundred pages. Google writes that guide for sites of roughly a million pages changing weekly, or 10,000 or more changing daily.⁹ Programmatic SEO can turn a small site into a large one overnight, though, so if you are heading past those numbers, read IndexNow, instant indexing and crawl budget strategy before you publish, not after.

Then measure. Search Console's Page indexing report shows which submitted URLs Google indexed and why the others were not. Google itself says "Don't expect every URL on your site to be indexed", and that "Crawled, currently not indexed" is common and not always a problem.¹⁰ But across a set of programmatic pages, the share that makes it in is one of the most honest signals you will get:

$$i = \frac{u_{\text{indexed}}}{u_{\text{submitted}}}$$

Here $i$ is the index rate, $u_{\text{indexed}}$ is the number of URLs from the sitemap that the Page indexing report shows as indexed, and $u_{\text{submitted}}$ is the number of URLs in that sitemap. It is another plain ratio, covered in Foundations: ratios and rates.

Worked example: Aoife's old town pages against the new ones

Two years before this story, a previous developer had built 60 "removals to [town]" pages for Aoife, each a swapped noun template. The Page indexing report shows 11 of them indexed. The rest sit under "Crawled, currently not indexed" or "Duplicate without user selected canonical", which means Google decided another page said the same thing.¹⁰ So $i = 11 / 60 = 0.18$.

Three months after the 31 new pages go live, the report shows 29 of them indexed. So $i = 29 / 31 = 0.94$.

The index rate is not the goal, since an indexed page can still help nobody. But when Google declines four pages in five, it is telling you, as politely as it can, that it has seen these pages before.

Where AI and automation help, and where they stop

Automation runs all through this system, from the queries that build the table to the job that rebuilds pages when new moves come in. AI tools fit too, on the right side of one line. Google's guidance on generative AI is short and worth reading in full. It says AI "can be particularly useful when researching a topic, and to add structure to original content", and in the same breath that generating "many pages without adding value for users may violate" the scaled content abuse policy.¹¹ It also says what to do about accuracy: generative models "don't retrieve facts, but predict a likely sequence of words", so "It is critical to manually factcheck and review all AI generated content", including titles, meta descriptions, structured data and image alt text, because those appear in search too.¹¹

Worked example: the crew's notes

Aoife's crew leads write their notes on a phone in the van, in shorthand. "Medieval Mile end, narrow lanes, book the morning slot or you'll be stuck in traffic." She uses an AI tool to turn each note into two clean sentences in the site's style, with the instruction to change the wording and never the facts. Then she reads each result against the original note before it goes into the table.

The tool did the typing, the crew supplied the knowledge, and Aoife checked the join. Asked instead to "write a paragraph about moving to Kilkenny", the same tool would produce the average of everything ever written about Kilkenny. Same tool, opposite result. The difference is where the facts came from.

Prune what does not earn its place

A programmatic system is never finished. Data changes and some pages never find readers. So every quarter I look at each page's impressions and clicks in Search Console, and at whether it still passes the threshold, and each page gets one of three answers.

Keep it, and refresh it when the data moves. Merge it, when two pages answer the same question for the same people: the weaker page redirects to the stronger with a 301, which Google treats as a strong signal that the target should be the one in its results.¹² Or remove it, returning a 404 or 410, when it has nothing left to say and no natural home to redirect to.

Worked example: the 60 old pages

Aoife's 60 old swapped noun pages have to go somewhere. For the 19 towns that now have a proper new page, the old URL redirects with a 301 to the new one, so any links and bookmarks still land somewhere useful. The other 41 towns have no new page, so their old URLs return 410, gone. She does not dump them all on the home page, which would let down every visitor who asked about one town.

Where programmatic SEO fits, and where it does not

It fits wherever a business holds a table of facts that people search for one row at a time, and the rows really differ. Integration pages for a software product, where each page explains what the connection actually does and how to set it up. Compatibility pages for a parts shop, where each page lists what fits and what does not. Routes, branches with their real stock and hours, prices that were really paid.

It does not fit where only the keyword varies. Town pages for a service that is the same everywhere, "best [product] for [job title]" pages with nothing behind the job title, translated copies of one article for markets you do not serve. If you cannot say what is true of this row and no other, you do not have a programmatic page. You have a doorway.

An example: a year later Aoife considers a second set of pages, "house clearance in [town]". She has done 14 clearances in total, spread over nine towns, so the most any town has is three. Not one row passes the threshold, and the idea goes back in the drawer until the table can carry it. That is the system working, not failing.

The old version of this article opened by saying search engines reward scale. They do not. They reward pages worth visiting, and scale is only good news when each page is. Aoife ended up with 31 pages instead of 400, and every one of them says something no other page on the internet can say. That is the only scale that lasts.

Footnotes

1. Danny Sullivan and Chris Nelson, Google Search's guidance about AI generated content, Google Search Central Blog, 8 February 2023, https://developers.google.com/search/blog/2023/02/google-search-and-ai-content

2. Google, Spam policies for Google web search (sections on doorway abuse, scaled content abuse, scraping and thin affiliation), Google Search Central documentation, accessed 8 November 2026, https://developers.google.com/search/docs/essentials/spam-policies

3. Google Search Central, What web creators should know about our March 2024 core update and new spam policies, 5 March 2024, https://developers.google.com/search/blog/2024/03/core-update-spam-policies

4. Google, A guide to Google Search ranking systems (introduction and helpful content system), Google Search Central documentation, accessed 8 November 2026, https://developers.google.com/search/docs/appearance/ranking-systems-guide

5. Google, Creating helpful, reliable, people first content, Google Search Central documentation, accessed 8 November 2026, https://developers.google.com/search/docs/fundamentals/creating-helpful-content

6. Google, Build and submit a sitemap, Google Search Central documentation, accessed 8 November 2026, https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap

7. Google, Manage your sitemaps with a sitemap index file, Google Search Central documentation, accessed 8 November 2026, https://developers.google.com/search/docs/crawling-indexing/sitemaps/large-sitemaps

8. Google, Block Search indexing with noindex, Google Search Central documentation, accessed 8 November 2026, https://developers.google.com/search/docs/crawling-indexing/block-indexing

9. Google, Crawl budget management for large sites, Google Search Central documentation, accessed 8 November 2026, https://developers.google.com/search/docs/crawling-indexing/large-site-managing-crawl-budget

10. Google, Page indexing report, Search Console Help, accessed 8 November 2026, https://support.google.com/webmasters/answer/7440203

11. Google, Google Search's guidance on using generative AI content on your website, Google Search Central documentation, accessed 8 November 2026, https://developers.google.com/search/docs/fundamentals/using-gen-ai-content

12. Google, How to specify a canonical URL with rel="canonical" and other methods, Google Search Central documentation, accessed 8 November 2026, https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls