Building custom AI recommendations in Solidus

What Solidus gives you, and what it leaves to you

Solidus is the open source eCommerce framework on Ruby on Rails that I build on. It gives you a solid core: products, variants, orders, payments, shipping, promotions. It does not give you a recommendation engine.

What it does give you is close to the edge of one. The starter frontend, the storefront the Solidus installer copies into your own application, shows a "You May Also Like" row on every product page.¹ Behind it sits a small override on Spree::Product: take the product's categories (Solidus calls them taxons), collect the other products in them, keep the first three.² That is honest and useful, and it knows nothing about what people actually buy.

Solidus also does not record what people look at. Its event bus announces things like a finalised order, a shipped carton or a cancelled order, and nothing about a product view.³ So the signals a recommender learns from are yours to collect, and the intelligence on top is yours to build.

I like that. A recommender that only counts which products share a basket, without asking why, is not enough. Building your own means you know what it does, you own the data, and you tune it to your shop.

Worked example: the shop in this article

Marina Tea is an invented tea merchant in Cork, run by Niamh on a Solidus shop at marinatea.example. It sells 240 products: loose leaf teas, teapots, infusers, mugs and gift tins. Over the last six months it took 9,000 orders, at about €31 each.

Today, a customer looking at Earl Grey sees three other products from the "Black tea" taxon. Niamh's question is the one this article answers: can the shop show the infuser that Earl Grey buyers actually go on to buy, and show a returning customer the teas that suit them, without breaking the law or the site?

Three ways to recommend, and the question each one answers

There are three classic approaches, and each answers a different question.⁴⁵

Co occurrence asks: what tends to be bought together with this product? It needs only orders.

Collaborative filtering asks: what did people with similar behaviour go on to like? It learns from everyone's behaviour at once, and it knows nothing about the products themselves.

Content based filtering asks: what is described like the things this person liked? It reads the catalogue, and it knows nothing about other customers.

A hybrid blends the second and third, because each covers the other's blind spot. Collaborative filtering cannot recommend a product nobody has touched yet. Content based filtering never strays far from what the customer already knows.⁶⁷ The rest of this article builds all of it, in that order.

Start with what is bought together, and correct for popularity

Co occurrence is where Amazon's well known item to item approach starts: for each product, find the products that customers tend to purchase together, compute a similarity, and build a table of similar items in advance.⁴ The naive version just counts. The problem is that counting rewards whatever is popular. Every product looks related to the bestseller, because the bestseller is in everyone's basket.

The fix is to compare the share against the base rate.⁸ The confidence of the rule "bought $A$, so also bought $B$" is a conditional probability, and its interest is how far that probability sits above the share of all orders that contain $B$ anyway:

$$\text{conf}(A \to B) = P(B \mid A) = \frac{n_{AB}}{n_A}, \qquad \text{interest}(A \to B) = P(B \mid A) - P(B)$$

Here $n_A$ is the number of orders containing product $A$, $n_{AB}$ the number containing both $A$ and $B$, $P(B \mid A)$ the share of $A$ orders that also contain $B$, and $P(B)$ the share of all orders that contain $B$. An interest near zero means $B$ turns up with $A$ no more often than it turns up anywhere. The ideas underneath are in Foundations: probability and Foundations: base rates.

Worked example: the infuser and the gift bag

At Marina Tea, Earl Grey is in 1,200 of the 9,000 orders. The glass infuser mug is in 900 orders, and 300 of them also contain Earl Grey. The paper gift bag is in 2,700 orders, and 420 of them contain Earl Grey.

Confidence for the infuser: $300 / 1{,}200 = 0.25$. For the gift bag: $420 / 1{,}200 = 0.35$. Counting alone says the gift bag is the better recommendation.

Now the base rates. The infuser is in $900 / 9{,}000 = 0.10$ of all orders, the gift bag in $2{,}700 / 9{,}000 = 0.30$.

Interest for the infuser: $0.25 - 0.10 = 0.15$. For the gift bag: $0.35 - 0.30 = 0.05$. The gift bag is simply popular. The infuser is what Earl Grey buyers specifically go on to want. That is the row worth showing.

Collaborative filtering learns a taste for every customer

Co occurrence works product by product. Collaborative filtering goes one step further and learns a short list of numbers for every customer and every product, so that a customer's numbers line up with the products they would like. Those lists are called embeddings, or latent factors: points in a shared space, where things consumed together sit close.⁵ The method became famous through the Netflix Prize, and Koren, Bell and Volinsky describe two ways to fit it: stochastic gradient descent and alternating least squares, ALS for short.⁹

Shop data is implicit. People rarely rate a teapot, they view it, add it to the basket, buy it or ignore it. Hu, Koren and Volinsky adapted the method for exactly that.¹⁰ Every customer and product pair gets a preference $p_{ui}$, 1 if the customer interacted with the product at all and 0 otherwise, and a confidence that grows with the evidence:

$$c_{ui} = 1 + \alpha r_{ui}, \qquad \min_{x, y} \sum_{u, i} c_{ui} \left( p_{ui} - x_u^{\top} y_i \right)^2 + \lambda \left( \sum_u \lVert x_u \rVert^2 + \sum_i \lVert y_i \rVert^2 \right)$$

Here $u$ is a customer and $i$ a product, $r_{ui}$ the observed interaction strength, $\alpha$ how fast confidence grows with it, and $c_{ui}$ the resulting confidence. $x_u$ is the customer's vector of factors and $y_i$ the product's, and $x_u^{\top} y_i$ their dot product, the model's predicted preference. $\lambda$ is the regularisation weight that keeps the vectors small, and $\lVert x_u \rVert^2$ the squared length of a vector. ALS fixes all product vectors and solves for the customers, then fixes the customers and solves for the products, and repeats.¹⁰ The vector and matrix basics are in Foundations: matrices.

Worked example: one customer, three products

Niamh's tracker gives a purchase a weight of 5 and a product view a weight of 1 (the example's own weights, set out in the next section). A customer in Douglas bought Earl Grey once and looked at it twice, so $r_{ui} = 5 + 1 + 1 = 7$. With $\alpha = 1$, the default in the library used below, the confidence is $c_{ui} = 1 + 1 \times 7 = 8$. For a teapot she never touched, $r_{ui} = 0$, $p_{ui} = 0$ and $c_{ui} = 1$. The model cares eight times as much about getting Earl Grey right.

Now pretend the model learned just two factors. Her vector is $x_u = (0.9, 0.2)$. Earl Grey is $(0.8, 0.1)$, the infuser mug $(0.7, 0.3)$, the matcha whisk $(0.1, 0.9)$.

Earl Grey: $0.9 \times 0.8 + 0.2 \times 0.1 = 0.74$. Infuser: $0.9 \times 0.7 + 0.2 \times 0.3 = 0.69$. Whisk: $0.9 \times 0.1 + 0.2 \times 0.9 = 0.27$. She already has Earl Grey, so it is filtered out, and the infuser leads. Nobody told the model that an infuser goes with loose leaf tea. It learned it from what everyone else did.

I use ALS rather than a deep learning model for most Solidus shops. ALS handles implicit, sparse data well, and any one customer has touched a tiny fraction of the catalogue. Neural approaches can do better, but they want far more data and compute than a shop of Niamh's size has.

Content based filtering reads the catalogue

Content based filtering turns each product's text into a vector and recommends the nearest ones. The standard recipe is TF IDF, short for term frequency, inverse document frequency: a word scores high for a product when it is frequent in that product's text and rare across the catalogue. Two vectors are then compared by the angle between them:

$$\cos(\mathbf{a}, \mathbf{b}) = \frac{\mathbf{a} \cdot \mathbf{b}}{\lVert \mathbf{a} \rVert \, \lVert \mathbf{b} \rVert}$$

Here $\mathbf{a}$ and $\mathbf{b}$ are the two products' word vectors, $\mathbf{a} \cdot \mathbf{b}$ their dot product, and $\lVert \mathbf{a} \rVert$ and $\lVert \mathbf{b} \rVert$ their lengths. The result runs from 0, nothing in common, to 1, the same direction.⁵ The scikit-learn vectoriser scales every row to length 1 by default, so the cosine is just the dot product.¹¹ The vector arithmetic is in Foundations: matrices.

It works from the first day, for a new customer as soon as they view one product, and for a new product as soon as it is in the catalogue. Its weakness is that it can only build on interests the customer already has.⁶ And it is only as good as the product text. "Black tea. 100g." gives it nothing to hold on to. Tasting notes, origin, strength and occasion give it plenty. The same richer data helps AI assistants find your products, which is the subject of how generative AI is changing product discovery.

Worked example: three teas as word counts

Take three words, "assam", "breakfast" and "green", and count them in three descriptions. Irish Breakfast is $\mathbf{a} = (2, 1, 0)$, Assam Gold is $\mathbf{b} = (1, 1, 0)$, and Sencha is $\mathbf{s} = (0, 0, 3)$.

Irish Breakfast and Assam Gold: the dot product is $2 \times 1 + 1 \times 1 + 0 \times 0 = 3$. The lengths are $\sqrt{5} = 2.236$ and $\sqrt{2} = 1.414$. The cosine is $3 / (2.236 \times 1.414) = 0.949$, very similar.

Irish Breakfast and Sencha: the dot product is $0$, so the cosine is 0. A new Assam added tomorrow gets recommended next to Irish Breakfast on its first day, before anyone has bought it.

Record the signals first, and only with a yes

Recommendations are only as good as the signals you feed them, and Solidus records orders, not views.³ So the first job is a small event table. I keep it deliberately thin: who, which product, what kind of event, how strong, when. No referrer, no free text, nothing the model does not use.

# db/migrate/20260223090000_create_recommendation_tables.rb
class CreateRecommendationTables < ActiveRecord::Migration[7.1]
  def change
    create_table :recommendation_interaction_events do |t|
      t.references :user, null: false, index: true
      t.references :product, null: false, index: true
      t.string :event_type, null: false
      t.decimal :weight, precision: 4, scale: 2, null: false
      t.timestamps
    end

    create_table :recommendation_item_scores do |t|
      t.references :product, null: false, index: true
      t.bigint :recommended_product_id, null: false
      t.float :score, null: false
      t.string :model_version, null: false, index: true
      t.timestamps
    end

    create_table :recommendation_user_scores do |t|
      t.references :user, null: false, index: true
      t.references :product, null: false
      t.float :score, null: false
      t.string :model_version, null: false, index: true
      t.timestamps
    end

    # spree_users with solidus_auth_devise; use your own user table otherwise
    add_column Spree.user_class.table_name, :recommendations_consent, :boolean,
      default: false, null: false
  end
end

# app/models/recommendations.rb and friends
module Recommendations
  class InteractionEvent < ApplicationRecord
    self.table_name = "recommendation_interaction_events"
  end

  class ItemScore < ApplicationRecord
    self.table_name = "recommendation_item_scores"
  end

  class UserScore < ApplicationRecord
    self.table_name = "recommendation_user_scores"
  end

  class EventTracker
    # Implicit feedback weights: a stronger signal of interest gets more.
    WEIGHTS = { purchase: 5.0, add_to_cart: 3.0, product_view: 1.0 }.freeze

    def track(event_type:, product:, user:)
      return unless user&.recommendations_consent?

      InteractionEvent.create!(
        user_id: user.id,
        product_id: product.id,
        event_type: event_type.to_s,
        weight: WEIGHTS.fetch(event_type)
      )
    end

    # One row per customer and product, summed, for training.
    def interaction_rows(since: 6.months.ago)
      InteractionEvent
        .where(created_at: since..)
        .group(:user_id, :product_id)
        .sum(:weight)
        .map { |(user_id, product_id), weight| [user_id, product_id, weight.to_f] }
    end
  end
end

Two things changed from the first version of this article. The user check used Spree::User, which only exists with one particular authentication gem. Solidus lets you bring your own user class, so the code asks the user object itself.¹² And anonymous visitors are no longer tracked at all, which is the subject of the section on consent below.

Worked example: one week of Niamh's signals

A signed in customer who has said yes views the Darjeeling page three times, adds it to the basket once and buys it. The table gets five rows. Summed for training: $3 \times 1 + 1 \times 3 + 1 \times 5 = 11$. Another customer views it once and leaves: 1. The model sees the first as a strong preference and the second as a faint one. These weights are the example's starting point, a judgement call, and the A/B test later is how to find out whether they work.

Hook into the product page, the basket and the finished order

With the starter frontend the storefront controllers live in your own application, so there is nothing to decorate. You edit ProductsController directly.¹ The consent check sits in StoreController, which every storefront controller inherits from.

# app/controllers/store_controller.rb (add to the existing class)
helper_method :personalisation_consent?

# Your consent banner decides the cookie name and value. A signed in
# customer's choice is also stored on the account, for background jobs.
def personalisation_consent?
  spree_current_user&.recommendations_consent? &&
    cookies[:consent_personalisation] == "granted"
end

# app/controllers/products_controller.rb (add to the existing class)
after_action :track_product_view, only: :show

private

def track_product_view
  return unless personalisation_consent?

  Recommendations::EventTracker.new.track(
    event_type: :product_view, product: @product, user: spree_current_user
  )
end

# app/controllers/cart_line_items_controller.rb (add to the existing class)
after_action :track_cart_add, only: :create

private

def track_cart_add
  return unless personalisation_consent? && @line_item && @order.errors.none?

  Recommendations::EventTracker.new.track(
    event_type: :add_to_cart, product: @line_item.product, user: spree_current_user
  )
end

Purchases come from the event bus. A Solidus subscriber includes Omnes::Subscriber, names the event it handles, and reads the order from the payload. It also has to be subscribed to Spree::Bus in an initializer, inside to_prepare so it survives code reloading.¹³ The first version of this article left that registration out, and without it the subscriber never runs.

# app/subscribers/recommendations/order_subscriber.rb
module Recommendations
  class OrderSubscriber
    include Omnes::Subscriber

    handle :order_finalized,
      with: :track_purchases,
      id: :recommendations_track_purchases

    def track_purchases(event)
      order = event.payload[:order]
      return unless order.user

      tracker = EventTracker.new
      order.line_items.each do |item|
        tracker.track(event_type: :purchase, product: item.product, user: order.user)
      end
    end
  end
end

# config/initializers/omnes.rb
Rails.application.config.to_prepare do
  Recommendations::OrderSubscriber.new.subscribe_to(Spree::Bus)
end

The guides say to prefer the event bus and the built in hooks over overriding Solidus classes, and to keep overrides small and descriptively named when you need one.¹⁴ This setup needs none.

Train both models in one Python script

The training runs as a batch job, not per page view. A Rake task writes two CSV files, the script trains on them and writes two more, and Rails imports the results. Calling Python on every page load adds latency for nothing, and the first version of this article admitted as much in its pitfalls while its code did exactly that.

The script uses the implicit library for ALS, which implements the Hu, Koren and Volinsky method,¹⁵ and scikit-learn for TF IDF. One correction matters here. Since implicit 0.5, fit takes a matrix with customers as rows and products as columns.¹⁶ The first version transposed the matrix before fitting, which on any current version trains the model on the wrong axis.

"""Train the collaborative and the content model, write recommendation tables.

Usage: python3 lib/recommendations/train.py tmp/recs
Reads interactions.csv (user_id, product_id, weight) and products.csv
(product_id, text). Writes item_recs.csv and user_recs.csv.
Tested with implicit 0.7, scikit-learn 1.9 and pandas 3.1.
"""
import sys
from pathlib import Path

import numpy as np
import pandas as pd
from implicit.als import AlternatingLeastSquares
from scipy.sparse import csr_matrix
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import linear_kernel

N = 12


def train_als(interactions, factors=50, iterations=30, regularization=0.01, alpha=1.0):
    users = interactions["user_id"].unique()
    items = interactions["product_id"].unique()
    user_pos = pd.Series(np.arange(len(users)), index=users)
    item_pos = pd.Series(np.arange(len(items)), index=items)
    user_items = csr_matrix(
        (
            interactions["weight"].to_numpy(dtype=np.float32),
            (
                user_pos[interactions["user_id"]].to_numpy(),
                item_pos[interactions["product_id"]].to_numpy(),
            ),
        ),
        shape=(len(users), len(items)),
    )
    model = AlternatingLeastSquares(
        factors=factors,
        iterations=iterations,
        regularization=regularization,
        alpha=alpha,
        use_gpu=False,
        random_state=42,
    )
    # Since implicit 0.5, fit takes users as rows and products as columns.
    # The old advice to transpose the matrix first now trains on the wrong axis.
    model.fit(user_items, show_progress=False)
    return model, users, items, user_items


def als_user_recs(model, users, items, user_items, n=N):
    ids, scores = model.recommend(
        np.arange(len(users)), user_items, N=n, filter_already_liked_items=True
    )
    rows = [
        (users[u], items[i], float(s))
        for u, (id_row, score_row) in enumerate(zip(ids, scores))
        for i, s in zip(id_row, score_row)
        if i >= 0 and np.isfinite(s)
    ]
    return pd.DataFrame(rows, columns=["user_id", "product_id", "score"])


def als_similar(model, items, n=N):
    ids, scores = model.similar_items(np.arange(len(items)), N=n + 1)
    rows = [
        (items[i], items[j], float(s))
        for i, (id_row, score_row) in enumerate(zip(ids, scores))
        for j, s in zip(id_row, score_row)
        if j >= 0 and j != i
    ]
    return pd.DataFrame(rows, columns=["product_id", "recommended_id", "score"])


def content_similar(products, n=N):
    tfidf = TfidfVectorizer(
        stop_words="english", ngram_range=(1, 2), min_df=2, max_df=0.95, sublinear_tf=True
    )
    matrix = tfidf.fit_transform(products["text"])
    ids = products["product_id"].to_numpy()
    rows = []
    for start in range(0, matrix.shape[0], 1000):
        # Rows are L2 normalised, so the dot product is the cosine similarity.
        block = linear_kernel(matrix[start:start + 1000], matrix)
        for offset, sims in enumerate(block):
            i = start + offset
            sims[i] = -1.0  # never recommend the product itself
            for j in np.argsort(-sims)[:n]:
                if sims[j] > 0:
                    rows.append((ids[i], ids[j], float(sims[j])))
    return pd.DataFrame(rows, columns=["product_id", "recommended_id", "score"])


def normalise(df, key):
    df = df[df["score"] > 0].copy()
    df["score"] = df["score"] / df.groupby(key)["score"].transform("max")
    return df


def blend(collab, content, key, w_collab=0.6, w_content=0.4, n=N):
    pair = [key, "recommended_id"]
    merged = normalise(collab, key).merge(
        normalise(content, key), on=pair, how="outer", suffixes=("_cf", "_cb")
    ).fillna(0.0)
    # Fewer than five collaborative candidates: lean on content instead.
    thin = merged.groupby(key)["score_cf"].transform(lambda s: (s > 0).sum() < 5)
    wc = np.where(thin, 0.3, w_collab)
    wt = np.where(thin, 0.7, w_content)
    merged["score"] = wc * merged["score_cf"] + wt * merged["score_cb"]
    merged = merged.sort_values([key, "score"], ascending=[True, False])
    return merged.groupby(key).head(n)[pair + ["score"]]


def main(folder):
    folder = Path(folder)
    interactions = pd.read_csv(folder / "interactions.csv")
    products = pd.read_csv(folder / "products.csv").fillna("")

    model, users, items, user_items = train_als(interactions)
    content = content_similar(products)

    item_recs = blend(als_similar(model, items), content, "product_id")
    item_recs.to_csv(folder / "item_recs.csv", index=False)

    # Content candidates for a person: products similar to their five strongest.
    strongest = interactions.sort_values("weight", ascending=False).groupby("user_id").head(5)
    by_content = (
        strongest.merge(content, on="product_id")
        .groupby(["user_id", "recommended_id"], as_index=False)["score"].sum()
    )
    seen = interactions.rename(columns={"product_id": "recommended_id"})[["user_id", "recommended_id"]]
    by_content = by_content.merge(seen, how="left", indicator=True)
    by_content = by_content[by_content["_merge"] == "left_only"].drop(columns="_merge")

    collab = als_user_recs(model, users, items, user_items).rename(
        columns={"product_id": "recommended_id"}
    )
    user_recs = blend(collab, by_content, "user_id").rename(
        columns={"recommended_id": "product_id"}
    )
    user_recs.to_csv(folder / "user_recs.csv", index=False)
    print(f"{len(users)} customers, {len(items)} products with interactions, "
          f"{len(products)} products in the catalogue")


if __name__ == "__main__":
    main(sys.argv[1])

Try it on a small made up file first, say 800 customers and 240 products. The products nobody has touched still get recommendations, from their text alone, which is the cold start case the hybrid exists for.

Blend the two so each covers the other's gap

The blend function does the hybrid. It scales each method's scores so the best candidate for a product or a customer is 1, then adds them with weights: 0.6 for collaborative and 0.4 for content. Where collaborative filtering has fewer than five candidates, the weights swap to 0.3 and 0.7, because its few scores are not worth trusting yet. Those weights are where I start, not a law. The A/B test decides.

Worked example: an old favourite and a new arrival

For the Earl Grey page, the infuser mug scores 1.0 from collaborative filtering and 0.5 from content: $0.6 \times 1.0 + 0.4 \times 0.5 = 0.80$. A new Lady Grey, added last week, has no collaborative score at all but the best content score, 1.0: $0.6 \times 0 + 0.4 \times 1.0 = 0.40$. The infuser leads and the newcomer still makes the row.

Now a product with only three collaborative candidates. The weights become 0.3 and 0.7. The infuser: $0.3 \times 1.0 + 0.7 \times 0.5 = 0.65$. Lady Grey: $0.3 \times 0 + 0.7 \times 1.0 = 0.70$. When the purchase data is thin, the catalogue gets the louder voice.

Serve from Rails with a version, a cache and fallbacks

Rails reads the imported tables. Every import gets a new model version, the cache key includes it, and so a retrain never needs a cache purge. The first version of this article called Rails.cache.delete_matched, which not every cache store supports, and several helper methods it never defined. This version is complete.

# app/models/recommendations/engine.rb
module Recommendations
  Result = Struct.new(:product_ids, :strategy, :model_version)

  class Engine
    def initialize(limit: 8)
      @limit = limit
      @version = ItemScore.maximum(:model_version)
    end

    def for_product(product)
      ids = cached(:product, product.id) do
        ItemScore.where(product_id: product.id, model_version: @version)
          .order(score: :desc).limit(@limit).pluck(:recommended_product_id)
      end
      return Result.new(ids, "similar_products", @version) if ids.size >= @limit / 2

      # similar_products comes from the starter frontend's override
      taxon_ids = product.similar_products(@limit).map(&:id)
      Result.new((ids + taxon_ids).uniq.first(@limit), "taxon_fallback", @version)
    end

    def for_user(user, consent:)
      return popular unless user && consent

      ids = cached(:user, user.id) do
        UserScore.where(user_id: user.id, model_version: @version)
          .order(score: :desc).limit(@limit).pluck(:product_id)
      end
      ids.size >= @limit / 2 ? Result.new(ids, "personalised", @version) : popular
    end

    def for_cart(order)
      return popular if order.nil? || order.line_items.empty?

      in_cart = order.line_items.map { |item| item.variant.product_id }
      ids = ItemScore.where(product_id: in_cart, model_version: @version)
        .where.not(recommended_product_id: in_cart)
        .group(:recommended_product_id)
        .order(Arel.sql("MAX(score) DESC"))
        .limit(@limit)
        .pluck(:recommended_product_id)
      ids.any? ? Result.new(ids, "cart_complementary", @version) : popular
    end

    # Bestsellers of the last 30 days. Product#orders runs through every
    # variant including the master, so single variant products count too.
    def popular
      ids = cached(:popular, :all) do
        Spree::Product.joins(:orders)
          .where(spree_orders: { completed_at: 30.days.ago.. })
          .group("spree_products.id")
          .order(Arel.sql("COUNT(spree_line_items.id) DESC"))
          .limit(@limit)
          .pluck("spree_products.id")
      end
      Result.new(ids, "popularity", @version)
    end

    private

    def cached(kind, id, &block)
      Rails.cache.fetch(["recs", @version, kind, id, @limit], expires_in: 1.hour, &block)
    end
  end
end

The popularity query fixes two quiet bugs from the first version. It joined variants, which in Solidus excludes the master variant, so a product sold without options never counted as a bestseller. Product#orders goes through variants_including_master instead.¹⁷ And it filtered on orders, which is not the joined table's name. Rails needs spree_orders there.

Worked example: three visitors on one afternoon

A first time visitor lands on the Earl Grey page. Item scores exist, eight of them, so the row is "similar products", the same for everyone, built from what all customers bought.

A minute later someone opens Lady Grey, imported yesterday, with two item scores. Two is fewer than half of eight, so the starter frontend's taxon products fill the row up.

A signed in customer who said yes to personalisation opens the homepage. She has 12 user scores, so she sees her own row. Her neighbour, signed in but without that yes, sees the bestsellers.

The diagram reads from the top: consent comes first, collection second, and the heavy work happens once a week, away from the customer. The decision box is the part shoppers feel. Either the model has enough to say, or the page falls back to something sensible, and never to an empty row.

Render the row without slowing the page

A helper turns the engine's result into products, loads them in one query with their images, and drops anything no longer available. The first version preloaded images on the product, but in Solidus images is delegated to the master variant, not an association, so Rails cannot preload it by that name.¹⁷

# app/helpers/recommendations_helper.rb
module RecommendationsHelper
  def recommended_products(context:, limit: 8)
    engine = Recommendations::Engine.new(limit: limit)
    result =
      case context
      when :product_page then engine.for_product(@product)
      when :cart_page then engine.for_cart(current_order)
      when :homepage then engine.for_user(spree_current_user, consent: personalisation_consent?)
      end
    return [] if result.nil? || result.product_ids.empty?

    products = Spree::Product.where(id: result.product_ids)
      .includes(master: :images).index_by(&:id)

    result.product_ids.filter_map do |id|
      product = products[id]
      next unless product&.available?

      { product: product, strategy: result.strategy, model_version: result.model_version }
    end
  end
end

<%# app/views/products/_recommendations.html.erb %>
<% recommendations = recommended_products(context: context) %>
<% if recommendations.any? %>
  <section class="recommendations"
           data-strategy="<%= recommendations.first[:strategy] %>"
           data-model-version="<%= recommendations.first[:model_version] %>">
    <h2><%= title %></h2>
    <ul>
      <% recommendations.each do |rec| %>
        <% product = rec[:product] %>
        <li>
          <%= link_to product_path(product, rec: rec[:strategy]) do %>
            <% image = product.images.first %>
            <%= image_tag(image.url(:small), alt: "") if image %>
            <span><%= product.name %></span>
            <span><%= product.display_price %></span>
          <% end %>
        </li>
      <% end %>
    </ul>
  </section>
<% end %>

The starter frontend's routes live in your application, so the link is product_path, not spree.product_path. The image has empty alt text on purpose: the product name is right next to it inside the same link, so a screen reader would otherwise read it twice.

Prove it with an A/B test before you trust it

Never ship recommendations without measuring them. Build the test in from the first day. Here the test compares the starter frontend's taxon row (control) with the hybrid row, for signed in customers who said yes. Assignment hashes the customer id, so the same person always sees the same version.

# app/models/recommendations/experiment.rb
module Recommendations
  class Experiment
    NAME = "product_page_recs"
    VARIANTS = { control: 0.5, hybrid: 0.5 }.freeze

    def self.variant_for(user)
      bucket = Digest::SHA256.hexdigest("#{NAME}:#{user.id}").to_i(16) % 1000 / 1000.0
      cumulative = 0.0
      VARIANTS.each do |variant, share|
        cumulative += share
        return variant if bucket < cumulative
      end
      :control
    end

    # Needs a table recommendation_ab_events (experiment, variant, event, user_id)
    class Event < ApplicationRecord
      self.table_name = "recommendation_ab_events"
    end

    def self.log(user:, event:)
      Event.create!(experiment: NAME, variant: variant_for(user).to_s,
        event: event.to_s, user_id: user.id)
    end

    def self.results(since: 30.days.ago)
      counts = Event.where(experiment: NAME, created_at: since..)
        .group(:variant, :event).distinct.count(:user_id)
      VARIANTS.keys.map do |variant|
        seen = counts.fetch([variant.to_s, "impression"], 0)
        bought = counts.fetch([variant.to_s, "purchase"], 0)
        { variant: variant, customers: seen, purchasers: bought,
          rate: seen.zero? ? 0.0 : bought.fdiv(seen) }
      end
    end
  end
end

Call Experiment.log(user:, event: :impression) where the row renders and event: :purchase in the order subscriber. To decide whether a difference is real, compare the two purchase rates with a two proportion test:

$$z = \frac{p_2 - p_1}{\sqrt{\bar{p}(1 - \bar{p})\left(\frac{1}{n_1} + \frac{1}{n_2}\right)}}$$

Here $p_1$ and $p_2$ are the purchase rates in control and in the hybrid, $n_1$ and $n_2$ the number of customers in each, and $\bar{p}$ the pooled rate of both groups together. A $z$ beyond 1.96 either way is significant at 5%. The reasoning, and why to fix the sample size before the test starts, are in Foundations: hypothesis tests and Foundations: sample size.

Worked example: eight weeks at Marina Tea

1,600 of Niamh's consenting customers visit in eight weeks, 800 in each version. 160 buy in control, 192 with the hybrid. The rates are $p_1 = 160 / 800 = 0.20$ and $p_2 = 192 / 800 = 0.24$. The pooled rate is $\bar{p} = 352 / 1{,}600 = 0.22$.

The standard error: $\sqrt{0.22 \times 0.78 \times (1/800 + 1/800)} = \sqrt{0.1716 \times 0.0025} = 0.0207$.

So $z = (0.24 - 0.20) / 0.0207 = 1.93$, a two sided p value of about 0.053. Tempting, and not yet significant at 5%. The hybrid looks better, and a shop this size cannot tell it from luck in eight weeks. Niamh does not call it early. She keeps the test running to the sample size she fixed before it started, because stopping on the first good looking day is how tests lie.

Retrain on a schedule, not on a whim

One Rake task does the round trip: export, train, import. I run it weekly for most shops, daily for a shop with high volume and fast changing stock.

# lib/tasks/recommendations.rake
require "csv"

namespace :recommendations do
  desc "Export, train and import recommendation models"
  task retrain: :environment do
    dir = Rails.root.join("tmp/recs")
    FileUtils.mkdir_p(dir)

    CSV.open(dir.join("interactions.csv"), "w") do |csv|
      csv << %w[user_id product_id weight]
      Recommendations::EventTracker.new.interaction_rows.each { |row| csv << row }
    end

    CSV.open(dir.join("products.csv"), "w") do |csv|
      csv << %w[product_id text]
      Spree::Product.includes(:taxons).find_each do |product|
        description = ActionController::Base.helpers.strip_tags(product.description.to_s)
        csv << [product.id, [product.name, description, product.taxons.map(&:name).join(" ")].join(" ").squish]
      end
    end

    system("python3", Rails.root.join("lib/recommendations/train.py").to_s, dir.to_s, exception: true)
    Recommendations::Importer.new(dir).call
  end
end

# app/models/recommendations/importer.rb
require "csv"

module Recommendations
  class Importer
    def initialize(dir)
      @dir = Pathname(dir)
      @version = Time.current.strftime("%Y%m%d%H%M")
    end

    def call
      ActiveRecord::Base.transaction do
        load_rows(ItemScore, "item_recs.csv") do |row|
          { product_id: row["product_id"].to_i,
            recommended_product_id: row["recommended_id"].to_i,
            score: row["score"].to_f }
        end
        load_rows(UserScore, "user_recs.csv") do |row|
          { user_id: row["user_id"].to_i, product_id: row["product_id"].to_i,
            score: row["score"].to_f }
        end
        ItemScore.where.not(model_version: @version).delete_all
        UserScore.where.not(model_version: @version).delete_all
      end
    end

    private

    def load_rows(model, file)
      now = Time.current
      CSV.foreach(@dir.join(file), headers: true).each_slice(5_000) do |batch|
        model.insert_all(batch.map { |row| yield(row).merge(model_version: @version, created_at: now, updated_at: now) })
      end
    end
  end
end

The import runs in one transaction, so a page never sees half a model. The new version string sorts after the old one, the engine picks it up, and the old cache entries simply stop being asked for.

The cold start problem, honestly

This is the hardest problem in recommendation systems, and the code above only softens it.

A new customer with no history gets bestsellers, or the item to item row on whatever product they are looking at. If they arrived from a category or a campaign, that context is the best clue you have. None of this is personal. It is just better than random.

A new product has no interactions, so collaborative filtering cannot place it: there is no embedding to look up.⁷ Content based filtering can, from its text, and the blend gives it a place in the row from the first day.

A small catalogue, under about 100 products, gives ALS too little to find patterns in. For a very small range, a hand picked "goes well with" list may beat any model. Do not take a machine learning sledgehammer to a walnut.

Seasonal ranges reset every season, and last winter's product ids say nothing about this winter's. Content based filtering copes better, because new products are placed by their attributes.

My rule of thumb for collaborative filtering: at least 500 to 1,000 orders across 50 or more products with tracking before the first training run. Below that, start with co occurrence and content.

Worked example: new gift tins before Christmas

In October Niamh adds 30 Christmas gift tins. On day one, none has a single order. Each gets its row from content similarity ("spiced", "gift", "black tea"), weighted 0.7 because collaborative filtering has nothing. By mid November, after the weekly retrains, the best sellers among them have enough orders for collaborative scores, and the weights move back to 0.6 and 0.4 on their own.

Five pitfalls I watch for

Popularity bias. Collaborative filtering recommends popular products more, because they appear in more patterns. Research on it finds recommendations concentrated on popular items even for customers whose own taste runs to the long tail.¹⁸ Watch the share of your catalogue that ever appears in a row, and show a less popular item now and then on purpose.

The filter bubble. Content based filtering keeps a running shoe buyer among running shoes. The hybrid softens it, it does not remove it.

Latency. That is why everything heavy runs weekly and the page only reads a table through a cache.

Odd correlations. The model finds patterns, not reasons, and sometimes the pattern is nonsense. Correlation is not causation. Have a person look at a sample of rows after every retrain.

Offline scores flatter. Precision, recall and their cousins measured on past data are an imperfect stand in for what happens live,¹⁹ and in one simulation study better offline scores brought diminishing returns online.²⁰ Trust the A/B test, not the notebook.

Consent comes before the first tracked view

This part is short, because it is covered properly elsewhere. For an Irish shop two sets of rules meet here.

The first is regulation 5(3) of the ePrivacy Regulations 2011: storing information on a visitor's device, or reading information already stored there, needs the visitor's consent and clear, comprehensive information about the purpose.²¹ The only way round it is regulation 5(5): storage purely to carry a communication, or what is strictly necessary for a service the user explicitly asked for.²¹ The Data Protection Commission (DPC) draws that line with examples that fit this case closely. A session cookie that keeps track of the items in a basket is strictly necessary and needs no consent. A cookie that recognises a visitor's browser over time to remember their preferences for them needs consent, however helpful it may be.²² Analytics cookies need consent as well, so there is no measurement exception to lean on.²² A row chosen from what the shop has learned about someone over many visits is the remembered preference, not the basket.

So the item to item row, the same for everyone, needs no profile. The personal row needs a yes. And the Solidus guest_token cookie, which keeps the basket working for a guest,²³ is not a free tracking id: it was set for the basket, the one purpose that can make it strictly necessary, and following someone's browsing with it is a different purpose. That is why the code above tracks only signed in customers who said yes, and nobody else.

The second is the GDPR. A taste vector built from someone's behaviour is profiling, which the law defines as using personal data to analyse or predict things like personal preferences, interests or behaviour.²⁴ The DPC makes the same link from the cookie side: information derived from tracking technologies that is used to profile people is personal data, and the GDPR applies to it.²² The first version of this article filed recommendations under Article 22. That was wrong. Article 22 is about decisions based solely on automated processing, including profiling, that produce legal effects or similarly significantly affect the person.²⁵ A tea carousel is not that. What does apply: if the recommendations feed marketing emails, a customer can object to that profiling at any time, and then it stops.²⁶ And the data has to be limited to what the purpose needs and kept no longer than necessary.²⁷ The six month training window and the thin event table are how the code meets that.

Erasure is one method:

module Recommendations
  def self.erase(user)
    InteractionEvent.where(user_id: user.id).delete_all
    UserScore.where(user_id: user.id).delete_all
    Experiment::Event.where(user_id: user.id).delete_all
    user.update!(recommendations_consent: false)
  end
end

The cached row expires within the hour. For the full picture, read the consent setup for Ireland, Germany and Austria, event tracking and consent mode, personalising without being creepy and GDPR compliant AI features.

Worked example: who gets which row at Marina Tea

Of Niamh's 3,000 signed in customers, suppose 1,800 say yes in the banner. Those 1,800 get tracked, trained on and shown their own row. The other 1,200 still get the item to item row on product pages and bestsellers on the homepage, with nothing recorded about them. One of the 1,800 later asks to be forgotten: Recommendations.erase runs, her events and scores go, and the next retrain does not know she existed.

Where it all runs

For a typical Solidus shop the pieces are few. The Python script lives in the Rails repository and runs on the same server as the Rake task. For a large catalogue, or training that takes long enough to matter, move it to its own small service, but avoid an architecture nobody can maintain. The scores sit in Postgres, the rows in the Rails cache, often the Redis you already run for Sidekiq. The retrain is a scheduled job. Four numbers tell you it is healthy: the share of product views that show a row, the click rate on rows, the purchase rate from rows, and the days since the last retrain.

What I would build first

If you are starting from nothing on a Solidus shop:

1. Week one: the consent wiring and the event tracker. You cannot train on data you never collected, and every day you wait is data you do not have.
2. Week two: the content model and the item to item row on product pages. It works with the catalogue you already have.
3. Weeks four to six: once a few hundred orders have tracking, the collaborative model and the hybrid on product pages and the homepage.
4. Week eight: the A/B test against the taxon row, measured in revenue, not clicks.
5. After that: retrain weekly, watch the four numbers, tune the weights from the test results, add the basket and email rows.

Every shop is different. A fashion shop with 5,000 SKUs and high returns needs different tuning from a specialist food shop with 200 products and loyal regulars. The framework handles both. You tune the weights, the training window and the fallbacks to the shop in front of you.

Footnotes

1. Solidus, solidus_starter_frontend README, GitHub, 2026, https://raw.githubusercontent.com/solidusio/solidus_starter_frontend/main/README.md

2. Solidus, solidus_starter_frontend, templates/app/overrides/product_featured_similar_products.rb, GitHub, 2026, https://raw.githubusercontent.com/solidusio/solidus_starter_frontend/main/templates/app/overrides/product_featured_similar_products.rb

3. Solidus, core/lib/spree/core/engine.rb (the events registered on Spree::Bus), v4.8.1, GitHub, 2026, https://raw.githubusercontent.com/solidusio/solidus/v4.8.1/core/lib/spree/core/engine.rb

4. Greg Linden, Brent Smith and Jeremy York, Amazon.com Recommendations: Item-to-Item Collaborative Filtering, IEEE Internet Computing, January/February 2003, https://www.cs.umd.edu/~samir/498/Amazon-Recommendations.pdf

5. Google, Recommendation Systems: Candidate generation overview, Google for Developers, 2026, https://developers.google.com/machine-learning/recommendation/overview/candidate-generation

6. Google, Recommendation Systems: Content-based filtering advantages and disadvantages, Google for Developers, 2026, https://developers.google.com/machine-learning/recommendation/content-based/summary

7. Google, Recommendation Systems: Collaborative filtering advantages and disadvantages, Google for Developers, 2026, https://developers.google.com/machine-learning/recommendation/collaborative/summary

8. Jure Leskovec, Anand Rajaraman and Jeff Ullman, Mining of Massive Datasets, Frequent Itemset Mining and Association Rules (lecture slides), Stanford University, http://www.mmds.org/mmds/v2.1/ch06-assocrules.pdf

9. Yehuda Koren, Robert Bell and Chris Volinsky, Matrix Factorization Techniques for Recommender Systems, IEEE Computer, 2009, https://datajobs.com/data-science-repo/Recommender-Systems-%5BNetflix%5D.pdf

10. Yifan Hu, Yehuda Koren and Chris Volinsky, Collaborative Filtering for Implicit Feedback Datasets, IEEE International Conference on Data Mining, 2008, http://yifanhu.net/PUB/cf.pdf

11. scikit-learn developers, TfidfVectorizer, scikit-learn documentation, 2026, https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html

12. Solidus, core/app/models/spree/user_class_handle.rb and Spree.user_class, v4.8.1, GitHub, 2026, https://raw.githubusercontent.com/solidusio/solidus/v4.8.1/core/lib/spree/user_class_handle.rb

13. Solidus Guides, Subscribing to events, 2026, https://guides.solidus.io/customization/subscribing-to-events

14. Solidus Guides, Customizing the core, 2026, https://guides.solidus.io/customization/customizing-the-core

15. Ben Frederickson, implicit: AlternatingLeastSquares API documentation, 2026, https://benfred.github.io/implicit/api/models/cpu/als.html

16. Ben Frederickson, implicit CHANGELOG, v0.5.0 breaking API changes (model.fit takes a user_items matrix), GitHub, https://raw.githubusercontent.com/benfred/implicit/main/CHANGELOG.md

17. Solidus, core/app/models/spree/product.rb, v4.8.1, GitHub, 2026, https://raw.githubusercontent.com/solidusio/solidus/v4.8.1/core/app/models/spree/product.rb

18. Himan Abdollahpouri, Masoud Mansoury, Robin Burke and Bamshad Mobasher, The Unfairness of Popularity Bias in Recommendation, arXiv, 2019, https://arxiv.org/abs/1907.13286

19. Balázs Hidasi and Ádám Tibor Czapp, Widespread Flaws in Offline Evaluation of Recommender Systems, arXiv, 2023, https://arxiv.org/abs/2307.14951

20. Karl Krauth and others, Do Offline Metrics Predict Online Performance in Recommender Systems?, arXiv, 2020, https://arxiv.org/abs/2011.07931

21. European Communities (Electronic Communications Networks and Services) (Privacy and Electronic Communications) Regulations 2011, S.I. No. 336 of 2011, regulations 5(3) and 5(5), Irish Statute Book, 2011, https://www.irishstatutebook.ie/eli/2011/si/336/made/en/print

22. Data Protection Commission, Guidance note: Cookies and other tracking technologies, DPC, 2020, https://www.dataprotection.ie/sites/default/files/uploads/2020-04/Guidance%20note%20on%20cookies%20and%20other%20tracking%20technologies.pdf

23. Solidus, core/app/helpers/spree/core/controller_helpers/auth.rb (set_guest_token), v4.8.1, GitHub, 2026, https://raw.githubusercontent.com/solidusio/solidus/v4.8.1/core/app/helpers/spree/core/controller_helpers/auth.rb

24. European Parliament and Council, Regulation (EU) 2016/679 (General Data Protection Regulation), Article 4, point 4 (profiling), EUR-Lex, 2016, https://eur-lex.europa.eu/eli/reg/2016/679/oj/eng

25. European Parliament and Council, Regulation (EU) 2016/679 (General Data Protection Regulation), Article 22, automated individual decision making, including profiling, EUR-Lex, 2016, https://eur-lex.europa.eu/eli/reg/2016/679/oj/eng

26. European Parliament and Council, Regulation (EU) 2016/679 (General Data Protection Regulation), Article 21, right to object, paragraphs 2 and 3, EUR-Lex, 2016, https://eur-lex.europa.eu/eli/reg/2016/679/oj/eng

27. European Parliament and Council, Regulation (EU) 2016/679 (General Data Protection Regulation), Article 5(1), points (c) and (e), EUR-Lex, 2016, https://eur-lex.europa.eu/eli/reg/2016/679/oj/eng