Content-Based Filtering & the Two-Tower Model¶
Collaborative filtering asks "who is like you?" Content-based filtering asks "what is the item made of, and what do you already like?" This page covers content-based scoring and its signature trade-offs (cold start solved for items, serendipity lost), then the two-tower architecture — MF's dot-product skeleton with the ID lookups upgraded to feature-eating neural nets — which is where CF and content-based finally fuse into one model.
Rapid Recall
Content-based represents each item by its features, builds a user profile by aggregating the feature vectors of liked items, and scores by cosine — solving new-item cold start outright but only softening new-user cold start, and it can never be serendipitous (filter bubble). Two-tower is MF generalized: \(p_u=f_\theta(\text{user features})\) and \(v=g_\phi(\text{item features})\), two networks that meet only at a dot product, trained contrastively with in-batch negatives (plus logQ correction). That separation is the whole point — item tower runs offline into an ANN index, user tower runs online — which is exactly why it's a coarse retrieval model that can't learn user×item cross-features (ranking's job).
§1 Content-based filtering¶
Content-based uses features of items, never anyone else's data.
How it works, start to end¶
- Represent each item by its features — genre, cast, director, year, text embedding of synopsis, even image/video embeddings. This vector exists the moment the item is created, before anyone touches it.
- Build a user profile from items they engaged with — aggregate (often a weighted average by rating/watch-time/recency) of the feature vectors of liked items. The profile lives in the same feature space as items.
- Score by similarity — cosine between user profile and each item vector. High cosine = "made of the stuff you historically liked."
Variant: instead of one profile vector, train a small per-user classifier/regressor on liked vs. not-liked using item features. More expressive (can learn nonlinear "likes action unless horror").
Solving cold start — be precise about which kind¶
| Cold-start type | Content-based? | Why |
|---|---|---|
| New item | ✅ Solved cleanly | A new movie has genre/cast/synopsis instantly — compute its vector day zero, before anyone watches. |
| New user | ⚠️ Partially / not really | Zero interactions → no profile to aggregate. Needs a few clicks to bootstrap. Zero-history still defeats it. |
Accurate statement: content-based solves new-item cold start outright and only softens new-user cold start — because item features exist independent of behavior, but a user profile still requires some behavior.
Strong vs weak¶
| Strong when | Weak when |
|---|---|
| Fresh / high-churn catalogs (news, jobs, new launches) | Over-specialization / filter bubble — the signature failure |
| Rich item features available | Features are poor or expensive (just ID + title) |
| Niche / solo users (no crowd needed) | Can't learn quality/taste subtleties features don't encode |
| Explainability ("because it's sci-fi like X") | New user with zero history |
The defining weakness — no serendipity
Because it only recommends items similar to your past, it can never surprise you. Watch one cooking video → cooking videos forever. It cannot discover that cooking-watchers also love a travel show — that cross-domain leap requires behavioral co-occurrence (CF). This is why content-based is almost never used alone.
Where it's used¶
Almost never alone — as one source in a hybrid candidate generator, as the cold-start bridge (new items enter via content until they accumulate CF embeddings), and in pure-content domains (news, jobs, Spotify audio-features for new tracks).
Mental model: CF and content-based are complementary, not competing. CF gives serendipity + latent quality but dies on cold items; content-based handles cold items + explains itself but traps you in a bubble. Every serious system runs both — which leads straight to two-tower, which absorbs both.
§2 The two-tower architecture¶
Two-tower = MF's trainable-embedding + dot-product skeleton, with the ID lookups upgraded to feature-eating neural networks, trained contrastively. It is where CF and content-based fuse, and it is the 2026 default retrieval model.
The key mapping
MF is two-tower's degenerate special case. In MF, \(p_u\) = "look up user u's row." Two-tower replaces that with \(p_u = f_\theta(\text{features of user }u)\), a network. Same dot-product scoring; vastly richer inputs.
Step by step¶
- Feed each tower features, not just IDs — user tower eats id + context + recent history; item tower eats id + content features. Content-based features live in the item tower; collaborative signal lives in training + user history. Both paradigms absorbed into one model.
- Each tower → a fixed vector in a shared space (same \(k\), so dot product is meaningful). Usually MLPs; modern variants use transformers on user history.
- Score = dot product (or cosine) \(\;\text{score}(u,i)=u\cdot v\). Richness moved into the towers; the comparison stayed simple — on purpose.
- Train it contrastively — the part with teeth.
Training: the negative sampling problem¶
- Logs give only positives (clicks/watches). You must manufacture negatives.
- In-batch negatives — in a batch of 1000 positive (user, item) pairs, treat the other 999 items as negatives for each user. Cheap, because their vectors are already computed. The workhorse trick.
- Loss — sampled softmax / InfoNCE-style contrastive: for a user, push the true item's dot product up and negatives down. Same contrastive idea as embedding training in RAG/NLP.
Depth signal — logQ correction
Popular items appear as positives constantly, so they appear as in-batch negatives constantly → the model over-penalizes them. Fix: logQ correction — subtract a term proportional to the item's sampling frequency from its logit. Naming this signals you've actually studied two-tower.
Serving — why two towers, not one network¶
The towers never interact until the final dot product, and that separation is the entire reason two-tower is usable at retrieval scale:
- Item tower offline — items don't change per request. Push all 10M items through once → 10M vectors → ANN index. Never re-run at request time.
- User tower online — only the user vector needs computing (context/history just changed). One forward pass.
- Retrieval = ANN search — items with highest dot product to \(u\). Sub-50 ms over millions.
If you instead used one network on user+item jointly → that's a cross-encoder (the ranking model): far more accurate because features interact early, but nothing is precomputable, so scoring 10M/request is impossible. The separation IS the precomputability.
Strong vs weak¶
| Strong | Weak |
|---|---|
| Web-scale retrieval, sub-50 ms (its reason to exist) | Can't model fine user–item feature interactions — towers meet only at a dot product |
| Cold start both kinds (features flow through towers) | Dot-product ceiling — one number expresses all relevance |
| Heterogeneous features (text/image/categorical/sequence) | Negative sampling is fiddly & high-stakes |
| One model, both signals (no separate CF + content pipelines) | Truly cold user → falls back to popularity; index goes stale |
Why it's a retrieval model and not a ranking model
Because user and item only meet at a dot product, it can't learn cross-features like "this user likes horror only on weekends on mobile." Those get crushed into independent vectors before they meet. It's a recall-optimized, coarse model by design — great for "get the plausible few hundred," not for "perfectly order them." Ranking (the cross-encoder) recovers what the separation throws away.
Interview questions¶
Q1: Why does content-based solve new-item but not new-user cold start? A new item has features (genre, cast, synopsis) the instant it's created, so you can compute its vector before anyone interacts. A new user has no interactions to aggregate into a profile, so there's nothing to match — only a few clicks bootstrap it, but zero history still defeats it.
Q2: Why can't content-based produce serendipitous recommendations, and which method can? It only recommends items similar to your past consumption (similarity in feature space), so it can never surprise you with cross-domain leaps. Discovering that cooking-watchers also love a travel show requires behavioral co-occurrence — that's collaborative filtering.
Q3: Why must the two towers stay separate until the final dot product? Separation lets you precompute all item vectors offline into an ANN index and compute only the user vector online — enabling sub-50 ms retrieval over millions. Fusing them early (one joint network) gives richer feature interactions and higher accuracy but makes nothing precomputable, so scoring millions per request becomes impossible. The separation IS the precomputability.
Q4: What is the in-batch negative trick, and what bias does it introduce? Within a batch of positive (user,item) pairs, use the other items in the batch as negatives for each user — cheap because their vectors are already computed. Bias: popular items appear as in-batch negatives constantly → over-penalized. Fix: logQ correction (subtract a term ∝ sampling frequency from the logit).
Q5: Why is two-tower a retrieval model and not a ranking model? Because user and item meet only at a dot product, it can't model fine cross-features ("likes horror only on weekends on mobile") — those get crushed into independent vectors before meeting. It's a recall-optimized, coarse model; ranking (the cross-encoder) recovers the interactions it sacrificed.