Netflix tests language model for content recommendations, reports improved results

Netflix is testing GenRec, a proprietary language model system designed to replace its traditional, hand-built recommendation logic. GenRec converts user watch history into plain text, eliminating the need for thousands of manually engineered features. In a four-week online A/B test covering ten percent of traffic, GenRec improved home screen user behavior by 0.115 percent and a core long-term metric by 0.006 percent, while requiring 40 times less labeled training data in its second stage.

GenRec language model system

• ▪To manage compute costs, GenRec runs on vLLM in a mode where the model reads the input once and scores all candidates in a single pass without generating text.
• ▪Netflix is testing a proprietary language model system called GenRec as an alternative to its traditional hand-built recommendation logic.
• ▪GenRec is trained in two stages, starting with an unnamed open-weight language model fine-tuned on Netflix data, followed by specialized training to turn it into a recommendation ranker.

Natural language recommendation approach

• ▪To prevent GenRec from suggesting non-existent titles, Netflix integrates a separate component that only scores real catalog entries.
• ▪GenRec converts user watch history, plays, durations, ratings, list additions, and drop-offs into plain text dialogue instead of dense numerical vectors.
• ▪Netflix filters interaction data aggressively to keep high-signal events like long watch sessions while dropping brief taps or quick scrolls to fit GenRec's context window.

A/B testing results

• ▪In a four-week online A/B experiment on about ten percent of traffic, GenRec improved a short-term home screen user behavior metric by 0.115 percent and a long-term core metric by 0.006 percent.
• ▪GenRec delivered approximately 1.6 percent better offline ranking quality compared to Netflix's existing production recommendation system.
• ▪The recommendation-specific fine-tuning in GenRec's second stage adds 35 to 50 percent to the base model's performance, widening to about 80 percent when the base model is two weeks old.

Training efficiency improvements

• ▪GenRec required roughly 40 times fewer labeled examples in its second training stage compared to the existing production system to achieve its ranking quality.
• ▪Netflix's traditional recommendation system relies on thousands of hand-crafted features, making it expensive to onboard new content types like games, live formats, or podcasts.

Netflix recommendation infrastructure evolution

• ▪Netflix introduced an opt-in mobile feature called Downloads For You on February 22, 2021, which automatically downloads recommended content to Android devices based on viewing history.
• ▪In 2020, Netflix utilized Google's BERT language model to process human-written title summaries and feed machine-readable representations to downstream models.