Netflix tests language model for content recommendations, reports improved results

Netflix is testing GenRec, a proprietary language model system designed to replace its complex, hand-crafted recommendation logic. By converting user interactions into plain text dialogue, GenRec simplifies onboarding for new formats like games and podcasts. In a four-week online A/B test covering ten percent of traffic, GenRec achieved statistically significant improvements, raising a short-term home screen metric by 0.115 percent and a long-term core metric by 0.006 percent.

GenRec language model system

• ▪To prevent GenRec from recommending non-existent titles, Netflix integrated a separate component that restricts recommendations to verified catalog entries.
• ▪The GenRec system is designed to address the high onboarding costs of new content types like games, live formats, or podcasts associated with Netflix's traditional recommendation system.
• ▪Netflix developed and tested GenRec, a proprietary language model system designed as an alternative to its traditional hand-built recommendation logic.

Natural language user data encoding

• ▪To manage context window limits, Netflix filters user interaction text by retaining high-signal events like long watch sessions while dropping brief taps or quick scrolls.
• ▪Netflix's GenRec converts user interactions, such as watch durations, thumbs up or down, and drop-offs, into plain text dialogue rather than dense numerical vectors.

A/B test performance metrics

• ▪In offline testing, GenRec delivered a 1.6 percent improvement in ranking quality compared to Netflix's active production recommendation system.
• ▪In a four-week online A/B experiment on approximately ten percent of Netflix traffic, GenRec improved a short-term home screen metric by 0.115 percent and a long-term core metric by 0.006 percent.

Context engineering paradigm shift

• ▪Netflix views GenRec as part of an industry shift toward context engineering, aligning with other recommendation research projects such as PLUM, GLIDE, and OneRec-Think.
• ▪The shift toward systems like GenRec transitions engineering efforts from building custom architectures to determining which signals to include in the model's input.

Two-stage model training process

• ▪Netflix trains GenRec in a two-stage process, starting with fine-tuning an unnamed open-weight language model on Netflix catalog and user data.
• ▪The recommendation-specific fine-tuning in GenRec's second stage adds 35 to 50 percent to the base model's performance, widening to about 80 percent when the base model is two weeks old.
• ▪The second stage of GenRec training turns the base model into a recommendation ranker, requiring roughly 40 times fewer labeled examples than the traditional system to achieve its offline ranking quality.