Rendered at 14:14:22 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
lqstuart 6 minutes ago [-]
Netflix has imo the worst recommendations on the entire internet. They are a Java shop that’s 5-10 years behind the industry in AI, and now they’re trying to use the most grossly inefficient and overcomplicated method out there to try to be trendy instead of just using a DLRM like everyone else was in 2020. What could go wrong?
mvkel 21 minutes ago [-]
Netflix had an algorithm contest, when was it, like 20 years ago? A team made a highly effective content recommendation system and Netflix never used it. Why? It killed revenue.
Separately, there simply isn't enough content to recommend. It's not like you need help finding the perfect Wikipedia article. At any one time, there are ~50 things a user would probably watch.
Feels more like a soft pivot to cash in on AI valuations since Hollywood is cratering. Probably worth the investment.
jubilanti 9 minutes ago [-]
> A team made a highly effective content recommendation system and Netflix never used it. Why? It killed revenue.
Nope. This is an internet urban legend. The Netflix prize was only ever for marketing and recruitment. They never intended to deploy whatever winning weights that win because they overfitted on the training and validation data. The dataset they released was not their actual full dataset. It contained zero personal information, just (userid, movieid, rating, date) which is only ever going to get you so far compared to what they do internally.
And it wasn't "a team" that beat the baseline of a ranking algorithm Netflix never deployed, dozens of teams were within fractions of a point from each other. Hence the over fitting strategy.
jmbwell 23 minutes ago [-]
Okay. Well so if the model will tend to converge on the median of a user’s interests as measured by engagement… the user would have to depart from the recommendations to increase the odds of being recommended something surprising, no?
It’s interesting to see how Netflix evaluates RecSys internally, inferring from how they are comparing GenRec to it. In both cases the premise seems to be that users mainly want to engage with more of the same.
I would be even more interested in some kind of comparison to Netflix’s much earlier system for exchanging reviews and recommendations among a user’s human social group. Seems like there could be some insights around the input of fresh signals from aligned but not strictly conforming participants.
wolttam 33 minutes ago [-]
It feels icky to me that they feed a log of what you’ve watched, when, and on what device, over to an LLM just to get back some recommendations.
Even though It’s probably exactly the type of thing I would do on a locally hosted LLM
jhack 23 minutes ago [-]
This sounds like data they've already been collecting, they're just using it in a different way.
alt227 35 minutes ago [-]
Here we go, the start of LLMs plugged into everything.
Do Netflix recommendations really need to use LLMs?
This post paints a very altruistic picture of how recommendations can be fed from user history, preferences, device and environment context etc. However they make no mention of needing to advertise content from paid clients, promote new releases, and increase views on certain underperforming content. I'm assuming that once their fancy LLM spits out some user recommendations, they are then run through another process to 'commercialise' the results before displaying them to the user.
ofjcihen 22 minutes ago [-]
I’m curious as to what readers think Large Language Models will bring to the recommendations table that our current machine learning doesn’t.
trollbridge 20 minutes ago [-]
I’m curious too, as ML is pretty amazing; LLMs are one application of ML; LLMs have a much narrower good set of uses than broad ML.
You can use LLMs for things they are a bad fit for. I know someone who uses it like a spreadsheet to add sums of numbers, etc.
cjs_ac 5 minutes ago [-]
It will bring the ability to tell investors that Netflix is using Large Language Models.
porridgeraisin 1 minutes ago [-]
In this setting, they are essentially using it as a feature extractor.
As for moving to this versus your bespoke feature extractor, _given_ that your existing features and their compositions are not antithetical to their language representation, an LLM will be an equal or better feature extractor. But for example, if your collaborative filter has learnt features that smartly recommending a show with one text feature "sci-fi" to people that like "comedy" because of learned behaviour despite the labelID, then you have to verbalise this feature "scifi,comedy,laugh track" or maybe providing samples of the subtitles of the show or add a "frequently co-watched with" section to the prompt, to effectively get an LLM to do the same thing [1]
In many cases, people have potentially verbalizable features and feature compositions in their existing systems even if it may not exactly be optimal.
[1] or integrate the cf embedding in a myriad of ways. Thinking out loud tool calls may work but it will break their prefill-only requirement.
[2] composition mentioned everywhere above is crucial! Provided you can verbalise your important features, LLMs can perform very strong deductions and compositions out of the box.
Conol_ai 27 minutes ago [-]
[flagged]
BosunoB 23 minutes ago [-]
Spotify seems to use an LLM for their new DJ feature, and it honestly is way better than any other music recommendation service I've ever tried. It will give you music that actually sounds like the music you ask for.
What I'm excited for is Bumble's AI matchmaker. LLM categorization may actually be the key to a decent dating app.
eterm 14 minutes ago [-]
I've not seen any difference between the spotify "AI DJ" and their daily playlists, they literally just pick a song you like and run similar.
The only difference is that every so often the DJ yaps at you and rotates to a different playlist.
It's still absolutely nothing like a DJ, and spotify daily recommendations are still hopeless compared to what Google Music ( rip ) used to do, which actually was able to deliver diverse playlists based on recommendations.
Just chopping between different homogenous blocs is not a good experience.
Separately, there simply isn't enough content to recommend. It's not like you need help finding the perfect Wikipedia article. At any one time, there are ~50 things a user would probably watch.
Feels more like a soft pivot to cash in on AI valuations since Hollywood is cratering. Probably worth the investment.
Nope. This is an internet urban legend. The Netflix prize was only ever for marketing and recruitment. They never intended to deploy whatever winning weights that win because they overfitted on the training and validation data. The dataset they released was not their actual full dataset. It contained zero personal information, just (userid, movieid, rating, date) which is only ever going to get you so far compared to what they do internally.
And it wasn't "a team" that beat the baseline of a ranking algorithm Netflix never deployed, dozens of teams were within fractions of a point from each other. Hence the over fitting strategy.
It’s interesting to see how Netflix evaluates RecSys internally, inferring from how they are comparing GenRec to it. In both cases the premise seems to be that users mainly want to engage with more of the same.
I would be even more interested in some kind of comparison to Netflix’s much earlier system for exchanging reviews and recommendations among a user’s human social group. Seems like there could be some insights around the input of fresh signals from aligned but not strictly conforming participants.
Even though It’s probably exactly the type of thing I would do on a locally hosted LLM
This post paints a very altruistic picture of how recommendations can be fed from user history, preferences, device and environment context etc. However they make no mention of needing to advertise content from paid clients, promote new releases, and increase views on certain underperforming content. I'm assuming that once their fancy LLM spits out some user recommendations, they are then run through another process to 'commercialise' the results before displaying them to the user.
You can use LLMs for things they are a bad fit for. I know someone who uses it like a spreadsheet to add sums of numbers, etc.
As for moving to this versus your bespoke feature extractor, _given_ that your existing features and their compositions are not antithetical to their language representation, an LLM will be an equal or better feature extractor. But for example, if your collaborative filter has learnt features that smartly recommending a show with one text feature "sci-fi" to people that like "comedy" because of learned behaviour despite the labelID, then you have to verbalise this feature "scifi,comedy,laugh track" or maybe providing samples of the subtitles of the show or add a "frequently co-watched with" section to the prompt, to effectively get an LLM to do the same thing [1]
In many cases, people have potentially verbalizable features and feature compositions in their existing systems even if it may not exactly be optimal.
[1] or integrate the cf embedding in a myriad of ways. Thinking out loud tool calls may work but it will break their prefill-only requirement.
[2] composition mentioned everywhere above is crucial! Provided you can verbalise your important features, LLMs can perform very strong deductions and compositions out of the box.
What I'm excited for is Bumble's AI matchmaker. LLM categorization may actually be the key to a decent dating app.
The only difference is that every so often the DJ yaps at you and rotates to a different playlist.
It's still absolutely nothing like a DJ, and spotify daily recommendations are still hopeless compared to what Google Music ( rip ) used to do, which actually was able to deliver diverse playlists based on recommendations.
Just chopping between different homogenous blocs is not a good experience.