Suggestions
Andrew Yaroshevsky
AI/ML Director at Google | ex- Apple, Amazon | Y Combinator alum | Investor | 🇺🇦
Andrew Yaroshevsky is a seasoned GM leader with a touch of entrepreneurial spirit and over two decades of work experience in the consumer-facing ventures and B2B SaaS enterprises across four countries. He specializes in developing machine-learned products, utilizing his rich background of 15 years in the ML industry.
Andrew's leadership philosophy revolves around crafting compelling visions that inspire a sense of purpose, driven by his ability to think big. He is known for his positive orientation, profound self-awareness, and the ability to build cohesive teams, attract top-tier professionals, and mentor and cultivate leaders.
With a career highlighted by key roles at tech giants like Google, Apple, Amazon, Yandex, and Chatfuel, Andrew emphasizes making a meaningful impact over empire-building, showcasing the value of substantial contributions without the need for excessive headcount expansion. His educational background includes a Master's degree in Computer Science from the National Metallurgical Academy of Ukraine and a Bachelor's degree from Paton College of Welding and Electronics.
Highlights
Exploring the next frontier in RecSys: How are LLMs redefining Recommendation products?
Zooming out a bit from my previous post on fine-tuning LLMs for Recommender Systems tasks (RecSys), I want to provide a brief of the most popular ways of leveraging LLMs for this today more generally. There are mainly three categories.
🔸 𝟭. 𝗟𝗟𝗠-𝗣𝗼𝘄𝗲𝗿𝗲𝗱 𝗥𝗲𝗰𝗦𝘆𝘀: A fascinating shift from traditional methods, this approach employs LLMs directly for recommendation tasks without retraining new models. Instead, it leverages prompts (Liu et al. 2023a; Gao et al. 2023; Dai et al. 2023; Chen 2023) or minor fine-tuning (Zhang et al. 2023; Kang et al. 2023; Bao et al. 2023) to translate RecSys challenges into natural language tasks. By designing prompts for scenarios like rating prediction and sequential recommendation, among others, this method uses few-shot prompting to inject user interaction insights into the model, helping LLMs in capturing user preferences and needs more accurately.
🔸 𝟮. 𝗟𝗟𝗠𝘀 𝗮𝘀 𝗥𝗶𝗰𝗵 𝗙𝗲𝗮𝘁𝘂𝗿𝗲 𝗦𝗼𝘂𝗿𝗰𝗲𝘀: Viewed as a sophisticated feature extractor (Wu et al. 2021; Qiu et al. 2021; Yao et al. 2022; Muhamed et al. 2021; Xiao et al. 2022), this paradigm enriches traditional RecSys with LLM-derived embeddings. By inputting user and item features into LLMs, it fetches nuanced embeddings that traditional models then use for enhanced recommendation accuracy. Moreover, certain methods (Liu et al. 2023b; Wang et al. 2022, 2023) innovate by generating tokens from these features, mining for semantic clues to user preferences that can significantly inform the recommendation process.
🔸 𝟯. 𝗟𝗟𝗠𝘀 𝗮𝘀 𝗚𝘂𝗶𝗱𝗶𝗻𝗴 𝗔𝗴𝗲𝗻𝘁𝘀: Stepping into an agent's role, LLMs orchestrate the recommendation pipeline, overseeing everything from data collection to feature engineering and the scoring/ranking mechanism. This model (Andreas 2022; Bao et al. 2023; Hou et al. 2023; Lin et al. 2023; Gao et al. 2023; Friedman et al. 2023) enables LLMs to adapt seamlessly to the recommendation context, managing the intricacies of user interaction and system response to deliver precise recommendations.
Have I left anything out?
#AI #MachineLearning #Personalization #RecSys #3MinPapers
How to supercharge recommender systems with LLMs?
Just read a compelling white paper on RecSysLLM, a model that directly integrates LLMs with recommender systems.
But first, some context.
Compared with recommendation models based on large language models (LLMs), conventional recommendation models trained from scratch using architectures like Transformer, BERT, RNN, CNN have several key limitations.
🔸 First, they lack a deep understanding of context and semantics that comes from pretraining a large model on diverse corpora. As a result, they struggle to truly comprehend user preferences and behavioral sequences.
🔸 Second, they have minimal ability to generate novel, high-quality recommendations since they are not optimized for free-form text generation. LLMs, in contrast, can produce human-like recommendations by leveraging their generative capabilities.
🔸 Third, conventional models have difficulty effectively leveraging multiple data modalities like text, images, audio, etc. LLMs are adept at multimodal processing due to pretraining objectives that learn connections between modalities.
🔸 Finally, LLMs can seamlessly adapt to new downstream recommendation tasks through simple fine-tuning, whereas conventional models require extensive retraining. For example, BERT4Rec (Sun et al. 2019) employs deep bidirectional self-attention to model user behavior sequences. They are trained solely based on the recommendation data without the general knowledge corpus, resulting in a limited understanding and reasoning of behavior sequence data and an inability to empower downstream tasks better. In summary, recommendation models based on pretrained LLMs are more contextual, creative, versatile, and adaptable compared to conventional models trained from scratch.
So here comes the RecSysLLM from this paper.
🔹 RecSysLLM merges LLMs' semantic depth with recommendation data, aiming for pinpoint accuracy in user suggestions.
🔹 Employs entity-level masking and span ordering, customizing LLMs for nuanced recommendation tasks.
🔹 On Amazon's Beauty dataset, RecSysLLM achieved an HR@10 of 0.0512 and NDCG@10 of 0.0375, outperforming baselines and illustrating its predictive prowess.
🔹 Real-World Application: Tested on Alipay's complex user data, showcasing its capability to handle large-scale, real-world scenarios effectively.
Check-out the link in comments below 👇🏼
#AI #MachineLearning #Personalization #3MinPapers
