Suggestions
Andrei Lopatenko 🇺🇦
VP AI & Engineering | Co-Founder | Keynote speaker | Ex-Google, Apple, WML
Andrei Lopatenko is a distinguished VP of Engineering with a rich history of leading successful teams and driving revenue growth for renowned companies like Google, Apple, WalmartLabs, eBay, and Recruit Holdings.
With a PhD in Computer Science and a specialization in data science and machine learning, Andrei has spearheaded the development of machine learning-based systems that cater to billions of users and generate hundreds of millions of dollars in revenue.
His expertise spans across machine learning, natural language processing, predictive analysis, and the creation of large-scale, high-load systems, fostering interdepartmental collaborations between business, product, science, engineering, and operations.
In addition to his corporate roles, Andrei co-founded a Conversational AI startup that was later acquired by Facebook, demonstrating his entrepreneurial acumen.
A global leader, Andrei has transformed numerous organizations into top-tier engineering product entities, excelling in the creation of NLP-based consumer solutions and enterprise applications like Call Center AI and Call Analytics.
Throughout his distinguished career, Andrei has been honored with prestigious accolades such as the Google Operating Committee Awards, Zillow the Shelfie award, and the ICON Analytics award.
Moreover, as a sought-after speaker, Andrei has delivered engaging talks, tutorials, and keynote speeches at various conferences on topics ranging from machine learning and NLP to MLOps and search engines.
Highlights
Besides a comprehensive review of models, there is a list of evaluation sets and methods they use in medicine to evaluate language models See Table 2, column Experimental datasets Clinical Insights: A Comprehensive Review of Language Models in Medicine.
This research presents a novel approach to evaluating language models, conducted by the University of Toronto and the Vector Institute. The authors propose a method for generating human-readable "Report Cards" that assess the capabilities of 'student models' (language models being evaluated). These report cards are designed to be faithful, specific, and interpretable by humans. They provide a detailed analysis of the models' performance across multiple skills and topics. The generation of these cards is automated using a larger 'judge' language models, which evaluate the 'student' models. Extensive details on the implementation are provided in a comprehensive appendix. This method offers is a significant complementary method to traditional score-based evaluations used in large-scale benchmarks. The report card system can be tailored to assess model competencies relevant to specific business needs, enabling the generation of immediately interpretable assessments that can be shared across research, product development, and business teams. https://lnkd.in/gnYDkPQf
