Sign In

Subhabrata Mukherjee

Senior Researcher at Microsoft Research (MSR)

Subhabrata (Subho) Mukherjee is no longer a Senior Researcher at Microsoft Research (MSR). As of July 2023, he became the Co-founder and Chief Scientific Officer at Hippocratic AI, a startup focused on building a safety-oriented large language model for the healthcare industry.12

Prior to his current role, Mukherjee was a Principal Researcher at Microsoft Research (MSR) in Redmond from April 2019 to July 2023. During his time at Microsoft, he led cross-team initiatives on Efficient AI at Scale, focusing on generative large foundation models.12

Mukherjee's background includes:

  1. Education:

    • Ph.D. in Computer Science from Max Planck Institute for Informatics (2013-2017), graduating summa cum laude1
    • M.Tech in Computer Science from Indian Institute of Technology, Bombay (2010-2012)1
    • B.E. in Computer Science from Jadavpur University (2006-2010)1
  2. Professional experience:

    • Machine Learning Scientist at Amazon (2017-2019)1
    • Research Engineer at IBM Research (2012-2013)1
    • Intern at Google Research (2015)1
  3. Achievements:

    • 2018 SIGKDD Doctoral Dissertation Runner-up Award1
    • 2022 MIT Technology Review Innovators under 35 Semi-finalist12

His research focuses on aligning large foundation models to human reasoning, preferences, and safety, with interests in efficient pre-training, instruction-tuning, reinforcement learning, conversational alignment, and hybrid specialization.2

Highlights

Jun 23 · linkedin

💡Source: Conversation with Bing Chat about Orca We asked Bing about our recent language model Orca, what makes its learning process unique and the promises it holds for the future. This is its response (references in comment).

🚀Orca is a new AI model developed by Microsoft Research that can learn from explanations, step-by-step thought processes, and other complex instructions. Orca is designed to overcome the limitations of smaller models by imitating the reasoning process of large foundation models like GPT-4.  Orca shows impressive results despite being a small model with 13 billion parameters.

👉Some of the unique features of Orca are: 

  • It learns from rich signals from GPT-4 including explanation traces and chain of thoughts, guided by teacher assistance from ChatGPT. 
  • It uses fewer computational resources to run due to its smaller size. 
  • It performs competitively on academic exams such as the SAT, LSAT, GRE, and GMAT. 
  • It outperforms other open-source models, earning the top spot across many benchmarks.     👉Some of the similarities between Orca and human learning process are: 
  • It learns by imitating a more advanced tutor and asking for explanations. 
  • It can understand the principles and logic behind different tasks by learning from explanations and simpler to more complex reasoning, just like humans can learn better from relevant explanations that connect examples to task principles.  
  • It learns by building on prior knowledge and applying it to new domains.     👉Some of the key results of Orca are:

🎯- Orca outperforms open source models like Vicuna by upto 100% on hard benchmarks  🎯- Orca reaches parity with ChatGPT on the Big-Bench Hard (BBH) benchmark in zero-shot settings and shows a 4-point gap on the AGIEval human-centric benchmark in professional and academic examinations such as SAT, LSAT, GRE, and GMAT.  🚲 - Orca lags behind more powerful models like GPT-4 on tasks that require natural language understanding, generation, or common sense.     👉Orca will revolutionize the field of AI by: 

  • Enhancing the reasoning and comprehension capabilities of smaller models through imitation learning. 
  • Enabling researchers to tune these models based on their needs. 
  • Promoting progressive learning from step-by-step explanations, whether these are generated by humans or more advanced AI models.

⚡Some of the limitations and future work for improving Orca are:

  • Orca may not be able to generalize well to domains or tasks that are not well represented by the large language models.
  • Orca may not be able to capture the full diversity and richness of natural language, especially for creative or generative tasks.
  • Orca may need to address ethical and social issues related to the use and misuse of large language models, such as bias, fairness, privacy, and accountability.   cc my co-authors: Arindam, Ganesh, Sahaj, Hamid, Ahmed

#microsoft #microsoftresearch #ai #orca #chatgpt #gpt4

Related Questions

What are the main goals of Hippocratic AI?
How did Subhabrata Mukherjee's work at Microsoft contribute to Efficient AI at Scale?
What was the focus of Subhabrata Mukherjee's PhD dissertation at the Max Planck Institute?
What are some notable projects Subhabrata Mukherjee has worked on at Amazon?
How does Hippocratic AI ensure the safety and responsibility of its large language model?
Subhabrata Mukherjee
Subhabrata Mukherjee, photo 1
Subhabrata Mukherjee, photo 2
Add to my network

Location

Greater Seattle Area