Sign In

Rahul Modak

Chief Data Scientist & Founder Bayesian Analytics

Rahul Modak is a co-founder and a key figure at Bayesian Analytics, where he focuses on developing large language models (LLMs) for customer data platforms and visualization. His work involves leveraging Bayesian methods and Direct Preference Optimization (DPO) techniques to enhance AI models, particularly in refining their alignment with human preferences and improving user engagement through custom visualizations. This innovative approach aims to simplify model fine-tuning while maintaining computational efficiency.12

In his role, Modak is actively engaged in research and development, exploring how Bayesian statistics can address challenges in generative AI, such as the common issue of model hallucinations, thereby improving the accuracy and reliability of AI outputs.2 His contributions to the field are underscored by his background in machine learning and data analytics, which he applies to drive significant business outcomes for clients.

Modak's expertise and leadership in these areas position him as an influential figure in the intersection of AI and data analytics, particularly within the context of enhancing customer data platforms.

Highlights

Nov 15 · linkedin

Simplifying ChatGPT and Generative AI training

In today's post we will briefly touch upon the Gen AI training process which is what every organisation is attempting these days. To better understand, imagine LLMs like ChatBots as clever robots that guess what word comes next in a sentence, just like completing a sentence in a game. They learn through a method where they get points for right guesses and lose points for the wrong ones — think of it as a sophisticated word game with rewards and penalties. This training method, known as RLHF (Reinforcement Learning with Human Feedback), relies on people to judge the guesses.

The tech behind this training is complex and expensive, using powerful GPU and billions of dollars, making it a key factor in the competition. It's so crucial that even governments are keeping an eye as well as executive orders around it. Almost all big models so far have been using this approach in one way or other!

Recently, a French startup called Mistral introduced a game-changer: DPO (Decision Policy Optimization), which sets rules for making these guesses, simplifying the process. This new method allows even smaller models to achieve accuracy close to giants like OpenAI's GPT-4, which has 175 billion parameters, compared to Mistral's leaner 7 billion .

Both of these are methods to evaluate human feedback but DPO eliminates need to train a reward model and feedback in DPO is simple Yes-No (binary)

However, no model is perfect. Sometimes in both approaches, without rules or human feedback for guidance, models can make errors or "hallucinate" answers. In fact there is a leader board now for hallucinations on git hub!

So our team at Bayesian Analytics is working to see how this hallucination problem can be addressed using Bayesian statistics and improve accuracy and we do hope to release our training & model in near future.

Oct 28 · linkedin

**** Simplifying the Concepts - Gaussian Processes ****

In today's post, we will simplify and explain Gaussian Processes (GP) in very simple terms. GPs are a substantial improvement in tackling classification and regression problems. There's also a Bayesian version of GP, which addresses uncertainty in the GPs. Let's begin.

Though mathematically complex, a Gaussian Process (GP) can be broken down into simpler terms. It's a collection of random variables, any finite number of which have a joint Gaussian distribution. It's defined by a mean function and a covariance function. There are also complex concepts like Kernel methods and Reproducing Kernel Hilbert Space (RKHS) that underlie these methods, but we'll skip the mathematical theory for now.

When we face a regression problem, we aim to fit a line that best describes a set of points. We adjust the line up and down (altering the slope) or left and right (altering the intercept) until we get the best fit.

Rubber Sheet Analogy*

Now, let's imagine a smooth, flexible rubber sheet. Consider the data points as pegs fixed on a board. When you lay the rubber sheet over the pegs, the rubber sheet will stretch and bend to cover as many pegs as possible, adapting its shape to the pegs' arrangement. In this analogy:

Rubber Sheet: Represents the functions in the Gaussian Process. Like the rubber sheet can take many shapes, the GP considers many possible functions that could fit the data.

Pegs: Represent the data points guiding the shape the rubber sheet (or the functions in the GP) will take.

Stretching and Bending: Represent how the functions in the GP adapt to fit the data, controlled by the parameters of the RBF kernel:

Amplitude Parameter: Controls how "tall" the rubber sheet can stretch, similar to the vertical stretching of the sheet. Length-Scale Parameter: Controls how "wide" the rubber sheet can stretch, similar to the horizontal stretching of the sheet. RBF Kernel: Acts like the material of the rubber sheet—it defines how flexible and smooth the sheet is, influencing how well it can adapt to cover the pegs. There are many other kernels we can use.

Better Fitting: Like the rubber sheet adapts its shape to cover the pegs better than a rigid board, the GP, guided by the RBF kernel, can potentially provide a better fit to the data compared to a rigid linear regression model.

****Extending the Analogy for Bayesian GPs ****

In a Bayesian setting, we add a level of uncertainty to our model. We're not just looking for a single "best" rubber sheet to fit the data, but considering a range of possible rubber sheets, each with different stretching and bending levels.

To try out these in python ecosystem, we can start with sklearn.gaussian_process module and then try GPy or GPFlow or pymc3. Any complex classification/regression problem with sparse data is a good candidate for experimentation.

Related Questions

What are some key projects Rahul Modak has led at Bayesian Analytics?
How does Rahul Modak's work with Bayesian and DPO techniques impact customer data platforms?
What is the significance of integrating Pyro and PyTorch in Rahul Modak's AI projects?
Can you provide examples of custom visualizations improved by Rahul Modak's methods?
How does Rahul Modak's approach to AI differ from other large language model training methods?
R
Add to my network

Location

Thane, Maharashtra, India