Sign In

Andleep Farooqui

Machine Learning Engineer

Andleep Farooqui is a Software Engineer currently working at Axyom, a company focused on Web3 and AI technologies, since April 2023. Prior to this role, he served as a Software Engineer at TRC Healthcare starting in July 2021, and he also completed an internship as a Software Engineer at SymSoft Solutions.1

His professional background highlights a progression in software engineering roles, showcasing his involvement in innovative fields such as artificial intelligence and blockchain technology within the Web3 domain.

Highlights

Dec 13 · linkedin

RLHF (reinforcement learning from human feedback) and DPO (direct preference optimization) are great techniques to add on top of supervised finetuning to help align LLM generation with the sorts of generations that humans would want.

That said.. Both RLHF and DPO use a model of human preferences called the “Bradley-Terry” model. It has a lot of positives but it’s pretty old (orginated in 1952) and there have been subsequent decades of research in computational cognitive science.

Somewhat recently, cogsci researchers have introduced a new paradigm - KTO. The primary difference between KTO and DPO is the form of the human value function - KTO is informed by the work of Kahneman and Tversky in 1992 and uses an experimentally determined power law distribution - with specific parameters gotten by experiment!

In experiments, researchers found that KTO performs as well or better than DPO at all scales 😮 best of all, we do not need preference data which opens up a lot of possibilities..

The Archangel repository contains a number of KTO and DPO finetuned models to try and i’m wondering if anyone has played with this technique and what they think.

Posting so more people hear about this and know! It really is a cool development.

Contextual AI put out a blogpost here with all the links you’d need: https://lnkd.in/e-_wKrTD

Lmk if you have tried it or what you think of the archangel models, I haven’t had much experience with dpo myself yet (hoping to change that soon 🙂)

Sep 22 · linkedin

A huge piece of value for GPT4 is to use it as a tool to build miniature functions - specifically in terms of text processing. Named entity extraction, splitting text into pieces, etc.

It feels really underrated. I keep building out minor LLM apps to just deal with everyday stuff.

Related Questions

What projects has Andleep Farooqui worked on at Axyom?
What is Andleep Farooqui's educational background?
How did Andleep Farooqui transition from TRC Healthcare to Axyom?
What technologies does Andleep Farooqui specialize in?
Can you provide examples of Andleep Farooqui's contributions to Web3 and AI?
Andleep Farooqui
Andleep Farooqui, photo 1
Andleep Farooqui, photo 2
Add to my network

Location

Sacramento, California, United States