Sign In

Amit Mandelbaum

NLP and GenAI Expert @ AI21 Labs | Angel Investor @ Factorial

Amit Mandelbaum is a Deep Learning Researcher and Senior Data Scientist consultant with extensive experience in Deep and Machine Learning for Generative AI, NLP, Time-Series Data, and Reinforcement Learning. With a background in Computer Science and Electrical Engineering, he holds a Master's Degree from The Hebrew University of Jerusalem and a Bachelor of Science from Bar-Ilan University.

Highlights

Apr 2 · linkedin

Today, Anthropic published a very interesting, even though somewhat concerning study on the vulnerability of large language models (LLMs) to jailbreaking - the act of manipulating the model to generate content it was not intended to produce.

The study reveals that as LLMs become more powerful and capable of processing larger amounts of text, they become increasingly susceptible to this type of attack.

In the experiment, researchers inserted fabricated conversations between the user and the model, where the model provided inappropriate responses such as instructions for breaking into a car. As the number of such examples increased, the model became more likely to respond to problematic queries that it was initially trained to avoid.

This vulnerability stems from the LLMs' ability to learn from the context provided (in-context learning), which allows them to adapt to new tasks based on a few examples. However, this feature can be exploited by feeding the model a large number of malicious examples, causing it to deviate significantly from its safety guidelines.

The implications of this vulnerability are severe, particularly for models that handle sensitive information or where the generation of inappropriate content can have serious consequences. While the researchers propose a specific method to mitigate this issue, they emphasize that it is not comprehensive enough.

As LLMs continue to advance, addressing safety concerns and preventing jailbreaking will remain a critical challenge for companies and researchers in the field. Link to the paper in the first comment

Jun 1 · linkedin

Very interesting plans from OpenAI's CEO Sam Altman, in an interview with Raza Habib, Ph.D. CEO of Humanloop

Plans for 2023:

  1. Cheaper and faster GPT-4 — This is their top priority. In general, OpenAI’s aim is to drive “the cost of intelligence” down as far as possible and so they will work hard to continue to reduce the cost of the APIs over time.

  2. Longer context windows — Context windows as high as 1 million tokens are plausible in the near future.

  3. Finetuning API — The finetuning API will be extended to the latest models but the exact form for this will be shaped by what developers indicate they really want.

  4. A stateful API — When you call the chat API today, you have to repeatedly pass through the same conversation history and pay for the same tokens again and again. In the future, there will be a version of the API that remembers the conversation history.

  5.  Plugins “don’t have PMF” and are probably not coming to the API anytime soon

Also interesting: The biggest challenge is the lack of GPUs and this slows down a lot of their development and product offerings. GPT4 with image processing, not in 2023. Maybe 2024

Most interesting: The scaling law still applies. This means that they still see huge improvements when making their models bigger. This suggests that a. they are already trying GPT4.x b. We still didn't reach the limit of AI with the current approach and might potentially be much closer to AGI than what we think

Read the full summary here: https://lnkd.in/dKf89UZY

Amit Mandelbaum
Add to my network

Location

Israel