Superfast AI 1/11/2023

Welcome to 2023! Today we dive into 2022 paper highlights, RLHF, and AI Interpretability.

January 11, 2023

Welcome to 2023, everyone! It’s going to be a big year and we have a lot to cover. Today we’ll talk about 2022 paper highlights, RLHF, and AI Interpretability. Let’s dive in!

🗞 News

Embeddings

OpenAI released a new embedding model which is simpler, more capable and more cost effective to use. (link)

Papers from 2022

Highlights:

  • A ConvNet for the 2020s (link) — new SOTA architecture for vision models

  • MaxViT: Multi-Axis Vision Transformer (link) — local vs global attention in vision models

  • High-Resolution Image Synthesis with Latent Diffusion Models (link) — aka Stable Diffusion

Dive into the full list here!

3 Headlines and a Lie

Can you spot the fake headline?

  • South Park Creators Land a $20M Investment for Their Deepfake VFX Studio (link)

  • China Is Building a Parallel Generative AI Universe to Compete with Western Counterparts (link)

  • You Can Now Use AI to Detect AI Generated Text (link)

  • Cineffusion: Runway ML Launches New Product That Leverages Diffusion Models to Create Synthetic Movies (link)

📚 Concepts & Learning

Reinforcement Learning with Human Feedback (RLHF)

What is 2 + 2?

Ask ChatGPT and you'll find its performance is impressive! Ask the original GPT-3... and it's not even close. Many researchers have cited Reinforcement Learning with Human Feedback (RLHF) as a game-changing ingredient towards ChatGPT's improvement over previous models.

How? ML models have moved from simple autocompletion to a better understanding of prompt intention. Check out the example below for a side-by-side comparison.

If you're interested in the latest research and development in machine learning, definitely check out Surge AI's RLHF blog post series! (link)

Large Language Models or *Small* Language Models

Is bigger always better?

In December, Stanford and Mosaic ML trained a small medical language model called PubMedGPT 2.7B. What’s particularly notable about this model is its size and training volume: it’s 2.7B parameters trained on 16B abstracts and 5M full articles (roughly 50B tokens) from the Pile dataset. This training size is smaller than it’s notable counterparts: GPT3 2.7B trained on 6x more tokens and GPT-J trained on 8x more tokens. It trained on 128 A100 GPUs for 6.25 days, which is still a non-trivial amount of compute.

The small language model results

  • PubMedGPT sets a new state of the art standard on MedQA-USML evaluations (scoring 50.3%).

  • It scores slightly worse than Facebook’s Galactica on the PubMedQA (scoring 74.4% vs Galactica’s 77.6%).

  • It scores 96.4% on BioASQ.

This all sounds great, but just this week Google and DeepMind just launched MedPaLM, a new large (540B) language model for medical applications (tweet and arxiv).

The large language model results

  • It sets a new state of the art standard in 7 medical question-answering tasks.

  • It also scores 67% on MedQA USMLE, which is a 17% improvement over prior results from PubMedGPT

The upshot

Given the cost, time and demands of building a LLM, researchers might opt to build small language models* for specific use cases. LLMs would have more general intelligence and can be used in a wide variety of use cases, while SLMs* can be deployed in niche segments. Medical expertise is one example where highly focused training (and resulting model) might make a lot of sense. But if LLMs continue to win out on results, and researchers have the budget and velocity to execute at speed, we may continue to see LLMs dominate.

*not an official term

Check out the full articles here:

Scaling Datasets

Will we run out of data? An analysis of the limits of scaling datasets in Machine Learning (link)

AI Interpretability (shallow dive)

As AI becomes more capable, it becomes important to understand and potentially predict how models make their decisions. Interpretability is particularly important for AI safety research, which seeks to align AI actions with human values.

AI Paperclip Maximizer

A common parable referenced in AI safety is the “AI paperclip maximizer.” It’s a short story that illustrates the potential dangers of misaligned Artificial General Intelligence (AGI). It goes like this: A model is programmed to maximize the production of paperclips. At first, it produces a small, reasonable amount of paperclips, but soon it become unwieldy and consumes more and more of Earth’s resources in the pursuit of paperclip production. Eventually, there’s nothing left but paperclips. This parable is intended to illustrate the idea that if an AI is given a goal that is too narrowly defined and lacks safety constraints, it could end up causing unintended harm in its pursuit of that goal.

Interpretability’s Role

If we had been able to predict that the paperclip maximizer would continue production until the end of time, we probably wouldn’t have turned it on. Instead, we may have tweaked the model before launching it to cease production after 1M paperclips or after accumulating a $1M profit. Or we may have set guidelines to continue producing as long as it didn’t violate other human values we have (such as property rights).

There are many methods being researched to progress AI interpretability. Here are a few if you’re interested in diving in deeper:

  • Neel Nanda’s Mechanistic Interpretability Explainer & Glossary (link)

  • Neel Nanda’s Favourite Mechanistic Interpretability Papers (link)

  • MIT: Building explainability into the components of machine-learning models (link)

  • Quanta: Interview with Been Kim on interpretability (link)

  • Hase and Shen: 70 Summaries of recent interpretability papers (link)Two recommended overview papers:

    • The Mythos of Model Interpretability (link)

    • Towards A Rigorous Science of Interpretable Machine Learning (link)

Additional Resources

  • What is NLP? Summary by Andrew Ng (link)

  • The power of small datasets in ML training (link)

  • Data-centric AI resources (link)

🎁 Miscellaneous

Rick and Mortify

Check out this Rick and Mortify episode generator. This team built a generator to create never-before-seen episodes of Rick and Morty leveraging generative AI. (link)

Here’s a link to my storyboard about AI turning everything on Rick and Morty’s planet into a paperclip. I don’t think it’s quite there yet, but it took me less than 30 seconds to generate:

ChatGPT

  • A list of ChatGPT use cases (link).

That’s it! Have a great week and see you next Sunday! 👋

Apologies for the delay this week given the holiday season! We’ll jump back into the regular schedule this Sunday. Cheers!

Thanks for reading Superfast AI. If you enjoyed this post, feel free to share it with any AI-curious friends. Cheers!