- Superslow AI Newsletter
- Posts
- Superfast AI (12/3/2022)
Superfast AI (12/3/2022)
Hi everyone, welcome to the first Superfast AI newsletter. Today we’ll cover ChatGPT, NeurIPS, and Galactica… superfast. I had a lot of fun trying out a few generative AI products as well, which we’ll cover. Okay, let’s dive in.
🗞 News
ChatGPT
OpenAI has trained a model called ChatGPT. It can:
engage in dialogue in a conversational, friendly manner
answer follow-up questions
admit its mistakes
challenge incorrect premises
reject inappropriate requests
The model was trained using Reinforcement Learning from Human Feedback (RLHF), using the same methods as InstructGPT. This release is the latest step in OpenAI's iterative deployment of generative AI systems. (Try it here!)
All the best examples of ChatGPT, from OpenAI:
— Ben Tossell (@bentossell)
10:56 AM • Dec 1, 2022
NeurIPS Conference
NeurIPS is one of the largest and best known conference for ML research, and it was held in New Orleans last week.
Galactica
A version of Galactica, an ML model that summarizes and generates information about scientific research papers, was released last month. It received a fair amount of criticism, and the beta has since been withdrawn. Here’s a thread with some key critiques:
I asked #Galactica about some things I know about and I'm troubled. In all cases, it was wrong or biased but sounded right and authoritative. I think it's dangerous. Here are a few of my experiments and my analysis of my concerns. (1/9)
— Michael Black (@Michael_J_Black)
6:47 AM • Nov 17, 2022
MIT Tech Review (Why Meta’s latest large language model survived only three days online)
A blog from Alberto Romero (Galactica: What Dangerous AI Looks Like)
Other News
Stable Diffusion 2.0 is now live! Stability AI wrote an accompanying blog post on how the new model will enable high-quality image creation in fewer steps with new samplers. They also included a safety filter and inpainting features in the new version. (Link)
Disney’s new neural network can change an actor’s age with ease (Link)
Elon Musk claims Neuralink is about ‘six months’ away from first human trial (Link)
Black Mirror Tech
Are you looking at Billboards, or are Billboards looking at you? (Link)
Billboards are finding ways to deliver targeted advertising by assessing how much time you spend engaging with their ads. Two examples from The Batch:
One advertisement promoted Emoji Movie by showing passersby with their faces overlaid by emojis.
Another that was intended to raise awareness of suicide risk depicted a girl with a sad expression who smiled when people looked at the ad.
📚 Concepts & Learning
Model Training
Compute is expensive. How do you train your model effectively while minimizing spend (Link to Google Research’s blog). Two methods:
UL2R: a lightweight second stage of pre-training that focuses on denoising your data
Flan: fine-tuning a language model on a collection of datasets that are instruction-style (think prompts, “Write me an essay…”, “I’m looking for…”)
Related: GPT-3 quality models for <$500k (Mosaic ML) and Chinchilla Scaling Laws (Arxiv)
Generative Text vs Search
An ongoing debate: Will language models like ChatGPT replace search engines like Google?
Google is done.
Compare the quality of these responses (ChatGPT)
— josh (@jdjkelly)
6:29 PM • Nov 30, 2022
My guess is no. Google continues to provide valuable offerings with search discovery. To test out that theory, let’s try a little generative text vs search experiment.
Prompt: What are some great Christmas gift suggestions you can give me?
100 Best Christmas Gifts 2022 (Parade)
Every Holiday Gift Guide (NY Mag)
75 Unique Christmas Gifts for Everyone (Cosmopolitan)
You.com
Neeva
Overall, search vs text generation serve different purposes. Search optimizes for discovery, while generation optimizes for delivery.
NeRF
What is NeRF? A way to turn 2D images into 3D videos. (Research paper)
Neural Radiance Fields (NeRF) is a computer graphics rendering technique. It uses a deep neural network to learn the underlying physical properties of a 3D scene, and is able to produce fabricated (but realistic) scenes from its understanding of light, color and texture at any particular camera location. This removes the need for complex ray tracing algorithms, and can greatly reduce the production cost for digital work. NeRF has been shown to outperform traditional rendering methods in terms of both image quality and computational efficiency. Here’s a creative example:
1/ Now you can create "drone" shots from your phone footage, thanks to NeRF
Collab with @justLV / software @nerfstudioteam / see below for our process
#NeRF #neuralrendering#artificialintelligence#instantNeRF— Karen X. Cheng (@karenxcheng)
3:19 PM • Oct 25, 2022
AI Theory
AI teaches AI theory
Prompt: Can you explain what the attention mechanism is from the paper “Attention is All You Need” without using technical jargon? (source: ChatGPT).
Bonus: A breakdown of stable diffusion with visuals (Link)
🎁 Miscellaneous
Upgrade your emails
I mentor a young lad with poor literacy skills who is starting a landscaping business. He struggles to communicate with clients in a professional manner.
I created a GPT3-powered Gmail account to which he sends a message. It responds with the text to send to the client.
— Danny Richman (@DannyRichman)
9:56 AM • Dec 1, 2022
Deploying no-code and GPT-3 to send and automate professional emails (Link)
Rap Inspo
AI that can generate rap inspiration for you (Link)
Generative Images
Prompt: japanese noodle shop, neon lighting, customers sitting outside, rain falling, dramatic lighting, digital painting, oil painting, digital art, concept art, trending on artstation
[Original, DALL-E 2, Stable Diffusion’s DreamStudio]
That’s it! Have a great week and see you next Sunday! 👋
🎤 ChatGPT, can you tell me a joke?
