Which matters more? Pre-training compute or test-time compute? What are the scaling laws that maximize performance over constrained compute budgets?
Interpreting AI 2027's timelines in light of METRs long task research
What are embeddings and how do they work? Let's dive in.
Diving into Meta's Llama 2 model and how it compares to SOTA open- and closed-source LLMs.
Bigger isn't always better in LLMs. Larger context windows fail across a variety of LLM tests.
Models suffer from catastrophic forgetting and data poisoning when trained on synthetic data, new research shows.
GPT-4 is rumored to be a Mixture-of-Experts, totaling 1.76T parameters. How does MoE work and why is it so powerful?
How red-teaming regulates LLMs, synthetic vs real data and fun out-painting with Photoshop.
What are the Chinchilla scaling laws? How to read DeepMind's paper on Compute-Optimal Scaling Laws
