Modern Chaos
Posts
MC.46: Is Llama 3.1 the End of Closed-Source AI Dominance?

MC.46: Is Llama 3.1 the End of Closed-Source AI Dominance?

Explore the impact of Meta's Llama 3.1 release, featuring a 405B parameter model rivaling top closed-source AI systems. Discover how this open-source breakthrough could reshape AI research, innovation, and industry competition.

Samuel ROY
July 25, 2024

In partnership with

Hey there,

Meta released Llama 3.1, including a 405B parameter model that's as good as the best closed-source systems. This is big - it's the first time a top-level language model is open for anyone to use and study.

Huge congrats to @AIatMeta on the Llama 3.1 release!
Few notes:
Today, with the 405B model release, is the first time that a frontier-capability LLM is available to everyone to work with and build on. The model appears to be GPT-4 / Claude 3.5 Sonnet grade and the weights are… x.com/i/web/status/1…
— Andrej Karpathy (@karpathy)
8:13 PM • Jul 23, 2024

Key points:

405B model matches or beats GPT-4 and Claude 3.5 Sonnet on many benchmarks
8B and 70B versions also released, with the 8B model showing impressive capabilities when fine-tuned
128K token context window
Built-in tool use for search, Python, Wolfram Alpha etc.
Extensive use of synthetic data for improved performance across many domains

Now researchers and companies have access to top-tier AI that was previously locked away.

It's like when Unix was released for open source software. Meta is positioning itself as the open foundation for AI, unlike other major players.

This could lead to:

Faster research and innovation
More competition and lower prices for AI services
Regulatory challenges, as the EU considers 405B+ models "high risk"

Meta is betting that openness will drive progress and make them a leader, a strategy praised by many, like George Hotz, questioning the ethics of researchers working at closed labs.

Not only were the 405B Llama weights released, they also released a paper explaining how it was made. Nice!
How does any self respecting ML researcher still work at a closed lab? You aren't saving the world from evil AI, you are doing useless duplicate work.
— George Hotz 🌑 (@realGeorgeHotz)
6:38 PM • Jul 23, 2024

Meta is also releasing tools like llama-agentic-system for complex tasks. This is similar to their strategy with React, aiming to create a full AI development stack. The repo got 1,000+ stars in less than 24 hours.

What do you think? Are closed AI labs doomed? Let me know in the feedback section!

Cheers,

Sam

Learn AI in 5 Minutes a Day

AI Tool Report is one of the fastest-growing and most respected newsletters in the world, with over 550,000 readers from companies like OpenAI, Nvidia, Meta, Microsoft, and more.

Our research team spends hundreds of hours a week summarizing the latest news, and finding you the best opportunities to save time and earn more using AI.

What do you think of this email?

You can add more feedback after choosing an option, this helps a lot 👍

Enjoyed this newsletter? Forward it to a friend and have them signup here.

Until next Thursday 🎉

Reply

or to participate.