AI’s evidence engine – how Epoch is mapping machine progress

Epoch AI, launched three years ago, stands as a nonprofit research organization focused on improving society’s understanding of AI’s trajectory through data-driven research and public knowledge sharing. Their commitment to presenting unbiased evidence allows them to inform critical decisions about artificial intelligence development without advocating for specific outcomes, positioning them as an important independent voice in the rapidly evolving AI landscape.

The big picture: Epoch AI works to understand and communicate AI’s development trajectory by conducting research and sharing findings with a broad audience that includes policy experts, journalists, and AI developers.

• The organization maintains neutrality on whether AI development will ultimately benefit society, focusing instead on providing factual, evidence-based information.
• Their commitment to transparency means they share most research publicly, helping to inform high-stakes decisions about artificial intelligence.

Key focus areas: Epoch concentrates on four primary activities to fulfill its mission of tracking and understanding AI’s evolution.

• They maintain an open-source intelligence program tracking AI models, hardware, and supercomputers, regularly publishing short analyses of important trends.
• The organization develops benchmarks like FrontierMath (commissioned by OpenAI) to measure advanced AI capabilities in areas such as mathematics.
• They provide independent evaluations of publicly available AI models through a maintained dashboard that illustrates capability trends.
• Epoch offers consultation services and conducts commissioned research for organizations across various sectors, providing data collection, benchmark development, and model evaluation.

Guiding principles: The organization follows clear priorities when selecting projects and partnerships.

• Projects are evaluated based on their potential to improve understanding of AI’s trajectory and inform important decisions about artificial intelligence.
• Public transparency remains a core commitment, with most research findings shared openly rather than kept private.
• Epoch maintains neutrality regarding AI’s ultimate societal impact, focusing on evidence rather than advocacy.

Why this matters: As AI capabilities rapidly advance, independent organizations tracking development trajectories provide crucial context for policymakers, businesses, and the public to make informed decisions.

• Epoch’s focus on public transparency and neutrality offers a counterbalance to potentially biased perspectives from organizations with direct financial stakes in AI development.
• Their benchmark development work, like FrontierMath, creates standardized ways to measure AI progress in complex domains, helping establish objective assessments of capability advancements.