Reading today's open-closed performance gap
The open-closed performance gap is driven by evolving benchmarks, differing training domains, and industry shifts, with open models struggling to keep up due to limited evidence on their capabilities in newer domains. Leading open model labs are supported by data industry dynamics, creating a disparity in performance and relevance. Economic pressures and the adoption of coding and terminal tasks favor open models, but their ability to maintain performance in complex, agentic tasks remains a challenge.
More from Interconnects
I wrote an AI textbook — how long until AI can do it better?
Reflections on AI's writing ability and how AI models get more capable.
interconnects.ai··Mads Kristian Nylund·AI EducationAI GovernanceAI Ethics
5 useful things you'll learn in my new post-training textbook (shipping now!)
After a few long years of finding time to document my lessons from training open models, my post-training book is done!
interconnects.ai··Mads Kristian Nylund·AI EducationAI DevelopmentAI Tools
Lessons from the hacks
Musings on model alignment, what determines safety, and where we go from here.
interconnects.ai··Mads Kristian Nylund·AI InfrastructureAI GovernanceAI Ethics