Latest open artifacts (#21): Open model bonanza! Gemma 4, DeepSeek V4, Kimi K2.6, MiMo 2.5, GLM-5.1 & others. On CAISI's V4 assessment.
The evaluation of open models like Gemma 4, DeepSeek V4, and GLM-5.1 revealed significant performance gaps compared to closed models, with open models lagging in benchmarks and real-world tasks. The assessment highlighted the lack of standardized benchmarks, such as those used in Claude Code or OpenCode, which exacerbates the disparity in capabilities. Open models, while showing promise in certain areas, still face challenges in performance and scalability, emphasizing the need for more comprehensive and fair benchmarking frameworks.
More from Interconnects
I wrote an AI textbook — how long until AI can do it better?
Reflections on AI's writing ability and how AI models get more capable.
interconnects.ai··Mads Kristian Nylund·AI EducationAI GovernanceAI Ethics
5 useful things you'll learn in my new post-training textbook (shipping now!)
After a few long years of finding time to document my lessons from training open models, my post-training book is done!
interconnects.ai··Mads Kristian Nylund·AI EducationAI DevelopmentAI Tools
Lessons from the hacks
Musings on model alignment, what determines safety, and where we go from here.
interconnects.ai··Mads Kristian Nylund·AI InfrastructureAI GovernanceAI Ethics