Overfitting in LLMs is not as problematic as everyone claims. Here are my predictions for 2026:
Overfitting in LLMs is not as problematic as everyone claims. Here are my predictions for 2026:
Think about how we learn. What is overfit learning if not our school grading system? We teach in a highly specific way, expecting students to apply exact techniques to a test. We all follow this pattern for at least 15 years of our lives.
The hope is that by applying the same formulas repeatedly, some will eventually discover how to actually learn and find the techniques that work for them, whether that's singing a math formula or rewriting a whiteboard three times.
Humanity has almost always taught and learned this way. It is nearly impossible to avoid overfitting in AI when we haven't even discovered a better way for ourselves.
This overfit learning has brought us a society of specialized jobs, and that will hold true for AI as well. If I were to bet on anything for 2026, it would be the rise of specialized models.
Models won't just get larger; they will get smaller and more specialized in specific techniques or tasks. If a task is in high demand by the community, there will surely be a model dedicated to it.
Consider these examples:
- Chat Specialists: This is the most obvious case. We will see models with pre-built actions and high accuracy specifically for Sales or Customer Success.
- Assessment Resolvers: As benchmarking for AI and humans becomes more common, we'll see specialist models designed to navigate diverse question types, incorporating images, complex math, and multi-choice logic.
- AI Reviewers: We are currently overwhelmed by reviews for everything: films, PC boards, best smartphones. AI can perform this flawlessly, provided it has the right context and direction to create a high-quality critique.
In what other areas do you think AI will become a specialist?