LLMs don't trust AI
I've noticed that when I ask an LLM to implement an AI-powered system, it usually does it in the wrong way. It wraps the model in so many guardrails and hard-coded rules that the AI part loses its point. It almost becomes a deterministic system.
I have to tell it "let AI do its thing" and the result gets much better. Maybe it's the training data: most of the code it learned from predates LLMs, when good software meant deterministic software. Or maybe it just doesn't trust AI enough. I bet I'm not the only one who's noticed this.