Lab

This is a working notebook of experiments, models, and observations.


AI Design as Evergreen Training

I'd like to propose to all the trainers, communicators and designers out there that there be a minimum standard for AI agent/pipeline design: never build an AI system with fewer than two visible stages between input and output.

The communication around AI tends to present the idea that it can make decisions and solve problems in one move: You ask, it thinks, it answers. But there's quite a big and as it turns out unnecessary hole in that design: mysticism/anthropomorphism. When we collapse all the elements of a task into a single prompt, the impression the users is left with is that AI is 'smart' enough to figure out all these things, all the steps needed, what order, what the parameters are etc. Even if you're a truly brilliant prompt designer who tends to be able to work magic with a single prompt to get exactly what you want every time, you're still telling the user or anyone who sees your system, that the AI is doing all the work. You're making your own work invisible and by doing so I'd argue you're also making it less effective. Or at the very least you're making your users more dependent on you, which makes you a bottleneck. Great for job security, terrible for operational contingency planning, and really terrible for scaling your company's AI adoption.

So the proposal is this: If you have a simple prompt that goes input > LLM > Output, add a step. Make it input > LLM > non-operational step > Output. By adding another step, even if it has no function, you're communicating that something complex is happening or could happen here. In other words, the intent is to demonstrate to users and those who request you build something that 1-step is not the default. it's not even common. in many cases the AI harness is doing deterministic tasks before and after the LLM gets involved. Get them to respect your work, but also teach them that AI is more than a brilliant machine. This will help stop them from over-estimating its 'intelligence' and may even help to reduce a few errors here and there.