AI Coding as engineering
Andrej Karpathy: From Vibe Coding to Agentic Engineering w/ Stephanie Zhan - YouTube Stop Prompting Claude. Use Karpathy’s Method Instead. - YouTube
Just from picking apart how Karpathy works, you can see that the deciding factor in using AI is still being clear about your own requirements. Look at his first layer, it says so plainly, right? AI isn’t a person. It can’t do a great job of understanding what you want, so the very first layer asks you for a definite goal and a spec built around that goal. The way I see it, you don’t even necessarily have to write that spec yourself, but the point is that you have to review it the whole way and stay in it the whole way, checking whether the spec the AI produced actually matches what I want and whether it’s heading toward the goal I want. If the spec isn’t definite enough, the AI really can’t do much of a job of building the thing you want.
His second layer is the details of getting that spec implemented. What makes a good prompt, for instance? A good prompt tells it plainly and clearly what to do, instead of, ugh, make this better for me. But what counts as better? The AI has no way of knowing what kind of good you have in mind. So getting your requirements across to the AI plainly and clearly is the best way to get the result I want out of it.
In other words, on these two things, the way the person running AI at a top company works and the way I’ve felt out day to day are broadly the same. I’m not polishing my own badge here, it’s just that this at least shows my understanding and use of AI is pointed the right way. Mostly because this is my own sore spot too. I keep catching myself saying, make it a bit better for me, take it in that direction. And the vaguer my requirements are, the harder it is for me to lay out what I personally need, the more its output makes my blood boil, the less I can hold it together. Whereas the clearer I am, the more definite the direction I give, the more orderly it is, laid out sorted into categories, the better it builds.
Obviously this isn’t only about how strong the model is. The other side of it is, look, understanding between two people is already hard, let alone with an AI. However strong it gets, there’s no way for it to know what my personal definition of good actually is. It’ll probably build whatever is biggest inside its statistical model, and that might not be the effect I want at all.
Take the classic car wash prompt. Even with Opus 4.7, if you ask should I walk or drive, it’ll answer that I should walk, because the distance is too short. What I think is going on underneath is that the AI hasn’t really seen into, or inferred, the situation I didn’t tell it about: I’m going to get my car washed, but my car isn’t already parked at the car wash. It assumes by default that I’ve already dropped the car off, and anyway it doesn’t think I should be driving over a distance this short, that driving over isn’t good advice. Actually the moment the user says should I drive or walk, that already hints the car isn’t sitting in the lot, but subtext this vague clearly can’t set off its reasoning. Can I call the model itself stupid, or not smart? Not really. It can untangle nested code many times more complicated than this logically. The problem here is clearly that vague, unclear phrasing left the Agent unable to work out what my actual situation was, so of course it couldn’t correctly arrange what I should do next based on that situation.
But this kind of inference about subtext and unstated circumstances gets better as the models get better. Take 4.8 now. It still opens by telling you you’re better off walking, but it also gives a very definite conditional in the answer: if the thing being washed is your car, then you can’t leave the car at home, you definitely have to drive, because the car is the object being washed; but if you just want to go ask about prices, or just check something, then walking is better, because the distance doesn’t call for driving. So the newest model really does give the best result. The first answer it throws out is still walk, but it’s obviously assuming I’m asking about the second case, and it also says plainly that if my car hasn’t been driven over, driving is still best when the point is to wash it. The newer model is more aware of the boundary conditions on its own answer, unlike the older one that just handed you a result without saying which situation calls for what.
The last layer is an effective workspace, a context environment that lets the Agent do its best work, things like standard MD files, enough reference material, the project’s background, the process it went through before. These are all constraints on it in the context sense. In a way it’s the same constraint the Harness is trying to achieve, except here it means constraints from context, the working environment specifically. However good the Harness is, if you haven’t given it a decent working environment it still won’t do the job well.
He also singles out the knowledge base, but to me a knowledge base is at bottom one part of that context environment, and it’s only that a good one makes both lookup and upkeep more convenient. From the most lightweight angle, a standard MD-level knowledge base is already enough, unless the project really is enormous in scale and the knowledge involved really is scattered across a huge number of files. In that case you might genuinely need a dedicated knowledge base tool.
Like Skills made specifically for the work we ourselves do, instead of always reaching for generic Skills, because that thing is by nature a generic workflow and can’t be applied to your own workflow. What I’ve got here is very particular small details, operations, team style, and a generic one definitely can’t be the best fit, so what I need is to build the Skills that work best for me, right? So having your own customized Skills matters a lot, and building them isn’t especially hard either, right? First I, well, if I’ve already written my past working process down into a document, then I can have it produce a Skill for me straight off that past process, which is exactly why recording the intermediate process matters so much. Once those intermediate steps have turned into a context environment, it can make tacit knowledge explicit, and then pull all that made-explicit tacit knowledge together into Skills you can use generally. Writing down how you work is important, and it’s a good way to get more out of AI.
And the last one, make the AI’s boundaries definite. What it can do, what it can’t do, you have to tell it very clearly. This can be a list, something I build up as I go: oh, it’s no good here, no good there, and I patch it in bit by bit. This is still part of context management, it’s just that here it means Rules in particular, a context document of its own. Of course if there isn’t that much of it, there’s nothing wrong with writing it into Agents.MD, but the important thing is having a definite boundary like this at all.
But for the hardest constraints, say, I just won’t allow you to read a certain file, not under any circumstances, I just won’t allow you to use certain commands, not under any circumstances, that’s when you need Hooks. You have to use a hook to intercept it outright rather than trusting the AI’s own behavior. Because in some cases, even after you’ve said don’t do this thing, it does it anyway. Don’t look at that file, there’s sensitive material in there, and it looks anyway. Don’t delete it, and it deletes it. It might even be that because you said all this, it went into its context, and now it’s more likely than usual to do those very things.
So the best approach is a mandatory programmatic hook that intercepts it directly and makes execution impossible in the first place, which is part of permission management in itself. Rather than trusting that the AI, having been given my constraints, won’t do it, use hard permission management so it can’t, which is the safer move. There’s one line from Karpathy that cuts deepest: You can outstore your thinking, but you can’t outstore your understanding.
And that’s exactly my own view, that some things absolutely cannot be outsourced. Once my own understanding and my own thinking (even though literally he folds thinking in here, I personally read that more as a procedural process, not as my thinking about direction, more like thinking about the concrete implementation) get outsourced too, then the one who eventually withers and gets replaced is me. The core point is always keeping the human primary in the process of working with AI, or in human-machine interaction generally.
Using tools to extend our own capability has always been where we humans, where humans hold the advantage over other animals, so there’s no need to refuse tools. But making sure the tool serves me rather than the other way round is, to me, the most important part of using tools. Though people and tools do shape each other, how I use a tool and make it fit me, and in the other direction the tool shapes me. The AI era needs an AI-era system for developing people and an AI-era way of thinking, but it will never replace thinking and understanding themselves. You have to definitely know and understand your own goal before AI can serve you better; you have to clearly understand what the AI is actually doing before you can keep your own system sound. A good engineer has to know his own boundaries, and that’s what it takes to build a good system. There’s no such thing as an absolutely sound system, of course, but you can manage to build one whose boundaries are clear enough, and that’s enough.
All of the above is precisely why Agentic engineering, set against Vibe Coding, gets called engineering. What it does is optimization in the engineering sense, and that’s why it’s called engineering. Classic Vibe Coding is just a wishing machine, I hand it the thing and hope it automatically turns out a result good enough. But that isn’t how you can work now, and if you want a system that runs well in production, Vibe Coding alone isn’t enough. I need engineering means of all kinds to make sure the AI can run smoothly, so it gradually builds what I asked for.
So we can say quite naturally that today’s Agentic engineers are doing engineering, and not what some people sneer at, oh, you just make wishes at an AI. Anyone who’s really used AI understands that if you stop at the Vibe Coding level you can’t build a decent production-grade system. What people inside the top companies are actually using, what they’ve been shifting toward, is doing one engineering optimization after another inside an Engineering framework.
Translation note. I wrote this in Chinese. This English version is an LLM translation, so the wording is not mine even though the thinking is. Original: 作为工程的 AI Coding.