We’re about to learn a painful lesson about delayed gratification in software engineering.

New data from China, 26,811 students tracked January 2023 through June 2025. Students using AI for homework saw their scores jump 20 percent. Completion time dropped nearly half. They aced the assignments.

Then exam season came. Those same students scored 20 to 40 percent worse when they couldn’t use the tool.

The homework phase is over. The exam phase is coming.

We’re doing this in software right now. Vibe coding feels incredible. Features ship fast. Nobody’s asking what happens in Month 18 when the original dev has left and nobody understands the codebase.

Commercial pilots fly with autopilot for most of every flight. They’re required to maintain manual flying proficiency regardless. If the system fails mid-air and the pilot can’t take over, people die.

Most teams using AI right now have forgotten how to fly manually. They’ve become passengers in their own systems. The autopilot flies, nobody checks instruments, and the first sign of trouble will be a breach notice or outage.

Three rules:

  1. Command the mission. Define architecture before prompting. Ambiguity kills in code and in flight. Delegate selectively. Offload mechanical work. Keep design and security reviews human. Verify everything. Audit before production.
  1. Never trust the automation without checking instruments.
  1. Quick wins feel good. Sustainable engineering feels boring. Boring keeps systems standing.

Organisations surviving the next two years won’t ship the fastest. They’ll be the ones who remember how to fly without the aids.

people insisting that you actually be skilled, independently of your tools, doesn’t make them Luddites. Rather, being unable to do so makes you a phony.

  • balsoft@lemmy.ml
    2·
    10 hours ago

    The main problem with AI is that it’s pretending to understand more than it actually does right now. It can’t do software architecture. It can’t really write maintainable code without a lot of re-prompting and corrections. And yet it will happily make a bunch of those decisions for you when you ask it to write you an app.

    Compilers are well-defined tools. “Human-readable” code goes in, “machine-readable” code comes out, according to some spec somewhere. Yes, compilers can be nondeterministic and they do a lot of weird stuff sometimes and they choose the wrong approach often, so some developers still need to understand assembly and CPU cache and SIMD and other features of the hardware our code runs on. At the very least every developer needs to understand the performance tradeoffs they’re making, or we end up with a JS-based start menu in Win11. But at least compilers have a well-defined class of inputs and outputs.

    For AI these issues are amplified a lot. They are non-deterministic and unbounded in what layer of software they operate on or produce. I’d argue every developer who uses AI needs to understand software architecture, approaches to software development, and actually read and understand the code that the AI produced, at a minimum. Otherwise the whole industry will become saturated with and built on top of code no-one understands or can maintain, and it will literally collapse in on itself at some point. Carefully reviewed, selected, maintained code has always been precious, it will become a golden nugget in a sea of pig manure.

    • mindbleach@sh.itjust.works
      1·
      2 hours ago

      It can’t really write maintainable code without a lot of re-prompting and corrections.

      Which is a skillset being learned. One which users are presumably better at than non-users, since they’re practicing it. If the chatbot-that-codes Just Worked, there’d be nothing to discuss; we could consider human coding successfully completed. The people who don’t move up to proper computer science can get on with whatever they wanted software for. We only teach them all of this because they’ve got shit to do, and up until now, it’s been really really hard. Like if beginner astronomy meant telescope engineering.

      Already the key complaint has become “maintainability.” That’s quite a high-level criticism. When I sass specific compilers, it’s for using register-juggling macros that could have been one hardware instruction… not for adding a feature from a one-sentence description, but in a way that will complicate any future refactor. Or would, if refactoring was not also something you can tell the chatbot to do.

      Compiler developers are a hardened core of ultranerds. I couldn’t do it above a toy level. If I needed to understand how they work, in detail, then I would be in trouble. Thankfully it is sufficient to abstract all of their work into ‘C goes in ARM comes out.’ Or at least to mumble about intermediate representations and distract people with Godbolt before running away. Sure, ‘add a button above the widget’ is more pluripotent than a K&R-standard Hello World, but even the foundation you celebrate is layer upon layer of Don’t Worry About It.

      AI coding right now is a blunt instrument. Those often suffice. And this one can already be used to remove layers. To obviate a fat framework. To translate to a faster language. To golf. Or it could format your hard drive if a website tells it to, IDFK. It is a higher class of footgun. We take those.