It’s shocking to me that people even ask this type of question. How do you not see the difference between a machine that will hallucinate something random if it doesn’t know the answer vs a human that will logic through things and find the correct answer.
Because I've seen the results ? Failure mode of LLMs are unintuitive, and the ability to grasp the big picture is limited (by context mostly I'd say), but I find CC to follow instructions better than 80% of people I've worked with. And the amount of mental stamina it would take to grok that much context even when you know the system vs. what these systems can do in minutes.
As for the hallucinations - you're there to keep the system grounded. Well the compiler is, then tests, then you. It works surprisingly well if you monitor the process and don't let LLM wander off when it gets confused.
Because humans also make stupid random mistakes, and if your test suite and defensive practices don't catch it, the only difference is the rate of errors.
It may be that you've done the risk management, and deemed the risk acceptable (accepting the risk, in risk management terms) with human developers and that vibecoding changes the maths.
But that is still an admission that your test suite has gaping holes. If that's been allowed to happen consciously, recorded in your risk register, and you all understand the consequences, that can be entirely fine.
But the problem then isn't reflecting a problem with vibe coding, but a risk management choice you made to paper over test suite holes with an assumed level of human dilligence.
> How do you not see the difference between a machine that will hallucinate something random if it doesn’t know the answer vs a human...
Your claim here is that humans can't hallucinate something random. Clearly they can and do.
> ... that will logic through things and find the correct answer.
But humans do not find the correct answer 100% of the time.
The way that we address human fallibility is to create a system that does not accept the input of a single human as "truth". Even these systems only achieve "very high probability" but not 100% correctness. We can employ these same systems with AI.
Almost all current software engineering practices and projects rely on humans doing ongoing "informal" verification. The engineers' knowledge is integral part of it and using LLMs exposes this "vulnerability" (if you want to call it that). Making LLMs usable would require such a degree of formalization (of which integration and end-to-end tests are a part), that entire software categories would become unviable. Nobody would pay for an accounting suite that cost 10-20x more.
Which interestingly is the meat of this article. The key points aren’t that “vibe coding is bad” but that the design and experience of these tools is actively blinding and seductive in a way that impairs ability to judge effectiveness.
Basically, instead of developers developing, they've been half-elevated to the management class where they manage really dumb but really fast interns (LLM's).
But they dont get the management pay, and they are 100% responsible for the LLMs under them. Whereas real managers get paid more and can lay blame and fire people under them.
Humans who fail to do so find the list of tasks they’re allowed to do suddenly curtailed. I’m sure there is a degree of this with LLMs but the fanboys haven’t started admitting it yet.
> It’s shocking to me that people even ask this type of question. How do you not see the difference between a machine that will hallucinate something random if it doesn’t know the answer vs a human that will logic through things and find the correct answer.
I would like to work with the humans you describe who, implicitly from your description, don't hallucinate something random when they don't know the answer.
I mean, I only recently finished dealing with around 18 months of an entire customer service department full of people who couldn't comprehend that they'd put a non-existent postal address and the wrong person on the bills they were sending, and this was therefore their own fault the bills weren't getting paid, and that other people in their own team had already admitted this, apologised to me, promised they'd fixed it, while actually still continuing to send letters to the same non-existent address.
Don't get me wrong, I'm not saying AI is magic (at best it's just one more pair of eyes no matter how many models you use), but humans are also not magic.
Humans are accountable to each other. Humans can be shamed in a code review and reprimanded and threatened with consequences for sloppy work. Most,
humans once reprimanded , will not make the same kind of mistake twice.
> Humans can be shamed in a code review and reprimanded and threatened with consequences for sloppy work.
I had to not merely threaten to involve the Ombudsman, but actually involve the Ombudsman.
That was after I had already escalated several times and gotten as far as raising it with the Data Protection Officer of their parent company.
> Most, humans once reprimanded , will not make the same kind of mistake twice.
To quote myself:
other people in their own team had already admitted this, apologised to me, promised they'd fixed it, while actually still continuing to send letters to the same non-existent address.
> How do you not see the difference between a machine that will hallucinate something random if it doesn’t know the answer vs a human that will logic through things and find the correct answer.
I see this argument over and over agin when it comes to LLMs and vibe coding. I find it a laughable one having worked in software for 20 years. I am 100% certain the humans are just as capable if not better than LLMs at generating spaghetti code, bugs, and nonsensical errors.
It's shocking to me that people make this claim as if humans, especially in some legacy accounting system, would somehow be much better at (1) recognizing their mistakes, and (2) even when they don't, not fudge-fingering their implementation. Like the criticisms of agents are valid, but the incredulity that they will ever be used in production or high risk systems to me is just as incredible. Of course they will -- where is Opus 4.6 compared to Sonnet 4? We've hit an inflection point where replacing hand coding with an agent and interacting only via prompt is not only doable, highly skilled people are already routinely doing it. Companies are already _requiring_ that people do it. We will then hit an inflection point at some time soon where the incredulity at using agents even in the highest stakes application will age really really poorly. Let's see!
Your point is the speculative one, though. We know humans can and have built incredibly complex and reliable systems. We do not have the same level of proof for LLMs.
Claims like your should wait at least 2-3 years, if not 5.
That is also speculative. Well let's just wait and see :) but the writing is on the wall. If your criticism is where we're at _now_ and whether or not _today_ you should be vibe coding in highly complex systems I would say: why not? as long as you hold that code to the same standard as human written code, what is the problem? If you say "well reviews don't catch everything" ok but the same is true for humans. Yes large teams of people (and maybe smaller teams of highly skilled people) have built wonderfully complex systems far out of reach of today's coding agents. But your median programmer is not going to be able to do that.
Your comment is shocking to me. AI coding works. I have seen it with my own eyes last week and today.
I can therefore only assume that you have not coded with the latest models. If you experiences are with GPT 4o or earlier all you have only used the mini or light models, then I can totally understand where you’re coming from. Those models can do a lot, but they aren’t good enough to run on their own.
The latest models absolutely are I have seen it with my own eyes. Ai moves fast.
I don’t understand how you’re concluding that the biopsy shouldn’t have happened from that anecdote? Just because a test result is negative doesn’t mean that it shouldn’t have happened. That’s not how practicing medicine works.
Hence the N=1 I started the statement with. If we had a thousand similar situations and it turned out that in, say, 1% of cases there was a lung tumor, then we could debate whether it's worth putting 990 people through a biopsy they don't need so 10 can find out they have a tumor. Maybe that gives us the opportunity to discuss waiting a month and taking another chest x-ray (which has its own negatives) vs. doing an immediate biopsy.
But we don't have 1000 similar cases. As I said, we have 1: mine. And it turned out negative. So 1 person was put through a biopsy they didn't need, and 0 people found out they have a tumor.
My pessimism is mostly rooted in the VC economics of it all. The vision is great, but its a busy space and there's no actual product or business. They basically wrote the guy a check to build the spaceship in space.
Definitely more common to just use streaming services like Spotify, but some people do own their music library. There’s probably a decent sized overlap of those who buy music and those who self host things like jellyfin
Humans have been around for thousands of years. Look at what we've accomplished in the last hundred. We have artificial heart pumps now. In the next two hundred years, if cancer research doesn't slow down too much and if we find some quick fixes for neurodegeneration, I think it's entirely plausible that 90 will become the new 60. I doubt I'll be around for it, and we might never hit the "life extension outpaces people reaching their life expectancy" medical immortality Holy Grail; but in the abstract, there is hope.
On the other hand...all of the medical advances up till now mean some of us (who live in the right place and have enough money) will live better, but up to now, we don't really live longer. People have lived into their 90s for centuries, but a microscopically tiny number even now live into, say, their late 100s. The oldest was 122. And there's nothing concrete on the horizon that says "if we solve this problem, we'll live to 125", much less 200 or 500. If we cured cancer and heart disease tomorrow, that wouldn't change.
Sometime in the next 5 minutes, in evolutionary timescale terms.
We built the first calculating machines yesterday, and a few hours later they took us to the moon. Now we’ve got vastly more powerful ones in our pockets and they have the sum total of all human knowledge and infinite patience for our questions.
Give it a few more minutes. We’ll know soon enough if the sand we’re imbuing with life is our salvation or our doom or something else entirely.
They don't have the sum total of all human knowledge: a lot isn't digitised. Even a large portion of academic knowledge is tied up in oral tradition: how much more is this the case for other fields of endeavour? One cannot learn the local social conventions about waiting tables from reading Not Always Right.
Even in domains where (virtually) all the knowledge is available, and most tasks are exact variations of what has come before, like programming, the most powerful AI systems are mediocre, bordering on competent. Outside this idealised case, they may have "infinite patience for our questions" (up to the token limit, anyway), but they largely lack the capacity to provide answers.
Medical research is about the best example you could pick for something that current-gen AI systems cannot do. Most of the information about the human body is located in human bodies, and wholly inaccessible to every AI system. An extremely important part of medical research is identifying when the established consensus is wrong: how is AI to do that?
There is no reason to believe that LLMs will ever meaningfully contribute to medicine, in much the same sense there is no reason to believe that lawn ornaments will. Pen-and-paper calculations, and the engineering / manufacturing / etc work of humans, took us to the moon: the computers acted as batch processors and task schedulers, nothing more. Medical research done by humans is responsible for the past century of medical improvements. As much as I like computers, they won't be people for the foreseeable future.
Death is horrifying, but an unfounded belief that AI will save you is not a healthy coping mechanism. If you're looking for religion, there are far better ones. And if you don't think you're looking for religion, perhaps the "death gives life meaning" philosophies might suffice? All Men are Mortal by Simone de Beauvoir was presumably some comfort to its author, who also wrote:
> There is no such thing as a natural death: nothing that happens to a man is ever natural, since his presence calls the world into question. All men must die: but for every man his death is an accident and, even if he knows it and consents to it, an unjustifiable violation.
I said something similar in a different thread but the joy of actually physically writing code is the main reason why I became a software developer. I think there is some beauty to writing code. I enjoy typing the syntax, the interaction with my IDE, debugging by hand (and brain) rather than LLM, even if it's less efficient. I still use AI, but I do find it terribly sad that this type of more "manual" programming seems to be being forced out.
I also enjoy walking more than driving, but if I had to travel 50 miles every day for my job, I would never dream of going on foot. Same goes for AI for me. If I can finish a project in half the time or less, I still feel enough accomplishment and on top of that I will use the gained free time for self actualisation. I like my job and I love coding and solving challenging problems, but I also love tons of other stuff that could use more of my attention. AI has created an insane net positive value for me so far. And I see tons of other people who could also benefit from it the same way, if only they spent a bit more time learning how to use it effectively. Considering how everyone and their uncle thinks they need to chime in on what AI is or is not or what it can or can not do, I find most people have frustratingly little insight into what you can actually do already. Even the people working at companies like Amazon or MS who claim to work on AI integrations sometimes seem to be missing some essentials.
I don’t really understand your point about AI freeing up your time to do other stuff at your job. Does your employer let you work less hours since you’re finishing projects sooner? Mine certainly doesn’t, and I’d rather be coding than doing the other parts of my job. But maybe I’m misunderstanding why you were trying to say?
I also would rather a project take longer and struggle through it without using AI as I find joy in the process. But as I said in my original post I understand that type of work appears to be coming to an end.
Maybe think of it this way: If your job is delivering mail and the only transport you have is a horse, your life would probably suck a lot. You'd barely be home and you still wouldn't cover that much distance. Now, if someone gave you a car for your job, you would not just cover a lot more distance and deliver a lot more mail (=happy employer), you would also get to sleep at home every day and not be exposed to the weather all the time (=happy employee).
This is how all industrialisation/automation works in general. A lot more stuff gets produced and people still get to have more quality of life. Code is just another product in the end. Your employer will get more product and you will get to worry about less when you're at home. Imagine you could be a 10x engineer as a normal guy without working 100 hour weeks. And the true 10x guys who still work 100 hours will be able to change the world.
Where I live we have labor rights and if your job is delivering mail and you're given a horse for it, you'd only be expected to deliver as much mail as you can in your contractual work time, which is then limited by the legal limits (8 hours/workday, up to 6 days/week). So you'd be home for most of the day, but delivering less letters per day.
(where I live a car would also be slower for delivering mail than a horse, most delivery people are given trikes, but alas)
This is how all industrialization/automation works in general: When you have a way to deliver faster/more, you're given more mail to deliver in your work time. Your pay does not go up, but any given road blockage or instance of traffic makes you fall behind quota significantly more. You're not paid by how many letters you deliver, but by the hours you work. Maybe you even make less as there's less overtime. Post will then proceed to simply employ less people over time as each employee is made to deliver more letters, then maybe you're part of the people whose jobs are cut. Or they might just reduce wages for everyone anyways, as now the job is much more accessible and there's more supply of labor than there is demand.
This is not an argument against industrialization or automation, but your perspective of what would happen if we had more industrialization is... very narrow.
We must consider the potential future where there's simply not enough work for most people to do (a realistic future now), and how we'll prevent that from going the same way it would currently go (losing income -> losing domicile -> starvation/freezing/etc).
Yes. Drugs are too damn expensive and innovation is too damn slow. Better to allow higher risk options. And if you don't like it, you are always free not to use them.
"If you don't like it don't take it" ignores all of human history. A huge majority of substances can be fatal if ingested. Are you sure you want to live in a world where Elon can speed run figuring out what won't kill "most" people?
I'm on death row. So are you. The only thing that has a chance of giving me a stay of execution is rapid technological advancement so I'm willing to accept a lot of risk here. It's a matter of self preservation.
reply