We can borrow some math from Nyquist and Shannon to understand how much information can be transmitted over a noisy channel and potentially overcome the magic ruler uncertainty from the article:
Loosely this means that if we're above the Shannon Limit of -1.6 dB (below a 50% error rate), then data can be retransmitted some number of times to reconstruct it by:
number of retransmissions = log(desired confidence)/log(odds of failure)
Where confidence for n sigma, using the cumulative distribution function phi is:
confidence = 1 - phi(sigma)
So for example, if we want to achieve the gold standard 5 sigma confidence level of physics for a discovery (an uncertainty of 2.87x10^-7), and we have a channel that's n% noisy, here is a small table showing the number of resends needed:
Error rate Number of resends
0.1% 3
1% 4
10% 7
25% 11
49% ~650
In practice, the bit error rate for most communication channels today is below 0.1% (dialup is 10^-6 to 10^-4, ethernet is around 10^-12 to 10^-10). Meaning that to send 512 or 1500 byte packets for dialup and ethernet respectively results in a cumulative resend rate of around 4% (dialup) and 0.1% (ethernet).
Just so we have it, the maximum transmission unit (MTU), which is the 512 or 1500 bytes above, can be calculated by:
MTU in bits = (desired packet loss rate)/(bit error rate)
So (4%)/(10^-5) = 4000 bits = 500 bytes for dialup and (0.0000001)/(10^-11) = 10000 bits = 1250 bytes for ethernet. 512 and 1500 are close enough in practice, although ethernet has jumbo frames now since its error rate has remained low despite bandwidth increases.
So even if AI makes a mistake 10-25% of the time, we only have to re-run it about 10 times (or run 10 individually trained models once) to reach a 5 sigma confidence level.
In other words, it's the lower error rate achieved by LLMs in the last year or two that has provided enough confidence to scale their problem solving ability to any number of steps. That's why it feels like they can solve any problem, whereas before that they would often answer with nonsense or give up. It's a little like how the high signal to noise ratio of transistors made computers possible.
Since GPU computing power vs price still doubles every 2 years, we only have to wait about 7 years for AI to basically get the answer right every time, given the context available to it.
For these reasons, I disagree with the premise of the article that AI may never provide enough certainty to provide engineering safety, but I appreciate and have experienced the sentiment. This is why I estimate that the Singularity may arrive within 7 years, but certainly within 14 to 21 years at that rate of confidence level increase.
The idea being that business logic gets written in synchronous blocking functional logic equivalent to Lisp, which is conceptually no different than a spreadsheet. Then real-world side effects get handled by imperative code similar to Smalltalk, which is conceptually similar to a batch file or macro. A bit like pure functional executables that only have access to STDIN/STDOUT (and optionally STDERR and/or network/file streams) being run by a shell.
I think of these like backend vs frontend, or nouns vs verbs, or massless waves like photons vs massive particles like nucleons. Basically that there is no notion of time in functional programming, just state transitions where input is transformed into output (the code can be understood as a static graph). While imperative programming deals with state transformation where statically analyzing code is as expensive as just running it (the code must be traced to be understood as a graph). In other words, functional code can be easily optimized and parallelized, while imperative code generally can't be.
So in model-view-controller (MVC) programming, the model and view could/should be functional, while the controller (event handler) could/should be imperative. I believe that there may be no way to make functional code handle side effects via patterns like monads without forcing us to reason about it imperatively. Which means that impure functional languages like Haskell and Scala probably don't offer a free lunch, but are still worth learning.
Why this matters is that we've collectively decided to use imperative code for almost everything, relegating functional code to the road not taken. Which has bloated nearly all software by perhaps 10-100 times in terms of lines of code, conceptual complexity and even execution speed, making perhaps 90-99% of the work we do a waste of time or at least custodial.
It's also colored our perception of what programming is. "Real work" deals with values, while premature optimization deals with references and pointers. PHP (which was inspired by the shell) originally had value-passing semantics for arrays (and even subprocess fork/join orchestration) via copy-on-write, which freed developers from having to worry about efficiency or side effects. Unfortunately it was corrupted through design by committee when PHP 5 decided to bolt-on classes as references rather than unifying arrays and objects by making the "[]" and "." operators largely equivalent like JavaScript did. Alternative implementations like Hack could have fixed the fundamentals, but ended up offering little more than syntactic sugar and the mental load of having to consider an additional standard.
To my knowledge there has never been a mainstream FCIS language. ClojureScript is maybe the closest IMHO, or F#. Because of that, I mostly use declarative programming in my own work (where the spec is effectively the behavior) so that the internals can be treated as merely implementation details. Unfortunately that introduces some overhead because technical debt usually must be paid as I go, rather than left for future me. Meaning that it really only works well for waterfall, not agile.
I had always hoped to win the internet lottery so that I could build and test some of these alternative languages/frameworks/runtimes and other roads not taken by tech. The industry's failure to do that has left us with effectively single-threaded computers which run around 100,000 times slower today (at 100 times the cores per decade) than they would have if we hadn't abandoned true multicore superscalar processing and very large scale integration (VLSI) in the early 2000s when most R&D was outsourced or cancelled after the Dot Bomb and the mobile/embedded space began prioritizing lower cost and power usage.
GPUs kept going though, which is great for SIMD, but doesn't help us as far as getting real work done. AI is here and can recruit them, which is great too, but I fear that they'll make all code look like its been pair-programmed and over-engineered, where the cognitive load grows beyond the ability of mere humans to understand it. They may paint over the rot without renovating it basically.
I hope that there's still time to emulate a true multiple instruction multiple data (MIMD) runtime on SIMD hardware to run fully-parallelized FCIS code potentially millions of times faster than anything we have now for the same price. I have various approaches in mind for that, but making rent always comes first, especially in inflationary times.
It took me over 30 years to really understand this stuff at a level where I could distill it down to these (inadequate) metaphors. So maybe this is TMI, but I'll leave it here nonetheless in the hopes that it helps someone manifest the dream of personal supercomputing someday.
It took me about 15 years (out of 20 in the industry) to arrive at similar ideas. Interestingly, I heard all the arguments many times before but somewhat obscured by the way function programming speaks of things.
For the purpose of this game spliting things into core/shell makes certain things super easy: saving and restoring state, undo, debugging, testing, etc.
And one more bit, relevant to this new reality we find outselves in. Having a bunch of pure functions merged into a very focused DSL makes it easy to extend the systems through LLMs: a description of well-understood inputs and outputs fits into limited context windows.
By the way.
It is true that dedicated languages never arrived but FCIS is not a language feature, it's more like a architectural paradigm.
Or maybe a conscious decision, as neoconservative Robert Kagan writes:
"President Trump has managed in just one year to destroy the American order that was and has weakened America's ability to protect its interests in the world that will be. Americans thought defending the liberal world order was too expensive. Wait until they start paying for what comes next,"
Just wanted to +1 this as a deep thinker who disagrees with the blog post's conclusion. I remember back on the years and decades I wasted dealing with the conceptual flaws inherent to nearly all software, and it breaks my heart.
90-99% of programming is a waste of time. Most apps today have less than a single spreadsheet page of actual business logic. The rest is boilerplate. Conjuring up whatever arcane runes are needed to wake a slumbering beast made of anti-patterns and groupthink.
For me, AI offers the first real computer that I've had access to in over 25 years. Because desktop computing stagnated after the 2000 Dot Bomb, and died on the table after the iPhone arrived in 2007. Where we should have symmetric multiprocessing with 1000+ cores running 100,000 times faster for the same price, we have the same mediocre quad core computer running about the same speed as its 3 GHz grandfather from the early 2000s. But AI bridges that divide by recruiting video cards that actually did increase in speed, albeit for SIMD which is generally useless for desktop computing. AI liberates me from having to mourn that travesty any longer.
I think that people have tied their identity to programming without realizing that it's mostly transcribing.
But I will never go back to manual entry (the modern equivalent of punch cards).
If anything, I can finally think deeply without it costing me everything. No longer having to give my all just to tread water as I slowly drown in technical debt and deadlines which could never be met before without sacrificing a part of my psyche in the process.
What I find fascinating is that it's truly over. I see so clearly how networks of agents are evolving now, faster than we can study, and have already passed us on nearly every metric. We only have 5-10 years now until the epiphany, the Singularity, AGI.
It's so strange to have worked so hard to win the internet lottery when that no longer matters. People will stop buying software. Their AI will deliver their deepest wish, even if that's merely basic resources to survive, that the powers that be deny us to prop up their fever dream of late-stage crony capitalism under artificial scarcity.
Everything is about to hit the fan so hard, and I am so here for it.
> 90-99% of programming is a waste of time. Most apps today have less than a single spreadsheet page of actual business logic.
I would very much like to know the kind of app you’ve seen. It’s very hard to see something like mpv, calibre, abiword, cmus,… through that lens. Even web apps like forgejo, gonic, sr.ht, don’t fit into that view.
Fair enough. I meant social network websites and social media apps like Facebook and TikTok that could have been made in a weekend using HyperCard, FileMaker, Microsoft Access, etc, if we had real reactive backends similar to Firebase, Airtable and Zapier, which come so close to almost working for normal people but miss the mark fundamentally somehow.
I know that programming has gone terribly wrong, but it's hard for me to articulate how, because it's all of it - the entire frontend web development ecosystem, mobile development languages and frameworks, steep learning curve languages like Rust that were supposed to make things easier but put the onus on the developer to get the busywork right, everything basically. It's like trying to explain screws to craftsmen only familiar with nails.
In the simplest terms, it's because corporations are driving the development of those tools and vacuuming up all profits on the backs of open sources maintainers working in their parents' basements, rather than universities working from first principles to solve hard problems and give solutions away to everyone for free for the good of society. We've moved from academia to slavery and call it progress.
Seconded. Another way of saying this is that avoiding politics benefits the status quo.
Since the status quo is inherently conservative, that has a stifling effect on innovation - which is inherently liberal. Which is ironic for a site dedicated to disruption. Hence the cognitive dissonance.
I try to entertain opposing viewpoints in all of my comments, even if I don't always agree with them. So while I find it most practical to live conservatively, that doesn't mean that I wish that for the world. It's important to remember that FDR - a liberal - was one of Reagan's heroes. I think that we can imagine a Star Trek style post-scarcity geopolitical reality without abandoning the ethos which got us this far.
Now, regardless of all that, I still think that HN has the best ranking algorithm around. So I would say that if it wants to get serious about getting back to meritocracy, funding real work on hard problems, setting a positive example through intellectual honesty, etc, then it should consider revising its flagging policy.
A proof of concept might be to move flagged posts below the fold past slot 31, rather than removing them completely. Then they could bubble back up on their own merit. Or maybe each flag costs 10 slots, something like that. And all flags should go through human review to prevent gaming, if they don't already.
With nobody at Apple handling the engineering problem of implementing user requests, we're stuck with what we got. So I highly recommend App Tamer by St. Clair Software (no affiliation), which lets you set how much CPU percentage each process can use:
It does cost $14.95 USD, but it's given me my computer back for years now. I have Spotlight Indexer set to 10%, although I'm using an old version of macOS and don't know if that's mdutil now or if Apple has outsmarted its throttling. I also set web browsers to 10% when they're in the background. And you can always message the developer with feature requests.
A bit of a rant: I honestly feel that we've done process scheduling wrong in most OSs and apps. It should have always been up to the user, along with granting permissions as needed. And I can't believe that no web browser has implemented turning JavaScript off after perhaps 10 seconds for example, so that we can have as many tabs open as we want. Instead we've let the technology order us what to do. It's all just so wrong. But the barriers to entry for writing a new browser are so high that only large organizations can do it, and they choose not to, so help isn't coming. Although I think with the arrival of AI, we're going to start seeing real software again that makes a mockery of the status quo and hopefully eats its lunch.
I've been Apple fan for all those years but it is becoming harder and harder. M1 will be my first HW replaced before it stops working. Just because of software.
Funny, I'm the opposite. Since information wants to be free, and storage/compute get more affordable every year, then really everything ever posted on the web should be mirrored somewhere, like Neocities.
I grew up in the 80s when office software and desktop publishing were popular. Arguably MS Access, FileMaker and HyperCard were more advanced in some ways than anything today. There was a feeling of self-reliance before the internet that seems to have been lost. To me, there appears to be very little actual logic in most websites, apps and even games. They're all about surveillance capitalism now.
Now that AI is here, I hope that hobbyists begin openly copying websites and apps. All of them. Use them as templates and to automate building integration tests. Whatever ranking algorithm that HN uses, or at least the part(s) they haven't disclosed, should be straightforward to reverse engineer from the data.
That plants a little seed in the back of every oligopoly's psyche that ensh@ttification is no longer an option.
If "information wants to be free," doesn't that cut both ways? It applies equally to the personal data that I don't want to upload to an age gate as it does to the information that people want to keep behind an age gate.
Came here to say this. Journalism is increasingly seen as part of the commons (public good), like utilities. Under free market forces, it turns into propaganda for capitalists (moneyed interests - not workers), the same way that private utilities charge extortionary prices because people have little alternative.
So the litmus test I use is that if a politician works to undermine public funding of journalism, then they're the product of lobbyists, or at least beholden to moneyed interests in some way, and not a public servant.
I've been following a story where Elon Musk's xAI is building an 88 acre solar farm next to its Colossus data center near Memphis TN after public outrage due to running 35 methane gas turbines without a permit, which increased NOx emissions enough to allegedly impact health:
88 acres = 356,124 m2
4.56 kWh/m2 per day solar insolation (4.5 is typical for much of the US)
4.56 kWh/m2 per day \* 356,124 m2 = 1,623,924 kWh/day = 67,664 kW = 67.66 MW average
1000 W/m2 \* 356,124 m2 = 356 MW peak
They're estimating that they'll get 30 MW on average from that, but I'd estimate more like 15 MW at a solar panel efficiency just over 20%. Still, the total cost for that power should be less than for turbines, since solar is now the cheapest electricity other than hypothetical nuclear (assuming an ideal breeder or waste-consuming reactor and excluding mining/waste externalities/insurance).
30 MW is still only 10% of the the 300 MW used by the data center. But there's lots of land out there, so roughly 1000 acres per data center doesn't seem that extreme to me. That's a 4 km2 or 1.5 mile2 lot, or about 2 km or 1.25 miles on a side.
Basically every GPU server uses 1 kW (about 1 space heater), which puts into perspective just how much computing power is available at these data centers. Running a GPU continuously at home would need 24 kWh/day, so with > 20% efficiency panels that's 4.5*.2 = 0.9 kWh/m2 per day, so 26.67 m2, so at 2 m2 per commercial solar panel and assuming that my math is right: that's about 14 panels considering nights and seasons.
It's interesting to think just how many panels it takes to run a GPU or space heater continuously, even when they put out 500 W or 250 W/m2 peak. And how cheap that electricity really is when it's sold for on the order of $0.15 per kWh, or $3.60 per day.
I've found that the very best way to save on your electric bill is to have a few south-facing slider doors and windows, which is like running a space heater every square meter of window. There's just no way that any other form of power generation can compete with that. Also, I feel that we're doing it wrong with solar. This analysis shows just how much better alternatives like trough solar and concentrated solar (mirrors towards solar panels) might be cost-wise. On an ironic note, solar panels now cost less than windows by area, and probably mirrors.
I'll never forget the feeling of the whoosh when I was working as a furniture mover in the early 2000s and felt the implosion when a cardboard box collapsed and dumped a large CRT TV face-down on the driveway, blowing our hair back. When the boss asked what happened to the TV, I said it fell, and our lead man (who had set it on the box) later thanked me for putting it so diplomatically.
That was nothing compared to the time the CAT scan machine fell face down off the lift gate on the back of the delivery truck because our driver pushed the wrong button and tipped it instead of lowering it, but I missed the flack from that because I was on a move somewhere thankfully. Afterwords he was forever known as the quarter million dollar man.
Oops just saw this! I never heard, last time I saw it, it was sitting busted up against a wall with blankets thrown over it. The warehouse usually ate breakages because that was cheaper than insurance overhead in the long run.
https://en.wikipedia.org/wiki/Nyquist_rate
https://en.wikipedia.org/wiki/Noisy-channel_coding_theorem
Loosely this means that if we're above the Shannon Limit of -1.6 dB (below a 50% error rate), then data can be retransmitted some number of times to reconstruct it by:
Where confidence for n sigma, using the cumulative distribution function phi is: So for example, if we want to achieve the gold standard 5 sigma confidence level of physics for a discovery (an uncertainty of 2.87x10^-7), and we have a channel that's n% noisy, here is a small table showing the number of resends needed: In practice, the bit error rate for most communication channels today is below 0.1% (dialup is 10^-6 to 10^-4, ethernet is around 10^-12 to 10^-10). Meaning that to send 512 or 1500 byte packets for dialup and ethernet respectively results in a cumulative resend rate of around 4% (dialup) and 0.1% (ethernet).Just so we have it, the maximum transmission unit (MTU), which is the 512 or 1500 bytes above, can be calculated by:
MTU in bits = (desired packet loss rate)/(bit error rate)
So (4%)/(10^-5) = 4000 bits = 500 bytes for dialup and (0.0000001)/(10^-11) = 10000 bits = 1250 bytes for ethernet. 512 and 1500 are close enough in practice, although ethernet has jumbo frames now since its error rate has remained low despite bandwidth increases.
So even if AI makes a mistake 10-25% of the time, we only have to re-run it about 10 times (or run 10 individually trained models once) to reach a 5 sigma confidence level.
In other words, it's the lower error rate achieved by LLMs in the last year or two that has provided enough confidence to scale their problem solving ability to any number of steps. That's why it feels like they can solve any problem, whereas before that they would often answer with nonsense or give up. It's a little like how the high signal to noise ratio of transistors made computers possible.
Since GPU computing power vs price still doubles every 2 years, we only have to wait about 7 years for AI to basically get the answer right every time, given the context available to it.
For these reasons, I disagree with the premise of the article that AI may never provide enough certainty to provide engineering safety, but I appreciate and have experienced the sentiment. This is why I estimate that the Singularity may arrive within 7 years, but certainly within 14 to 21 years at that rate of confidence level increase.
reply