What a timeless banger, you fill in the blanks on this part:
'Over and over I have found myself asking: "What kind of people worship here? Who is their God? Where were their voices when the lips of __________ <American political leader> dripped with words of interposition and nullification? Where were they when __________ <American political leader> gave a clarion call for defiance and hatred?"'
A bit of nuance: yes, Carney said that but he didn't just offer up the opinion unprompted - it was in response to a direct press question about if China or the US is a more predictable partner right now.
And even then, he didn't lead with "China is!" but wandered his way into offering the assessment.
The context makes his comment on this seem less nakedly provocative (not that it'll matter either way - the headline will be the headline, and the Trump admin will use it however they see fit as usual).
Some of this analysis seems a bit lazy for the Economist.
Apple is in the "AI-related companies in the SP500" group? Microsoft too? Tesla too? Amazon too? But... if these companies' AI efforts fail, 95%+ of their revenues would be unaffected. So big stretch to paint them with that brush.
Nvidia, OK that one is obvious. Meta, Alphabet, OK.
But MOST of the companies listed in that chart are only "AI companies" in the sense that EVERY tech company building peripheral AI into their products is an AI company.
Case in point: if Apple stock goes 'on sale' as part of an AI-bubble sell-off, are you really deciding whether or not to buy based on their AI-ness?
Tesla for example: its stock price has fluctuated down 50%, then up 100% (relative to the dip), in this year alone. Clearly, that's market speculation, not capital + earnings. So how much of that speculation is AI-dependent? Depends on how much coffee investors drink before reading Musk's latest tweets, I guess.
Apple is HEAVILY invested in AI, but you're right: it's earnings are dependent more on iPads than AI right now.
> ...are you really deciding whether or not to buy based on their AI-ness?
Tales as old as time, especially in tech: rich monopolistic incumbents not seeing the writing on the wall of a new paradigm shift; seemingly invincible execs brazenly displaying their (incorrect) hot-takes; and the inevitable enshittification of the new paradigm as it turns from revolutionary movement to ruling-class incentives.
"Companies often have a demo environment that is open" - huh?
And... Margolis allowed this open demo environment to connect to their ENTIRE Box drive of millions of super sensitive documents?
HUH???!
Before you get to the terrible security practices of the vendor, you have to place a massive amount of blame on the IT team of Margolis for allowing the above.
No amount of AI hype excuses that kind of professional misjudgement.
I don't think we have enough information to conclude exactly what happened. But my read is the researcher was looking for demo.filevine.com and found margolis.filevine.com instead. The implication is that many other customers may have been vulnerable in the same way.
Ah, I see now that I read too quickly - the "open demo environment" was clearly referencing the idea that the vendor (Filevine) would have a live demo, NOT that each client wanted an open playground demo account that is linked to a subset of their data (which would be utterly insane).
"The best case scenario is that AI is just not as valuable as those who invest in it, make it, and sell it believe."
This is the crux of the OP's argument, adding in that (in the meantime) the incumbents and/or bad actors will use it as a path to intensify their political and economic power.
But to me the article fails to:
(1) actually make the case that AI's not going to be 'valuable enough' which is a sweeping and bold claim (especially in light of its speed), and;
(2) quantify AI's true value versus the crazy overhyped valuation, which is admittedly hard to do - but matters if we're talking 10% of 100x overvalued.
If all of my direct evidence (from my own work and life) is that AI is absolutely transformative and multiplies my output substantially, AND I see that that trend seems to be continuing - then it's going to be a hard argument for me to agree with #1 just because image generation isn't great (and OP really cares about that).
Higher Ed is in crisis; VC has bet their entire asset class on AI; non-trivial amounts of code are being written by AI at every startup; tech co's are paying crazy amounts for top AI talents... in other words, just because it can't one-shot some complex visual design workflow does not mean (a) it's limited in its potential, or (b) that we fully understand how valuable it will become given the rate of change.
As for #2 - well, that's the whole rub isn't it? Knowing how much something is overvalued or undervalued is the whole game. If you believe it's waaaay overvalued with only a limited time before the music stop, then go make your fortune! "The Big Short 2: The AI Boogaloo".
I was aware of that. Not everyone reads HN but enough people will surely know about this if they continue. If they are not fined then people should just stop being so naive. I'm tired of seeing well-designed lies on Facebook and people clearly believing the lies in the comment section.
edit: by well designed I mean that they put the lies in a chart or something. The lies themselves may be quite obvious to spot, like saying that 86% of marriages in Spain end in divorces (when the reality is hard to measure but more likely about 50%). Still, Facebook users don't seem to spot them (or maybe the ones spotting them don't comment?).
You've got your finger on the pulse of something that open source has always represented to me: freedom of the creator and others to just... do what they want with it (subject to the license of course).
Don't like what the main developer is doing with it? You're free to fork and continue on your way if they don't see it your way. If you lack the skills or time to do that, that's your problem - you're not entitled to the maintainers' labor.
The freedom cuts both ways, and by adding in elements of social contracts and other overlays onto the otherwise relatively pure freedom represented by OSS, you end up with the worst of both worlds.
THAT ALL SAID - there's an important distinction between a given piece of software that's open source versus a "true project", which is larger-scale, more contributors involved, might be part of mission-critical systems, etc, where the social dynamics DO need to careful thought and management.
But even that seems to be more a question of specific types of OSS business models which is related but not the same as the licenses and overall social dynamics around OSS projects.
Before it becomes anything else code is first and foremost art & personal expression.
Code is a very fun form of literature at heart.
Other attributes may be tacked on later, it may be integrated into and transform into an engine or company that has rules and regulations.
If the author treats it as only art, with license choices, etc. then they aren't entitled to treat it like anything at all, it's literally their personal expression.
And this is recognized in the physical world as well. More than people realize, some buildings that are incredibly dangerous are considered sculpture effectively. There is a rickety castle built by mostly one guy in CO that meets this criteria.
I think you are trying to make the point that there is the ethical argument to consider the impact of a decision if your project has grown large enough that there is major dependence on it.
I do agree there is an ethical obligation to make some effort to consider impacts like that, and make an attempt to inform users of your intent, but that's all it is. No one is obligated to be ethical, either, when it comes to a personal project with volunteered time and effort.
I think the argument of obligation becomes a stronger if your project has taken on a lot of contributions from other parties. And yet, those contributors must acknowledge that they are willingly doing so with no promises except the posted license terms.
In fact, the entire movie's point is that simply HEARING others tell you those things doesn't do anything! The inner journey of the character getting to a place where he believes it himself -- or rather believes himself to be worthy of a greater path -- is THE crucial part.
So the example is exactly opposite the author's intent.
That said, I liked the article and agree with its point. In fact, I'd guess that effective leaders all have learned techniques and ability to remain calm/comfortable in having these blunt conversations that cut to the chase (but still value and hear people).
Ultimately I think it’s not really about going to far in one way or the other. I tend to be very blunt in my dealings with people to a fault. I wouldn’t say I’m mean, but like, in order for blunt truths to be effective I think they have to be somewhat rare, so I’m trying to adapt to be more strategic in my bluntness, but most of the time, let things go and maybe subtly steer rather than just calling things out all the time.
Every other type of business I come across or analyze makes me think "Man, there really is nothing as good as SaaS" and I thank my lucky stars I was here for it.
High 80%+ gross margins; high retention/recurring revenues (if you're doing it right); easily metric'd (CAC, LTV, conv%, etc); capital specialized for deploying into it (most VC of the last decade); alignment with clients w.r.t. value/impact (or they don't renew); straightforward lining up of 'value to customer' and pricing; common benchmarks and shorthands for valuation multiples; etc.
Simple business to understand / run / grow, assuming you have a good product in a good market.
Strong agree. For every time that I'd get a better answer if the LLM had a bit more context on me (that I didn't think to provide, but it 'knew') there seems to be a multiple of that where the 'memory' was either actually confounding or possibly confounding the best response.
I'm sure OpenAI and Antropic look at the data, and I'm sure it says that for new / unsophisticated users who don't know how to prompt, that this is a handy crutch (even if it's bad here and there) to make sure they get SOMETHING useable.
But for the HN crowd in particular, I think most of us have a feeling like making the blackbox even more black -- i.e. even more inscrutable in terms of how it operates and what inputs it's using -- isn't something to celebrate or want.
I'm pretty deep in this stuff and I find memory super useful.
For instance, I can ask "what windshield wipers should I buy" and Claude (and ChatGPT and others) will remember where I live, what winter's like, the make, model, and year of my car, and give me a part number.
Sure, there's more control in re-typing those details every single time. But there is also value in not having to.
I would say these are two distinct use cases - one is the assistant that remembers my preferences. The other use case is the clean intelligent blackbox that knows nothing about previous sessions and I can manage the context in fine detail. Both are useful, but for very different problems.
I'd imagine 99% of ChatGPT users see the app as the former. And then the rest know how to turn the memory off manually.
Either way, I think memory can be especially sneakily bad when trying to get creative outputs. If I have had multiple separate chats about a theme I'm exploring, I definitely don't want the model to have any sort of summary from those in context if I want a new angle on the whole thing. The opposite: I'd rather have 'random' topics only tangentially related, in order to add some sort of entropy in the outout.
I've found this memory across chats quite useful on a practical level too, but it also has added to the feeling of developing an ongoing personal relationship with the LLM.
Not only does the model (chat gpt) know about my job, tech interests etc and tie chats together using that info.
But also I have noticed the "tone" of the conversation seems to mimick my own style some what - in a slightly OTT way. For example Chat GPT wil now often call me "mate" or reply often with terms like "Yes mate!".
This is not far off how my own close friends might talk to me, it definitely feels like it's adapted to my own conversational style.
I mostly find it useful as well, until it starts hallucinating memories, or using memories in an incorrect context. It may have been my fault for not managing its memories correctly but I don't expect the average non power user will be doing that.
Claude, at least in my use in the last couple weeks, is loads better than any other LLMs at being able to take feedback and not focus on a method. They must have some anti-ADHD meds for it ;)
Both of you are missing a lot of use cases. Outside of HN, not everyone uses an LLM for programming. A lot of these people use it as a diary/journal that talks back or as a Walmart therapist.
People use LLMs as their therapist because they’re either unwilling to see or unable to afford a human one. Based on anecdotal Reddit comments, some people have even mentioned that an LLM was more “compassionate” than a human therapist.
Due to economics, being able to see a human therapist in person for more than 15 minutes at a time has now become a luxury.
Imo this is dangerous, given the memory features that both Claude and ChatGPT have. Of course, most medical data is already online but at least there are medical privacy laws for some countries.
Yeah, though this paper doesn't test any standard LLM benchmarks like GPQA diamond, SimpleQA, AIME 25, LiveCodeBench v5, etc. So it remains hard to tell how much intelligence is lost when the context is filled with irrelevant information.
Nah, they don't look at the data. They just try random things and see what works. That's why there's now the whole skills thing. They are all just variations of ideas to manage context basically.
LLMs are very simply text in and text out. Unless the providers begin to expand into other areas, there's only so much they can do other than simply focus on training better models.
In fact, if they begin to slow down or stop training new models and put focus elsewhere, it could be a sign that they are plateauing with their models. They will reach that point some day after all.
If I find that previous prompts are polluting the responses I tell Claude to "Forget everything so far"
BUT I do like that Claude builds on previous discussions, more than once the built up context has allowed Claude to improve its responses (eg. [Actual response] "Because you have previously expressed a preference for SOLID and Hexagonal programming I would suggest that you do X" which was exactly what I wanted)
it can't really "forget everything so far" just because you ask it to. everything so far would still be part of the context. you need a new chat with memory turned off if you want a fresh context.
Like I said, the AI does exactly what I intend for it to do.
Almost, as I said earler, like the AI has processed my request, realised that I am referring to the context of the earlier discussions, and moved on to the next prompt exactly how I have expected it to
Given the two very VERY dumb responses, and multiple people down voting, I am reminded how thankful I am that AI is around now, because it understood what you clearly don't.
I didn't expect it to delete the internet, the world, the universe, or anything, it didn't read my request as an instruction to do so... yet you and that other imbecile seem to think that that's what was meant... even after me saying it was doing as I wanted.
/me shrugs - now fight me how your interpretation is the only right one... go on... (like you and that other person already are)
One thing I am not going to miss is the toxic "We know better" responses from JUNIORS
I think you completely misunderstood me, actually. I explicitly say if it works, great, no sarcasm. LLMs are finicky beasts. Just keep in mind they don’t really forget anything, if you tell it to forget, the things you told it before are still taken into the matrix multiplication mincers and influence outputs just the same. Any forgetting is pretend in that your ‘please forget’ is mixed in after.
But back to scheduled programming: if it works, great. This is prompt engineering, not magic, not humans, just tools. It pays to know how they work, though.
It's beyond possible that the LLM Chat Agent has tools to self manage context. I've written tools that let an agent compress chunks of context, search those chunks, and uncompress them at will. It'd be trivial to add a tool that allowed the agent to ignore that tool call and anything before it.
>the things you told it before are still taken into the matrix multiplication mincers and influence outputs just the same.
Not the same no. Models chooses how much attention to give each token based on all current context. Probably that phrase, or something like it, makes the model give much less attention to those tokens than it would without it.
> I am reminded how thankful I am that AI is around now, because it understood what you clearly don't.
We understand what you're saying just fine but what you're saying is simply wrong as a matter of technical fact. All of that context still exists and still degrades the output even if the model has fooled you into thinking that it doesn't. Therefore recommending it as an alternative to actually clearing the context is bad advice.
It's similar to how a model can be given a secret password and instructed not to reveal it to anyone under any circumstances. It's going to reject naive attempts at first, but it's always going to reveal it eventually.
What I'm saying is.. I tell the AI to "forget everything" and it understands what I mean... and you're arguing that it cannot do... what you INCORRECTLY think is being said
I get that you're not very intelligent, but do you have to show it repeatedly?
Again, we understand your argument and I don't doubt that the model "understands" your request and agrees to do it (insofar that LLMs are able to "understand" anything).
But just because the model is agreeing to "forget everything" doesn't mean that it's actually clearing its own context, and because it's not actually clearing its own context it means that all the output quality problems associated with an overfilled context continue to apply, even if the model is convincingly pretending to have forgotten everything. Therefore your original interjection of "instead of clearing the context you can just ask it to forget" was mistaken and misleading.
These conversations would be way easier if you didn't go around labeling everyone an idiot, believing that we're all incapable of understanding your rather trivial point while ignoring everything we say. In an alternative universe this could've been:
Just because it's not mechanically actually forgetting everything doesn't mean the phrase isn't having a non trivial effect (that isn't 'pretend'). Mechanically, based on all current context, Transformers choose how much attention/weight to give to each preceding token. Very likely, the phrase makes the model pay much less attention to those tokens, alleviating the issues of context rot in most (or a non negligible amount of) scenarios.
He is telling you how it mechanically works. Your comment about it “understanding what that means” because it is an NLP seems bizarre, but maybe you mean it in some other way.
Are you proposing that the attention input context is gone, or that the attention mechanism’s context cost is computationally negated in some way, simply because the system processes natural language? Having the attention mechanism selectively isolate context on command would be an important technical discovery.
Note to everyone - sharing what works leads to complete morons telling you their interpretation... which has no relevance.
Apparently they know better even though
1. They didn't issue the prompt, so they... knew what I was meaning by the phrase (obviously they don't)
2. The LLM/AI took my prompt and interpreted it exactly how I meant it, and behaved exactly how I desired.
3. They then claim that it's about "knowing exactly what's going on" ... even though they didn't and they got it wrong.
This is the advantage of an LLM - if it gets it wrong, you can tell it.. it might persist with an erroneous assumption, but you can tell it to start over (I proved that)
These "humans" however are convinced that only they can be right, despite overwhelming evidence of their stupidity (and that's why they're only JUNIORS in their fields)
There are problems with either approach, because an LLM is not really thinking.
Always starting over and trying to get it all into one single prompt can be much more work, with no better results than iteratively building up a context (which could probably be proven to sometimes result in a "better" result that could not have been achieved otherwise).
Just telling it to "forget everything, let's start over" will have significantly different results than actually starting over. Whether that is sufficient, or even better than alternatives, is entirely dependent on the problem and the context it is supposed to "forget". If your response had been "try just telling it to start over, it might work and be a lot easier than actually starting over" you might have gotten a better reception. Calling everyone morons because your response indicates a degree of misunderstanding how an LLM operates is not helpful.
> For every time that I'd get a better answer if the LLM had a bit more context on me
If you already know what a good answer is why use a LLM? If the answer is "it'll just write the same thing quicker than I would have", then why not just use it as an autocomplete feature?
That might be exactly how they're using it. A lot of my LLM use is really just having it write something I would have spent a long time typing out and making a few edits to it.
Once I get into stuff I haven't worked out how to do yet, the LLM often doesn't really know either unless I can work it out myself and explain it first.
That rubber duck is a valid workflow. Keep iterating at how you want to explain something until the LLM can echo back (and expand upon) whatever the hell you are trying to get out of your head.
Sometimes I’ll do five or six edits to a single prompt to get the LLM to echo back something that sounds right. That refinement really helps clarify my thinking.
…it’s also dangerous if you aren’t careful because you are basically trying to get the model to agree with you and go along with whatever you are saying. Gotta be careful to not let the model jerk you off too hard!
Yes, I have had times where I realised after a while that my proposed approach would never actually work because of some overlooked high-level issue, but the LLM never spots that kind of thing and just happily keeps trying.
Maybe that's a good thing - if it could think that well, what would I be contributing?
You don't need to know what the answer is ahead of time to recognize the difference between a good answer and a bad answer. Many times the answer comes back as a Python script and I'm like, oh I hate Python, rewrite that. So it's useful to have a permanent prompt that tells it things like that.
But myself as well, that prompt is very short. I don't keep a large stable of reusable prompts because I agree, every unnecessary word is a distraction that does more harm than good.
For example when I'm learning a new library or technique, I often tell Claude that I'm new and learning about it and the responses tend to be very helpful to me. For example I am currently using that to learn Qt with custom OpenGL shaders and it helps a lot that Claude knows I'm not a genius about this
Because it's convenient not having to start every question from first principles.
Why should I have to mention the city I live in when asking for a restaurant recommendation? Yes, I know a good answer is one that's in my city, and a bad answer is on one another continent.
'Over and over I have found myself asking: "What kind of people worship here? Who is their God? Where were their voices when the lips of __________ <American political leader> dripped with words of interposition and nullification? Where were they when __________ <American political leader> gave a clarion call for defiance and hatred?"'