“You’re not [X]—you’re [Y]” is the one that drives me nuts. [X] is typically some negative characterization that, without RLHF, the model would likely just state directly. I get enough politics/subtext from humans. I’d rather the LLM just call it straight.
It’s kind of intriguing that predicting the future state of any quantum system becomes almost trivial—assuming you can diagonalize the Hamiltonian. But good luck with that in general. (In other words, a “simple” reference frame always exists via unitary conjugation, but finding it is very difficult.)
“Live next to your relatives because if they get in a car accident and you live across the country you won’t be there to tell them goodbye.”
^Another one I’ve never understood. Like geez, hopefully my daughter doesn’t give up her life dreams just based on the possibility I might be in a freak accident one day...
When my dad was battling cancer a decade ago, the distance from Seattle to Vancouver was maddening. In the end, I regretted not just taking some months off of school to spend time with him in what would be his last year. But now, it's my mom's time around... and I haven't moved back or taken extensive leave. But I'm calling her often enough that we're talking about daily life more more than cancer; enough that we run out of things to talk about. Proximity is nice, but contact is the most important thing.
Interesting. I agree with most items on this page as overrated, but with an L5-S1 disc herniation, the Aeron is about the only chair I can sit in for an extended period of time without hurting. Then again, I haven't tried dozens of office chairs, but at least for me it was worth the purchase cost.
Yep. I have an Ecobee currently and have had a Nest previously. Am totally perplexed why people like these things. Just opened the Ecobee app and literally at the top is an ad saying "The Holiday sale is here. Shop now." Irritates me to no end.
the worst is that it doesn’t show up immediately, but after a second and it’s location is where I just went to tap to enter my thermostat, forcing me to exit back. I now try and just use the app via Apple Home for a better experience.
> If spacetime had a discrete character at scales like the inverse of the universe scale we would see dispersion of light as it traveled cosmological distances and we do not observe this. It is technically possible that the discreteness scale is much, much smaller than the inverse universe scale, of course, but at this point it seems pointless to me to entertain discrete models
A computational universe does not strictly imply discrete spacetime. You can most certainly still have a continuous universe—at least from the perspective of the beings that inhabit it. By way of analogy, consider the fact that ZFC proves the existence of uncomputable real numbers yet itself has a countable model (presuming it is consistent).
As a non-mathematician, I’m confused why so many people think the conjecture (whether true or false) is provable within PA. To me, it seems like something that would be very nicely just right outside the boundary of PA’s capability, sort of like how proving all Goodstein sequences terminate requires transfinite induction up to ε_0. Add that to the fact that the Collatz Conjecture seems to fall in the same “category” of problem as the Turing machines that the Busy Beaver project is having a hard time proving non-halting behavior of, and the heuristic arguments all seem to point to: Collatz is independent of PA.
But I’m interested in hearing the counterarguments that Collatz likely is provable within PA and why this would be the case.
The Collatz conjecture is "obviously" (i.e. probabalistically) true, so it's frustrating to not be able to turn that into a proof. PA doesn't matter, there's no known approach to proving it with stronger theories either. Of course many other propositions like the twin prime conjecture are in the same situation.
Goodstein's theorem by contrast is obviously provable in slightly stronger theories than PA, and it involves a fast-growing sequence which suggests it's out of weaker theories' reach. In fact it encodes ordinals up to eps_0 in a natural way, so its equivalence to CON(PA) is unsurprising. The Collatz conjecture is nothing like that. It's beguilingly simple by comparison.
> Eliminate redundant matrix operations (like two transposes next to each other)
In 2016, I was trying to construct orthogonal irreducible matrix representations of various groups (“irreps”). The problem was that most of the papers describing how to construct these matrices used a recursive approach that depended on having already constructed the matrix elements of a lower dimensional irrep. Thus the irrep dimension n became quite an annoying parameter, and function calls were very slow because you had to construct the irrep for each new group element from the ground up on every single call.
I ended up using Julia’s @generated functions to dynamically create new versions of the matrix construction code for each distinct value of n for each type of group. So essentially it would generate “unrolled” code on the fly and then use LLVM to compile that a single time, after which all successive calls for a specific group and irrep dimension were extremely fast. Was really quite cool. The only downside was that you couldn’t generate very high dimensional irreps because LLVM would begin to struggle with the sheer volume of code it needed to compile, but for my project at the time that wasn’t much of a concern.
Hmm... I agree with parts of this and disagree with other parts. In my experience "cross-functional collaboration" splits into two distinct components: leadership and information. Anecdotal, but when leadership is split into too many people at the same level who are each in charge of a domain (that requires heavy interaction with the other domains), nothing gets accomplished—analysis paralysis and politics takes over. You absolutely need one specific person as the final decision maker. They should carefully consider all input from various sources and then make a final decision in a timely fashion. If it turns out to be the wrong path, that's fine, just reverse course quickly as well.
On the other hand, information silos are absolutely horrible. The most effective companies I've worked at have always had tons of information freely available to all employees. Unless there are privacy, cybersecurity, antitrust, or similar risks involved, every employee should have access to all information across all teams. It should be easily searchable as well. There are certainly exceptions—Apple seems to function well despite all the secrecy. But most companies aren't Apple, and I don't think it's generally a good strategy.
Haha, I like to joke that we were on track for the singularity in 2024, but it stalled because the research time gap between "profitable" and "recursive self-improvement" was just a bit too long that we're now stranded on the transformer model for the next two decades until every last cent has been extracted from it.
There's massive hardware and energy infra built out going on. None of that is specialized to run only transformers at this point, so wouldn't that create a huge incentive to find newer and better architectures to get the most out of all this hardware and energy infra?
Only being able to run transformers is a silly concept, because attention consists of two matrix multiplications, which are the standard operation in feed forward and convolutional layers. Basically, you get transformers for free.
reply