I keep seeing everyone guessing what the margin is on inference. You say that it's largely break even. We have this person in the thread claim 80% margin (https://news.ycombinator.com/item?id=45462442).
Why would they be make believe? I am generalizing across multiple companies and simplifying it as much as possible as we won’t know all the cost buckets but we do know at current costs the bare inference cost of keeping the machines running is being covered.
Rate cuts are priced in at about 100% for a 1/4 point. It might go up for a 1/2 point, but that may also signal fear which could have the opposite effect. I think it's hard to say at this point.
The Federal Reserve operates under a "dual mandate" from Congress to promote maximum employment and stable prices (low inflation). As unemployment rises, the Fed will lower interest rates to stimulate investment (increase employment opportunities).
If AI is the cause, it will only stimulate investment in more AI and accelerate the layoffs though. And because they don't have other tools, looks like this is exactly what they are going to do. Investment tends to concentrate massively at the festest-growing trend which this time is just replacing workers with AI.
Which will deepen the hole, until we crest over the AI peak of inflated expectations, bottom out in the trough of disillusionment, and start creeping up the slope of enlightenment. The next 12 months should be worsening unemployment, more rate cuts, higher AI stock prices. Once we bottom out in disillusionment (or companies run out of AI capital), the stock will plummet again, AI companies will die off, unemployment reduces, and rates return to previous levels.
That's the "exists-in-a-vacuum" picture anyway. Stimulus, tariffs, a new war, or some other bullshit will change the results.
That really just feels like they're operating with the mandate to keep the cost of labor low, and the main way to do that is by not paying people enough.
"The same influx of immigrant workers that helped fill job openings also dampened wage pressures across the affected industries and states. At the industry level, sectors with some of the highest immigrant workforce growth, such as construction and manufacturing, saw the sharpest deceleration in wage growth (specifically, average hourly earnings) from 2021 to 2023."
Maybe I should expand it to say it sounds like they're under a mandate to keep people more-or-less economically stuck.
They need to keep employment high and keep prices stable. Their main lever for controlling these two things is the prime interest rate. They kept that rate low for a long time. Capital now thinks that being able to get money for cheap is the norm. If it can't get money for cheap, well, that's a problem, because capital's mandate is to get more capital.
If you're being incentivized to keep prices low and employment high by this institution, while also trying to accumulate capital for yourself, you are more likely to employ people at lower wages in order to keep prices low instead of taking the hit in reduced capital accumulation to employ people at higher wages while keeping prices low. Furthermore, any sort of sane prime interest rate now seems high to capital, so that additional cost is also factored in as why costs must rise and wages must drop to keep the accumulation rate of capital as high as possible.
Is there anything wrong with accumulation of capital? In and of itself, no, but when you have people with net worths in the hundreds of billions of dollars making the decisions on how to allocate resources for their own continued benefit, well, you get a reduction in economic returns for the rest of the economy.
There are definitely some similarities. I think the main difference is the compatibility of Wasm / WASI. Often for unikernel / libOS, you would need to recompile applications to target the specific unikernel / libOS. The goal is that you should be able to take a component that you would run with Wasmtime or other component model compatible runtime and be able to run that on Hyperlight-Wasm.
As for executing a tree of connected components, in the current state of Hyperlight-Wasm you'd probably want to take a collection of components and compose them together using something like https://github.com/bytecodealliance/wac to create a final component composed together from multiple components.
Thank you for the illuminating references! So this sockets example is a host that runs on a raw vm (x86_64-unknown-none target) and spawns sandboxes. To get at my original idea, I could imagine a sandbox with special functions imported from the host that can spawn new sandboxes. This could be complicated by the fact that spawning these sandboxes and hooking up inputs/outputs seems to be statically compiled...
I wondered how components were supposed to be used. So wac takes a group of wasm components and merges them into a new .wasm file with inputs/outputs mapped according to a defined .wac file. Then you can just run the new .wasm file. Does this not obviate the isolation benefits of having separate wasm modules? Is there a path to running a tree of components directly while maintaining the sandbox between them?
A Wasm component running inside of Wasmtime is just fine. However, when you start to use resources from outside of Wasm, e.g. systems, network interfaces, GPUs, etc., Wasmtime uses OS resources from the host that it is running upon. If this host is running on your trusted compute base, then it implies you are trusting the host implementations in Wasmtime, which for some is just fine. However, Hyperlight-Wasm gives platform builders the ability to describe the interface between the guest and the host explicitly, so you could only expose the host functionality you would want with the trusted implementation you'd want. For example, if I'm building a FaaS, I may want to provide only an exported event handler and an imported key/value interface to the guest for which I've built a safe, multi-tenant implementation and strictly disallow all other host provided functionality.
Krustlet implemented the Kubelet API rather than the containerd-shim API. That means that with containerd shims, you can use the default K8s kubelet, containerd, and extend your containerd runtime with a shim for executing Wasm. Kubelet is much higher in the stack and requires much more to implement it and everything below it in the stack.
I would like to see remote comp be structured with a flat base rate across geos, and then have bonuses for locality to corporate centers. If you feel that it is worth it, you can choose to live close to the office. A flat base rate across geos compensates folks doing the same work for the same pay.
Maybe a good candidate in this organization is one that will leverage every edge they have to produce a desired result. Resourcefulness and risk tolerance are a potent pair.
These numbers are complete make believe.