I’m curious to hear more about phone-local assistants. I rather assumed only the latest hardware ( iPhone 15+, not sure on Android side) could do local inference. Is there a way to get something going on hardware a couple years old?
> Is there a way to get something going on hardware a couple years old?
Tensor accelerators are very recent thing, and GPU/WebGPU also recent.
RAM was also limited, 4Gb was long time barrier.
So, model should run on CPU and within 4Gb or even 2Gb.
Oh, I forget one important thing - couple years old mobile CPUs was also weak (and btw exception was iphone/ipad).
But, if you have gaming mobile (or iphone), which at that time was comparable to Notebooks, may run something like Llama-2 quantized to 1.8Gb at about 2 tokens per second, not very impressive, but could work.
Unfortunately, I could not remember, when median performance of mobile CPU become comparable to business Notebooks.
I think, Apple entered race for speed with iPhone X and iPad 3. For Androids things even worse, looks like median achieved Notebooks speed at about Qualcomm snapdragon 6xx.
FUTO voice typing runs local on my galaxy 20, so, yes. Also there are SPA that claim to load local that I have but I haven't tried that. There are small models, one I know of is 380M parameter, rather than 15B or 800B...
preemptively adding for us AMD users - it’s pretty seamless to get Ollama working with rocm, and if you have a card that’s a bit below the waterline (lowest supported is a 6800xt, i bought a 6750xt), you can use a community patch that will enable it for your card anyway:
I specifically recommend the method where you grab the patched rocblas.dll for your card model, and replace the one that Ollama is using, as someone who is technical but isn’t proficient with building from source (yet!)
Part of me is worried that we'll be giving kids a distorted perception of what AIs actually are - hell, the general adult public can't help but anthropomorphize them to a degree that I don't necessarily think is warranted.
The thing I dislike the most about Instagram is it seems like at some point it will be like "I'm done showing you content from your feed, here is generically popular stuff". There's a divider and a little message, but if I'm casually scrolling I might miss it and I'm on my timeline, if I wanted to be on Discover I'd be on Discover.
When TikTok runs out of relevant content and starts showing me generic/popular stuff; I say aloud "Ope, I'm out of tiktoks".
These days I don't check TikTok every day or I hit the normie soft cap after 5 minutes. My friends rely on me to be the "TikTok" guy since none of them have it installed for various reasons, so I "save them up" by waiting to view my fyp for longer periods of time.
I can confirm this - I got a 72 month loan on my current car when my finances were a lot worse and it was pitched to me as making it affordable. The price every month now is incredibly sustainable, but I have this suspicion that I'm way overpaying for what I bought.
Honestly, everything about that purchasing experience has taught me to steer clear of dealers unless I'm ready to buy on the spot and have done my own homework.
There's a lot to consider with financing a car that isn't necessarily obvious to most people. We've all heard that the car depreciates in value significantly the second you drive it off the lot. With a 72 month note, you are upside down on the note for a much much longer time. Since it is a car, there is a greater than zero chance of getting in a wreck. It could be a situation that after insurance pays off the deemed value, you are still owing the financing company for a car that you can no longer use. This is why gap insurance is important for the longer term financing.
I think the idea is good, and the economics don't immediately suggest that is impractical more than, say, the supply chain dynamics behind widespread gas stations to me. The issue is getting there, and while China has these manufacturers pushing the swap paradigm, we'll almost need someone with Tesla level deep pockets to get over the implementation hump in the US.
One of the economies of battery scale is durability, no? What you gain in cost savings up front you lose by needing to replace the batteries more frequently than N times the larger battery pack size.
Don't LIDAR/radar sensors (I'm not familiar with exactly what the options are) have benefits that vision doesn't have, like working in poor lighting/visibility? Why would Tesla move away from these sensors?
Lidar has advantages over cameras but it also has some downsides. Sunlight, rain, and snow can interfere with the sensors, as can other nearby lidar devices (though de-noising algorithms are always improving). There are also issues with object detection. Lidar gives you a point cloud, and you need software that can detect and categorize things from that point cloud. Because lidar is new, this problem hasn't had as much R&D put into it as similar problems in computer vision.
Then there's the issue of sensor fusion. Lidar requires sensor fusion because it can't see road lines, traffic signals, or signs. It also can't see lights on vehicles or pedestrians. So you still have to solve most of the computer vision issues and you have to build software that can reliably merge the data from both sensors. What if the sensors disagree? If you err on the side of caution and brake if either sensor detects an object, you get lots of false positives and phantom braking (increasing the chance of rear-end collisions). If you YOLO it, you might hit something. If you improve the software to the point that lidar and cameras never disagree, well then what do you need lidar for?
I think lidar will become more prevalent, and I wouldn't be surprised if Tesla added it to their vehicles in the future. But the primary sensor will always be cameras.
Because LIDAR is expensive and available in limited quantities. There's no way they could sell the amount of cars they are selling right now if each one came with a LIDAR.
Camera modules are cheap and available in huge quantities.
That's what I'm thinking of doing when I build PCs for my partner and I later this year. I've been looking at benchmarks and I'm not as worried about the top end of performance as much as I am Intel being about ready to release a new socket design next refresh.
