Never thought I would ever see this on a google owned websites!
Really? Google used to be famous not only for its errors, but for its creative error pages. I used to have a google.com bookmark that would send an animated 418.
Try mimalloc. I have prototyped a feature on top of mimalloc and while effort was a dead end, the code (this was around 2020) was nicely written and well maintained and it was fun to hack on it. When I swapped jemalloc in our system with mimalloc, it was on par if not better when it comes to fragmentation growth control and heap usage perspective.
Why do you think it is a huge tolerance ? (Just curious since it is not clear to me if that will lead to too much of reduction in numerical accuracy compared to the speedup)
The point is, this amount of error is huge for fp32, but may be expected for fp16. But then why compare to fp32 performance baselines? An algorithm that gives you the accuracy of fp16 should be compared to an fp16 baseline, and this may not be (it probably is not) a speedup at all, it's likely much slower.
My original question is to understand why it is considered as huge tolerance and what should be considered low tolerance. I am suspecting the paper’s intention is not to compare apples and oranges. They are trying to optimize fp32 baseline by sometime resorting using fp16 as long as the resultant solution’s numerical accuracy is within thr tolerance level. They are going for the “low hanging fruits” type of optimization.
I just read between the lines in the article that he must have gotten something like that since he said in the article that when he dropped out to be a poet he knew he would not have to attend college in two years.
Looking into it, I think they actually translate their version of it as GED too:
I haven't read the article yet but the moment I saw the writer is "Jordana Cepelewicz" I knew it is going to be great! I have read few of her past articles and they were excellent think pieces.
Sorry, there seems to be an error. Please try again soon.”
Never thought I would ever see this on a google owned websites!