OMLX Brings Optimized LLM Inference to Mac Users
OMLX is a new tool designed to optimize large language model inference specifically for Mac users. It promises to make running LLMs on Apple Silicon faster and more efficient.
#optimization
Optimization
28 stories tagged Optimization · page 2 of 2
DIVERSED: Relaxed Speculative Decoding Boosts LLM Inference Speed
Researchers introduce DIVERSED, a new method that relaxes strict token verification in speculative decoding to significantly increase acceptance rates. This approach bypasses the bottleneck of rigid distribution matching, offering faster LLM inference without sacrificing output quality.
Rethinking Generalization in Reasoning SFT
Researchers challenge the notion that supervised finetuning memorizes while reinforcement learning generalizes. They find that cross-domain generalization is conditional, influenced by optimization, data, and model capability. This challenges prevailing narratives in LLM post-training.
MegaTrain: Full Precision Training
MegaTrain enables full precision training of large language models on a single GPU. It stores parameters in host memory and uses GPUs as compute engines.