5 stories tagged Apple Silicon
ContextCore is an open-source project that enables AI agents to remember long conversations on Apple Silicon devices using GPU-accelerated context retrieval, making on-device AI far more practical for complex, multi-turn tasks.
Modular has updated its Max AI models to run on Apple Silicon GPUs, making powerful AI tools more accessible. This change allows Mac users to leverage their hardware for advanced AI tasks without needing external servers.
Hugging Face has ported its Transformers library to MLX, enabling faster AI model inference on Apple Silicon. This move aims to leverage Apple's neural engine for enhanced performance.
Researchers extended the MLX-LM framework to enable cross-tokenizer speculative decoding, improving inference speed for Polish language models on Apple Silicon. The study evaluated Bielik-11B-Instruct paired with three draft models, showing significant speedups.
OMLX is a new tool designed to optimize large language model inference specifically for Mac users. It promises to make running LLMs on Apple Silicon faster and more efficient.