#llm-inference

LLM Inference

2 stories tagged LLM Inference

researchApr 21, 2026

Cross-Family Speculative Decoding Boosts Polish LLM Performance on Apple Silicon

Researchers extended the MLX-LM framework to enable cross-tokenizer speculative decoding, improving inference speed for Polish language models on Apple Silicon. The study evaluated Bielik-11B-Instruct paired with three draft models, showing significant speedups.

via ArXiv cs.CL#speculative-decoding #apple-silicon #polish-language

researchApr 10, 2026

DIVERSED: Relaxed Speculative Decoding Boosts LLM Inference Speed

Researchers introduce DIVERSED, a new method that relaxes strict token verification in speculative decoding to significantly increase acceptance rates. This approach bypasses the bottleneck of rigid distribution matching, offering faster LLM inference without sacrificing output quality.

via ArXiv cs.CL#speculative-decoding #llm-inference #arxiv