https://dailypypy.org/tags/benchmark/ Recent content in Benchmark on 每日拍拍 Hugo -- gohugo.io zh-tw © 2026 每日拍拍 Mon, 28 Sep 2026 10:44:00 +0800 155076163427069952 154825760438254592 https://dailypypy.org/learn/mlx-lm-speculative-decoding/ Mon, 28 Sep 2026 10:44:00 +0800 https://dailypypy.org/learn/mlx-lm-speculative-decoding/ <!--- 1440x768 prompt: masterpiece, best quality, highres, clean anime illustration, japanese anime style, soft shading, flat color design, 1girl, black hair, green eyes, white off-shoulder shirt, black short skirt, wide shot, small figure, three-quarter view, standing with hands behind back between one small translucent glowing orb and one larger translucent glowing orb, bright analytical smile, looking at viewer, pastel rose quartz and soft teal background, subtle paired flowing light trails converging ahead without symbols, logos, markings, numbers, letters, or text, neat composition, detailed eyes, cute and smart vibe, minimal background, polished illustration, no text negative prompt: worst quality, bad eye, bad hand, extra limbs, manga, multiple views, monochrome, text, signature dedup note: This is a focused follow-up to ../mlx-lm-local-models/, ../mlx-lm-batch-inference/, ../mlx-lm-quantization-convert/, and ../mlx-lm-long-context-memory/. Those posts cover basic generation, repeatable prompt evaluation, weight quantization, and context/cache memory. This article instead centers target/draft compatibility, speculative token acceptance, draft-length tuning, end-to-end latency and throughput measurement, break-even analysis, and greedy quality parity. It does not repeat installation tours, model conversion recipes, generic sampling advice, or long-context cache tuning. source check: Reviewed against the current official MLX-LM generate.py, server documentation, release history, and public issue tracker on 2026-09-28. The current generate CLI exposes --draft-model and --num-draft-tokens; stream_generate requires the same tokenizer, and speculative decoding requires a trimmable prompt cache. Public reports also show that model/cache combinations and greedy-output parity must be verified on the exact installed version rather than assumed. ---> <p> <figure> <img class="my-0 rounded-md" loading="lazy" alt="featured" src="./featured.png" /> </figure> </p>