https://dailypypy.org/tags/long-context/ Recent content in Long Context on 每日拍拍 Hugo -- gohugo.io zh-tw © 2026 每日拍拍 Sat, 05 Sep 2026 10:11:00 +0800 155076163427069952 154825760438254592 https://dailypypy.org/learn/mlx-lm-long-context-memory/ Sat, 05 Sep 2026 10:11:00 +0800 https://dailypypy.org/learn/mlx-lm-long-context-memory/ <!--- 1440x768 prompt: masterpiece, best quality, highres, clean anime illustration, japanese anime style, soft shading, flat color design, 1girl, black hair, green eyes, white off-shoulder shirt, black short skirt, upper body, three-quarter view, leaning forward while gently holding a long curled blank paper ribbon with both hands, curious thoughtful expression, looking at viewer, pastel lavender background, subtle floating translucent memory layers and soft looping ribbons without symbols or text, neat composition, detailed eyes, cute and smart vibe, minimal background, polished illustration, no text negative prompt: worst quality, bad eye, bad hand, extra limbs, manga, multiple views, monochrome, text, signature dedup note: This is a long-context inference follow-up to ../mlx-lm-local-models/, ../mlx-lm-local-api-server/, and ../mlx-lm-quantization-convert/. The general inference post only trims recent turns, the server post covers HTTP-level automatic prefix caching, and the quantization post compares model-weight artifacts with one KV-memory reminder. This article instead builds an end-to-end token budget, explains prefill versus decode memory, estimates KV size, compares prefill chunking, rotating cache, KV-cache quantization, and reusable on-disk prompt caches, then defines a long-context benchmark matrix. It does not repeat model installation basics, server deployment, tool calling, weight conversion, or general batch inference. source check: Reviewed against the current official MLX-LM README, generate.py, cache implementation, multi-turn prompt-cache example, and the selected model config on 2026-09-05. The current CLI exposes max-kv-size, prefill-step-size, prompt-cache-file, kv-bits, kv-group-size, and quantized-kv-start; model/cache compatibility still needs measurement instead of assumption. ---> <p> <figure> <img class="my-0 rounded-md" loading="lazy" alt="featured" src="./featured.png" /> </figure> </p>