https://dailypypy.org/tags/mlx/ Recent content in Mlx on 每日拍拍 Hugo -- gohugo.io zh-tw © 2026 每日拍拍 Sat, 05 Sep 2026 10:11:00 +0800 155076163427069952 154825760438254592 https://dailypypy.org/learn/mlx-lm-long-context-memory/ Sat, 05 Sep 2026 10:11:00 +0800 https://dailypypy.org/learn/mlx-lm-long-context-memory/ <!--- 1440x768 prompt: masterpiece, best quality, highres, clean anime illustration, japanese anime style, soft shading, flat color design, 1girl, black hair, green eyes, white off-shoulder shirt, black short skirt, upper body, three-quarter view, leaning forward while gently holding a long curled blank paper ribbon with both hands, curious thoughtful expression, looking at viewer, pastel lavender background, subtle floating translucent memory layers and soft looping ribbons without symbols or text, neat composition, detailed eyes, cute and smart vibe, minimal background, polished illustration, no text negative prompt: worst quality, bad eye, bad hand, extra limbs, manga, multiple views, monochrome, text, signature dedup note: This is a long-context inference follow-up to ../mlx-lm-local-models/, ../mlx-lm-local-api-server/, and ../mlx-lm-quantization-convert/. The general inference post only trims recent turns, the server post covers HTTP-level automatic prefix caching, and the quantization post compares model-weight artifacts with one KV-memory reminder. This article instead builds an end-to-end token budget, explains prefill versus decode memory, estimates KV size, compares prefill chunking, rotating cache, KV-cache quantization, and reusable on-disk prompt caches, then defines a long-context benchmark matrix. It does not repeat model installation basics, server deployment, tool calling, weight conversion, or general batch inference. source check: Reviewed against the current official MLX-LM README, generate.py, cache implementation, multi-turn prompt-cache example, and the selected model config on 2026-09-05. The current CLI exposes max-kv-size, prefill-step-size, prompt-cache-file, kv-bits, kv-group-size, and quantized-kv-start; model/cache compatibility still needs measurement instead of assumption. ---> <p> <figure> <img class="my-0 rounded-md" loading="lazy" alt="featured" src="./featured.png" /> </figure> </p> https://dailypypy.org/learn/mlx-lm-quantization-convert/ Tue, 25 Aug 2026 10:44:00 +0800 https://dailypypy.org/learn/mlx-lm-quantization-convert/ <!--- 1440x768 prompt: masterpiece, best quality, highres, clean anime illustration, japanese anime style, soft shading, flat color design, 1girl, black hair, green eyes, white off-shoulder shirt, black short skirt, dutch angle, cowboy shot, leaning sideways while balancing two blank translucent geometric model blocks on open palms, playful analytical raised eyebrow and confident half-smile, looking at viewer, pastel dusty rose background, subtle floating layered chip shapes and tiny comparison dots without symbols or text, neat composition, detailed eyes, cute and smart vibe, minimal background, polished illustration, no text negative prompt: worst quality, bad eye, bad hand, extra limbs, manga, multiple views, monochrome, text, signature dedup note: This is a quantization-specific follow-up to ../python-mlx/ and ../mlx-lm-local-models/. The older MLX overview shows one basic 4-bit conversion command, while the MLX-LM inference post only recommends pre-quantized models. This article focuses on reproducible Hugging Face-to-MLX conversion, 4/8-bit and group-size trade-offs, built-in mixed-bit recipes, artifact validation, and an A/B evaluation harness for size, peak memory, speed, and task quality. It does not repeat general MLX arrays, model selection, chat templates, streaming, server, batch-runner, embeddings, or LoRA workflows. ---> <p> <figure> <img class="my-0 rounded-md" loading="lazy" alt="featured" src="./featured.png" /> </figure> </p> https://dailypypy.org/learn/mlx-lm-local-api-server/ Thu, 20 Aug 2026 11:47:00 +0800 https://dailypypy.org/learn/mlx-lm-local-api-server/ <!--- 1440x768 prompt: masterpiece, best quality, highres, clean anime illustration, japanese anime style, soft shading, flat color design, 1girl, black hair, green eyes, white off-shoulder shirt, black short skirt, wide shot, small figure, three-quarter side view, sitting cross-legged while typing on a slim laptop, focused curious half-smile, glancing toward viewer, pastel apricot background, subtle floating connected nodes and soft glowing rounded panels without symbols or text, neat composition, detailed eyes, cute and smart vibe, minimal background, polished illustration, no text negative prompt: worst quality, bad eye, bad hand, extra limbs, manga, multiple views, monochrome, text, signature dedup note: This is a service-integration follow-up to ../mlx-lm-local-models/. The older post covers model selection, CLI/Python generation, chat templates, streaming, and a local chat CLI. This article starts at the HTTP boundary: mlx_lm.server, OpenAI-compatible clients, health checks, automatic prefix/KV cache reuse, tool-calling compatibility, concurrency, observability, and safe network exposure. It does not repeat general inference, batch evaluation, embeddings, quantization, or LoRA workflows. ---> <p> <figure> <img class="my-0 rounded-md" loading="lazy" alt="featured" src="./featured.png" /> </figure> </p> https://dailypypy.org/learn/mlx-lora-finetune-local/ Wed, 12 Aug 2026 12:30:00 +0800 https://dailypypy.org/learn/mlx-lora-finetune-local/ <!--- 1440x768 prompt: masterpiece, best quality, highres, clean anime illustration, japanese anime style, soft shading, flat color design, 1girl, black hair, green eyes, white off-shoulder shirt, black short skirt, side profile, leaning forward, determined look, looking over shoulder at viewer, pale apricot background, subtle tiny adapter modules and dataset cards floating nearby without text, neat composition, detailed eyes, cute and smart vibe, minimal background, polished illustration, no text negative prompt: worst quality, bad eye, bad hand, extra limbs, manga, multiple views, monochrome, text, signature dedup note: Planned 2026-06-16 topic `mlx-lora-finetune-local` is a specific follow-up to ../mlx-lm-local-models/ and ../mlx-lm-batch-inference/. The older posts cover local inference, chat templates, streaming, JSONL batch inference, sampling grids, and repeatable evaluation. This article focuses on LoRA/QLoRA adapter training: dataset schema, train/valid/test split, `mlx_lm.lora`, adapter outputs, prompt masking, generation with adapters, fusing, and memory knobs. It does not repeat the general MLX-LM inference workflow. ---> <p>本地 LLM 跑起來以後，下一個很自然的問題是： 「那我可以讓模型更懂自己的任務嗎？」 可以。 但拍拍君先把期待值放準一點。 LoRA 微調不是魔法。 它不是把一個小模型訓練成萬能博士。 它比較像是在模型旁邊加一組可拆卸的小筆記，讓模型在某個格式、語氣、分類規則、抽取任務上更穩。 這篇要做的是 MLX-LM 的 LoRA 入門工作流：</p> https://dailypypy.org/learn/mlx-lm-batch-inference/ Wed, 10 Jun 2026 11:06:00 +0800 https://dailypypy.org/learn/mlx-lm-batch-inference/ <!--- 1440x768 prompt: masterpiece, best quality, highres, clean anime illustration, japanese anime style, soft shading, flat color design, 1girl, black hair, green eyes, white off-shoulder shirt, black short skirt, from above, sitting on stool, hand on chin thinking, serious focused look, looking at viewer, pale coral background, subtle floating JSONL cards and sampling slider shapes without text, neat composition, detailed eyes, cute and smart vibe, minimal background, polished illustration, no text negative prompt: worst quality, bad eye, bad hand, extra limbs, manga, multiple views, monochrome, text, signature follow-up note: 這篇是 `mlx-lm-local-models` 的具體後續，焦點放在可重跑的批次資料集、prompt template、sampling grid、JSONL 輸出與本機評測，不重複前文的安裝、第一個 prompt、聊天 CLI 與串流輸出教學。 ---> <h2 class="relative group">一. 前言：聊天很酷，批次推論才是真的工作流 <div id="一-前言聊天很酷批次推論才是真的工作流" class="anchor"></div> <span class="absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100"> <a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e4%b8%80-%e5%89%8d%e8%a8%80%e8%81%8a%e5%a4%a9%e5%be%88%e9%85%b7%e6%89%b9%e6%ac%a1%e6%8e%a8%e8%ab%96%e6%89%8d%e6%98%af%e7%9c%9f%e7%9a%84%e5%b7%a5%e4%bd%9c%e6%b5%81" aria-label="定位點">#</a> </span> </h2> <p>第一次把本地模型跑起來時，大家通常會先打開聊天介面。 這很合理，因為你想確認模型能不能回答問題、中文會不會怪、速度是不是能接受。 可是拍拍君要說一句有點掃興但很實用的話：真正讓本地 LLM 變成工具的，常常不是聊天，而是批次推論。 例如把 200 篇短筆記整理成標籤、幫一批 issue 產生摘要、比較不同 prompt template 的輸出穩定度，或對同一組測試問題跑 temperature 參數實驗。 聊天是即興對話；批次推論是可以版本控制、可以重跑、可以比較的實驗流程。 如果你還沒看過 MLX-LM 的基本用法，可以先看前一篇 <a href="../mlx-lm-local-models/">MLX-LM 實戰：在 Apple Silicon 上跑本地模型推論</a>。 那篇處理的是「怎麼跑起來」。 今天這篇處理的是「跑起來之後，怎麼把它變成穩定的小型生產線」。 我們會做一個小但完整的批次 runner：</p> https://dailypypy.org/learn/mlx-lm-local-models/ Tue, 19 May 2026 10:42:00 +0800 https://dailypypy.org/learn/mlx-lm-local-models/ <!--- 1440x768 prompt: masterpiece, best quality, highres, clean anime illustration, japanese anime style, soft shading, flat color design, 1girl, black hair, green eyes, white off-shoulder shirt, black short skirt, side profile, looking over shoulder, holding a small laptop, serious focused look, looking at viewer, pale coral background, subtle floating chip shapes and terminal panels without text, neat composition, detailed eyes, cute and smart vibe, minimal background, polished illustration, no text negative prompt: worst quality, bad eye, bad hand, extra limbs, manga, multiple views, monochrome, text, signature dedup note: This is a specific follow-up to ../python-mlx/ and ../mlx-embeddings-semantic-search/. The older posts cover MLX basics and embedding search; this post focuses on mlx-lm text-generation inference, streaming, chat templates, and local CLI workflow. ---> <p> <figure> <img class="my-0 rounded-md" loading="lazy" alt="featured" src="./featured.png" /> </figure> </p> https://dailypypy.org/learn/local-ai-app-architecture/ Sun, 10 May 2026 12:23:00 +0800 https://dailypypy.org/learn/local-ai-app-architecture/ <!--- 1440x768 prompt: masterpiece, best quality, highres, clean anime illustration, japanese anime style, soft shading, flat color design, 1girl, black hair, green eyes, white off-shoulder shirt, black short skirt, wide shot, small figure, side profile, looking over shoulder, gentle confident smile, looking at viewer, pastel peach background, subtle floating app windows, neural nodes, tiny llama silhouettes and laptop shapes without text, neat composition, detailed eyes, cute and smart vibe, minimal background, polished illustration, no text negative prompt: worst quality, bad eye, bad hand, extra limbs, manga, multiple views, monochrome, text, signature ---> <p> <figure> <img class="my-0 rounded-md" loading="lazy" alt="featured" src="./featured.png" /> </figure> </p> https://dailypypy.org/learn/mlx-embeddings-semantic-search/ Tue, 05 May 2026 11:13:00 +0800 https://dailypypy.org/learn/mlx-embeddings-semantic-search/ <!--- 1440x768 prompt: masterpiece, best quality, highres, clean anime illustration, japanese anime style, soft shading, flat color design, 1girl, black hair, green eyes, white off-shoulder shirt, black short skirt, cowboy shot, hand on chin thinking, determined curious smile, looking at viewer, pastel pistachio background, subtle floating vector nodes and tiny database cards without text, neat composition, detailed eyes, cute and smart vibe, minimal background, polished illustration, no text negative prompt: worst quality, bad eye, bad hand, extra limbs, manga, multiple views, monochrome, text, signature ---> <p> <figure> <img class="my-0 rounded-md" loading="lazy" alt="featured" src="./featured.png" /> </figure> </p> https://dailypypy.org/learn/local-ai-apple-workflow/ Fri, 01 May 2026 11:16:00 +0800 https://dailypypy.org/learn/local-ai-apple-workflow/ <!--- 1440x768 prompt: masterpiece, best quality, highres, clean anime illustration, japanese anime style, soft shading, flat color design, 1girl, black hair, green eyes, white off-shoulder shirt, black short skirt, from below low angle, pointing upward, explaining, confident bright smile, looking at viewer, pastel periwinkle background, subtle floating phone and laptop silhouettes without text, neat composition, detailed eyes, cute and smart vibe, minimal background, polished illustration, no text negative prompt: worst quality, bad eye, bad hand, extra limbs, manga, multiple views, monochrome, text, signature ---> <p> <figure> <img class="my-0 rounded-md" loading="lazy" alt="featured" src="./featured.png" /> </figure> </p> https://dailypypy.org/learn/python-mlx/ Thu, 26 Feb 2026 10:19:00 +0800 https://dailypypy.org/learn/python-mlx/ <!--- 1024x620 prompt: masterpiece, best quality, highres illustration, green eyes, white off-shoulder shirt, 1girl, flat color, lavender background, simple design, black hair, leaning forward, curious expression, from above, cute anime girl, japanese anime style, soft shading, detailed eyes, black short skirt, apple logo icon, neural network, sparkles, laptop on desk negative prompt: worst quality, bad eye, bad hand, extra limbs, manga, multiple views, monochrome, text, signature ---> <h2 class="relative group">一、前言 <div id="一前言" class="anchor"></div> <span class="absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100"> <a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e4%b8%80%e5%89%8d%e8%a8%80" aria-label="定位點">#</a> </span> </h2> <p>嗨，大家好！我是拍拍君 🍎</p>