https://dailypypy.org/tags/streaming/ Recent content in Streaming on 每日拍拍 Hugo -- gohugo.io zh-tw © 2026 每日拍拍 Thu, 24 Sep 2026 10:11:00 +0800 155076163427069952 154825760438254592 https://dailypypy.org/learn/polars-lazyframe-query-optimization/ Thu, 24 Sep 2026 10:11:00 +0800 https://dailypypy.org/learn/polars-lazyframe-query-optimization/ <!--- 1440x768 prompt: masterpiece, best quality, highres, clean anime illustration, japanese anime style, soft shading, flat color design, 1girl, black hair, green eyes, white off-shoulder shirt, black short skirt, side profile, looking over shoulder, full body, kneeling beside three floating translucent data panels connected by soft flowing ribbons, calm confident smile, looking at viewer, pastel lavender and pale cyan background, subtle funnel-shaped light motif without symbols, logos, markings, numbers, letters, or text, neat composition, detailed eyes, cute and smart vibe, minimal background, polished illustration, no text negative prompt: worst quality, bad eye, bad hand, extra limbs, manga, multiple views, monochrome, text, signature dedup note: Planned 2026-09-24 topic `polars-lazyframe-query-optimization` passed a strict 171-post inventory and current-source scan. ../python-polars/ teaches general DataFrame expressions, group-by, and only a short Lazy Evaluation introduction; ../python-pyarrow/ focuses on Arrow schemas, Parquet datasets, and cross-tool exchange; ../duckdb-remote-parquet-s3/ discusses DuckDB-specific remote pushdown. This follow-up instead teaches scan-based logical plans, predicate/projection/slice pushdown, optimizer barriers, explain, reproducible benchmarks, explicit streaming execution, native sinks, and query-plan QA. It cross-links the general Polars article without repeating DataFrame basics or pandas comparisons. source check: Reviewed against the current official Polars LazyFrame, collect, explain, streaming, and sink documentation on 2026-09-24. Examples target Polars 1.44.2, use `engine="streaming"` instead of the older `streaming=True`, avoid the deprecated `LazyFrame.profile()`, and treat plan text as diagnostic output rather than a stable API contract. ---> <h2 class="relative group">一. 前言：快，不該只靠換一台更大的機器 <div id="一-前言快不該只靠換一台更大的機器" class="anchor"></div> <span class="absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100"> <a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e4%b8%80-%e5%89%8d%e8%a8%80%e5%bf%ab%e4%b8%8d%e8%a9%b2%e5%8f%aa%e9%9d%a0%e6%8f%9b%e4%b8%80%e5%8f%b0%e6%9b%b4%e5%a4%a7%e7%9a%84%e6%a9%9f%e5%99%a8" aria-label="定位點">#</a> </span> </h2> <p>嗨，我是拍拍君。 你把一份幾 GB 的 Parquet 讀進來，接著篩選日期、挑五個欄位、做聚合， 最後只輸出幾百列。 但程式一開始就把整份資料塞進記憶體。 這不是資料太大，是執行順序太誠實了。 Polars 的 <code>LazyFrame</code> 會先記錄「想做什麼」，組成 logical plan， 等到 <code>collect()</code> 或 <code>sink_*()</code> 才最佳化並執行。 因此引擎有機會：</p>