first weekly roundup.
the MLX world moves fast and the news is scattered across GitHub, X, and Hugging Face.
so every friday I'm going to pull it together here in one place.
if I missed something, drop it in the replies and I'll add it.
1. mlx-lm now does tensor parallel inference across Macs
this is the big one.
Awni Hannun (the MLX lead at Apple) posted that the latest mlx-lm release adds tensor parallel LLM inference on top of the new low latency JACCL backend in MLX.
it also picked up Transformers v5 support.
Angelos Katharopoulos showed Devstral writing a quicksort in C++ about 1.7x faster on two M3 Ultras vs one.
that's the first time splitting a single model across two Macs has been a normal, supported thing instead of a science project.
if you have two Macs with Thunderbolt 5 and a big model that won't fit on one, this is for you.
Awni's post
Angelos' post
mlx-lm releases
2. batch generation doubles throughput on GLM 4.7
second demo from Awni.
GLM 4.7 at 6-bit on an M3 Ultra, batch of 4, runs at almost double the throughput of a single generation (around 30 tok/sec total).
so if you're serving a model to more than one person, or running a bunch of agents, batching is basically free speed.
3. new MLX quants worth trying
a few uploads on Hugging Face that caught my eye this week.
context-1 at 6-bit (mlx-community). a 20B agentic search model built on gpt-oss-20b. about 16 GB on disk, 131k context. the card reports 127 tok/sec generation and 17 GB peak memory, so this fits a 32 GB machine.
link
Qwen3.8 27B at 4-bit (lmstudio-community). 16.1 GB, and it does vision plus text. good candidate for the 32 GB and 48 GB crowd.
link
Cohere Transcribe at 8-bit (mlx-community). int8 speech to text, 4.13 GB. if you're doing transcription on device and Whisper isn't cutting it, worth a look.
link
GLM 5.3 Flash mixed 4/8-bit (pipenetwork). the monster. 320B total, 18B active, squeezed from 642 GB down to 182 GB. this one is for the Mac Studio Ultra owners only.
link
4. if you want to understand the distributed stuff
the WWDC26 session on distributed inference and training with MLX is the best background for item 1.
it covers JACCL, RDMA over Thunderbolt 5, and running a 27B Qwen and the 1 trillion parameter Kimi across a cluster with roughly 3x speedup.
that's the week.
what are you running right now, and on what chip?
I'd love to get real tokens per second numbers from real machines in the replies.
Stay Frosty,