Cognition & Deployment Engineering
Long-form thinking on AI deployment engineering, cognitive literacy, and workflow redesign — deep, sourced, and built for the real world.
5 articles
Multi-GPU Chassis Benchmark: 287ms TTFT and 310/310 Zero Errors on a 122B MoE Model
We ran a full inference benchmark of a 122B-parameter MoE model (10B active, Int4) on our in-house liquid-cooled multi-GPU chassis: 164 tok/s output at the production concurrency point, 287ms median time-to-first-token, zero errors across 310 requests, and five out of five quality tests passed. Full numbers and the concurrency knee analysis below.
A Production RAG Upgrade in 48 Hours: Cross-Encoder Reranking, vLLM Migration, v1.0.0
Over 48 hours on 2026-07-24/25 we shipped a complete production RAG upgrade on a single DGX Spark: cross-encoder reranking wired into the chat path, the inference engine migrated to vLLM, and a 17-question golden set established as a blocking CI gate. Single-stream TTFT fell 72%, c=2 aggregate throughput doubled, and the primary serving path stayed up throughout.
GB10 Inference Engine Shootout: llama.cpp -np4 vs vLLM vs NVIDIA's Official NGC Container
On the same DGX Spark (GB10, 128GB unified memory), we ran GPT-OSS-120B behind llama.cpp -np4, a self-built vLLM image, and NVIDIA's official NGC vLLM container, head-to-head on the real chat pipeline — not a synthetic benchmark. Verdict: the self-built vLLM image won, for a reason that cuts against the "official image is faster" intuition.
TEI Has No arm64 Image: Running bge-reranker-v2-m3 on DGX Spark with llama.cpp
HuggingFace Text Embeddings Inference (TEI) ships no linux/arm64 image, so it cannot start on DGX Spark (GB10, ARM64). Here is how we served a cross-encoder reranker from the llama.cpp image we already run, shipping with zero business-code changes, and how we sized the batch parameters for real RAG chunk lengths.
人工智能前沿部署时代的「基础认知」全景解析与组织流程重塑报告
前沿部署工程师（FDE）在客户真实场景中锻造 AI 商业价值：本文以「10-20-70 法则」揭示成败七成取决于人与流程，并系统构建覆盖技术基石、工作流重塑与组织治理的三层「基础认知」框架。