Kimi K3 on B300×8: MXFP4 Memory and Day-0 Paths

Kimi K3 from Moonshot AI is an open-weight MoE at roughly 2.8T total / 104B active parameters. Released weights are MXFP4 with a disk footprint around 1.56–1.68TB, so the first self-host fork is practical: H100 80GB×8 cannot hold the weights, while B300×8 may fit a single node.

📑Table of Contents

1. Official Kimi K3 specs: downloadable is not production-ready
2. Can one B300×8 node actually fit it? Memory and Day-0 evidence
3. Serving paths compared: SGLang, vLLM, and Dynamo
4. Pre-adoption checklist and next actions
5. FAQ
6. Bottom line

The short answer is conditional yes. If you have MXFP4 weights plus B300 288GB×8 (about 2.3TB aggregate HBM), single-node Day-0 validation is realistic. Moonshot’s 64+ accelerator guidance and multi-node recipes from vLLM / NVIDIA Dynamo still belong on a different axis: lab proof versus production throughput.

Decision materials in this article

This article maps the following decision material before you buy more GPUs:

• Official footprint and memory floor
• How to read a B300×8 Day-0 run
• SGLang vs vLLM vs Dynamo paths
• A go/no-go checklist you can apply on your own stack

---

Official Kimi K3 specs: downloadable is not production-ready

Fixed model-card facts

Start with the fixed card, not social headlines. The Hugging Face moonshotai/Kimi-K3 model card lists:

• 2.8T total / 104B activated
• 896 experts with 16 selected plus 2 shared
• 1,048,576 context
• MoonViT-V2 (401M)
• MXFP4 weights / MXFP8 activations via QAT
• Kimi K3 License

---

Architecture and the first bottleneck

Architecturally, keep these points in view:

• Most attention layers are linear KDA (fixed-size state)
• Periodic Gated MLA (compressed KV)
• The hybrid design limits pure KV blow-up at long context
• The first bottleneck is usually whether weights fit at all
• Sparse MoE does not automatically shrink resident memory

---

Independent coverage vs sustained SLA

Independent coverage converges on the same operational picture (innovaTopia, PC Watch):

• Open weights
• MXFP4 near 1.56TB
• 64+ accelerator guidance
• Engine paths such as vLLM / SGLang

Being able to download the package is not the same as meeting a sustained SLA.

---

License forks for ops and legal timing

License terms create another fork. The Runpod technical FAQ describes:

• A MIT-derived Kimi K3 License that is broadly usable
• Extra display/conditions at large MaaS/MAU or revenue thresholds
• Internal eval and public large-scale SaaS should not share the same legal checklist timing

---

Spec summary table

Item	Value	Source
Total / active	2.8T / 104B	HF model card
MoE	896 experts, 16 active + 2 shared	HF
Attention	69 KDA + 24 Gated MLA	HF
Context	1,048,576	HF / Dynamo
Weight format	MXFP4 (QAT) / MXFP8 activations	HF
Disk ballpark	~1.56TB (reports/FAQ) / 1680GB (vLLM Recipes)	innovaTopia, Runpod, vLLM
Vision	MoonViT-V2 401M	HF

Source: Hugging Face moonshotai/Kimi-K3 (as of July 2026).

---

Can one B300×8 node actually fit it? Memory and Day-0 evidence

Memory math first

Memory math is blunt.

• An FP8-style footprint near 2.8TB does not fit eight B300 GPUs (~2.3TB)
• MXFP4 moves the main weights into roughly 1.56–1.68TB
• Headroom is decided by KV, KDA state, and runtime slack

---

Fixstars Day-0 environment

The Fixstars Zenn Day-0 write-up (origin measurement; attribute it) reports this environment:

• B300 SXM6×8 box (288GB×8=2304GB)
• SGLang 0.5.16 via lmsysorg/sglang:kimi-k3
• CUDA 13

---

Fixstars Day-0 measurements

Key measurements:

• Measured weights: ~195.86 GB/GPU, ~1.57TB total
• Cold start: ~88.8 minutes total (~81.4 minutes load). Restarts are expensive
• Steady-state per rank: ~248/275 GB used; Mamba/KDA state 17.13GB; MLA KV 1.70GB (65,984 tokens/rank)
• Example effective context with DCP8: 65,984 × 8 = 527,872 tokens at --mamba-full-memory-ratio=5
• Lesson learned: ratio=5 over-allocates KDA state. Cookbook defaults sit near ~0.9 and should be derived from average request length

---

Independent FAQ fit point

Independent operator guidance agrees on the fit point. Runpod’s FAQ explicitly calls 8×B300 (288GB, ~2.3TB aggregate) the simplest fit for weights plus KV.

• H100 8×80GB is treated as a non-starter for weights, matching JA reporting
• Day-0 boot success is not 1M context at high concurrency forever
• mamba-full-memory-ratio, DCP, and engine choice change usable context and parallelism
• If cold start is about an hour, do not design a workflow that restarts the stack casually

---

Memory ballpark table

Lens	Ballpark	Implication
Weights	MXFP4 ~1.56–1.68TB	Design point for 8×B300
Node HBM	~2.3TB (288×8)	Budget for KV/state/NCCL
Cold start	tens of minutes to ~90 min	Poor fit for frequent restarts
Long-context tuning	lower mamba-full-memory-ratio	Shift budget from KDA state toward MLA KV

Sources: Fixstars Zenn measurements, Runpod Kimi K3 FAQ, SGLang cookbook (July 2026).

---

Serving paths compared: SGLang, vLLM, and Dynamo

Engine docs do not describe the same “minimum viable” shape. Choose by topology and ops model, not brand preference.

SGLang cookbook

SGLang cookbook (docs.sglang.io) highlights:

• Day-0 tags such as lmsysorg/sglang:kimi-k3
• B300 1×8 in its hardware matrix
• Unified / Prefill / Decode modes
• Latency / throughput presets
• DSPARK
• A --mamba-full-memory-ratio calculator
• Good fit for single-node long-context tuning and Fixstars’ DCP-oriented Day-0 path

---

vLLM Recipes

vLLM Recipes (recipes.vllm.ai) centers on:

• vllm/vllm-openai:kimi-k3
• An MXFP4 1680GB overview figure
• Prerequisites that lean toward at least 8×GB300 and multi-node for real production traffic
• Deeper TP / TEP / DEP and PD-disaggregation profiles
• Prefer when OpenAI-compatible serving and cluster standardization matter more than one-box Day-0 experiments
• Budget for tool-call format drift with schema validation and retries

---

NVIDIA Dynamo

NVIDIA Dynamo (docs.nvidia.com) targets:

• Kubernetes Dynamo+vLLM on GB200 / GB300 NVL72
• Aggregated and disaggregated topologies
• Examples: 16×GB200 TP16, or 16×GB300 as two 8-GPU replicas
• Production-rack recipe with FP8 KV and 1M context assumptions
• Not a laptop-adjacent single B300 lab story

Source

---

Time-bound engine choice note

Fixstars’ Day-0 note also explains a time-bound engine choice:

• At that moment vLLM lacked DCP
• MLA KV was replicated across GPUs
• SGLang+DCP looked better for long context

Re-check current engine matrices before locking a stack.

---

Path comparison at a glance

Axis	SGLang cookbook	vLLM Recipes	NVIDIA Dynamo
Typical shape	B300 1×8 in validation matrix	multi-node / 8×GB300 posture	GB200/GB300 racks + K8s
Strengths	DCP, mamba ratio, DSPARK path	OpenAI-compatible serve, broad HW picker	KV-aware routing, disagg ops
Best for	Day-0 single-node tuning	Cluster TP/EP production	Large agentic + platform ops
Watch-outs	Bad ratio shrinks context capacity	Tool-parser drift, interconnect	Heavy prerequisites, long first launch

---

Serving-path sources

Sources: SGLang cookbook, vLLM Recipes, NVIDIA Dynamo Kimi-K3 (July 2026).

---

Pre-adoption checklist and next actions

What readers need after the article is a recorded go / no-go, not another summary of the headline.

Hardware and purpose checks

1. GPUs: Can you secure B300 288GB×8 (or equivalent ~2.3TB class)? H100 80GB×8 alone is no-go for weights.
2. Storage/network: 1.5TB+ download plus local or shared cache. Expect retries right after release.
3. Purpose: Lab Day-0 validation versus high-concurrency production. For the latter, re-estimate with 64+ guidance and Dynamo/GB300 multi-replica shapes.

---

Engine, ops, and evaluation checks

4. Engine: Single-node long-context efficiency → SGLang+DCP first, freeze ratio from average request length. Cluster standardization → vLLM/Dynamo official recipes.
5. Ops tolerance: Can you accept ~1 hour cold starts? Always-on reasoning tokens, tool-parser checks, and license thresholds need owners.
6. Evaluation: Measure TTFT/ITL/throughput on your workload. Do not promote vendor or blog numbers into SLA targets unchanged.

---

Recommended next steps

Suggested sequence:

1. Confirm moonshotai/Kimi-K3 and the license on Hugging Face
2. Generate a B300 1×8 command path from the SGLang cookbook and lock ratio from average request length
3. Short-prompt smoke → long context / tool calls → DSPARK if needed
4. Escalate to 16×B200, multi-replica GB300, or Dynamo disaggregation when the single node is insufficient

---

Separate API decision track

API pricing and product-fit questions are a separate decision track. For fee/API caveats around Kimi K3 as a service choice, pair this with the related “Kimi K3 adoption judgment” article.

---

FAQ

Does Kimi K3 run on one B300×8 node?

With MXFP4 open weights, measurements and independent FAQs support weight fit. Usable context and concurrency still depend on KV/state split, cold start, and workload.

---

How many GPUs does the vendor recommend?

Coverage often cites Moonshot’s 64+ accelerator guidance. Treat single-node Day-0 and recommended production scale as different questions.

---

SGLang or vLLM?

Prefer SGLang cookbook when Day-0 single-node DCP/mamba tuning matters most. Prefer vLLM-family paths for cluster norms and Dynamo integration. Always re-check current DCP support tables.

---

If I can download it, can I use it immediately?

Not really. Terabyte-scale load means long cold starts. Plan tuning around expensive restarts.

---

What about H100×8?

Independent reports treat H100 8×80GB as unable to hold the weights. Check generation and HBM capacity first.

---

Bottom line

Even at 2.8T class, MXFP4 makes 8×B300 single-node Day-0 a realistic validation line. Success conditions are MXFP4 weights, enough HBM, and engine-specific state/KV design—not a viral headline.

Next actions: inventory GPUs, lock cookbook ratios, run smoke tests, then escalate to multi-node/Dynamo only if needed. Recording go/no-go against the checklist moves the discussion from “it boots” to “it fits this use case.”

Related articles:

• 11300
• 11301
• 13549