Every week seems to bring a bigger open-weights model, but every so often a release lands that actually changes the conversation. Tencent Hunyuan Hy4, released and open-sourced on August 28, is one of those. The headline numbers are impressive on their own — 770 billion total parameters, roughly 49 billion active per forward pass, and a context window stretching past one million tokens. But the numbers aren’t the story.
The story is the two claims wrapped around them: that Hunyuan Hy4 is the first Hunyuan model to participate in optimizing its own training and inference pipeline, and that it can orchestrate multiple OpenAI Codex sessions well enough to beat Codex alone on eight coding benchmarks. An open model that helps train itself and manages closed models as subcontractors? That’s a genuinely new shape of claim, and it deserves a closer look than a press-release skim.
In this post we’ll break down the architecture, dig into the 1M-token context engineering, separate the verifiable parts of the self-training narrative from the marketing, examine the orchestration benchmarks, and get practical about how to run Hunyuan Hy4 locally. Let’s dig in.
What Hunyuan Hy4 Actually Is: A 770B Open-Source MoE Model With 49B Active Parameters
Hy4 is a Mixture-of-Experts (MoE) model, and the total-versus-active parameter split is the key to understanding why anyone would ship a 770B model in 2026.
In a dense model, every parameter fires on every token. A 770B dense model would need an absurd amount of memory and burn a full 770B parameters’ worth of compute per token. MoE models work differently: the model contains many “expert” feed-forward blocks, but a learned router activates only a small subset for each token. Hunyuan Hy4 activates roughly 49B parameters per forward pass — about 6.4% of the total parameter count.
That ratio has a concrete practical meaning:
- Quality scales with total parameters. More experts means more learned “knowledge circuits”; the model’s capacity for world knowledge and rare-domain reasoning grows with the full 770B.
- Cost tracks active parameters. FLOPs per token are proportional to the ~49B active subset, not 770B. Inference is dramatically cheaper than a dense model of equivalent quality.
- Memory is the catch. You still have to store all 770B parameters. MoE shifts the bottleneck from compute to memory capacity — more on that when we get to self-hosting.
The model is available on Hugging Face under Tencent’s Hunyuan open-license terms, with weights downloadable on day one — which is what makes it a true open-source 770B model rather than a weights-behind-an-API release. That same-day availability matters: the fastest-growing open models now hit hundreds of thousands of downloads within 48 hours, and Hy4’s uptake has been right on that pattern. There’s also a low-friction hosted path — Tencent is offering free access through its WorkBuddy and CodeBuddy products for two weeks, which we’ll evaluate in the deployment section.
If you’ve followed the Hunyuan line, you’ll notice the scale jump. Previous Hunyuan flagship releases sat in the low-hundreds-of-billions range; Hunyuan Hy4 puts Tencent firmly in the same weight class as the largest open-weights releases from DeepSeek and Alibaba’s Qwen team.
The 1M-Token Context: How Tencent Engineered an Open-Source 1M Token Context Model and What It’s Good For
A context window exceeding one million tokens puts the Hunyuan Hy4 preview in the top tier of open models for input length. For calibration: 1M tokens is roughly 750,000 words, or a mid-sized codebase of 50,000–100,000 lines — plus documentation, tests, and commit history.
Long context is famously where demos shine and reality bites, so it’s worth asking what the window is actually good for and what it costs.
Where million-token context genuinely helps
- Codebase-scale reasoning. Feeding an entire repository in one prompt and asking cross-cutting questions — “which modules call this deprecated API,” “where does auth logic leak outside its boundary” — is the killer use case. Smaller windows force RAG-style retrieval, which is workable but loses holistic structure.
- Multi-document analysis. Legal discovery, financial filings, academic literature reviews — tasks where the relationships between documents matter more than any single one.
- Long agent sessions. Coding agents accumulate enormous context. A model that doesn’t degrade as the transcript grows can sustain longer autonomous runs.
The honest trade-offs
Long context is not free, and “1M supported” does not mean “1M equally reliable.” Two things to watch:
- Attention economics. Self-attention scales quadratically with sequence length unless the model uses sparse, sliding-window, or otherwise optimized attention. Hy4’s long-context engineering (Tencent cites chunked pretraining and position-encoding extensions, the standard toolkit) makes 1M tokens possible, but effective recall in the middle of very long inputs — the classic “lost in the middle” problem — is something you should test against your workload, not a leaderboard’s.
- Latency and cost. Prefilling a 900K-token prompt is a serious compute event. Even at aggressive attention kernels, first-token latency on huge inputs is measured in tens of seconds to minutes unless the provider runs heavy prompt caching. If your workflow is interactive, budget for it.
The practical sweet spot for most teams: use the full window for batch analysis and agent-state consolidation, and keep interactive sessions in the 100–200K range where quality and speed are both comfortable.
“It Helped Train Itself”: Dissecting the Hunyuan Hy4 Self-Training Claim
Here’s the claim that lit up the discourse: Tencent says Hunyuan Hy4 is the first Hunyuan model that participated in optimizing its own training and inference pipeline.
The phrasing is doing a lot of work, so let’s parse it carefully.
What this most plausibly means
“Optimizing the training and inference pipeline” is not the same as “the model improved its own weights.” Based on how the claim is framed, the believable version looks like this:
- Synthetic data generation. The model (or a predecessor in the Hunyuan line) generated training data — instruction following, code reasoning trajectories, preference pairs — that was filtered and incorporated into later training stages. This is now industry-standard, and it is a form of self-participation, but it has precedent in several major labs’ pipelines.
- Data filtering and quality scoring. Large models are routinely used to grade and de-duplicate training corpora. Letting a Hunyuan model score Hunyuan’s own data is a modest step beyond that.
- Inference-system optimization. Using model feedback to tune serving configurations — batching strategies, kernel selection, speculative-decoding draft behavior — is genuinely useful engineering and plausibly what “optimizing the inference pipeline” refers to.
What it does not (yet) demonstrate
Nobody has shown, for Hy4, a closed self-improvement loop where a model’s own outputs materially raise its measured capabilities across successive training rounds without human-driven direction. What’s verifiable today: the model exists, the benchmarks Tencent publishes can be independently replicated by anyone who downloads the weights, and the pipeline claims can be audited only to the extent Tencent discloses methodology. Everything beyond that is narrative.
That said, be careful not to dismiss it either. If your data pipeline, filtering, and serving stack are all shaped by a model in your own family, you’ve built a feedback loop with real flywheel potential — even if each revolution of the wheel is human-supervised. The meaningful question isn’t “is this AGI-adjacent self-improvement?” (no) but “does self-generated training data compound across model generations?” (increasingly, for frontier labs, yes).
My take: treat “helped train itself” as a legitimate description of modern data-centric training with a good press spin — and treat the next Hunyuan release, where that flywheel gets a full cycle to spin, as the real test.
Hunyuan Hy4 Benchmarks vs Codex: Hy4 as an Orchestrator vs. Codex Alone
The benchmark claim that stands out isn’t a MMLU number — it’s this: Tencent benchmarks Hunyuan Hy4 as an orchestrator of multiple OpenAI Codex sessions, and asserts that Hy4-plus-Codex beats Codex alone on eight coding benchmarks.
Why orchestration is an interesting capability claim
The standard pattern for coding agents has been one model doing the planning and the edits. The orchestration pattern is different: a large “manager” model decomposes a task, spawns multiple worker instances (here, Codex), reviews their outputs, merges results, and iterates. It’s the repo of ideas behind multi-agent frameworks, but with a frontier open model playing the manager role natively.
We’ve covered the evolution of AI coding tools before on this blog — see our earlier deep dive on Cursor’s Composer model.
If the claim holds up, it’s notable for two reasons:
- It’s a capability, not just knowledge. Orchestration tests decomposition, coordination, and evaluation — skills that don’t show up in single-turn QA benchmarks. A model good at managing parallel workers is valuable even if its raw per-token knowledge isn’t SOTA.
- It’s a business signal. Tencent positioning its open model as a manager of OpenAI’s closed model is a striking inversion of the usual dependency direction. An open-weights model acting as the control plane for closed-model labor is exactly the “open model at the top of the stack” future open-source advocates have been predicting.
How much stock to put in vendor benchmarks
Standard disclaimers apply, loudly. Tencent-selected benchmarks, vendor-run evaluations, and “beats Codex alone” (note: alone, not beats Hy4-orchestrated-Codex with a different manager) are all decisions that shape the headline. The good news with open weights: you can check. The honest read is that orchestration gains are real but task-dependent — multi-agent setups shine on large, parallelizable tasks and can underperform a single strong model on tightly coupled problems where coordination overhead dominates.
Compared with the open models we’ve covered previously — DeepSeek, GLM, and Mistral: Hy4’s differentiation isn’t raw benchmark placement — those models remain excellent — it’s the combination of extreme context length, extreme scale, and an explicit orchestration story. None of the prior open-weights releases led with “we manage other models better than they manage themselves.”
How to Run Hunyuan Hy4 Locally: Hardware, Quantization, and Deployment Options
Now the practical question many readers are asking — how to run Hunyuan Hy4 locally? Can you host this thing? The answer is a tiered maybe. If you’ve already followed a guide on how to run smaller open models like Gemma 4 locally or squeezed a 400B-parameter LLM onto consumer hardware, the workflow is familiar — only the hardware bill changes.
The full-precision reality check
At BF16, 770B parameters means roughly 1.5 TB of weight memory before you reserve anything for KV cache and activations. That translates to:
- A single-node 8× H200 (1.1 TB total HBM) won’t fit it comfortably at full precision. You’re looking at multi-node setups, or 8× B300 / MI400-class hardware with headroom.
- Memory-bandwidth matters as much as capacity for MoE models — with only 49B active parameters, tokens-per-second is bandwidth-limited during decode, so HBM3e-class bandwidth is your friend.
Realistically, full-precision self-hosting is for companies and labs with serious GPU budgets. That’s fine — that’s who day-one open releases are for.
Quantization paths for smaller setups
This is where MoE structure helps you:
- FP8: 770B parameters at 1 byte each ≈ 770 GB. Still multi-GPU (a 4× H200 node gets close), but a vastly more achievable target, and MoE models are known to be relatively robust to FP8 quantization since most experts see sparse activation.
- Aggressive 4-bit: roughly 385–450 GB depending on the scheme. That’s a 2–3 node A100/H100 cluster or an 8× A100 80GB node at the edge of feasibility. Quality loss on a model this large is typically modest, though routing-sensitive layers deserve careful handling.
- Expert offloading: Because only ~49B parameters are active per token, CPU-offloading or NVMe-streaming expert weights becomes viable for patient, batch-oriented workloads. Throughput suffers; the setup cost drops enormously. For a low-traffic internal tool sleeping between requests, this can be the pragmatic choice.
For oversized open-weights models, multi-GPU memory planning is its own discipline, and we’ll cover quantization formats in detail in a future post.
Practical rule of thumb for capacity planning: KV cache grows with context length, and at 1M-token contexts the cache, not the weights, becomes the dominant consumer. If you actually plan to serve million-token prompts, budget cache memory separately and consider limiting maximum context per instance.
The zero-hardware path
If none of that sounds like a fun weekend, Tencent’s two-week free trial through WorkBuddy and CodeBuddy is the sensible first stop. You get to test the 1M-context behavior and the orchestration workflows against your real tasks before committing hardware or money. Given how heterogeneous long-context quality is across workloads, evaluating before buying is not optional — it’s the whole game.
What Hunyuan Hy4 Means for the Open-Weights Race
Zoom out, and the timing of the Hunyuan Hy4 release tells you as much as its specs.
The China open-weights push keeps accelerating
Tencent, DeepSeek, and Alibaba are now shipping frontier-class open models on overlapping cadences, each release ratcheting the open-availability floor higher. For any startup or enterprise building on open weights, the practical effect is compounding leverage: capability that required a six-figure GPU cluster and a closed-API relationship two years ago now ships as a Hugging Face download. The previous open-weights MoE flagship we deep-dived, Mistral 3’s 675B MoE, now sits one tier down on total scale. Hy4’s 770B/49B-active design, with its 1M-token window, extends that floor into territory — extreme scale and extreme context together — that open weights hadn’t jointly claimed before.
The timing collision with the closed-lab story
The release landed in the same news cycle as turbulence at OpenAI around its relationship with coding-tool vendors (see our Cursor Composer 2.5 review for context) and a high-profile GPU supply deal reshuffle. That collision is not accidental in effect, whatever it is in intent. Every story about closed-ecosystem friction is, for open-weights labs, free advertising: “whatever happens to the closed vendors’ relationships, the weights are yours.” An open model that benchmarks itself as the orchestrator of a closed model’s sessions is the sharpest possible version of that pitch.
Is “free for two weeks” a pattern?
The WorkBuddy/CodeBuddy free trial is the interesting go-to-market wrinkle. We’ve seen open-weights releases paired with limited-time free hosting before, and it’s easy to read cynically. But the strategic logic is sound: the expensive part of competing with closed labs isn’t the weights — it’s distribution and trust. Free trials buy evaluation cycles from exactly the developers who write the benchmarks, blog posts, and fine-tunes that define a model’s reputation. Expect this “open weights + free hosted window” pattern to become standard operating procedure for every major China-lab release this year.
The bottom line
Hunyuan Hy4 isn’t just another big model. It’s the largest-scale, longest-context entry in the open-weights arena to date, it ships with a self-referential training narrative that previews where frontier labs generically are heading, and it makes an explicitly orchestration-first pitch — open model as the manager, closed models as the workforce. Whether the benchmark claims survive independent replication (and with open weights, you can be part of that replication) will determine how much of the hype sticks. But the direction of travel is unmistakable: the open-weights floor keeps rising, and this week it rose a lot.
If you take a test run through the free trial, or — braver — spin up a quantized deployment, I’d genuinely like to hear what you find on your own workloads, especially at the far end of that million-token window. Drop a comment or ping me; the community replication numbers are the benchmark that matters.
References and further reading
- Tencent Hunyuan — official site
- Tencent on Hugging Face — Hunyuan model weights
- OpenAI Codex
- Mixture-of-Experts Explained — Hugging Face blog
- DeepSeek — official site
- GLM by Z.ai
Please let us know if you enjoyed this blog post. Share it with others to spread the knowledge! If you believe any images in this post infringe your copyright, please contact us promptly so we can remove them.