Meta Releases Muse Spark Open: 816B MoE Tuned for Apple M5 Ultra

In a decisive escalation of the open-weight frontier race, Meta officially released Muse Spark Open, an 816-billion parameter Mixture-of-Experts (MoE) foundation model engineered for autonomous software engineering, deep multi-step reasoning, and long-horizon agentic workflows.

While the model’s raw scale positions it directly against proprietary frontier offerings from OpenAI, Anthropic, and Google, the defining breakthrough lies in how it runs. In an unexpected engineering partnership, Meta collaborated with Apple’s Machine Learning Research group to natively optimize Muse Spark Open for Apple silicon workstations.

Through custom kernels written for Apple’s MLX framework and multi-node Thunderbolt-based Remote Direct Memory Access (RDMA), developers can run the full 816B model locally on desktop clusters-specifically pairing dual 512GB M5 Ultra Mac Studios to establish an air-gapped, 1-terabyte unified memory workstation.

1. Architecture Breakdown: 816B MoE With 42B Active Parameters

Muse Spark Open builds upon the structural foundations established in Meta’s earlier closed-source Muse Wonder architecture, but transitions the entire model weights to an open-weight release hosted on Hugging Face under Meta’s Open Innovation Commercial License.

┌─────────────────────────────────────────────────────────────────────────────┐
│                 Meta Muse Spark Open: MoE Layer Routing                     │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│   [Multimodal Input: Code / Text / Audio / Video] (1 Million Token Context) │
│                                │                                            │
│                                ▼                                            │
│   ┌─────────────────────────────────────────────────────────────────────┐   │
│   │               Dynamic Gating & Routing Controller                   │   │
│   │   - Top-2 Expert Selection per Token across 64 Expert Sub-Networks  │   │
│   │   - 816B Total Parameters | 42B Sparsely Activated per Forward Pass │   │
│   └────────────────────────────┬────────────────────────────────────────┘   │
│                                │                                            │
│            ┌───────────────────┼───────────────────┐                        │
│            ▼                   ▼                   ▼                        │
│   ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐               │
│   │  Expert Cluster │ │  Expert Cluster │ │  Expert Cluster │               │
│   │    (01 to 21)   │ │    (22 to 43)   │ │    (44 to 64)   │               │
│   │  Syntax & Logic │ │  System Runtime │ │ Long-Term Memory│               │
│   │   Optimization  │ │   Tool Calling  │ │ Context Synthesis│               │
│   └────────┬────────┘ └────────┬────────┘ └────────┬────────┘               │
│            │                   │                   │                        │
│            └───────────────────┼───────────────────┘                        │
│                                │                                            │
│                                ▼                                            │
│   ┌─────────────────────────────────────────────────────────────────────┐   │
│   │                    KV-Cache Dynamic Quantization                    │   │
│   │   - FP8 Activation Tensors with Selective 4-Bit Sub-Layer Precision │   │
│   │   - Native Rotary Positional Embeddings (RoPE) up to 1M Tokens      │   │
│   └────────────────────────────┬────────────────────────────────────────┘   │
│                                │                                            │
│                                ▼                                            │
│   [Autonomous Tool Execution Engine: Terminal, Shell, Compiler & Browser]   │
└─────────────────────────────────────────────────────────────────────────────┘

The core architectural specifications include:

• Total Parameter Footprint: 816 billion parameters arranged in a 64-expert modular configuration.
• Active Parameters: 42 billion parameters activated per forward token pass, achieving high inference throughput without sacrificing model capacity.
• Context Length: 1 million tokens natively supported, utilizing grouped-query attention (GQA) with dynamic KV-cache compression.
• Multimodal Ingestion: Native cross-attention layers supporting high-resolution image sequences, arbitrary audio spectrograms, and raw video frames alongside code repositories.
• Agentic Primitives: Built-in function-calling primitives optimized for terminal interaction, git operations, and continuous test-driven execution loops.

Unlike dense models that require uniform GPU memory allocation across the entire network, Muse Spark Open’s sparse activation allows individual experts to be distributed across distinct memory banks without blocking the primary token generation thread.

2. The Apple Silicon Breakthrough: Desktop Supercomputing via MLX

Historically, running an 800B+ parameter model required multi-GPU server nodes (such as 8x Nvidia H100 or B200 HGX systems) costing upwards of $300,000 and drawing more than 10 kilowatts of facility power.

Meta and Apple bypassed this data-center bottleneck by exploiting the unified memory architecture (UMA) of Apple silicon.

┌─────────────────────────────────────────────────────────────────────────────┐
│          Dual M5 Ultra Mac Studio Cluster (Thunderbolt 5 RDMA Ring)         │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│   ┌───────────────────────────────┐     ┌───────────────────────────────┐   │
│   │     Mac Studio Primary Node   │     │    Mac Studio Secondary Node  │   │
│   │   Apple M5 Ultra (32-Core CPU)│     │   Apple M5 Ultra (32-Core CPU)│   │
│   │   96-Core GPU / 48-Core NPU   │     │   96-Core GPU / 48-Core NPU   │   │
│   │   512 GB Unified Memory       │     │   512 GB Unified Memory       │   │
│   │   (1.2 TB/s Memory Bandwidth) │     │   (1.2 TB/s Memory Bandwidth) │   │
│   │                               │     │                               │   │
│   │   [Experts 01-32 Allocation]  │     │   [Experts 33-64 Allocation]  │   │
│   └───────────────┬───────────────┘     └───────────────┬───────────────┘   │
│                   │                                     │                   │
│                   │      Dual Thunderbolt 5 RDMA Ring   │                   │
│                   └◄═══════════════════════════════════►┘                   │
│                          (160 Gbps Low-Latency Interconnect)                │
│                                                                             │
│   Total Pooled System:                                                      │
│   - 1,024 GB (1 TB) Continuous Unified Memory Address Space                 │
│   - 2.4 TB/s Aggregate Memory Bandwidth                                     │
│   - Full 816B MoE FP8 / INT4 Inference: 28-34 Tokens/Second Sustained       │
└─────────────────────────────────────────────────────────────────────────────┘

Thunderbolt RDMA and MLX Kernel Optimization

By enhancing Apple’s open-source MLX machine learning framework, engineers established a peer-to-peer memory bridge over dual Thunderbolt 5 cables:

1. Zero-Copy Memory Addressing: The operating system maps the memory pools of two Mac Studios into a single, contiguous 1,024 GB (1 TB) virtual address space.
2. Dynamic Expert Offloading: The first 32 experts reside in Node 1’s memory, while experts 33 through 64 reside in Node 2. Gating vectors route token embeddings across the 160 Gbps Thunderbolt bus in microsecond intervals.
3. Bandwidth Saturation: Each M5 Ultra chip features 1.2 TB/s of unified memory bandwidth, yielding an aggregate 2.4 TB/s throughput that prevents memory-bus stalls during large-context retrieval.

In benchmarking tests conducted by Meta and verified by independent developers, a dual M5 Ultra (512GB each) cluster generated between 28 and 34 tokens per second at 4-bit precision (INT4 AWQ) and 16 to 19 tokens per second at 8-bit floating point (FP8), well exceeding typical human reading speeds.

3. Benchmark Evaluations: Open Weights vs. Frontier Closed APIs

Meta evaluated Muse Spark Open against both the leading open-weight models (such as Mistral Large 4) and top-tier proprietary APIs including OpenAI GPT-5.6 Sol and Google Gemini 4 Argon.

Muse Spark Open closes the gap between open-weight distributions and proprietary frontier intelligence. On SWE-bench Verified, the model scored 62.4%, coming within 3.4 percentage points of Claude Opus 5.5, while offering developers total ownership of the model weights and zero API rate limits.

4. Cloud Economics vs. Local Hardware TCO

For enterprise software development teams and specialized defense contractors, the ability to run an 816B frontier model on local hardware fundamentally alters the total cost of ownership (TCO) equation.

The Cloud API Expenditure Equation

A team of 50 senior engineers utilizing automated coding agents typically consumes roughly 40 million input tokens and 10 million output tokens per day. At prevailing frontier API rates ($3.00 per million input tokens, $12.00 per million output tokens), monthly operational expenses exceed:

• Daily API Cost: (40 × $3.00) + (10 × $12.00) = $240.00 / day
• Annual API Run Rate: $240.00 × 260 working days = $62,400 per engineer team
• Data Risk: All proprietary intellectual property, internal git branches, and security tokens traverse commercial cloud endpoints.

The Local Workstation Capital Investment

By contrast, acquiring two high-spec M5 Ultra Mac Studios with 512GB of unified memory represents a one-time capital expense:

• Dual M5 Ultra Mac Studio (512GB each): ~$16,000 to $18,000 capital expenditure.
• Power Consumption: Peak draw of approximately 450 watts per node (~$65/month in commercial electricity).
• Breakeven Window: Less than 4 months compared to cloud API billing for a mid-sized engineering team.
• Sovereignty: Complete immunity from cloud service interruptions, third-party policy changes, or regulatory data-transfer bans under the EU AI Act.

5. Strategic Considerations for Enterprise Engineering

The arrival of Muse Spark Open establishes several strategic inflection points for engineering leaders:

1. Air-Gapped Autonomous Coding: Defense, healthcare, and financial institutions restricted from utilizing commercial cloud APIs can now deploy cutting-edge, 800B-class coding agents entirely within air-gapped on-premises local area networks.
2. Democratization of Desktop Clusters: Apple’s collaboration with Meta positions the Mac Studio as a serious desktop alternative to enterprise Nvidia server racks for inference and fine-tuning.
3. Weight Customization via LoRA: Teams can fine-tune private Low-Rank Adaptation (LoRA) adapters directly on their proprietary internal codebases without risking IP exposure.

Frequently Asked Questions

What is Meta Muse Spark Open?

Muse Spark Open is an 816-billion parameter Mixture-of-Experts (MoE) foundation model developed by Meta. It activates 42 billion parameters per token, supports a 1-million token context window, and is designed specifically for autonomous coding, complex reasoning, and long-horizon agentic workflows.

What hardware is required to run Muse Spark Open locally?

Running the unquantized or FP8 model requires roughly 1 TB of memory, achievable by clustering two Apple M5 Ultra Mac Studios (512GB unified memory each) over a high-speed Thunderbolt 5 connection using Apple’s MLX framework. Quantized INT4 configurations can operate on single or multi-node configurations with at least 450 GB of memory.

How does Muse Spark Open compare to proprietary APIs like Claude Opus and Gemini 4 Argon?

Muse Spark Open achieves 62.4% on SWE-bench Verified and 91.8% on HumanEval+, performing within 3-4% of top proprietary commercial models while providing complete data privacy, zero API per-token costs, and uncapped local inference.

What license applies to the model weights?

Meta released Muse Spark Open under the Meta Open Innovation Commercial License, allowing developers and enterprises to download, inspect, fine-tune, and deploy the weights locally with broad commercial rights.