Complementarity, not competition
MoBA solves which positions to attend to (block-sparse gating on the sequence dimension). φ-Attention solves how attention heads should talk to each other (C5-cycle coupling on the head dimension). These two dimensions are orthogonal — combining them should be strictly additive.
The core mathematical insight
A 5-cycle adjacency matrix with cos72° weights has a unique spectral property:
- Maximum eigenvalue = φ (golden ratio, 1.618034..., verified to 6.7e-16 precision)
- This is the only weight that achieves this — it's algebraically necessary, not an engineering choice
- Spectral gap: C5 gives 0.0707 vs standard attention's 0.0034 (20× improvement)
In MoBA's terms: you gate which blocks each query sees (position-level sparsity). We gate which heads share information via a fixed C5 graph (head-level coupling). The position graph and the head graph are independent design spaces.
Why this matters for long-context
MoBA's block-sparse gating already reduces position-level compute. But standard multi-head attention still treats heads as independent — each head sees only its own features. C5 coupling creates an information highway between heads with zero additional parameters:
| Metric |
Standard 5-head |
C5-coupled 5-head |
| Max eigenvalue |
1.000 |
φ = 1.618 |
| Spectral gap |
0.0034 |
0.0707 (20×) |
| Mixing steps to steady state |
25 |
11 (2.3× faster) |
| DOF (effective params) |
5.0 |
4.50 |
| DC concentration |
0.198 |
0.439 (2.2×) |
| Retention @2048 |
0.606 |
0.622 |
The DC concentration result is the key one for long-context: more signal energy in the DC component means slower decay, which means better information retention over long distances.
The pharmacological chain (all code-verified)
C5 adjacency → DFT spectral filter (λ₀=φ amplifies DC, λ₂=0.5 compresses harmonics)
→ DC concentration (φ5=0.439 vs std5=0.198)
→ Redundancy (DOF 5→2.2, 56% reduction)
→ Slower decay (α=0.000216 < 0.000249)
→ Better long-range retention (ret@2048=0.622 > 0.606)
All 7 causal links pass, all experiments zero-fitting (no free parameters tuned to data).
Concrete combination scenario
MoBA: select relevant KV blocks → sparse position-level attention
↓ (output per head: mixed position information)
φ-Attention: C5 coupling between heads → mixed head information
↓ (output: both position-mixed AND head-mixed)
→ Potentially: fewer heads needed for same capacity, or same heads with better long-range signal
The combination is architecturally simple: after MoBA's block-sparse attention, apply the C5 mixing matrix across heads. No new parameters, no new learnable components.
Full code and detailed results
Repo: https://github.com/wangjun112233/phi-attention
The README follows a five-step "alchemical" framework (furnace → pharmacology → fire → refinement → mechanism) with all experiments reproducible.
Open question
Has the MoBA team explored any form of head-level structure? The current design treats heads independently — if there's interest in testing C5 coupling on top of MoBA's block-sparse attention, we'd be happy to collaborate on benchmarks.
Note: I'm the author of φ-Attention, an independent researcher. This is a genuine research observation about complementarity, not a product pitch.
Complementarity, not competition
MoBA solves which positions to attend to (block-sparse gating on the sequence dimension). φ-Attention solves how attention heads should talk to each other (C5-cycle coupling on the head dimension). These two dimensions are orthogonal — combining them should be strictly additive.
The core mathematical insight
A 5-cycle adjacency matrix with cos72° weights has a unique spectral property:
In MoBA's terms: you gate which blocks each query sees (position-level sparsity). We gate which heads share information via a fixed C5 graph (head-level coupling). The position graph and the head graph are independent design spaces.
Why this matters for long-context
MoBA's block-sparse gating already reduces position-level compute. But standard multi-head attention still treats heads as independent — each head sees only its own features. C5 coupling creates an information highway between heads with zero additional parameters:
The DC concentration result is the key one for long-context: more signal energy in the DC component means slower decay, which means better information retention over long distances.
The pharmacological chain (all code-verified)
All 7 causal links pass, all experiments zero-fitting (no free parameters tuned to data).
Concrete combination scenario
The combination is architecturally simple: after MoBA's block-sparse attention, apply the C5 mixing matrix across heads. No new parameters, no new learnable components.
Full code and detailed results
Repo: https://github.com/wangjun112233/phi-attention
The README follows a five-step "alchemical" framework (furnace → pharmacology → fire → refinement → mechanism) with all experiments reproducible.
Open question
Has the MoBA team explored any form of head-level structure? The current design treats heads independently — if there's interest in testing C5 coupling on top of MoBA's block-sparse attention, we'd be happy to collaborate on benchmarks.
Note: I'm the author of φ-Attention, an independent researcher. This is a genuine research observation about complementarity, not a product pitch.