NeurIPS 2026 / Cooperative multi-agent learning
Graduate School of Artificial Intelligence, UNIST
Better cooperation starts with where agents focus—and how long they sustain that focus.
Sparse rewards tell agents little about how to work together. Even when each agent can change the environment, their influence can remain scattered.
FIM adds an intrinsic learning signal that directs agents toward under-explored parts of the state space and rewards sustained collective influence. It combines entropy-based state selection with eligibility traces, and can augment different multi-agent learning backbones.
Cooperative multi-agent reinforcement learning (MARL) under sparse rewards remains fundamentally challenging because agents often fail to concentrate their influence, leading to insufficiently coordinated exploration. To address this, we propose the Focusing Influence Mechanism (FIM), a framework that encourages agents to focus their influence on under-explored parts of the state space through an entropy-based criterion, while leveraging eligibility traces to enable multiple agents to consistently align and sustain their influence on the same parts of the state space when beneficial, thereby promoting coordinated and persistent joint behavior. By emphasizing under-explored regions of the state space, FIM facilitates more efficient and structured exploration even under extremely sparse rewards. Across diverse MARL benchmarks, FIM consistently improves cooperative performance over strong baselines.
Choose meaningful dimensions. Sustain coordinated influence.
State Focusing Influence
Estimate the entropy of state transitions and give more weight to dimensions with limited observed variation under the current behavior policy.
Under-explored state dimensions receive larger weights wd.
Agent Focusing Influence
Estimate joint influence through counterfactual reasoning. Accumulate this influence with eligibility traces, encouraging the team to keep affecting shared targets over time.
Repeated collective influence increases the trace etd.
Together, SFI and AFI encourage focused, persistent cooperation under sparse rewards.
Performance curves from the paper, using sparse rewards.
Curves show the mean over 5 random seeds; shaded regions denote standard deviation. Select a figure to open its full-resolution version.
Increasing allied teams from 8 to 10 agents tests whether focused influence remains useful as coordination and state spaces grow.
| Agents | FIM win rate | Steps to converge |
|---|---|---|
| 8 | 0.82 ± 0.24 | ≈ 7M |
| 9 | 0.80 ± 0.25 | ≈ 11M |
| 10 | 0.79 ± 0.27 | ≈ 25M |
Final performance is preserved in these settings, while larger teams require longer training.
Additional evaluations use spatial binary states in Overcooked and continuous actions in the Multi-Agent Particle Environment.
| Environment / backbone | Baseline | + FIM |
|---|---|---|
| Overcooked IPPO · coord-ring | 195.7 ± 5.3 | 218.5 ± 4.7 |
| MPE MAPPO · continuous | −80.7 ± 19.6 | −63.4 ± 7.9 |
Returns are mean ± standard deviation; higher is better. MPE is evaluated at 1M steps. Reward scales differ across environments.
Read the full manuscript and explore the FIM implementation on GitHub.
@inproceedings{park2026focusing,
title={Focusing Influence Mechanism for Multi-Agent Reinforcement Learning},
author={Park, Yisak and Lee, Sunwoo and Han, Seungyul},
booktitle={Advances in Neural Information Processing Systems},
year={2026}
}