Retaining Suboptimal Actions to Follow Shifting Optima in Multi-Agent Reinforcement Learning

Keep alternative high-value actions available as the best joint strategy changes during training.

Successive Sub-value Q-learning learns multiple sub-value functions rather than relying on a single optimal action. A softmax behavior policy uses these alternatives to sustain exploration and adapt to changing value estimates.