Scaling in Games: Continued Pre-Training For Embodied Agents In Diverse Virtual Worlds

Introduction

Recent advances in large‑scale training have shown that scaling data volume, diversity, and model capacity can dramatically boost performance for embodied agents in physical settings. A new study asks whether the same scaling benefits carry over to virtual game worlds, which differ in visual style, physics, controls, and objectives. By systematically varying data quantity, world diversity, and model size across 120 configurations, the researchers map out how these factors influence both in‑corpus performance and transfer to unseen environments.

Scaling Data and Diversity

The experiments span six data budgets—from 50 to 1,600 hours of gameplay footage—and five mixtures ranging from a single‑world (Minecraft‑only) corpus to a multi‑world blend covering several distinct games. Results confirm the intuitive expectation that more data improves performance on the worlds seen during training. More interestingly, adding diverse worlds yields a noticeable boost when agents are evaluated on related unseen worlds, though the benefit diminishes for distant worlds that share fewer mechanics or visual cues with the training set. This suggests that diversity acts as a form of regularization, helping agents learn more generalizable representations without guaranteeing full cross‑domain transfer.

Model Size and Efficiency

Four model sizes were tested, from 0.8 B to 27 B parameters. Larger models consistently achieve higher data efficiency: they reach comparable performance to smaller models with roughly half the amount of training data. However, when training on a single world, the biggest models tend to over‑fit to that world’s specifics, leading to poorer generalization on distant worlds. This trade‑off highlights a nuanced interaction between capacity and data diversity—simply scaling up a model does not automatically yield broader competence.

Training Phases and Empirical Scaling Law

The authors identify two distinct phases during continued pre‑training. An early interface phase shows broad improvements across many worlds, indicating that agents are learning general interaction primitives (e.g., navigation, object manipulation). As training progresses, a world phase emerges where gains become concentrated on the worlds present in the training mix, and transfer to unrelated worlds plateaus. By fitting an empirical scaling law that relates validation loss to data volume, diversity, and model size, the study provides a quantitative tool for predicting returns on additional compute or data.

Competitive Performance and Remaining Gaps

When benchmarked against specialized Minecraft agents and frontier 3D‑world models, the best‑performing continued‑pre‑trained agents are competitive, especially in real‑time settings where latency matters. Nonetheless, persistent gaps remain in distant worlds, underscoring the limits of imitation‑based pre‑training alone. The authors argue that achieving truly generalist embodied agents will likely require richer training pipelines—perhaps incorporating curriculum learning, multimodal feedback, or interactive fine‑tuning beyond the static pre‑training phase.

Practical Takeaways

• Prioritize data diversity when the goal is cross‑world transfer; even modest inclusion of varied worlds can improve generalization.
• Leverage larger models for data‑efficient learning, but pair them with diverse curricula to avoid over‑fitting to a single environment.
• Monitor training phases; early gains may be broadly applicable, while later improvements could become world‑specific.
• Plan for post‑pre‑training refinement if agents must operate in highly novel or distant virtual settings.

Continued pre‑training offers a promising route to more capable embodied agents in games, but the journey toward universal virtual competence will require complementary strategies that go beyond sheer scale.