Tencent Hunyuan, Fudan and Tsinghua Release ExplorationBench for Measuring AI Exploration

Tencent Hunyuan, in collaboration with researchers from Fudan University and Tsinghua University, has released ExplorationBench, a new benchmark designed to measure how AI systems explore. The benchmark features verifiable Alien Worlds environments, comprising two sandboxes: AlienCode, with 31 hidden rule changes and 70 tasks, and AlienLogic, with 24 patched inference rules and 70 theorems. Every answer is graded by an interpreter or proof checker, eliminating the need for LLM-based judges.

Evaluating 10 frontier AI systems, the study reveals four key findings. First, receiving feedback is more effective than thinking alone: on AlienCode, the best performance rises from 15.7% to 89.0% after four rounds, while runs without feedback stay between 0.5% and 11.0%. Second, designing experiments matters—replaying a system's own best probes yields the same evidence, yet 9 of 10 systems perform worse on AlienCode than when they selected probes themselves. Third, knowing a rule is not using it: even when all required rules are correctly stated, tasks are solved only 73.4% of the time. Fourth, a single score hides significant variation, with the same system under the same budget ranging from 5.7% to 79.0%, and rankings barely transferring between the two worlds.

The paper, website, and leaderboard are now available; code will be released soon.