AA-Robotics v0.1 Benchmark Ranks GPT-6 Astra First in Real-Robot Tasks

On October 14, AI benchmarking organization Artificial Analysis released AA-Robotics v0.1, a new evaluation that tests frontier large language models on real robotic manipulation. Each model was given control of a Hiwonder NexArm six-axis robotic arm through a coding agent and had to complete eight zero-shot tasks, including pouring liquid, writing, and stacking blocks.

According to the initial results, GPT-6 Astra ranked first with 69.4 points, followed by GPT-6.1 Sol with 64.6 points. Gemini 4 Argon scored 55.3 points, while Claude Opus 5.5 and Claude Fable 5.1 scored 48.8 and 46.3 points, respectively. Although Astra led the overall ranking, it achieved perfect scores on only four of the eight tasks; Sol, in second place, earned perfect scores on five tasks.

The evaluation also highlighted notable gaps in time and cost. Apart from Gemini 4 Argon, which averaged about 52 minutes per task, the other four models took roughly half an hour per task, compared with a human baseline of about 20 seconds. In terms of API costs, Astra averaged $16.36 per task, while Sol cost only $1.93 despite a narrow 4.8-point score difference.

Artificial Analysis found that models spent around 60% of their time planning actions, with actual arm movement accounting for only about 13%, reflecting the still-limited efficiency of general-purpose models in physical environments. The benchmark offers a quantitative reference point in the ongoing debate between general models and specialized embodied AI systems. Chinese models were not included in this first batch, leaving open the question of whether Qwen, DeepSeek, or others will appear in future rounds.