Global Markets
Home›Global Markets›China›Robotics benchmark removed companies after “benchmark…
Robotics benchmark removed companies after “benchmark hacking” claims
The benchmark’s creators said they found evidence of benchmark manipulation and overhauled methodology, removing Spirit AI and other models from the official list.
A Chinese physical AI robotics start-up that briefly topped a global embodied intelligence benchmark has been removed from the official rankings after the benchmark creators cited evidence of “benchmark hacking,” according to a report by SCMP Economy.
SCMP Economy reports that Spirit AI, a Hangzhou-based firm founded in 2024, had overtaken Nvidia to take the top spot in June, before the benchmark’s creators overhauled their methodology and dropped the model from the official list just days later.
The update also removed other competitors, including a model from Chinese start-up X Square Robot that had ranked fourth, after the creators found problems with the benchmark results.
SCMP Economy adds that a lead author and UC Berkeley PhD student said the team retroactively removed evaluations from organizations they identified as engaging in benchmark manipulation, without naming specific companies.