Alibaba's GUI Agent Scores 92% on Real Android Devices. GPT-5.6 Wasn't Even Close.
Alibaba's open-weight Qwen-UI-Agent models use visual screen understanding to operate phones, desktops, and browsers, posting strong reported results on mobile, desktop, web, and visual-grounding benchmarks.
The race to build AI that can control computers by reading screens just got a serious open-source contender. Alibaba released Qwen-UI-Agent this week — two open-weight models that navigate phones, desktops, and web browsers by visually parsing what's on screen, then deciding what to click, type, or swipe. On the benchmarks that measure exactly that, it beat GPT-5.6 Sol and Claude Opus 4.8 by margins that are hard to dismiss.
What it does
Qwen-UI-Agent isn't a language model you prompt. It's an agent trained to look at a screen and act. Given a task — "book a flight," "fill in this form," "search for this document" — it reads the live display, figures out where to click or type, and executes. No APIs. No pre-installed hooks. Just vision and motor control applied to whatever software is running.
The two released models, MAI-UI-2B and MAI-UI-8B, are available on Hugging Face and GitHub under the Tongyi-MAI project. No commercial pricing has been announced; the weights are free to use and run locally.
The benchmark numbers
On MobileWorld — a standard benchmark for AI mobile-device control — Qwen-UI-Agent scored 82.1%, beating GPT-5.6 Sol by 12.0 percentage points and Claude Opus 4.8 by 14.6 points.
That margin gets wider when you move from synthetic tests to real hardware. On MobileWorld-Real, a variant using actual Android devices running over 400 tasks across more than 100 applications, the model hit 92.2%. Competitors including Gemini 3.1 Pro, Claude Opus 4.8, and GPT-5.6 Sol all trailed.
On desktop, the picture is slightly more mixed. OSWorld-Verified, which measures how well an agent can complete arbitrary tasks on a PC desktop environment, put Qwen-UI-Agent at 79.5% — strong enough for second place overall. Web navigation scored 73.6% on WebArena, ahead of Claude Opus 4.8 (+1.7pp), GPT-5.5 (+4.1pp), and Gemini 3.1 Pro (+8.3pp). For visual grounding — the sub-task of correctly identifying where on a screen an element lives — ScreenSpot-Pro came in at 81.5%.
The AndroidDaily benchmark returned the most striking number: 97.5% accuracy on everyday Android task sequences.
How it was built
Training data for Qwen-UI-Agent came from over 100 real mobile devices running 150 distinct applications. Alibaba built its own real-device benchmark with 400+ tasks to avoid the gap that often exists between lab simulations and actual phone behavior.
Reinforcement learning was applied at scale: approximately 10,000 simultaneous simulated environments were used to teach the model to handle task sequences exceeding 100 steps — a meaningful threshold, since most practical computer-use tasks involve dozens of back-and-forth interactions before completion.
The desktop benchmark noted that roughly 40% of tasks involve batch command-line operations, not just GUI clicks. The model handles both.
What this means
The "computer use" capability — AI that can operate software by reading a screen, without needing APIs — has been developing as its own frontier since late 2024. Claude Computer Use, GPT-4o's vision integrations, and various agent frameworks have been chipping at the problem. What makes Qwen-UI-Agent noteworthy isn't just the benchmark wins; it's that the weights are free and run locally.
Any developer with a GPU can now run an agent that outperforms GPT-5.6 Sol on mobile device automation. That changes the distribution dynamics significantly. Until now, computer-use capability at this performance level required API access to closed frontier models. Open weights mean it can be embedded in local tools, deployed in enterprise environments with strict data-residency requirements, and iterated on without per-token costs.
The 2B parameter variant is particularly significant on that front — small enough to run on consumer hardware, which raises the question of what happens when every laptop can host a model that autonomously navigates its own interface.
The bigger race
Qwen-UI-Agent arrives as Alibaba's Qwen family has been making steady moves up the benchmark leaderboards. An earlier August release of Qwen3.8-Max already pushed past GPT-5.6 Sol on text benchmarks. The GUI variant adds a different capability plane entirely: not what the model knows, but what it can physically do on a computer.
The competitive implication for OpenAI, Anthropic, and Google is a familiar one: the open-source community keeps closing the gap, and in some specific capability niches, it's now ahead. For anyone building automation pipelines, enterprise agents, or accessibility tools that need to control software visually, the calculus just shifted.
The bigger question is whether "screen reading" is the right frame at all. What Qwen-UI-Agent is really demonstrating is that a sufficiently capable agent doesn't need software to be designed for it. It can operate whatever humans can operate. That's a different kind of general-purpose capability — and it's now open-weight.
Sources
Alibaba Launches Qwen-UI-Agent, Surpassing GPT-5.6 and Claude 4.8: https://digitalphablet.com/ai/alibaba-launches-qwen-ui-agent-surpassing-gpt-5-6-and-claude-4-8/
Alibaba's screen-reading AI scores 92% on real phones — and runs on your hardware: https://www.martincid.com/technology-sv/alibaba-qwen-ui-agent-screen-control/
Qwen-UI-Agent Technical Report (arXiv): https://arxiv.org/html/2607.28227v1