Lusor + DeepSeek: 116/116 on AndroidWorld for $2.95

A phone agent needs more than a model that can reason about a screenshot. It needs useful tools, a clear view of the screen, and a way to check what happened after it acts. We built Lusor around that whole loop. With open-weight DeepSeek V4.1 Flash making the task decisions at low reasoning effort, Lusor passed all 116 AndroidWorld tasks on Mobilerun's grader for $2.95 in model costs, including its helper checks. Google's unmodified grader passed 114/116; on the other two, Lusor did the task but read the instruction differently from that grader (explained below).

The question this run answers: can a production phone agent complete these tasks reliably and cheaply without a flagship closed model choosing its actions? On this benchmark, it did. On the median task it was also 5.2× faster than the time Mobilerun publishes for it.

Why DeepSeek

DeepSeek V4.1 Flash is an open-weight model: its weights are published, so anyone can run it, and it is available off the shelf today. We ran it at low reasoning effort, the way Lusor runs it in production. That matters for a phone agent. People repeat phone tasks many times, and a task can take many model calls, so the cost and the wait of every call add up.

We judge the whole system by what it gets done: tasks completed, time taken and money spent. A model's place on a general intelligence leaderboard doesn't answer that. Other entries on AndroidWorld choose their actions with closed models: Mobilerun publishes its setup on GPT-6 Astra, and agent-qa on gpt-6-sol. This was not a controlled test of models inside one harness, so it doesn't show that DeepSeek is better than those models. It shows that this complete system reached a perfect score on Mobilerun's grader with an open-weight model making the decisions.

What Lusor's agent does around the model

DeepSeek is the decision-maker, but it does not work alone. Lusor gives it tools for reading and navigating apps, plus small, focused checks around particular actions and saved values. The reported cost includes those helper calls. This is a result for that complete system, not a claim about DeepSeek running without a harness.

Each step runs the same loop: read the screen, choose an action, act, then inspect the result.

• DeepSeek reads the goal and chooses the next action from Lusor's tools.
• Lusor gives it the screen as structured information and carries out phone actions, such as tapping, typing and scrolling, instead of asking the model to drive the phone with free-form text.
• List-reading tools gather what is on screen across a long list, page by page, so the agent can work across multiple screens of information.
• Focused checks compare certain actions and typed values against the request and what the agent has read. The model doesn't simply get to declare that it succeeded.
• Separately, the benchmark's own grader decides whether each task was done, whatever the agent says at the end.

What it looked like on three tasks

NotesIsTodo asks a question about a note and expects an answer, not an edit. While answering, the agent tried to tap the checkbox inside the Travel Itinerary row. Lusor refused the checkbox tap because it targeted a control inside the row without an explicit toggle instruction. It pointed the agent to the row itself, which the agent opened without changing the checkbox. The task passed in 15.2 s, against Mobilerun's published 85.9 s.

BrowserDraw asks the agent to draw on a web page. It worked from what was on screen and passed on both graders in 90.2 s, against Mobilerun's published 388.0 s. (One other browser task did read the page's code; see below.)

Not every task was faster. TasksHighPriorityTasks passed, but took 273.3 s against Mobilerun's published 193.5 s. Across the benchmark, Lusor was slower than Mobilerun's published time on 5 of the 116 tasks.

Why AndroidWorld

AndroidWorld is a public benchmark of 116 tasks in real Android apps, and its own code decides whether each task was done. Anyone can read its tasks and its graders, and each grader checks the phone's state, or the agent's answer, against the task. It never takes the agent's word that it finished.

What we ran

• All 116 AndroidWorld tasks, with random seed 42, on Android emulators.
• Lusor's production agent, the build in production that day, with the same general and Android guidance it uses in production: DeepSeek V4.1 Flash at low reasoning effort makes every decision, and Lusor's own small check models verify lists and saves. Their cost is included below. The agent also had access to the app-skill catalog shipped with that build.
• The agent runs on pi, an open-source (MIT) agent toolkit. Lusor adds the mobile layer: device tools, screen reading, the agent guidance and the checks that decide when a task is done.
• One try per task in the run reported here, and no task was rerun.
• Every task started from a phone state we checked against a known clean copy, and every task was recorded on video: 116 of 116.
• The tasks and grader from Mobilerun's fork of AndroidWorld, graded live. Afterwards we graded the same runs again with Google's original, unmodified grader.
• Run on 9 November 2026.

Results

Three of the 116 tasks are worded differently in Mobilerun's fork than in Google's original. Leave those three out and the scores are 113/113 on Mobilerun's grader and 111/113 on Google's.

The two tasks Google's grader read differently

Google's original grader failed two tasks that Lusor completed. In both, the grader reads the instruction more literally than Lusor did. Asked to save a recording named 2023_06_24_training.m4a, Lusor saved exactly that file, while the grader looks for the name typed in full, which the app turns into …m4a.m4a. Asked to log the transactions a file marks as reimbursable, Lusor copied each note as written, including the “Reimbursable.” marker, which the grader expects removed. Mobilerun's fork of the grader accepts Lusor's reading on both. Another agent that passed both tasks typed the full name and removed the marker.

One task used the page's code

BrowserMultiply asks the agent to click a button on a web page five times, remember the number each click shows, and enter their product. The agent saw four of the numbers on screen: 10, 4, 7 and 5. It tried to download the page, which was refused, and to view the page source, which came up blank. It then opened the page's HTML file in Markor, read the code that generates the numbers, and ran it to get the fifth: 3. It answered 4200, and both graders passed it.

The answer depended on reading the page's code. We count the task as passed, as both graders do, and disclose it here. We searched the logs of all 116 tasks: this is the only one in which the agent read page code. The other two browser tasks, BrowserDraw and BrowserMaze, were solved from the screen.

What the speed and cost mean

Speed: for each task we divided Lusor's time by the time Mobilerun publishes for it. The median of those 116 ratios is 0.19: on the median task Lusor was 5.2× faster. That is a median of per-task comparisons, not one median time divided by the other. Lusor was slower on 5 of the 116 tasks.

Cost: $2.95 is what the model calls cost for all 116 tasks. $2.75 of it was DeepSeek's decisions and $0.20 the helper models that read lists and run checks. It is not the price of a Lusor plan, and it leaves out the phones and servers.

Mobilerun does not publish its token use, so we estimated its cost: the number of actions it publishes for each task, times $0.0766 per call, the cost we measured running GPT-6 Astra at low effort in our own harness. That comes to about $108.70 for the same 116 tasks, roughly 37 times ours. It is our estimate, not Mobilerun's bill, and we only make this comparison with Mobilerun.

What it means in practice: this run did not need a flagship closed model choosing the actions to reach this score, at this measured cost.

What this doesn't show

• One seed on emulators. Other seeds, real phones, or apps outside AndroidWorld may give different results.
• Mobilerun's times come from its published runs, not from a run beside ours, and its cost is our estimate.
• It isn't a controlled comparison of models: other entries used different models in different harnesses.
• We ran this ourselves. The graders are public code, but this is not an independent review.
• Both products change every week.

What's next

Benchmark evidence comes first: task results, recordings, steps and methodology. We're preparing a cleaned-up open-source version of Lusor's harness, lusor-harness. It is coming; we are not announcing a release date yet.

Try it

Read how we ran it, then give Lusor one of your own phone tasks: it is free to start, with no credit card. Or call the public API from the developer section.