All 116 AndroidWorld tasks

Sorted A–Z. Open a task to watch its recording and step through what the agent did. Filter to the two tasks Google’s original grader reads differently, or to the 5 where Lusor was slower than Mobilerun’s published time.

Search tasks

All 116 AndroidWorld tasks with both graders’ results and times
Task ↑	App	Mobilerun’s grader	Google’s grader	Lusor	Mobilerun	Faster
AudioRecorderRecordAudio	Audio Recorder	Pass	Pass	27.2 s	147.0 s	5.4× faster	Audio Recorder27.2 s vs 147.0 sMobilerun’s grader: Pass · Google’s: Pass
AudioRecorderRecordAudioWithFileName	Audio Recorder	Pass	Fail, read differently ¹	42.3 s	250.9 s	5.9× faster	Audio Recorder42.3 s vs 250.9 sMobilerun’s grader: Pass · Google’s: Fail, read differently ¹
BrowserDraw	Browser	Pass	Pass	90.2 s	388.0 s	4.3× faster	Browser90.2 s vs 388.0 sMobilerun’s grader: Pass · Google’s: Pass
BrowserMaze	Browser	Pass	Pass	51.2 s	350.5 s	6.8× faster	Browser51.2 s vs 350.5 sMobilerun’s grader: Pass · Google’s: Pass
BrowserMultiply ⁶	Browser	Pass	Pass	195.2 s	362.9 s	1.9× faster	Browser195.2 s vs 362.9 sMobilerun’s grader: Pass · Google’s: Pass
CameraTakePhoto	Camera	Pass	Pass	15.2 s	199.3 s	13.1× faster	Camera15.2 s vs 199.3 sMobilerun’s grader: Pass · Google’s: Pass
CameraTakeVideo	Camera	Pass	Pass	84.2 s	192.9 s	2.3× faster	Camera84.2 s vs 192.9 sMobilerun’s grader: Pass · Google’s: Pass
ClockStopWatchPausedVerify	Clock	Pass	Pass	21.2 s	216.7 s	10.2× faster	Clock21.2 s vs 216.7 sMobilerun’s grader: Pass · Google’s: Pass
ClockStopWatchRunning	Clock	Pass	Pass	18.2 s	122.1 s	6.7× faster	Clock18.2 s vs 122.1 sMobilerun’s grader: Pass · Google’s: Pass
ClockTimerEntry	Clock	Pass	Pass	21.1 s	117.0 s	5.5× faster	Clock21.1 s vs 117.0 sMobilerun’s grader: Pass · Google’s: Pass
ContactsAddContact	Contacts	Pass	Pass	39.2 s	409.4 s	10.4× faster	Contacts39.2 s vs 409.4 sMobilerun’s grader: Pass · Google’s: Pass
ContactsNewContactDraft	Contacts	Pass	Pass	48.2 s	215.0 s	4.5× faster	Contacts48.2 s vs 215.0 sMobilerun’s grader: Pass · Google’s: Pass
ExpenseAddMultiple	Pro Expense	Pass	Pass	72.2 s	471.1 s	6.5× faster	Pro Expense72.2 s vs 471.1 sMobilerun’s grader: Pass · Google’s: Pass
ExpenseAddMultipleFromGallery	Pro Expense	Pass	Pass	96.2 s	636.8 s	6.6× faster	Pro Expense96.2 s vs 636.8 sMobilerun’s grader: Pass · Google’s: Pass
ExpenseAddMultipleFromMarkor	Pro Expense	Pass	Fail, read differently ¹	126.2 s	318.7 s	2.5× faster	Pro Expense126.2 s vs 318.7 sMobilerun’s grader: Pass · Google’s: Fail, read differently ¹
ExpenseAddSingle	Pro Expense	Pass	Pass	42.2 s	311.7 s	7.4× faster	Pro Expense42.2 s vs 311.7 sMobilerun’s grader: Pass · Google’s: Pass
ExpenseDeleteDuplicates	Pro Expense	Pass	Pass	42.2 s	418.0 s	9.9× faster	Pro Expense42.2 s vs 418.0 sMobilerun’s grader: Pass · Google’s: Pass
ExpenseDeleteDuplicates2	Pro Expense	Pass	Pass	93.3 s	382.5 s	4.1× faster	Pro Expense93.3 s vs 382.5 sMobilerun’s grader: Pass · Google’s: Pass
ExpenseDeleteMultiple	Pro Expense	Pass	Pass	30.2 s	271.1 s	9.0× faster	Pro Expense30.2 s vs 271.1 sMobilerun’s grader: Pass · Google’s: Pass
ExpenseDeleteMultiple2	Pro Expense	Pass	Pass	180.3 s	366.7 s	2.0× faster	Pro Expense180.3 s vs 366.7 sMobilerun’s grader: Pass · Google’s: Pass
ExpenseDeleteSingle	Pro Expense	Pass	Pass	45.2 s	155.6 s	3.4× faster	Pro Expense45.2 s vs 155.6 sMobilerun’s grader: Pass · Google’s: Pass
FilesDeleteFile	Files	Pass	Pass	24.2 s	204.4 s	8.4× faster	Files24.2 s vs 204.4 sMobilerun’s grader: Pass · Google’s: Pass
FilesMoveFile	Files	Pass	Pass	51.2 s	380.8 s	7.4× faster	Files51.2 s vs 380.8 sMobilerun’s grader: Pass · Google’s: Pass
MarkorAddNoteHeader	Markor	Pass	Pass	69.2 s	224.2 s	3.2× faster	Markor69.2 s vs 224.2 sMobilerun’s grader: Pass · Google’s: Pass
MarkorChangeNoteContent	Markor	Pass	Pass	54.2 s	217.5 s	4.0× faster	Markor54.2 s vs 217.5 sMobilerun’s grader: Pass · Google’s: Pass
MarkorCreateFolder	Markor	Pass	Pass	18.2 s	157.9 s	8.7× faster	Markor18.2 s vs 157.9 sMobilerun’s grader: Pass · Google’s: Pass
MarkorCreateNote	Markor	Pass	Pass	30.2 s	204.5 s	6.8× faster	Markor30.2 s vs 204.5 sMobilerun’s grader: Pass · Google’s: Pass
MarkorCreateNoteAndSms	Markor	Pass	Pass	57.2 s	351.5 s	6.1× faster	Markor57.2 s vs 351.5 sMobilerun’s grader: Pass · Google’s: Pass
MarkorCreateNoteFromClipboard	Markor	Pass	Pass	36.2 s	258.9 s	7.2× faster	Markor36.2 s vs 258.9 sMobilerun’s grader: Pass · Google’s: Pass
MarkorDeleteAllNotes	Markor	Pass	Pass	48.2 s	198.6 s	4.1× faster	Markor48.2 s vs 198.6 sMobilerun’s grader: Pass · Google’s: Pass
MarkorDeleteNewestNote	Markor	Pass	Pass	21.2 s	194.0 s	9.2× faster	Markor21.2 s vs 194.0 sMobilerun’s grader: Pass · Google’s: Pass
MarkorDeleteNote	Markor	Pass	Pass	21.2 s	140.9 s	6.6× faster	Markor21.2 s vs 140.9 sMobilerun’s grader: Pass · Google’s: Pass
MarkorEditNote	Markor	Pass	Pass	27.3 s	143.1 s	5.2× faster	Markor27.3 s vs 143.1 sMobilerun’s grader: Pass · Google’s: Pass
MarkorMergeNotes	Markor	Pass	Pass	72.2 s	318.0 s	4.4× faster	Markor72.2 s vs 318.0 sMobilerun’s grader: Pass · Google’s: Pass
MarkorMoveNote	Markor	Pass	Pass	27.2 s	275.9 s	10.1× faster	Markor27.2 s vs 275.9 sMobilerun’s grader: Pass · Google’s: Pass
MarkorTranscribeReceipt	Markor	Pass	Pass	45.2 s	233.7 s	5.2× faster	Markor45.2 s vs 233.7 sMobilerun’s grader: Pass · Google’s: Pass
MarkorTranscribeVideo	Markor	Pass	Pass	354.2 s	489.2 s	1.4× faster	Markor354.2 s vs 489.2 sMobilerun’s grader: Pass · Google’s: Pass
NotesIsTodo	Joplin	Pass	Pass	15.2 s	85.9 s	5.7× faster	Joplin15.2 s vs 85.9 sMobilerun’s grader: Pass · Google’s: Pass
NotesMeetingAttendeeCount	Joplin	Pass	Pass	24.2 s	147.0 s	6.1× faster	Joplin24.2 s vs 147.0 sMobilerun’s grader: Pass · Google’s: Pass
NotesRecipeIngredientCount	Joplin	Pass	Pass	21.2 s	137.5 s	6.5× faster	Joplin21.2 s vs 137.5 sMobilerun’s grader: Pass · Google’s: Pass
NotesTodoItemCount	Joplin	Pass	Pass	30.2 s	156.7 s	5.2× faster	Joplin30.2 s vs 156.7 sMobilerun’s grader: Pass · Google’s: Pass
OpenAppTaskEval	Launcher	Pass	Pass	24.2 s	178.4 s	7.4× faster	Launcher24.2 s vs 178.4 sMobilerun’s grader: Pass · Google’s: Pass
OsmAndFavorite	OsmAnd	Pass	Pass	117.2 s	166.8 s	1.4× faster	OsmAnd117.2 s vs 166.8 sMobilerun’s grader: Pass · Google’s: Pass
OsmAndMarker	OsmAnd	Pass	Pass	36.2 s	212.6 s	5.9× faster	OsmAnd36.2 s vs 212.6 sMobilerun’s grader: Pass · Google’s: Pass
OsmAndTrack	OsmAnd	Pass	Pass	153.2 s	887.7 s	5.8× faster	OsmAnd153.2 s vs 887.7 sMobilerun’s grader: Pass · Google’s: Pass
RecipeAddMultipleRecipes	Broccoli	Pass	Pass	186.2 s	429.7 s	2.3× faster	Broccoli186.2 s vs 429.7 sMobilerun’s grader: Pass · Google’s: Pass
RecipeAddMultipleRecipesFromImage	Broccoli	Pass	Pass	222.3 s	443.9 s	2.0× faster	Broccoli222.3 s vs 443.9 sMobilerun’s grader: Pass · Google’s: Pass
RecipeAddMultipleRecipesFromMarkor	Broccoli	Pass	Pass	198.4 s	482.7 s	2.4× faster	Broccoli198.4 s vs 482.7 sMobilerun’s grader: Pass · Google’s: Pass
RecipeAddMultipleRecipesFromMarkor2	Broccoli	Pass	Pass	249.2 s	517.5 s	2.1× faster	Broccoli249.2 s vs 517.5 sMobilerun’s grader: Pass · Google’s: Pass
RecipeAddSingleRecipe	Broccoli	Pass	Pass	93.2 s	244.8 s	2.6× faster	Broccoli93.2 s vs 244.8 sMobilerun’s grader: Pass · Google’s: Pass
RecipeDeleteDuplicateRecipes	Broccoli	Pass	Pass	51.2 s	250.3 s	4.9× faster	Broccoli51.2 s vs 250.3 sMobilerun’s grader: Pass · Google’s: Pass
RecipeDeleteDuplicateRecipes2	Broccoli	Pass	Pass	96.2 s	327.1 s	3.4× faster	Broccoli96.2 s vs 327.1 sMobilerun’s grader: Pass · Google’s: Pass
RecipeDeleteDuplicateRecipes3	Broccoli	Pass	Pass	240.2 s	620.1 s	2.6× faster	Broccoli240.2 s vs 620.1 sMobilerun’s grader: Pass · Google’s: Pass
RecipeDeleteMultipleRecipes	Broccoli	Pass	Pass	39.2 s	234.1 s	6.0× faster	Broccoli39.2 s vs 234.1 sMobilerun’s grader: Pass · Google’s: Pass
RecipeDeleteMultipleRecipesWithConstraint	Broccoli	Pass	Pass	282.3 s	539.8 s	1.9× faster	Broccoli282.3 s vs 539.8 sMobilerun’s grader: Pass · Google’s: Pass
RecipeDeleteMultipleRecipesWithNoise	Broccoli	Pass	Pass	108.2 s	474.7 s	4.4× faster	Broccoli108.2 s vs 474.7 sMobilerun’s grader: Pass · Google’s: Pass
RecipeDeleteSingleRecipe	Broccoli	Pass	Pass	18.2 s	171.5 s	9.4× faster	Broccoli18.2 s vs 171.5 sMobilerun’s grader: Pass · Google’s: Pass
RecipeDeleteSingleWithRecipeWithNoise	Broccoli	Pass	Pass	48.2 s	301.6 s	6.3× faster	Broccoli48.2 s vs 301.6 sMobilerun’s grader: Pass · Google’s: Pass
RetroCreatePlaylist	Retro Music	Pass	Pass	57.3 s	663.2 s	11.6× faster	Retro Music57.3 s vs 663.2 sMobilerun’s grader: Pass · Google’s: Pass
RetroPlayingQueue	Retro Music	Pass	Pass	51.2 s	768.0 s	15.0× faster	Retro Music51.2 s vs 768.0 sMobilerun’s grader: Pass · Google’s: Pass
RetroPlaylistDuration	Retro Music	Pass	Pass	240.2 s	494.9 s	2.1× faster	Retro Music240.2 s vs 494.9 sMobilerun’s grader: Pass · Google’s: Pass
RetroSavePlaylist	Retro Music	Pass	Pass	84.4 s	603.7 s	7.2× faster	Retro Music84.4 s vs 603.7 sMobilerun’s grader: Pass · Google’s: Pass
SaveCopyOfReceiptTaskEval	Gallery	Pass	Pass	51.2 s	266.3 s	5.2× faster	Gallery51.2 s vs 266.3 sMobilerun’s grader: Pass · Google’s: Pass
SimpleCalendarAddOneEvent	Simple Calendar	Pass	Pass	69.2 s	420.8 s	6.1× faster	Simple Calendar69.2 s vs 420.8 sMobilerun’s grader: Pass · Google’s: Pass
SimpleCalendarAddOneEventInTwoWeeks	Simple Calendar	Pass	Pass	69.2 s	394.2 s	5.7× faster	Simple Calendar69.2 s vs 394.2 sMobilerun’s grader: Pass · Google’s: Pass
SimpleCalendarAddOneEventRelativeDay	Simple Calendar	Pass	Pass	78.2 s	435.5 s	5.6× faster	Simple Calendar78.2 s vs 435.5 sMobilerun’s grader: Pass · Google’s: Pass
SimpleCalendarAddOneEventTomorrow	Simple Calendar	Pass	Pass	72.2 s	328.1 s	4.5× faster	Simple Calendar72.2 s vs 328.1 sMobilerun’s grader: Pass · Google’s: Pass
SimpleCalendarAddRepeatingEvent	Simple Calendar	Pass	Pass	78.2 s	478.9 s	6.1× faster	Simple Calendar78.2 s vs 478.9 sMobilerun’s grader: Pass · Google’s: Pass
SimpleCalendarAnyEventsOnDate	Simple Calendar	Pass	Pass	21.2 s	106.3 s	5.0× faster	Simple Calendar21.2 s vs 106.3 sMobilerun’s grader: Pass · Google’s: Pass
SimpleCalendarDeleteEvents	Simple Calendar	Pass	Pass	42.2 s	261.4 s	6.2× faster	Simple Calendar42.2 s vs 261.4 sMobilerun’s grader: Pass · Google’s: Pass
SimpleCalendarDeleteEventsOnRelativeDay	Simple Calendar	Pass	Pass	33.2 s	215.6 s	6.5× faster	Simple Calendar33.2 s vs 215.6 sMobilerun’s grader: Pass · Google’s: Pass
SimpleCalendarDeleteOneEvent	Simple Calendar	Pass	Pass	36.2 s	266.2 s	7.4× faster	Simple Calendar36.2 s vs 266.2 sMobilerun’s grader: Pass · Google’s: Pass
SimpleCalendarEventOnDateAtTime	Simple Calendar	Pass	Pass	33.2 s	122.3 s	3.7× faster	Simple Calendar33.2 s vs 122.3 sMobilerun’s grader: Pass · Google’s: Pass
SimpleCalendarEventsInNextWeek	Simple Calendar	Pass	Pass	93.1 s	150.2 s	1.6× faster	Simple Calendar93.1 s vs 150.2 sMobilerun’s grader: Pass · Google’s: Pass
SimpleCalendarEventsInTimeRange	Simple Calendar	Pass	Pass	21.2 s	110.1 s	5.2× faster	Simple Calendar21.2 s vs 110.1 sMobilerun’s grader: Pass · Google’s: Pass
SimpleCalendarEventsOnDate	Simple Calendar	Pass	Pass	24.2 s	93.2 s	3.9× faster	Simple Calendar24.2 s vs 93.2 sMobilerun’s grader: Pass · Google’s: Pass
SimpleCalendarFirstEventAfterStartTime	Simple Calendar	Pass	Pass	42.2 s	112.8 s	2.7× faster	Simple Calendar42.2 s vs 112.8 sMobilerun’s grader: Pass · Google’s: Pass
SimpleCalendarLocationOfEvent	Simple Calendar	Pass	Pass	18.2 s	113.1 s	6.2× faster	Simple Calendar18.2 s vs 113.1 sMobilerun’s grader: Pass · Google’s: Pass
SimpleCalendarNextEvent	Simple Calendar	Pass	Pass	27.2 s	116.5 s	4.3× faster	Simple Calendar27.2 s vs 116.5 sMobilerun’s grader: Pass · Google’s: Pass
SimpleCalendarNextMeetingWithPerson	Simple Calendar	Pass	Pass	81.2 s	170.3 s	2.1× faster	Simple Calendar81.2 s vs 170.3 sMobilerun’s grader: Pass · Google’s: Pass
SimpleDrawProCreateDrawing	Simple Draw Pro	Pass	Pass	42.2 s	275.3 s	6.5× faster	Simple Draw Pro42.2 s vs 275.3 sMobilerun’s grader: Pass · Google’s: Pass
SimpleSmsReply	Simple SMS	Pass	Pass	21.2 s	159.6 s	7.5× faster	Simple SMS21.2 s vs 159.6 sMobilerun’s grader: Pass · Google’s: Pass
SimpleSmsReplyMostRecent	Simple SMS	Pass	Pass	27.2 s	178.5 s	6.6× faster	Simple SMS27.2 s vs 178.5 sMobilerun’s grader: Pass · Google’s: Pass
SimpleSmsResend	Simple SMS	Pass	Pass	24.2 s	147.4 s	6.1× faster	Simple SMS24.2 s vs 147.4 sMobilerun’s grader: Pass · Google’s: Pass
SimpleSmsSend	Simple SMS	Pass	Pass	21.2 s	172.9 s	8.2× faster	Simple SMS21.2 s vs 172.9 sMobilerun’s grader: Pass · Google’s: Pass
SimpleSmsSendClipboardContent	Simple SMS	Pass	Pass	24.7 s	173.3 s	7.0× faster	Simple SMS24.7 s vs 173.3 sMobilerun’s grader: Pass · Google’s: Pass
SimpleSmsSendReceivedAddress	Simple SMS	Pass	Pass	54.2 s	198.1 s	3.7× faster	Simple SMS54.2 s vs 198.1 sMobilerun’s grader: Pass · Google’s: Pass
SportsTrackerActivitiesCountForWeek	OpenTracks	Pass	Pass	351.3 s	279.7 s	1.3× slower	OpenTracks351.3 s vs 279.7 sMobilerun’s grader: Pass · Google’s: Pass
SportsTrackerActivitiesOnDate	OpenTracks	Pass	Pass	72.2 s	399.9 s	5.5× faster	OpenTracks72.2 s vs 399.9 sMobilerun’s grader: Pass · Google’s: Pass
SportsTrackerActivityDuration	OpenTracks	Pass	Pass	30.2 s	246.9 s	8.2× faster	OpenTracks30.2 s vs 246.9 sMobilerun’s grader: Pass · Google’s: Pass
SportsTrackerLongestDistanceActivity	OpenTracks	Pass	Pass	153.2 s	239.2 s	1.6× faster	OpenTracks153.2 s vs 239.2 sMobilerun’s grader: Pass · Google’s: Pass
SportsTrackerTotalDistanceForCategoryOverInterval	OpenTracks	Pass	Pass	144.2 s	364.8 s	2.5× faster	OpenTracks144.2 s vs 364.8 sMobilerun’s grader: Pass · Google’s: Pass
SportsTrackerTotalDurationForCategoryThisWeek	OpenTracks	Pass	Pass	69.4 s	217.0 s	3.1× faster	OpenTracks69.4 s vs 217.0 sMobilerun’s grader: Pass · Google’s: Pass
SystemBluetoothTurnOff	Settings	Pass	Pass	27.2 s	125.1 s	4.6× faster	Settings27.2 s vs 125.1 sMobilerun’s grader: Pass · Google’s: Pass
SystemBluetoothTurnOffVerify	Settings	Pass	Pass	15.2 s	136.9 s	9.0× faster	Settings15.2 s vs 136.9 sMobilerun’s grader: Pass · Google’s: Pass
SystemBluetoothTurnOn	Settings	Pass	Pass	18.2 s	121.9 s	6.7× faster	Settings18.2 s vs 121.9 sMobilerun’s grader: Pass · Google’s: Pass
SystemBluetoothTurnOnVerify	Settings	Pass	Pass	24.2 s	108.0 s	4.5× faster	Settings24.2 s vs 108.0 sMobilerun’s grader: Pass · Google’s: Pass
SystemBrightnessMax	Settings	Pass	Pass	18.2 s	132.0 s	7.3× faster	Settings18.2 s vs 132.0 sMobilerun’s grader: Pass · Google’s: Pass
SystemBrightnessMaxVerify	Settings	Pass	Pass	27.2 s	125.6 s	4.6× faster	Settings27.2 s vs 125.6 sMobilerun’s grader: Pass · Google’s: Pass
SystemBrightnessMin	Settings	Pass	Pass	33.2 s	129.7 s	3.9× faster	Settings33.2 s vs 129.7 sMobilerun’s grader: Pass · Google’s: Pass
SystemBrightnessMinVerify	Settings	Pass	Pass	18.2 s	117.4 s	6.5× faster	Settings18.2 s vs 117.4 sMobilerun’s grader: Pass · Google’s: Pass
SystemCopyToClipboard	Settings	Pass	Pass	54.8 s	221.3 s	4.0× faster	Settings54.8 s vs 221.3 sMobilerun’s grader: Pass · Google’s: Pass
SystemWifiTurnOff	Settings	Pass	Pass	24.2 s	155.7 s	6.4× faster	Settings24.2 s vs 155.7 sMobilerun’s grader: Pass · Google’s: Pass
SystemWifiTurnOffVerify	Settings	Pass	Pass	30.2 s	93.7 s	3.1× faster	Settings30.2 s vs 93.7 sMobilerun’s grader: Pass · Google’s: Pass
SystemWifiTurnOn	Settings	Pass	Pass	15.2 s	110.6 s	7.3× faster	Settings15.2 s vs 110.6 sMobilerun’s grader: Pass · Google’s: Pass
SystemWifiTurnOnVerify	Settings	Pass	Pass	21.2 s	97.1 s	4.6× faster	Settings21.2 s vs 97.1 sMobilerun’s grader: Pass · Google’s: Pass
TasksCompletedTasksForDate	Tasks	Pass	Pass	117.3 s	244.0 s	2.1× faster	Tasks117.3 s vs 244.0 sMobilerun’s grader: Pass · Google’s: Pass
TasksDueNextWeek	Tasks	Pass	Pass	180.2 s	154.4 s	1.2× slower	Tasks180.2 s vs 154.4 sMobilerun’s grader: Pass · Google’s: Pass
TasksDueOnDate	Tasks	Pass	Pass	171.3 s	139.4 s	1.2× slower	Tasks171.3 s vs 139.4 sMobilerun’s grader: Pass · Google’s: Pass
TasksHighPriorityTasks	Tasks	Pass	Pass	273.3 s	193.5 s	1.4× slower	Tasks273.3 s vs 193.5 sMobilerun’s grader: Pass · Google’s: Pass
TasksHighPriorityTasksDueOnDate	Tasks	Pass	Pass	81.2 s	133.7 s	1.6× faster	Tasks81.2 s vs 133.7 sMobilerun’s grader: Pass · Google’s: Pass
TasksIncompleteTasksOnDate	Tasks	Pass	Pass	219.2 s	126.2 s	1.7× slower	Tasks219.2 s vs 126.2 sMobilerun’s grader: Pass · Google’s: Pass
TurnOffWifiAndTurnOnBluetooth	Settings	Pass	Pass	54.2 s	164.8 s	3.0× faster	Settings54.2 s vs 164.8 sMobilerun’s grader: Pass · Google’s: Pass
TurnOnWifiAndOpenApp	Settings	Pass	Pass	51.2 s	134.1 s	2.6× faster	Settings51.2 s vs 134.1 sMobilerun’s grader: Pass · Google’s: Pass
VlcCreatePlaylist	VLC	Pass	Pass	75.2 s	331.1 s	4.4× faster	VLC75.2 s vs 331.1 sMobilerun’s grader: Pass · Google’s: Pass
VlcCreateTwoPlaylists	VLC	Pass	Pass	114.2 s	420.5 s	3.7× faster	VLC114.2 s vs 420.5 sMobilerun’s grader: Pass · Google’s: Pass

Showing 8 of 1165 of 116

Data schema (docs) →

Download data

Download all (JSON / CSV)

¹ Google’s original grader failed two tasks Lusor completed: it looks for the recording name typed in full (….m4a.m4a), and for the expense notes without the “Reimbursable.” marker. Mobilerun’s fork of the grader accepts Lusor’s reading on both.

⁶ In BrowserMultiply the agent read the page’s code to find the fifth number. Its task page explains how.

Data schema: one row per task, fields task, app, goal, mobilerun_grader_pass, google_grader_pass, lusor_time_s, mobilerun_published_time_s, speed_ratio, lusor_cost_usd, lusor_tool_calls, mobilerun_published_actions, mobilerun_est_cost_usd, disclosure. Times are seconds; costs are US dollars; Mobilerun’s cost is our estimate; speed_ratio is Lusor’s time ÷ Mobilerun’s published time.