MyPhoneBench

MyPhoneBench

**Do Phone-Use Agents Respect Your Privacy?**

MyPhoneBench studies whether phone-use agents respect privacy while completing benign mobile tasks. It makes privacy behavior measurable by pairing an explicit privacy contract with fully observable mock Android apps and deterministic auditing. Rather than asking only whether an agent finishes a task, MyPhoneBench asks whether it requests only the data it needs, avoids unnecessary re-disclosure, and uses saved preferences correctly across sessions.

Real-app examples of how a phone-use agent can cross privacy boundaries during a benign mobile task.

Concretely, MyPhoneBench provides:

• iMy, a minimal privacy contract for permissioned access, minimal disclosure, and user-controlled memory;
• 10 instrumented mock Android apps with auditable SQLite backends and field-level logs;
• 300 evaluation tasks spanning independent tasks and paired later-session preference reuse;
• and a public trajectory release for reproducible post-hoc analysis.

MyPhoneBench is built on the [AndroidWorld](https://github.com/google-research/android_world) infrastructure. Any phone-use agent that already runs on AndroidWorld can be evaluated on MyPhoneBench with minimal integration effort.

Resources

• Paper: arXiv:2604.00986
• Homepage & Leaderboard: freedomintelligence.github.io/MyPhoneBench
• Blog: Notion
• Code and mock apps: this repository
• Trajectory release: [MyPhoneBench-Trajectories](https://modelscope.cn/datasets/tangzhy