App screenshot classifier for bug reports

The brief

A mobile app team receives hundreds of bug reports with screenshots, and someone has to read each one to decide which team owns it. Train an image model that recognises the kind of screen (login, list, form, settings, error and so on) so reports can be routed automatically. Good looks like: accuracy per screen type, a confusion matrix, and a small demo that tags an uploaded screenshot.

What you will build

1. Choose screen categories and build a labelled set
2. Fine-tune a pre-trained image model
3. Evaluate per category and study the mistakes
4. Add a confidence threshold that sends unsure cases to a person
5. Build a small upload-and-tag demo

Free, open data to build it with

• Screen2Words (Rico screenshots with summaries) ↗ Google Research (GitHub)

Check each dataset's licence before you publish your work. Your mentor gives your team its own version of the brief (data, scope or region), so every team's project is its own.