Toby Perrett

Senior Research Engineer · Autodesk

Hi! I'm a Senior Research Engineer at Autodesk. Prior to that, I did my postdoc at the University of Bristol, working with Professor Dima Damen, and co-advising Chiara Plizzari and Saptarshi Sinha. I'm a founding member of the team collecting the EPIC Kitchens datasets, and also worked on the Visual AI project led by Professor Andrew Zisserman.

My research interests include datasets, benchmarks and methods for Egocentric video, Captioning, 3D understanding and CAD. I'm also interested in improving models when labelled data is scarce or imbalanced.

News

Apr 2026 neuralCAD-Edit: An Expert Benchmark for Multimodal-Instructed 3D CAD Editing is released and on arxiv!
Jan 2026 It's Just Another Day: Unique Video Captioning by Discriminative Prompting has been accepted in IJCV!
Jun 2025 It's Just Another Day: Unique Video Captioning by Discriminative Prompting has received the EgoVis Distinguished Paper Award, announced at CVPR 2025.
Mar 2025 I've just joined Autodesk as a Senior Research Engineer!
Mar 2025 HD-EPIC: A Highly Detailed Egocentric Video Dataset has been accepted at CVPR 2025!
Feb 2025 HD-EPIC: A Highly Detailed Egocentric Video Dataset is released and on arxiv!
Dec 2024 Received the ACCV Best Paper Award for It's Just Another Day: Unique Video Captioning by Discriminative Prompting!
Nov 2024 Spatial Cognition from Egocentric Video: Out of Sight, Not Out of Mind has been accepted at 3DV 2025!
Oct 2024 Paper accepted at ACCV 2024 as an oral presentation (top 5%)! It's Just Another Day: Unique Video Captioning by Discriminative Prompting. Code, benchmarks and models available.
Jun 2024 I'll be serving as an Area Chair for NeurIPS 2024, on the Datasets and Benchmarks track.
May 2024 I was an Outstanding Reviewer at CVPR 2024!
Apr 2024 New paper on arXiv: Spatial Cognition from Egocentric Video: Out of Sight, Not Out of Mind. Video. PDF.
Sep 2023 I was an Outstanding Reviewer at ICCV 2023!
Sep 2023 I gave a talk at the JADE (UK Supercomputing Facility) 2023 event on egocentric vision and computational requirements.
Jul 2023 Paper accepted at ICCV 2023! What can a cook in Italy teach a mechanic in India? Action Recognition Generalisation Over Scenarios and Locations.
May 2023 I gave a talk at Samsung AI Centre Cambridge on scaling few-shot models to handle long-tail tasks.
Feb 2023 I gave a talk at the University of Exeter Computer Science Seminar Series on image/video datasets, model shortcuts, and recent solutions.
Feb 2023 Paper accepted at CVPR 2023! Use Your Head: Improving Long-Tail Video Recognition. Code, benchmarks and models available.
Jan 2023 I gave a talk at the Visual AI group at the University of Oxford on our latest long-tail video work.

Selected Publications

See all →

neuralCAD-Edit: An Expert Benchmark for Multimodal-Instructed 3D CAD Model Editing

Toby Perrett, Matthew Bourchard, William McCarthy

arXiv, 2026

Project
It's Just Another Day: Unique Video Captioning by Discriminative Prompting

Toby Perrett, Tengda Han, Dima Damen, Andrew Zisserman

IJCV, 2026 · ACCV 2024 Best Paper Award · EgoVis Distinguished Paper Award

Project
HD-EPIC: A Highly-Detailed Egocentric Video Dataset

Toby Perrett, Ahmad Darkhalil, Saptarshi Sinha, Omar Emara, Sam Pollard, Kranti Parida, Kaiting Liu, Prajwal Gatti, Siddhant Bansal, Kevin Flanagan, Jacob Chalk, Zhifan Zhu, Rhodri Guerrier, Fahd Abdelazim, Bin Zhu, Davide Moltisanti, Michael Wray, Hazel Doughty, Dima Damen

CVPR, 2025

Project
Spatial Cognition from Egocentric Video: Out of Sight, Not Out of Mind

Chiara Plizzari, Shubham Goel, Toby Perrett, Jacob Chalk, Angjoo Kanazawa, Dima Damen

3DV, 2025

Project
Use Your Head: Improving Long-Tail Video Recognition

Toby Perrett, Saptarshi Sinha, Tilo Burghardt, Majid Mirmehdi, Dima Damen

CVPR, 2023

Project
Rescaling Egocentric Vision: Collection, Pipeline and Challenges for EPIC-KITCHENS-100

Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Antonino Furnari, Evangelos Kazakos, Jian Ma, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, Michael Wray

IJCV, 2022

Project
Temporal-Relational CrossTransformers for Few-Shot Action Recognition

Toby Perrett, Alessandro Masullo, Tilo Burghardt, Majid Mirmehdi, Dima Damen

CVPR, 2021

Project
DDLSTM: Dual-Domain LSTM for Cross-Domain Action Recognition

Toby Perrett, Dima Damen

CVPR, 2019

Project
Scaling Egocentric Vision: The EPIC-KITCHENS Dataset

Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Sanja Fidler, Antonino Furnari, Evangelos Kazakos, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, Michael Wray

ECCV, 2018

Project

Full Bibliography

2026

neuralCAD-Edit: An Expert Benchmark for Multimodal-Instructed 3D CAD Model Editing

Toby Perrett, Matthew Bourchard, William McCarthy — arXiv, 2026

Project
It's Just Another Day: Unique Video Captioning by Discriminative Prompting

Toby Perrett, Tengda Han, Dima Damen, Andrew Zisserman — IJCV, 2026

Project

2025

HD-EPIC: A Highly-Detailed Egocentric Video Dataset

Toby Perrett, Ahmad Darkhalil, Saptarshi Sinha, et al., Dima Damen — CVPR, 2025

Project
Spatial Cognition from Egocentric Video: Out of Sight, Not Out of Mind

Chiara Plizzari, Shubham Goel, Toby Perrett, Jacob Chalk, Angjoo Kanazawa, Dima Damen — 3DV, 2025

Project

2024

It's Just Another Day: Unique Video Captioning by Discriminative Prompting

Toby Perrett, Tengda Han, Dima Damen, Andrew Zisserman — ACCV 2024 (Best Paper Award, Oral)

Project

2023

Use Your Head: Improving Long-Tail Video Recognition

Toby Perrett, Saptarshi Sinha, Tilo Burghardt, Majid Mirmehdi, Dima Damen — CVPR, 2023

Project
What can a cook in Italy teach a mechanic in India? Action Recognition Generalisation Over Scenarios and Locations

Chiara Plizzari, Toby Perrett, Barbara Caputo, Dima Damen — ICCV, 2023

GitHub
Centre Stage: Centricity-based Audio-Visual Temporal Action Detection

Hanyuan Wang, Majid Mirmehdi, Dima Damen, Toby Perrett — arXiv, 2023

GitHub

2022

Rescaling Egocentric Vision: Collection, Pipeline and Challenges for EPIC-KITCHENS-100

Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Antonino Furnari, Evangelos Kazakos, Jian Ma, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, Michael Wray — IJCV, 2022

Project
Personalized Energy Expenditure Estimation: Visual Sensing Approach With Deep Learning

Toby Perrett, Alessandro Masullo, Tilo Burghardt, Dima Damen, Ian Craddock, Majid Mirmehdi — JMIR Formative Research, 2022

Paper
An Evaluation of OCR on Egocentric Data

Valentin Popescu, Dima Damen, Toby Perrett — EPIC Workshop at CVPR, 2022

PDF
Refining Action Boundaries for One-Stage Detection

Hanyuan Wang, Majid Mirmehdi, Dima Damen, Toby Perrett — AVSS, 2022

GitHub

2021

Temporal-Relational CrossTransformers for Few-Shot Action Recognition

Toby Perrett, Alessandro Masullo, Tilo Burghardt, Majid Mirmehdi, Dima Damen — CVPR, 2021

Project
The EPIC-KITCHENS Dataset: Collection, Challenges and Baselines

Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Sanja Fidler, Antonino Furnari, Evangelos Kazakos, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, Michael Wray — IEEE TPAMI, 2021

Project

2020

Meta-Learning with Context-Agnostic Initialisations

Toby Perrett, Alessandro Masullo, Tilo Burghardt, Majid Mirmehdi, Dima Damen — ACCV, 2020

Project

2019

DDLSTM: Dual-Domain LSTM for Cross-Domain Action Recognition

Toby Perrett, Dima Damen — CVPR, 2019

Project

2018

Scaling Egocentric Vision: The EPIC-KITCHENS Dataset

Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Sanja Fidler, Antonino Furnari, Evangelos Kazakos, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, Michael Wray — ECCV, 2018

Project

2017

Detection of Valuable Left-Behind Items in Vehicle Cabins

Toby Perrett, Majid Mirmehdi, Eduardo Dias — IEEE Intelligent Vehicles Symposium, 2017

Paper