Practical guidelines for validation of supervised machine learning models in accelerometer‐based animal behaviour classification
Abstract
Abstract Supervised machine learning has been used to detect fine‐scale animal behaviour from accelerometer data, but a standardised protocol for implementing this workflow is currently lacking. As the application of machine learning to ecological problems expands, it is essential to establish technical protocols and validation standards that align with those in other ‘big data’ fields. Overfitting is a prevalent and often misunderstood challenge in machine learning. Overfit models overly adapt to the training data to memorise specific instances rather than to discern the underlying signal. Associated results can indicate high performance on the training set, yet these models are unlikely to generalise to new data. Overfitting can be detected through rigorous validation using independent test sets. Our systematic review of 119 studies using accelerometer‐based supervised machine learning to classify animal behaviour reveals that 79% (94 papers) did not validate their models sufficiently well to robustly identify potential overfitting. Although this does not inherently imply that these models are overfit, the absence of independent test sets limits the interpretability of their results. To address these challenges, we provide a theoretical overview of overfitting in the context of animal accelerometry and propose guidelines for optimal validation techniques. Our aim is to equip ecologists with the tools necessary to adapt general machine learning validation theory to the…
How strong is the evidence?
Evidence Hub reads this as a review, done in computer models (in silico), not in people. Its evidence level is 5 of 5 (Very low · case / narrative / animal / in vitro / in silico). Case reports, narrative reviews, and animal, cell or computer-model studies give the weakest evidence for human health.
Read the paper
Related studies
- Improving the translation of search strategies using the Polyglot Search Translator: a randomized controlled trial
- Nutritional, Physical, Cognitive, and Combination Interventions and Frailty Reversal Among Older Adults: A Randomized Controlled Trial
- Clinical characteristics and risk factors associated with COVID-19 severity in patients with haematological malignancies in Italy: a retrospective, multicentre, cohort study
- A randomized, controlled trial of oral propranolol in infantile hemangioma
- Clinical phenotypes of delirium during critical illness and severity of subsequent long-term cognitive impairment: a prospective cohort study
- Increased physical activity improves sleep and mood outcomes in inactive people with insomnia: a randomized controlled trial
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.