Leave-one-subject-out cross-validation

A validation method that trains a model repeatedly, holding out all data from one subject at a time for testing. It estimates performance on previously unseen subjects.

Connect

Cross-validation: A family of methods that estimate predictive performance by repeatedly training and evaluating models on different data partitions. This method is a cross-validation design in which each fold contains one subject.
Human-subject research: Research involving data collected from or about human participants. The method evaluates whether models generalize between participants rather than between measurements.
K-fold cross-validation: A cross-validation method that partitions data into k folds and evaluates each fold once while training on the others. Ordinary folds can place the same subject in training and testing, unlike this design.
Pseudoreplication: The treatment of non-independent observations as if they were independent replicates. Subject-level folds avoid treating repeated measurements as independent evidence of new-person performance.
Data leakage: The use of information during model development that would not legitimately be available at prediction time. Splitting by subject helps prevent subject-specific information from leaking into test evaluations.
Neuroimaging: The use of imaging methods to measure the structure or activity of the nervous system. Neuroimaging datasets often contain many scans per person, making subject-level splits essential.
Leave-one-out cross-validation: A cross-validation method that holds out one observation at a time for testing. It holds out individual observations, not all observations from one subject.
Intraclass correlation: A statistic describing similarity among measurements from the same group or subject. Strong within-subject similarity explains why random observation splits can inflate performance estimates.
Training set: The portion of data used to fit a machine-learning model. Each iteration trains on every subject except the one reserved for testing.
Human activity recognition: The classification of human actions from sensor, video, or other measured data. A subject-held-out test measures recognition for people absent from training.