moabb.analysis.meta_analysis.compute_lowest_subject_scores

moabb.analysis.meta_analysis.compute_lowest_subject_scores(df, reference_pipeline, percentile=20)

Score pipelines on subjects selected from a reference pipeline.

For every dataset, rank subjects by the score of reference_pipeline and keep only the percentile percent that score lowest. Every pipeline is then averaged over that same cohort.

The supplied session scores are first macro-averaged per subject, so every subject weighs the same regardless of how many sessions it contributes. This helper does not recompute subject-level F1 scores from predictions.

Parameters:

• df (pandas.DataFrame) – results obtained by an evaluation, with at least the dataset, pipeline, subject and score columns

• reference_pipeline (str) – Pipeline whose per-subject scores define the lowest-performing cohort separately for each dataset. Every pipeline in a dataset must contain exactly the same subjects as this reference.

• percentile (float, default=20) – percentage of subjects to keep, in (0, 100]. The number of retained subjects is rounded up, and is at least one, so a value small enough always falls back to the single worst subject.

Returns:

scores – One row per (dataset, pipeline) pair, with the mean score over the retained subjects and the n_subjects that were retained.

Return type:

pandas.DataFrame

References

[1]

Gnassounou, T., Collas, A., Flamary, R., and Gramfort, A. (2025). PSDNorm: Test-Time Temporal Normalization for Deep Learning in Sleep Staging. https://arxiv.org/abs/2503.04582