Statistical Analysis¶

The MOABB codebase comes with convenience plotting utilities and some statistical testing. This tutorial focuses on what those exactly are and how they can be used.

```# Authors: Vinay Jayaram <vinayjayaram13@gmail.com>
#
# License: BSD (3-clause)
# sphinx_gallery_thumbnail_number = -2

import matplotlib.pyplot as plt
from mne.decoding import CSP
from pyriemann.estimation import Covariances
from pyriemann.tangentspace import TangentSpace
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis as LDA
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline

import moabb
import moabb.analysis.plotting as moabb_plt
from moabb.analysis.meta_analysis import (  # noqa: E501
compute_dataset_statistics,
find_significant_differences,
)
from moabb.datasets import BNCI2014001
from moabb.evaluations import CrossSessionEvaluation
from moabb.paradigms import LeftRightImagery

moabb.set_log_level("info")

print(__doc__)
```

Results Generation¶

First we need to set up a paradigm, dataset list, and some pipelines to test. This is explored more in the examples – we choose left vs right imagery paradigm with a single bandpass. There is only one dataset here but any number can be added without changing this workflow.

Create Pipelines¶

Pipelines must be a dict of sklearn pipeline transformer.

The CSP implementation from MNE is used. We selected 8 CSP components, as usually done in the literature.

The Riemannian geometry pipeline consists in covariance estimation, tangent space mapping and finally a logistic regression for the classification.

```pipelines = {}

pipelines["CSP+LDA"] = make_pipeline(CSP(n_components=8), LDA())

pipelines["RG+LR"] = make_pipeline(Covariances(), TangentSpace(), LogisticRegression())

pipelines["CSP+LR"] = make_pipeline(CSP(n_components=8), LogisticRegression())

pipelines["RG+LDA"] = make_pipeline(Covariances(), TangentSpace(), LDA())
```

Evaluation¶

We define the paradigm (LeftRightImagery) and the dataset (BNCI2014001). The evaluation will return a DataFrame containing a single AUC score for each subject / session of the dataset, and for each pipeline.

Results are saved into the database, so that if you add a new pipeline, it will not run again the evaluation unless a parameter has changed. Results can be overwritten if necessary.

```paradigm = LeftRightImagery()
dataset = BNCI2014001()
dataset.subject_list = dataset.subject_list[:4]
datasets = [dataset]
overwrite = True  # set to False if we want to use cached results
evaluation = CrossSessionEvaluation(
)

results = evaluation.process(pipelines)
```

Out:

```001-2014-CrossSession:   0%|          | 0/4 [00:00<?, ?it/s]/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):

001-2014-CrossSession:  25%|##5       | 1/4 [00:06<00:20,  6.81s/it]/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):

001-2014-CrossSession:  50%|#####     | 2/4 [00:13<00:13,  6.82s/it]

0%|                                              | 0.00/44.1M [00:00<?, ?B/s][A

0%|                                   | 1.02k/44.1M [00:00<1:34:16, 7.79kB/s][A

0%|                                     | 17.4k/44.1M [00:00<09:37, 76.3kB/s][A

0%|                                     | 33.8k/44.1M [00:00<07:28, 98.2kB/s][A

0%|                                      | 50.2k/44.1M [00:00<06:45, 109kB/s][A

0%|                                      | 82.9k/44.1M [00:00<04:36, 159kB/s][A

0%|                                      | 99.3k/44.1M [00:00<04:58, 147kB/s][A

0%|                                       | 132k/44.1M [00:00<04:03, 181kB/s][A

0%|▏                                      | 151k/44.1M [00:01<04:22, 168kB/s][A

0%|▏                                      | 181k/44.1M [00:01<03:53, 188kB/s][A

0%|▏                                      | 214k/44.1M [00:01<03:31, 207kB/s][A

1%|▏                                      | 247k/44.1M [00:01<03:19, 220kB/s][A

1%|▏                                      | 280k/44.1M [00:01<03:11, 229kB/s][A

1%|▎                                      | 312k/44.1M [00:01<03:06, 235kB/s][A

1%|▎                                      | 361k/44.1M [00:01<02:38, 277kB/s][A

1%|▎                                      | 411k/44.1M [00:01<02:22, 306kB/s][A

1%|▍                                      | 460k/44.1M [00:02<02:14, 325kB/s][A

1%|▍                                      | 525k/44.1M [00:02<01:55, 377kB/s][A

1%|▌                                      | 591k/44.1M [00:02<01:45, 412kB/s][A

1%|▌                                      | 640k/44.1M [00:02<01:48, 399kB/s][A

2%|▌                                      | 706k/44.1M [00:02<01:40, 430kB/s][A

2%|▋                                      | 787k/44.1M [00:02<01:29, 486kB/s][A

2%|▊                                      | 869k/44.1M [00:02<01:22, 526kB/s][A

2%|▊                                      | 951k/44.1M [00:03<01:17, 555kB/s][A

2%|▉                                     | 1.03M/44.1M [00:03<01:14, 575kB/s][A

3%|▉                                     | 1.13M/44.1M [00:03<01:09, 618kB/s][A

3%|█                                     | 1.23M/44.1M [00:03<01:05, 654kB/s][A

3%|█▏                                    | 1.34M/44.1M [00:03<00:59, 718kB/s][A

3%|█▎                                    | 1.46M/44.1M [00:03<00:55, 764kB/s][A

4%|█▍                                    | 1.61M/44.1M [00:03<00:48, 871kB/s][A

4%|█▌                                    | 1.75M/44.1M [00:03<00:44, 945kB/s][A

4%|█▌                                   | 1.92M/44.1M [00:04<00:40, 1.03MB/s][A

5%|█▊                                   | 2.11M/44.1M [00:04<00:36, 1.14MB/s][A

5%|█▉                                   | 2.25M/44.1M [00:04<00:37, 1.11MB/s][A

6%|██                                   | 2.46M/44.1M [00:04<00:32, 1.26MB/s][A

6%|██▏                                  | 2.67M/44.1M [00:04<00:30, 1.37MB/s][A

7%|██▍                                  | 2.88M/44.1M [00:04<00:28, 1.44MB/s][A

7%|██▋                                  | 3.13M/44.1M [00:04<00:26, 1.57MB/s][A

8%|██▊                                  | 3.34M/44.1M [00:05<00:25, 1.58MB/s][A

8%|███                                  | 3.61M/44.1M [00:05<00:23, 1.70MB/s][A

9%|███▎                                 | 3.90M/44.1M [00:05<00:21, 1.86MB/s][A

9%|███▌                                 | 4.18M/44.1M [00:05<00:20, 1.94MB/s][A

10%|███▊                                 | 4.49M/44.1M [00:05<00:19, 2.02MB/s][A

11%|████                                 | 4.83M/44.1M [00:05<00:17, 2.20MB/s][A

12%|████▎                                | 5.18M/44.1M [00:05<00:16, 2.32MB/s][A

13%|████▋                                | 5.52M/44.1M [00:05<00:16, 2.41MB/s][A

13%|████▉                                | 5.92M/44.1M [00:06<00:14, 2.54MB/s][A

14%|█████▎                               | 6.29M/44.1M [00:06<00:14, 2.64MB/s][A

15%|█████▋                               | 6.72M/44.1M [00:06<00:13, 2.82MB/s][A

16%|██████                               | 7.18M/44.1M [00:06<00:12, 3.02MB/s][A

17%|██████▍                              | 7.67M/44.1M [00:06<00:11, 3.22MB/s][A

19%|██████▉                              | 8.19M/44.1M [00:06<00:10, 3.45MB/s][A

20%|███████▎                             | 8.73M/44.1M [00:06<00:09, 3.64MB/s][A

21%|███████▊                             | 9.32M/44.1M [00:07<00:08, 3.89MB/s][A

23%|████████▎                            | 9.95M/44.1M [00:07<00:08, 4.14MB/s][A

24%|████████▉                            | 10.6M/44.1M [00:07<00:07, 4.35MB/s][A

26%|█████████▍                           | 11.3M/44.1M [00:07<00:06, 4.68MB/s][A

27%|██████████▏                          | 12.1M/44.1M [00:07<00:06, 4.98MB/s][A

29%|██████████▊                          | 12.9M/44.1M [00:07<00:05, 5.36MB/s][A

31%|███████████▌                         | 13.7M/44.1M [00:07<00:05, 5.60MB/s][A

33%|████████████▎                        | 14.7M/44.1M [00:07<00:04, 5.99MB/s][A

35%|█████████████                        | 15.6M/44.1M [00:08<00:04, 6.31MB/s][A

38%|█████████████▉                       | 16.6M/44.1M [00:08<00:04, 6.71MB/s][A

40%|██████████████▊                      | 17.7M/44.1M [00:08<00:03, 7.16MB/s][A

43%|███████████████▊                     | 18.8M/44.1M [00:08<00:03, 7.60MB/s][A

46%|████████████████▊                    | 20.1M/44.1M [00:08<00:02, 8.09MB/s][A

48%|█████████████████▉                   | 21.3M/44.1M [00:08<00:02, 8.49MB/s][A

51%|███████████████████                  | 22.7M/44.1M [00:08<00:02, 9.03MB/s][A

55%|████████████████████▏                | 24.1M/44.1M [00:09<00:02, 9.50MB/s][A

58%|█████████████████████▍               | 25.6M/44.1M [00:09<00:01, 10.1MB/s][A

62%|██████████████████████▊              | 27.2M/44.1M [00:09<00:01, 10.7MB/s][A

66%|████████████████████████▎            | 28.9M/44.1M [00:09<00:01, 11.3MB/s][A

70%|█████████████████████████▊           | 30.7M/44.1M [00:09<00:01, 12.1MB/s][A

74%|███████████████████████████▍         | 32.6M/44.1M [00:09<00:00, 12.8MB/s][A

79%|█████████████████████████████        | 34.7M/44.1M [00:09<00:00, 13.6MB/s][A

84%|██████████████████████████████▉      | 36.8M/44.1M [00:09<00:00, 14.5MB/s][A

89%|████████████████████████████████▉    | 39.2M/44.1M [00:10<00:00, 15.4MB/s][A

95%|██████████████████████████████████▉  | 41.6M/44.1M [00:10<00:00, 16.4MB/s][A

0%|                                              | 0.00/44.1M [00:00<?, ?B/s][A
100%|█████████████████████████████████████| 44.1M/44.1M [00:00<00:00, 47.6GB/s]

0%|                                              | 0.00/42.3M [00:00<?, ?B/s][A

0%|                                   | 1.02k/42.3M [00:00<1:32:00, 7.67kB/s][A

0%|                                     | 17.4k/42.3M [00:00<09:21, 75.3kB/s][A

0%|                                     | 33.8k/42.3M [00:00<07:15, 97.1kB/s][A

0%|                                      | 50.2k/42.3M [00:00<06:34, 107kB/s][A

0%|                                      | 82.9k/42.3M [00:00<04:28, 157kB/s][A

0%|                                      | 99.3k/42.3M [00:00<04:49, 146kB/s][A

0%|                                       | 132k/42.3M [00:00<03:56, 179kB/s][A

0%|▏                                      | 151k/42.3M [00:01<04:14, 166kB/s][A

0%|▏                                      | 181k/42.3M [00:01<03:46, 186kB/s][A

1%|▏                                      | 214k/42.3M [00:01<03:25, 205kB/s][A

1%|▏                                      | 247k/42.3M [00:01<03:13, 217kB/s][A

1%|▎                                      | 296k/42.3M [00:01<02:39, 264kB/s][A

1%|▎                                      | 329k/42.3M [00:01<02:42, 258kB/s][A

1%|▎                                      | 355k/42.3M [00:01<02:54, 241kB/s][A

1%|▎                                      | 394k/42.3M [00:01<02:43, 256kB/s][A

1%|▍                                      | 460k/42.3M [00:02<02:08, 326kB/s][A

1%|▍                                      | 509k/42.3M [00:02<02:03, 339kB/s][A

1%|▌                                      | 574k/42.3M [00:02<01:48, 385kB/s][A

2%|▌                                      | 640k/42.3M [00:02<01:39, 417kB/s][A

2%|▋                                      | 706k/42.3M [00:02<01:34, 439kB/s][A

2%|▋                                      | 771k/42.3M [00:02<01:31, 455kB/s][A

2%|▊                                      | 853k/42.3M [00:02<01:22, 502kB/s][A

2%|▊                                      | 935k/42.3M [00:03<01:17, 535kB/s][A

2%|▉                                     | 1.02M/42.3M [00:03<01:14, 551kB/s][A

3%|▉                                     | 1.10M/42.3M [00:03<01:12, 569kB/s][A

3%|█                                     | 1.20M/42.3M [00:03<01:06, 619kB/s][A

3%|█▏                                    | 1.30M/42.3M [00:03<01:03, 646kB/s][A

3%|█▎                                    | 1.39M/42.3M [00:03<01:00, 673kB/s][A

4%|█▎                                    | 1.51M/42.3M [00:03<00:56, 727kB/s][A

4%|█▍                                    | 1.61M/42.3M [00:04<00:55, 728kB/s][A

4%|█▌                                    | 1.72M/42.3M [00:04<00:52, 768kB/s][A

4%|█▋                                    | 1.84M/42.3M [00:04<00:50, 794kB/s][A

5%|█▊                                    | 1.98M/42.3M [00:04<00:45, 886kB/s][A

5%|█▉                                    | 2.11M/42.3M [00:04<00:43, 917kB/s][A

5%|██                                    | 2.26M/42.3M [00:04<00:41, 968kB/s][A

6%|██                                   | 2.43M/42.3M [00:04<00:38, 1.05MB/s][A

6%|██▎                                  | 2.59M/42.3M [00:04<00:36, 1.09MB/s][A

7%|██▍                                  | 2.77M/42.3M [00:05<00:34, 1.15MB/s][A

7%|██▌                                  | 2.95M/42.3M [00:05<00:32, 1.21MB/s][A

7%|██▊                                  | 3.15M/42.3M [00:05<00:30, 1.29MB/s][A

8%|██▉                                  | 3.36M/42.3M [00:05<00:28, 1.38MB/s][A

8%|███                                  | 3.57M/42.3M [00:05<00:26, 1.45MB/s][A

9%|███▎                                 | 3.79M/42.3M [00:05<00:25, 1.49MB/s][A

9%|███▍                                 | 3.98M/42.3M [00:05<00:25, 1.49MB/s][A

10%|███▋                                 | 4.23M/42.3M [00:06<00:23, 1.60MB/s][A

11%|███▉                                 | 4.49M/42.3M [00:06<00:22, 1.71MB/s][A

11%|████▏                                | 4.75M/42.3M [00:06<00:21, 1.79MB/s][A

12%|████▎                                | 5.00M/42.3M [00:06<00:20, 1.80MB/s][A

13%|████▋                                | 5.33M/42.3M [00:06<00:18, 2.00MB/s][A

13%|████▉                                | 5.69M/42.3M [00:06<00:16, 2.20MB/s][A

14%|█████▎                               | 6.06M/42.3M [00:06<00:15, 2.38MB/s][A

15%|█████▋                               | 6.47M/42.3M [00:06<00:13, 2.57MB/s][A

16%|██████                               | 6.92M/42.3M [00:07<00:12, 2.79MB/s][A

17%|██████▍                              | 7.32M/42.3M [00:07<00:12, 2.88MB/s][A

19%|██████▊                              | 7.83M/42.3M [00:07<00:10, 3.15MB/s][A

20%|███████▎                             | 8.37M/42.3M [00:07<00:09, 3.43MB/s][A

21%|███████▊                             | 8.95M/42.3M [00:07<00:09, 3.68MB/s][A

23%|████████▎                            | 9.54M/42.3M [00:07<00:08, 3.90MB/s][A

24%|████████▉                            | 10.2M/42.3M [00:07<00:07, 4.13MB/s][A

26%|█████████▍                           | 10.8M/42.3M [00:08<00:07, 4.36MB/s][A

27%|██████████                           | 11.6M/42.3M [00:08<00:06, 4.62MB/s][A

29%|██████████▊                          | 12.3M/42.3M [00:08<00:06, 4.95MB/s][A

31%|███████████▍                         | 13.1M/42.3M [00:08<00:05, 5.27MB/s][A

33%|████████████▏                        | 14.0M/42.3M [00:08<00:05, 5.60MB/s][A

35%|█████████████                        | 14.9M/42.3M [00:08<00:04, 5.97MB/s][A

37%|█████████████▊                       | 15.9M/42.3M [00:08<00:04, 6.34MB/s][A

40%|██████████████▊                      | 16.9M/42.3M [00:08<00:03, 6.72MB/s][A

42%|███████████████▋                     | 18.0M/42.3M [00:09<00:03, 7.14MB/s][A

45%|████████████████▋                    | 19.1M/42.3M [00:09<00:03, 7.51MB/s][A

48%|█████████████████▋                   | 20.3M/42.3M [00:09<00:02, 7.93MB/s][A

51%|██████████████████▊                  | 21.5M/42.3M [00:09<00:02, 8.35MB/s][A

54%|███████████████████▉                 | 22.8M/42.3M [00:09<00:02, 8.80MB/s][A

57%|█████████████████████▏               | 24.3M/42.3M [00:09<00:01, 9.38MB/s][A

61%|██████████████████████▌              | 25.8M/42.3M [00:09<00:01, 9.97MB/s][A

65%|███████████████████████▉             | 27.4M/42.3M [00:10<00:01, 10.7MB/s][A

69%|█████████████████████████▌           | 29.2M/42.3M [00:10<00:01, 11.4MB/s][A

73%|███████████████████████████          | 31.0M/42.3M [00:10<00:00, 12.1MB/s][A

78%|████████████████████████████▊        | 33.0M/42.3M [00:10<00:00, 12.9MB/s][A

83%|██████████████████████████████▋      | 35.1M/42.3M [00:10<00:00, 13.7MB/s][A

88%|████████████████████████████████▌    | 37.3M/42.3M [00:10<00:00, 14.5MB/s][A

94%|██████████████████████████████████▋  | 39.6M/42.3M [00:10<00:00, 15.4MB/s][A

99%|████████████████████████████████████▊| 42.1M/42.3M [00:10<00:00, 17.7MB/s][A

0%|                                              | 0.00/42.3M [00:00<?, ?B/s][A
100%|█████████████████████████████████████| 42.3M/42.3M [00:00<00:00, 46.0GB/s]
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):

001-2014-CrossSession:  75%|#######5  | 3/4 [00:44<00:17, 17.61s/it]

0%|                                              | 0.00/37.2M [00:00<?, ?B/s][A

0%|                                   | 1.02k/37.2M [00:00<1:19:23, 7.80kB/s][A

0%|                                     | 17.4k/37.2M [00:00<08:06, 76.4kB/s][A

0%|                                     | 33.8k/37.2M [00:00<06:17, 98.4kB/s][A

0%|                                      | 50.2k/37.2M [00:00<05:41, 109kB/s][A

0%|                                      | 82.9k/37.2M [00:00<03:52, 159kB/s][A

0%|                                      | 99.3k/37.2M [00:00<04:15, 145kB/s][A

0%|▏                                      | 132k/37.2M [00:00<03:27, 178kB/s][A

0%|▏                                      | 151k/37.2M [00:01<03:42, 166kB/s][A

0%|▏                                      | 181k/37.2M [00:01<03:17, 187kB/s][A

1%|▏                                      | 214k/37.2M [00:01<02:59, 206kB/s][A

1%|▎                                      | 247k/37.2M [00:01<02:48, 219kB/s][A

1%|▎                                      | 280k/37.2M [00:01<02:41, 228kB/s][A

1%|▎                                      | 329k/37.2M [00:01<02:15, 272kB/s][A

1%|▍                                      | 361k/37.2M [00:01<02:18, 265kB/s][A

1%|▍                                      | 411k/37.2M [00:01<02:03, 298kB/s][A

1%|▍                                      | 460k/37.2M [00:02<01:54, 321kB/s][A

1%|▌                                      | 509k/37.2M [00:02<01:48, 336kB/s][A

2%|▌                                      | 558k/37.2M [00:02<01:45, 348kB/s][A

2%|▋                                      | 624k/37.2M [00:02<01:32, 393kB/s][A

2%|▋                                      | 673k/37.2M [00:02<01:34, 387kB/s][A

2%|▊                                      | 738k/37.2M [00:02<01:26, 420kB/s][A

2%|▊                                      | 804k/37.2M [00:02<01:22, 443kB/s][A

2%|▉                                      | 886k/37.2M [00:03<01:12, 497kB/s][A

3%|▉                                      | 951k/37.2M [00:03<01:12, 498kB/s][A

3%|█                                     | 1.05M/37.2M [00:03<01:03, 570kB/s][A

3%|█▏                                    | 1.15M/37.2M [00:03<00:57, 622kB/s][A

3%|█▏                                    | 1.21M/37.2M [00:03<01:01, 585kB/s][A

4%|█▎                                    | 1.31M/37.2M [00:03<00:56, 631kB/s][A

4%|█▍                                    | 1.43M/37.2M [00:03<00:50, 703kB/s][A

4%|█▌                                    | 1.56M/37.2M [00:03<00:45, 790kB/s][A

5%|█▋                                    | 1.67M/37.2M [00:04<00:43, 814kB/s][A

5%|█▊                                    | 1.79M/37.2M [00:04<00:42, 831kB/s][A

5%|█▉                                    | 1.93M/37.2M [00:04<00:38, 915kB/s][A

6%|██▏                                   | 2.08M/37.2M [00:04<00:35, 976kB/s][A

6%|██▏                                  | 2.23M/37.2M [00:04<00:34, 1.02MB/s][A

6%|██▍                                  | 2.41M/37.2M [00:04<00:31, 1.12MB/s][A

7%|██▌                                  | 2.56M/37.2M [00:04<00:30, 1.12MB/s][A

7%|██▋                                  | 2.75M/37.2M [00:05<00:27, 1.23MB/s][A

8%|██▉                                  | 2.95M/37.2M [00:05<00:26, 1.31MB/s][A

8%|███▏                                 | 3.15M/37.2M [00:05<00:24, 1.36MB/s][A

9%|███▎                                 | 3.33M/37.2M [00:05<00:24, 1.37MB/s][A

9%|███▌                                 | 3.52M/37.2M [00:05<00:23, 1.40MB/s][A

10%|███▊                                 | 3.79M/37.2M [00:05<00:21, 1.58MB/s][A

11%|████                                 | 4.05M/37.2M [00:05<00:19, 1.70MB/s][A

12%|████▎                                | 4.36M/37.2M [00:05<00:17, 1.87MB/s][A

12%|████▌                                | 4.62M/37.2M [00:06<00:16, 1.93MB/s][A

13%|████▉                                | 4.95M/37.2M [00:06<00:15, 2.08MB/s][A

14%|█████▎                               | 5.31M/37.2M [00:06<00:13, 2.28MB/s][A

15%|█████▋                               | 5.69M/37.2M [00:06<00:12, 2.45MB/s][A

16%|██████                               | 6.10M/37.2M [00:06<00:11, 2.62MB/s][A

18%|██████▌                              | 6.54M/37.2M [00:06<00:10, 2.83MB/s][A

19%|██████▉                              | 6.98M/37.2M [00:06<00:10, 2.99MB/s][A

20%|███████▍                             | 7.46M/37.2M [00:07<00:09, 3.18MB/s][A

21%|███████▉                             | 7.96M/37.2M [00:07<00:08, 3.37MB/s][A

23%|████████▍                            | 8.52M/37.2M [00:07<00:07, 3.61MB/s][A

24%|█████████                            | 9.09M/37.2M [00:07<00:07, 3.85MB/s][A

26%|█████████▋                           | 9.72M/37.2M [00:07<00:06, 4.09MB/s][A

28%|██████████▎                          | 10.4M/37.2M [00:07<00:06, 4.35MB/s][A

30%|███████████                          | 11.1M/37.2M [00:07<00:05, 4.57MB/s][A

32%|███████████▋                         | 11.8M/37.2M [00:07<00:05, 4.85MB/s][A

34%|████████████▌                        | 12.6M/37.2M [00:08<00:04, 5.14MB/s][A

36%|█████████████▎                       | 13.4M/37.2M [00:08<00:04, 5.49MB/s][A

38%|██████████████▏                      | 14.2M/37.2M [00:08<00:03, 5.75MB/s][A

41%|███████████████                      | 15.2M/37.2M [00:08<00:03, 6.14MB/s][A

44%|████████████████                     | 16.2M/37.2M [00:08<00:03, 6.58MB/s][A

46%|█████████████████▏                   | 17.3M/37.2M [00:08<00:02, 7.03MB/s][A

49%|██████████████████▎                  | 18.4M/37.2M [00:08<00:02, 7.48MB/s][A

53%|███████████████████▌                 | 19.6M/37.2M [00:08<00:02, 7.96MB/s][A

56%|████████████████████▊                | 20.8M/37.2M [00:09<00:01, 8.41MB/s][A

60%|██████████████████████▏              | 22.2M/37.2M [00:09<00:01, 9.00MB/s][A

64%|███████████████████████▌             | 23.7M/37.2M [00:09<00:01, 9.58MB/s][A

68%|█████████████████████████            | 25.2M/37.2M [00:09<00:01, 10.2MB/s][A

72%|██████████████████████████▋          | 26.9M/37.2M [00:09<00:00, 10.9MB/s][A

77%|████████████████████████████▍        | 28.6M/37.2M [00:09<00:00, 11.6MB/s][A

82%|██████████████████████████████▎      | 30.4M/37.2M [00:09<00:00, 12.1MB/s][A

87%|████████████████████████████████▎    | 32.4M/37.2M [00:10<00:00, 12.9MB/s][A

93%|██████████████████████████████████▎  | 34.5M/37.2M [00:10<00:00, 14.8MB/s][A

98%|████████████████████████████████████▎| 36.4M/37.2M [00:10<00:00, 15.9MB/s][A

0%|                                              | 0.00/37.2M [00:00<?, ?B/s][A
100%|█████████████████████████████████████| 37.2M/37.2M [00:00<00:00, 43.4GB/s]

0%|                                              | 0.00/41.7M [00:00<?, ?B/s][A

0%|                                   | 1.02k/41.7M [00:00<1:29:54, 7.74kB/s][A

0%|                                     | 17.4k/41.7M [00:00<09:11, 75.6kB/s][A

0%|                                     | 33.8k/41.7M [00:00<07:08, 97.3kB/s][A

0%|                                      | 50.2k/41.7M [00:00<06:27, 107kB/s][A

0%|                                      | 82.9k/41.7M [00:00<04:24, 158kB/s][A

0%|                                      | 99.3k/41.7M [00:00<04:45, 146kB/s][A

0%|                                       | 132k/41.7M [00:00<03:52, 179kB/s][A

0%|▏                                      | 165k/41.7M [00:01<03:27, 200kB/s][A

0%|▏                                      | 198k/41.7M [00:01<03:16, 211kB/s][A

1%|▏                                      | 247k/41.7M [00:01<02:39, 260kB/s][A

1%|▎                                      | 296k/41.7M [00:01<02:21, 293kB/s][A

1%|▎                                      | 345k/41.7M [00:01<02:10, 316kB/s][A

1%|▎                                      | 378k/41.7M [00:01<02:20, 295kB/s][A

1%|▍                                      | 443k/41.7M [00:01<01:56, 355kB/s][A

1%|▍                                      | 525k/41.7M [00:02<01:35, 433kB/s][A

1%|▌                                      | 591k/41.7M [00:02<01:31, 450kB/s][A

2%|▌                                      | 656k/41.7M [00:02<01:28, 463kB/s][A

2%|▋                                      | 755k/41.7M [00:02<01:15, 545kB/s][A

2%|▊                                      | 837k/41.7M [00:02<01:12, 567kB/s][A

2%|▊                                      | 935k/41.7M [00:02<01:06, 616kB/s][A

3%|▉                                     | 1.05M/41.7M [00:02<00:59, 681kB/s][A

3%|█                                     | 1.16M/41.7M [00:02<00:55, 732kB/s][A

3%|█▏                                    | 1.28M/41.7M [00:03<00:52, 771kB/s][A

3%|█▎                                    | 1.41M/41.7M [00:03<00:48, 830kB/s][A

4%|█▍                                    | 1.54M/41.7M [00:03<00:45, 877kB/s][A

4%|█▌                                    | 1.69M/41.7M [00:03<00:42, 944kB/s][A

4%|█▋                                    | 1.82M/41.7M [00:03<00:41, 956kB/s][A

5%|█▊                                   | 2.00M/41.7M [00:03<00:37, 1.05MB/s][A

5%|█▉                                   | 2.18M/41.7M [00:03<00:34, 1.14MB/s][A

6%|██                                   | 2.38M/41.7M [00:04<00:31, 1.24MB/s][A

6%|██▎                                  | 2.61M/41.7M [00:04<00:28, 1.38MB/s][A

7%|██▌                                  | 2.85M/41.7M [00:04<00:25, 1.50MB/s][A

7%|██▊                                  | 3.11M/41.7M [00:04<00:23, 1.64MB/s][A

8%|███                                  | 3.39M/41.7M [00:04<00:21, 1.75MB/s][A

9%|███▎                                 | 3.70M/41.7M [00:04<00:19, 1.92MB/s][A

10%|███▌                                 | 4.03M/41.7M [00:04<00:18, 2.08MB/s][A

11%|███▉                                 | 4.39M/41.7M [00:04<00:16, 2.26MB/s][A

11%|████▏                                | 4.75M/41.7M [00:05<00:15, 2.40MB/s][A

12%|████▌                                | 5.13M/41.7M [00:05<00:14, 2.52MB/s][A

13%|████▉                                | 5.56M/41.7M [00:05<00:13, 2.73MB/s][A

14%|█████▎                               | 6.01M/41.7M [00:05<00:12, 2.94MB/s][A

16%|█████▊                               | 6.49M/41.7M [00:05<00:11, 3.09MB/s][A

17%|██████▏                              | 7.01M/41.7M [00:05<00:10, 3.33MB/s][A

18%|██████▋                              | 7.57M/41.7M [00:05<00:09, 3.57MB/s][A

20%|███████▏                             | 8.16M/41.7M [00:06<00:08, 3.82MB/s][A

21%|███████▊                             | 8.78M/41.7M [00:06<00:08, 4.08MB/s][A

23%|████████▎                            | 9.42M/41.7M [00:06<00:07, 4.30MB/s][A

24%|████████▉                            | 10.1M/41.7M [00:06<00:06, 4.60MB/s][A

26%|█████████▋                           | 10.9M/41.7M [00:06<00:06, 4.88MB/s][A

28%|██████████▎                          | 11.7M/41.7M [00:06<00:05, 5.19MB/s][A

30%|███████████                          | 12.5M/41.7M [00:06<00:05, 5.51MB/s][A

32%|███████████▉                         | 13.4M/41.7M [00:06<00:04, 5.91MB/s][A

35%|████████████▊                        | 14.4M/41.7M [00:07<00:04, 6.35MB/s][A

37%|█████████████▋                       | 15.5M/41.7M [00:07<00:03, 6.65MB/s][A

40%|██████████████▋                      | 16.6M/41.7M [00:07<00:03, 7.14MB/s][A

43%|███████████████▊                     | 17.8M/41.7M [00:07<00:03, 7.75MB/s][A

46%|████████████████▉                    | 19.2M/41.7M [00:07<00:02, 8.37MB/s][A

49%|██████████████████▏                  | 20.6M/41.7M [00:07<00:02, 9.01MB/s][A

53%|███████████████████▌                 | 22.1M/41.7M [00:07<00:02, 9.67MB/s][A

57%|████████████████████▉                | 23.7M/41.7M [00:08<00:01, 10.3MB/s][A

61%|██████████████████████▌              | 25.4M/41.7M [00:08<00:01, 11.0MB/s][A

65%|████████████████████████▏            | 27.3M/41.7M [00:08<00:01, 11.9MB/s][A

70%|█████████████████████████▉           | 29.2M/41.7M [00:08<00:00, 12.7MB/s][A

75%|███████████████████████████▊         | 31.4M/41.7M [00:08<00:00, 13.6MB/s][A

81%|█████████████████████████████▊       | 33.6M/41.7M [00:08<00:00, 14.7MB/s][A

86%|███████████████████████████████▉     | 36.0M/41.7M [00:08<00:00, 15.6MB/s][A

92%|██████████████████████████████████   | 38.5M/41.7M [00:09<00:00, 16.6MB/s][A

99%|████████████████████████████████████▍| 41.2M/41.7M [00:09<00:00, 17.6MB/s][A

0%|                                              | 0.00/41.7M [00:00<?, ?B/s][A
100%|█████████████████████████████████████| 41.7M/41.7M [00:00<00:00, 53.6GB/s]
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):
/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
LooseVersion(library.__version__) < LooseVersion(min_version):

001-2014-CrossSession: 100%|##########| 4/4 [01:11<00:00, 21.57s/it]
001-2014-CrossSession: 100%|##########| 4/4 [01:11<00:00, 17.93s/it]
```

MOABB Plotting¶

Here we plot the results using some of the convenience methods within the toolkit. The score_plot visualizes all the data with one score per subject for every dataset and pipeline.

```fig = moabb_plt.score_plot(results)
plt.show()
```

For a comparison of two algorithms, there is the paired_plot, which plots performance in one versus the performance in the other over all chosen datasets. Note that there is only one score per subject, regardless of the number of sessions.

```fig = moabb_plt.paired_plot(results, "CSP+LDA", "RG+LDA")
plt.show()
```

Statistical Testing and Further Plots¶

If the statistical significance of results is of interest, the method compute_dataset_statistics allows one to show a meta-analysis style plot as well. For an overview of how all algorithms perform in comparison with each other, the method find_significant_differences and the summary_plot are possible.

```stats = compute_dataset_statistics(results)
P, T = find_significant_differences(stats)
```

The meta-analysis style plot shows the standardized mean difference within each tested dataset for the two algorithms in question, in addition to a meta-effect and significance both per-dataset and overall.

```fig = moabb_plt.meta_analysis_plot(stats, "CSP+LDA", "RG+LDA")
plt.show()
```

The summary plot shows the effect and significance related to the hypothesis that the algorithm on the y-axis significantly outperformed the algorithm on the x-axis over all datasets

```moabb_plt.summary_plot(P, T)
plt.show()
```

Total running time of the script: ( 1 minutes 12.449 seconds)

Gallery generated by Sphinx-Gallery