Note

Click here to download the full example code

Within Session P300#

This example shows how to perform a within session analysis on three different P300 datasets.

We will compare two pipelines :

  • Riemannian geometry

  • XDAWN with Linear Discriminant Analysis

We will use the P300 paradigm, which uses the AUC as metric.

# Authors: Pedro Rodrigues <pedro.rodrigues01@gmail.com>
#
# License: BSD (3-clause)

import warnings

import matplotlib.pyplot as plt
import seaborn as sns
from mne.decoding import Vectorizer
from pyriemann.estimation import Xdawn, XdawnCovariances
from pyriemann.tangentspace import TangentSpace
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis as LDA
from sklearn.pipeline import make_pipeline

import moabb
from moabb.datasets import BNCI2014_009
from moabb.evaluations import WithinSessionEvaluation
from moabb.paradigms import P300

getting rid of the warnings about the future

warnings.simplefilter(action="ignore", category=FutureWarning)
warnings.simplefilter(action="ignore", category=RuntimeWarning)

moabb.set_log_level("info")

Create Pipelines#

Pipelines must be a dict of sklearn pipeline transformer.

pipelines = {}

We have to do this because the classes are called ‘Target’ and ‘NonTarget’ but the evaluation function uses a LabelEncoder, transforming them to 0 and 1

labels_dict = {"Target": 1, "NonTarget": 0}

pipelines["RG+LDA"] = make_pipeline(
    XdawnCovariances(
        nfilter=2, classes=[labels_dict["Target"]], estimator="lwf", xdawn_estimator="scm"
    ),
    TangentSpace(),
    LDA(solver="lsqr", shrinkage="auto"),
)

pipelines["Xdw+LDA"] = make_pipeline(
    Xdawn(nfilter=2, estimator="scm"), Vectorizer(), LDA(solver="lsqr", shrinkage="auto")
)

Evaluation#

We define the paradigm (P300) and use all three datasets available for it. The evaluation will return a DataFrame containing a single AUC score for each subject / session of the dataset, and for each pipeline.

Results are saved into the database, so that if you add a new pipeline, it will not run again the evaluation unless a parameter has changed. Results can be overwritten if necessary.

paradigm = P300(resample=128)
dataset = BNCI2014_009()
dataset.subject_list = dataset.subject_list[:2]
datasets = [dataset]
overwrite = True  # set to True if we want to overwrite cached results
evaluation = WithinSessionEvaluation(
    paradigm=paradigm, datasets=datasets, suffix="examples", overwrite=overwrite
)
results = evaluation.process(pipelines)

BNCI2014-009-WithinSession:   0%|          | 0/2 [00:00<?, ?it/s]/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'lampx.tugraz.at'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
  warnings.warn(


  0%|                                              | 0.00/18.5M [00:00<?, ?B/s]

  0%|                                      | 32.8k/18.5M [00:00<01:50, 167kB/s]

  1%|▎                                      | 144k/18.5M [00:00<00:32, 567kB/s]

  2%|▋                                      | 309k/18.5M [00:00<00:18, 974kB/s]

  3%|█▏                                    | 608k/18.5M [00:00<00:10, 1.68MB/s]

  6%|██▏                                  | 1.11M/18.5M [00:00<00:06, 2.78MB/s]

 11%|████▏                                | 2.09M/18.5M [00:00<00:03, 5.05MB/s]

 21%|███████▉                             | 3.96M/18.5M [00:00<00:01, 9.35MB/s]

 39%|██████████████▍                      | 7.22M/18.5M [00:00<00:00, 16.6MB/s]

 54%|████████████████████                 | 10.1M/18.5M [00:00<00:00, 20.2MB/s]

 71%|██████████████████████████▍          | 13.2M/18.5M [00:01<00:00, 23.7MB/s]

 88%|████████████████████████████████▌    | 16.3M/18.5M [00:01<00:00, 25.8MB/s]

  0%|                                              | 0.00/18.5M [00:00<?, ?B/s]
100%|█████████████████████████████████████| 18.5M/18.5M [00:00<00:00, 54.1GB/s]
/home/runner/work/moabb/moabb/moabb/datasets/preprocessing.py:273: UserWarning: warnEpochs <Epochs |  576 events (all good), 0 – 0.800781 s, baseline off, ~14.5 MB, data loaded,
 'Target': 96
 'NonTarget': 480>
  warn(f"warnEpochs {epochs}")
/home/runner/work/moabb/moabb/moabb/datasets/preprocessing.py:273: UserWarning: warnEpochs <Epochs |  576 events (all good), 0 – 0.800781 s, baseline off, ~14.5 MB, data loaded,
 'Target': 96
 'NonTarget': 480>
  warn(f"warnEpochs {epochs}")
/home/runner/work/moabb/moabb/moabb/datasets/preprocessing.py:273: UserWarning: warnEpochs <Epochs |  576 events (all good), 0 – 0.800781 s, baseline off, ~14.5 MB, data loaded,
 'Target': 96
 'NonTarget': 480>
  warn(f"warnEpochs {epochs}")
No hdf5_path provided, models will not be saved.
No hdf5_path provided, models will not be saved.
No hdf5_path provided, models will not be saved.
No hdf5_path provided, models will not be saved.
No hdf5_path provided, models will not be saved.
No hdf5_path provided, models will not be saved.

BNCI2014-009-WithinSession:  50%|#####     | 1/2 [00:12<00:12, 12.00s/it]/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'lampx.tugraz.at'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
  warnings.warn(


  0%|                                              | 0.00/18.5M [00:00<?, ?B/s]

  0%|                                      | 32.8k/18.5M [00:00<01:50, 167kB/s]

  1%|▎                                      | 148k/18.5M [00:00<00:31, 585kB/s]

  2%|▌                                      | 296k/18.5M [00:00<00:19, 919kB/s]

  3%|█▎                                    | 637k/18.5M [00:00<00:09, 1.79MB/s]

  6%|██▏                                  | 1.13M/18.5M [00:00<00:06, 2.82MB/s]

 11%|████▏                                | 2.08M/18.5M [00:00<00:03, 4.97MB/s]

 22%|████████▏                            | 4.09M/18.5M [00:00<00:01, 9.77MB/s]

 39%|██████████████▌                      | 7.29M/18.5M [00:00<00:00, 16.7MB/s]

 54%|███████████████████▉                 | 9.96M/18.5M [00:00<00:00, 19.7MB/s]

 71%|██████████████████████████▏          | 13.1M/18.5M [00:01<00:00, 23.4MB/s]

 87%|████████████████████████████████▎    | 16.2M/18.5M [00:01<00:00, 25.5MB/s]

  0%|                                              | 0.00/18.5M [00:00<?, ?B/s]
100%|█████████████████████████████████████| 18.5M/18.5M [00:00<00:00, 55.6GB/s]
/home/runner/work/moabb/moabb/moabb/datasets/preprocessing.py:273: UserWarning: warnEpochs <Epochs |  576 events (all good), 0 – 0.800781 s, baseline off, ~14.5 MB, data loaded,
 'Target': 96
 'NonTarget': 480>
  warn(f"warnEpochs {epochs}")
/home/runner/work/moabb/moabb/moabb/datasets/preprocessing.py:273: UserWarning: warnEpochs <Epochs |  576 events (all good), 0 – 0.800781 s, baseline off, ~14.5 MB, data loaded,
 'Target': 96
 'NonTarget': 480>
  warn(f"warnEpochs {epochs}")
/home/runner/work/moabb/moabb/moabb/datasets/preprocessing.py:273: UserWarning: warnEpochs <Epochs |  576 events (all good), 0 – 0.800781 s, baseline off, ~14.5 MB, data loaded,
 'Target': 96
 'NonTarget': 480>
  warn(f"warnEpochs {epochs}")
No hdf5_path provided, models will not be saved.
No hdf5_path provided, models will not be saved.
No hdf5_path provided, models will not be saved.
No hdf5_path provided, models will not be saved.
No hdf5_path provided, models will not be saved.
No hdf5_path provided, models will not be saved.

BNCI2014-009-WithinSession: 100%|##########| 2/2 [00:23<00:00, 11.77s/it]
BNCI2014-009-WithinSession: 100%|##########| 2/2 [00:23<00:00, 11.80s/it]

Plot Results#

Here we plot the results to compare the two pipelines

fig, ax = plt.subplots(facecolor="white", figsize=[8, 4])

sns.stripplot(
    data=results,
    y="score",
    x="pipeline",
    ax=ax,
    jitter=True,
    alpha=0.5,
    zorder=1,
    palette="Set1",
)
sns.pointplot(data=results, y="score", x="pipeline", ax=ax, palette="Set1")

ax.set_ylabel("ROC AUC")
ax.set_ylim(0.5, 1)

plt.show()
plot within session p300

Total running time of the script: ( 0 minutes 25.783 seconds)

Estimated memory usage: 12 MB

Gallery generated by Sphinx-Gallery