Find research datasets worth reusing

opencc-by-nc-sa-4.0Aug 2019View details →

Auditory Attention Detection Dataset KULeuven

<div>***************************************</div> Please cite the original paper where this data set was presented: Biesmans, W., Das, N., Francart, T., & Bertrand, A. (2016). Auditory-inspired speech envelope extraction methods for improved EEG-based auditory attention detection in a cocktail party scenario. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 25(5), 402-412. *************************************** IMPORTANT MESSAGE FROM THE AUTHORS (January 2024): We have observed that this dataset is widely used in research, establishing it as a standard benchmark for evaluating novel decoding strategies in auditory attention decoding (AAD). We emphasize the critical importance of rigorous cross-validation in such studies. In particular, researchers should be aware of two common and significant validation pitfalls: <ol> <li> Trial fingerprints: Avoid splitting data from the same experimental trial into training and testing segments. Classifiers can detect whether a test segment belongs to a specific trial if other, non-overlapping segments from that trial are in the training set. </li> <li> Gaze bias: AAD algorithms that directly classify EEG signals (often referred to as spatial AAD or SpAAD) without explicitly correlating the decoder output signal to the speech stimulus, should not be evaluated on this dataset, as it is affected by gaze-related bias. Instead, use the gaze-controlled dataset of Rotaru et al. available at <a href="https://zenodo.org/records/11058711">https://zenodo.org/records/11058711</a> </li> </ol> Further details on both issues are provided below. In the original study by Biesmans et al., which produced this dataset, linear correlation-based methods were employed, and a straightforward random cross-validation sufficed. However, with the recent surge in the application of machine learning techniques, particularly deep neural networks, in tackling the AAD challenge, a more stringent cross-validation approach becomes imperative. Deep networks are susceptible to overfitting to trial-specific patterns in EEG data, even from very brief segments (less than 1 second), leading to the ability to identify the trial source. Since a subject typically maintains attention to the same speaker throughout a trial, having knowledge of the trial effectively results in a perfect attention decoding. We observed that many research papers utilizing our dataset still adhere to the basic random cross-validation method, neglecting the separation of trials into training and testing sets. Consequently, these studies frequently report remarkably high AAD accuracies when using extremely short EEG segments (one or a few seconds). Nevertheless, research has demonstrated that such an approach yields inaccurate and excessively optimistic outcomes. Accuracies often plummet significantly, sometimes even falling below chance levels, when employing a proper cross-validation where this trial bias is removed (e.g., leave-one-trial-out, leave-one-story-out, or leave-one-subject-out cross-validation). This overfitting effect is described in: Corentin Puffay et al., "Relating EEG to continuous speech using deep neural networks: a review", Journal of Neural Engineering 20, 041003, 2023 DOI:10.1088/1741-2552/ace73f Moreover, it's important to note that AAD strategies which directly classify an EEG snippet, rather than explicitly computing a correlation between the decoder output and the corresponding speech envelopes, may be susceptible to an eye-gaze bias. This bias refers to the tendency of the subject to subtly and often unknowingly direct their gaze towards the attended speaker. Given that EEG equipment can inadvertently capture these gaze patterns, it becomes possible to leverage this gaze information, whether intentionally or unintentionally, to enhance AAD performance. It's crucial to highlight that there is a relatively strong eye gaze bias in this dataset (such gaze bias is present in the majority of public AAD datasets). This eye-gaze overfitting effects is discussed in: Rotaru et al. "What are we really decoding? Unveiling biases in EEG-based decoding of the spatial focus of auditory attention", Journal of Neural Engineering, vol. 21, 016017, DOI: https://doi.org/10.1088/1741-2552/ad2214. Also available on bioRxiv: https://doi.org/10.1101/2023.07.13.548824 To test whether your method is not using gaze as a shortcut, use the Rotaru et al. data set available at <a href="https://zenodo.org/records/11058711">https://zenodo.org/records/11058711</a> *************************************** Explanation about the data set: This work was done at ExpORL, Dept. Neurosciences, KULeuven and Dept. Electrical Engineering (ESAT), KULeuven. This dataset contains EEG data collected from 16 normal-hearing subjects. EEG recordings were made in a soundproof, electromagnetically shielded room at ExpORL, KULeuven. The BioSemi ActiveTwo system was used to record 64-channel EEG signals at 8196 Hz sample rate. The audio signals, low pass filtered at 4 kHz, were administered to each subject at 60 dBA through a pair of insert phones (Etymotic ER3A). The experiments were conducted using the APEX 3 program developed at ExpORL [1]. Four Dutch short stories [2], narrated by different male speakers, were used as stimuli. All silences longer than 500 ms in the audio files were truncated to 500 ms. Each story was divided into two parts of approximately 6 minutes each. During a presentation, the subjects were presented with the six-minutes part of two (out of four) stories played simultaneously. There were two stimulus conditions, i.e., `HRTF' or `dry' (dichotic).  An experiment here is defined as a sequence of 4 presentations, 2 for each stimulus condition and ear of stimulation, with questions asked to the subject after each presentation. All subjects sat through three experiments within a single recording session. An example for the design of an experiment is shown in Table 1 in [3]. The first two experiments included four presentations each.  During a presentation, the subjects were instructed to listen to the story in one ear, while ignoring the story in the other ear. After each presentation, the subjects were presented with a set of multiple-choice questions about the story they were listening to in order to help them stay motivated to focus on the task. In the next presentation, the subjects were presented with the next part of the two stories. This time they were instructed to attend to their other ear. In this manner, one experiment involved four presentations in which the subjects listened to a total of two stories, switching attended ear between presentations. The second experiment had the same design but with two other stories. Note that the Table was different for each subject or recording session, i.e., each of the elements in the table were permuted between different recording sessions to ensure that the different conditions (stimulus condition and the attended ear) were equally distributed over the four presentations. Finally, the third experiment included a set of presentations where the first two minutes of the story parts from the first experiment, i.e., a total of four shorter presentations, were repeated three times, to build a set of recordings of repetitions. Thus, a total of approximately 72 minutes of EEG was recorded per subject.  We refer to EEG recorded from each presentation as a trial. For each subject, we recorded 20 trials - 4 from  the first experiment, 4 from the second experiment, and 12 from the third experiment (first 2 minutes of the 4 presentations from experiment 1 X 3 repetitions). The EEG data is stored in subject specific mat files of the format 'Sx', 'x' referring to the subject number. The audio data is stored as wav files in the folder 'stimuli'. Please note that the stories were not of equal lengths, and the subjects were allowed to finish listening to a story, even in cases where the competing story was over. Therefore, for each trial, we suggest referring to the length of the EEG recordings to truncate the ends of the corresponding audio data. This will ensure that the processed data (EEG and audio) contains only competing talker scenarios. Each trial was high-pass filtered  (0.5 Hz cut off) and downsampled from the recorded sampling rate of 8192 Hz to 128 Hz. Artifacts were removed using the MWF-filtering method in [4]. Please get in touch with the team (of Prof. Alexander Bertrand or Prof. Tom Francart) if you wish to obtain the raw EEG data (without the mentioned high-pass filtering and artifact removal). Each trial (trial*.mat) contains the following information:  RawData.Channels : channel numbers (1 to 64) RawData.EegData :   EEG data (samples X channels) FileHeader.SampleRate : Sampling frequency of the saved data TrialID : a number between 1 to 20, showing the trial number attended_ear : the direction of attention of the subject. 'L' for left, 'R' for right stimuli : cell array with stimuli{1} and stimuli{2} indicating the name of audio files presented in the left ear and the right ear of the subject respectively condition : stimulus presentation condition. 'HRTF' - stimuli were filtered with HRTF functions to simulate audio from 90 degrees to the left and 90 degrees to the right of the speaker, 'dry' - a dichotic presentation in which there was one story track each presented separately via the left and the right earphones. experiment : the number of the experiment (1,2 or 3) part : part of the story track being presented (can be 1 to 4 for experiments 1 and 2, and 1 to 12 for experiment 3) attended_track : the attended story track. '1' for track 1 and '2' for track 2. Each track maintains continuity of the story. In Experiment 1, attention is always to track 1, and in Experiment 2, attention is always to track 2.  repetition : binary variable indicating where the trial is a repetition (of presented stimuli) or not. subject : subject id of the format 'Sx', 'x' being the subject number. The 'stimuli' folder contains wav files of the format: part{part number}_track{track number}_{condition}.wav. Although the folder contains stimuli with HRTF filtering as well, for the analysis, we have assumed knowledge of the original clean stimuli (i.e. stimuli presented under the 'dry' condition), and hence envelopes were extracted only from part{part number}_track{tracknumber}_dry.wav files. The Matlab file 'preprocess_data.m' gives an example of how the synchronization and preprocessing of EEG and audio data can be done as described in [5]. Dependency: AMToolbox. This dataset has been used in [3, 5-14] (not updated anymore).  [1] Francart, T., Van Wieringen, A., & Wouters, J. (2008). APEX 3: a multi-purpose test platform for auditory psychophysical experiments. Journal of neuroscience methods, 172(2), 283-293. [2] Radioboeken voor kinderen, <a href="http://radioboeken.eu/kinderradioboeken.php?lang=NL">http://radioboeken.eu/kinderradioboeken.php?lang=NL</a>, 2007 (Accessed: 30 March 2015) [3] Das, N., Biesmans, W., Bertrand, A., & Francart, T. (2016). The effect of head-related filtering and ear-specific decoding bias on auditory attention detection. Journal of neural engineering, 13(5), 056014. [4] Somers, B., Francart, T., & Bertrand, A. (2018). A generic EEG artifact removal algorithm based on the multi-channel Wiener filter. Journal of neural engineering, 15(3), 036007. [5] Das, N., Vanthornhout, J., Francart, T., & Bertrand, A. (2019). Stimulus-aware spatial filtering for single-trial neural response and temporal response function estimation in high-density EEG with applications in auditory research. bioRxiv 541318; doi: <a href="https://doi.org/10.1101/541318">https://doi.org/10.1101/541318</a> [6] Biesmans, W., Das, N., Francart, T., & Bertrand, A. (2016). Auditory-inspired speech envelope extraction methods for improved EEG-based auditory attention detection in a cocktail party scenario. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 25(5), 402-412. [7] Das, N., Van Eyndhoven, S., Francart, T., & Bertrand, A. (2016, August). Adaptive attention-driven speech enhancement for EEG-informed hearing prostheses. In 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (pp. 77-80). IEEE. [8] Van Eyndhoven, S., Francart, T., & Bertrand, A. (2016). EEG-informed attended speaker extraction from recorded speech mixtures with application in neuro-steered hearing prostheses. IEEE Transactions on Biomedical Engineering, 64(5), 1045-1056. [9] Das, N., Van Eyndhoven, S., Francart, T., & Bertrand, A. (2017, August). EEG-based attention-driven speech enhancement for noisy speech mixtures using N-fold multi-channel Wiener filters. In 2017 25th European Signal Processing Conference (EUSIPCO) (pp. 1660-1664). IEEE. [10] Narayanan, A. M., & Bertrand, A. (2018, July). The effect of miniaturization and galvanic separation of EEG sensor devices in an auditory attention detection task. In 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (pp. 77-80). IEEE. [11] Deckers, L., Das, N., Ansari, A. H., Bertrand, A., & Francart, T. (2018). EEG-based detection of the attended speaker and the locus of auditory attention with convolutional neural networks. bioRxiv 475673; doi: <a href="https://doi.org/10.1101/475673">https://doi.org/10.1101/475673</a> [12] Narayanan, A. M., & Bertrand, A. (2019). Analysis of miniaturization effects and channel selection strategies for EEG sensor networks with application to auditory attention detection. IEEE Transactions on Biomedical Engineering. [13] Geirnaert, S., Francart, T., & Bertrand, A. A New Metric to evaluate auditory attention detection performance based on a Markov chain. Accepted for publication in Proc. European Signal Processing Conference (EUSIPCO), A Coruna, Spain, Sep. 2019. [14] Geirnaert, S., Francart,T., Bertrand A. (2019). An  Interpretable performance metric for auditory attention decoding algorithms in  a  context  of  neuro-steered  gain  control. bioRxiv 745695; doi: <a href="https://doi.org/10.1101/745695">https://doi.org/10.1101/745695</a>

opencc-by-4.0Oct 2022View details →

Auditory Selective Attention Switch in a Virtual Reality Classroom Environment

General The audio-visual Auditory Selective Attention VR Proof of Concept (asaVRpoc) project serves to investigate the auditory selective attention switch in a close-to-real-life classroom setting. This dataset consists of a Unity project and Matlab code used to collect data on the voluntary switching of auditory selective attention in a virtual reality classroom scenario. The dataset contains: <ul> <li>Unity project for visual display and the experiment structure</li> <li>Matlab code for experiment preparation and HpFT measurement</li> <li>Data collected in the experiment (experiment performance, head tracking, questionnaires)</li> </ul> Experiment preparation using Matlab The code and software used to prepare the experiment is provided in the folder "matlab_asaVRpoc". The Matlab code used to prepare the trials for each participant as well as to measure the HpTFs. For the HpTF measurements, the  ITA Toolbox for Matlab was used and is provided (https://git.rwth-aachen.de/ita/toolbox commit hash: 598675ef704c178365f53d41e03ff4b11dea390f). A developmental version of Virtual acoustics (VA) 2020b (https://www.virtualacoustics.org/VA/overview/) is provided. Software requirements: <ul> <li>Matlab 2019a or higher</li> <li>ITA Toolbox for Matlab installed</li> </ul>   Experiment conduction in Unity The Unity project is provided in the folder "unity_pc_asaVRpoc". Therefore, a virtual classroom with some basic furniture is provided. The used models, prefabs and plugins can be found in the Assets folder. Note that the acoustic stimuli are NOT provided with this Unity project. The stimuli are available on request from the Institute for Hearing Technology and Acoustics, RWTH Aachen University. This Unity project was intended for the use in virtual reality using an HMD and respective controllers for input. However, it can also be used on a desktop pc. The mode can be changed using the "VRMode" toggle as described below. The audio reproduction is realized using the Unity plugin for Virtual Acoustics (VA, http://www.virtualacoustics.org/). Software requirements: <ul> <li>Unity 2019.4.21.f1.</li> <li>SteamVR 1.19.7</li> <li>Virtual Acoustics v2021a, VAUnity: https://git.rwth-aachen.de/ita/VAUnity</li> </ul>   Data evaluation The collected data is provided in the folder "dataEvaluation_asaVRpoc". This folder contains three types of data: the raw data collected in the experiment (reaction times and error rates), the head tracking data and responses from the simulator sickness questionnaire (before and after the experiment) and the igroup presence questionnaire (after the experiment). Matlab code for the evaluation of the head tracking data and the questionnaires is provided.

opencc-by-sa-4.0Apr 2024View details →

Audiovisual, Gaze-controlled Auditory Attention Decoding Dataset KU Leuven (AV-GC-AAD)

This dataset is described in detail in the following journal paper [1]: Rotaru, I., Geirnaert, S., Heintz, N., Van de Ryck, I., Bertrand, A., & Francart, T. (2024). What are we really decoding? Unveiling biases in EEG-based decoding of the spatial focus of auditory attention. Journal of Neural Engineering, 21(1), 016017. <a href="https://iopscience.iop.org/article/10.1088/1741-2552/ad2214/meta">https://iopscience.iop.org/article/10.1088/1741-2552/ad2214/meta</a> If using this dataset, please cite the original paper above and the current Zenodo repository. Note from the authors: Recent evaluations reveal that various published AAD (Auditory Attention Decoding) algorithms do not achieve significant above-chance performance on this AV-GC-AAD dataset, and in particular on the two gaze-incongruent conditions 'MovingVideo' and 'MovingTargetNoise'). This suggests that previously reported successes may have been largely influenced by eye gaze confounds present in other datasets, which can be exploited as shortcuts by machine learning algorithms. Despite these findings, poor performance on the AV-GC-AAD dataset is often dismissed, with reasons cited such as insufficient training data, high heterogeneity in audiovisual conditions, or the claim that participants were unable to focus their auditory attention due to the complexity of the instructions. To address these concerns, we provide a supplementary technical report (and accompanying code), showcasing results from a simple linear stimulus reconstruction AAD algorithm applied to this dataset. Our findings demonstrate that high AAD accuracy can be achieved within individual conditions, and that the model generalizes across conditions, new subjects, and even across different datasets. <a title="https://doi.org/10.48550/arxiv.2412.01401" href="https://doi.org/10.48550/arXiv.2412.01401" target="_blank" rel="noreferrer noopener">Report</a> | <a title="https://github.com/alexanderbertrandlab/linear-stimulus-reconstruction-aad-av-gc-aad-dataset" href="https://github.com/AlexanderBertrandLab/linear-stimulus-reconstruction-AAD-AV-GC-AAD-dataset" target="_blank" rel="noreferrer noopener">Matlab Code</a> Through this report, we aim to remove any doubts that the AV-GC-AAD dataset's limitations are the primary cause of AAD algorithms failing to exceed chance-level performance. Additionally, this report and its accompanying code offer a simple baseline evaluation procedure, which can serve as a minimal benchmark for testing more advanced AAD algorithms on this dataset. When reporting results on this data set, it is good practice to show performance for each condition separately, since 2 of the 4 conditions still contain gaze shortcuts, which could be exploited by machine learning algorithms.  ________________________________________________________________________________ Dataset description This work was performed at ExpORL, Dept. Neurosciences, KU Leuven and Dept. Electrical Engineering (ESAT), KU Leuven (Belgium), with the goal of investigating and controlling for the effect of gaze during a competing listening task. The full dataset contains EEG and EOG data collected from 16 normal-hearing subjects, during a competing listening task, where the subjects were instructed to focus on one of two competing speech signals. However, subjects 2, 5 and 6 were excluded from the online repository due to not consenting to sharing their data in a public database (cf. signed informed consents approved by KU Leuven Ethical Committee). EEG recordings were conducted in a soundproof, electromagnetically shielded room at ExpORL, KU Leuven. The BioSemi ActiveTwo system was used to record 64-channel EEG signals at 8196 Hz sample rate. Additionally, the participants' gaze movements were measured via 4 EOG (electrooculography) electrodes placed symmetrically around the eyes.  The audio signals were administered to each subject at 65 dB SPL through a pair of insert phones (Etymotic ER10). In some experimental trials, the video depicting the attended talker was also presented on the screen. The original presented speech and video stimuli (.wav and .mp4 files) are excluded from the dataset due to copyrights. However, the acoustic envelopes of the attended and unattended audio stimuli are calculated and included in the dataset (see below).  The experiments were conducted using custom-made Python scripts. The experimental trials were split into 2 blocks. Each block consisted of the following sequence of conditions: MovingVideo, MovingTargetNoise, NoVisuals, StaticVideo. The auditory task was the same for all conditions: the subjects had to attend to one of the two presented talkers, as indicated by an arrow on the screen. The visual task differed across conditions: <ul> <li>MovingVideo: the subjects had to follow the moving video of the to-be-attended speaker presented on a randomized horizontal trajectory on the screen.</li> <li>MovingTargetNoise: the subjects had to follow a moving cross-hair presented on a randomized horizontal trajectory on the screen.</li> <li>NoVisuals: a black screen was presented and the subjects had to fixate on an imaginary point in the center of the screen while minimizing the eye movements.</li> <li>StaticVideo: the subjects had to fixate the static video of the to-be-attended speaker presented on the same side with the audio stimulus of the attended speaker.</li> </ul> The full description of all experimental conditions can be consulted in [1]. Each trial/condition lasted for 10 minutes, with a spatial switch in attention after 5 minutes (i.e., the  presented speech stimuli  were programmed to swap sides - from L to R or vice versa, such that after the switch the subjects kept listening to the same speaker, but coming from the opposite spatial location). This means that the participant kept attending to the same speaker throughout an entire trial. To keep the subjects motivated, they had to answer one comprehension question related to the attended acoustic stimulus after each trial. For each subject, there is a .mat file containing the following variables: conditionID: the condition ID for each trial  data: the preprocessed EEG and EOG data for each trial (first 64 channels are EEG, last 4 are EOG) fs: the sampling rate of the EEG, EOG and stimuli envelopes initAttention: the initial spatial location of the attended stimulus for each trial metadata: the original metadata (e.g. channel names, triggers) saved in the raw .bdf files for each trial params: the filtering parameters used for each trial randomization: the randomization parameters (e.g. presented stimuli, attention switch times etc.) for each trial stimulus: the precalculated envelopes for the attended and unattended stimuli for each trial subjID: the anonymised ID of the current subject Preprocessing EEG and EOG All the following preprocessing steps were applied per trial. The EEG was initially downsampled using an antialiasing filter from 8192 Hz to 256 Hz. The data was then filtered between 1–40 Hz using a zero-phase Chebyshev filter (type II, with 80 dB attenuation at 10% outside the passband). Finally, downsampling to 128 Hz was performed to speed up computation. Speech envelopes extraction The original speech signals at 44100 Hz were downsampled to 8192 Hz (to match the EEG sampling rate). They were then passed through a gammatone filterbank, which roughly approximates the spectral decomposition as performed by the human auditory system. Per subband, the audio envelopes were extracted, and their dynamic range was compressed using a power-law operation with exponent 0.6 (as proposed in [2]). Each subband was then bandpass-filtered with the same filter used for the EEG data. The resulting subband envelopes were then summed to construct a single broadband envelope. Finally, the envelope signals were downsampled to 128 Hz to match the sampling rate of the preprocessed EEG. Notes <ol> <li>For subjects 1-3, 6 trials corresponding to 3 conditions (MovingVideo, NoVisuals, StaticVideo) were measured.</li> <li>For subjects 4-16, 8 trials corresponding to 4 conditions (MovingVideo, MovingTargetNoise, NoVisuals, StaticVideo) were measured.</li> <li>For subject 14, trial 2 from the StaticVideo condition was not recorded due to some technical problems.</li> <li>In the dataset, 'FixedVideo' is the alias name for the 'StaticVideo' condition described in [1].</li> <li>The EEG/EOG data was not referenced. Before further analysis, rereferencing the data (e.g., to an arbitrary EEG channel, or the common-average of all channels) is necessary to achieve a better common-mode rejection and thus increase the SNR of recorded data. (for details, see https://www.biosemi.com/faq/cms&drl.htm)</li> </ol> References [1] Rotaru, Iustina, et al. "What are we really decoding? Unveiling biases in EEG-based decoding of the spatial focus of auditory attention." Journal of Neural Engineering 21.1 (2024): 016017. [2] Biesmans, Wouter, et al. "Auditory-inspired speech envelope extraction methods for improved EEG-based auditory attention detection in a cocktail party scenario." IEEE Transactions on neural systems and rehabilitation engineering 25.5 (2016): 402-412.

opencc-by-nc-4.0Mar 2018View details →

EEG and audio dataset for auditory attention decoding

This dataset contains EEG recordings from 18 subjects listening to one of two competing speech audio streams. Continuous speech in trials of ~50 sec. was presented to normal hearing listeners in simulated rooms with different degrees of reverberation. Subjects were asked to attend one of two spatially separated speakers (one male, one female) and ignore the other. Repeated trials with presentation of a single talker were also recorded. The data were recorded in a double-walled soundproof booth at the Technical University of Denmark (DTU) using a 64-channel Biosemi system and digitized at a sampling rate of 512 Hz. Full details can be found in: <ul> <li>Søren A. Fuglsang, Torsten Dau & Jens Hjortkjær (2017): Noise-robust cortical tracking of attended speech in real-life environments. NeuroImage, 156, 435-444</li> </ul> and <ul> <li>Daniel D.E. Wong, Søren A. Fuglsang, Jens Hjortkjær, Enea Ceolini, Malcolm Slaney & Alain de Cheveigné: A Comparison of Temporal Response Function Estimation Methods for Auditory Attention Decoding. Frontiers in Neuroscience, <a href="https://doi.org/10.3389/fnins.2018.00531">https://doi.org/10.3389/fnins.2018.00531</a></li> </ul> The data is organized in format of the publicly available <a href="https://zenodo.org/record/1198430">COCOHA Matlab Toolbox</a>. The preproc_script.m demonstrates how to import and align the EEG and audio data. The script also demonstrates some EEG preprocessing steps as used the Wong et al. paper above. The AUDIO.zip contains wav-files with the speech audio used in the experiment. The EEG.zip contains MAT-files with the EEG/EOG data for each subject. The EEG/EOG data are found in data.eeg with the following channels: <ul> <li>channels 1-64: scalp EEG electrodes</li> <li>channel 65: right mastoid electrode</li> <li>channel 66: left mastoid electrode</li> <li>channel 67: vertical EOG below right eye</li> <li>channel 68: horizontal EOG right eye</li> <li>channel 69: vertical EOG above right eye</li> <li>channel 70: vertical EOG below left eye</li> <li>channel 71: horizontal EOG left eye</li> <li>channel 72: vertical EOG above left eye</li> </ul> The expinfo table contains information about experimental conditions, including what what speaker the listener was attending to in different trials. The expinfo table contains the following information: <ul> <li>attend_mf: attended speaker (1=male, 2=female)</li> <li>attend_lr: spatial position of the attended speaker (1=left, 2=right)</li> <li>acoustic_condition: type of acoustic room (1= anechoic, 2= mild reverberation, 3= high reverberation, see Fuglsang et al. for details)</li> <li>n_speakers: number of speakers presented (1 or 2)</li> <li>wavfile_male: name of presented audio wav-file for the male speaker</li> <li>wavfile_female: name of presented audio wav-file for the female speaker (if any)</li> <li>trigger: trigger event value for each trial also found in data.event.eeg.value</li> </ul> DATA_preproc.zip contains the preprocessed EEG and audio data as output from preproc_script.m. The dataset was created within the <a href="https://cocoha.org/">COCOHA</a><a href="https://cocoha.org/"> Project</a>: Cognitive Control of a Hearing Aid

opencc-by-4.0Sep 2022View details →

ESAA: an EEG-Speech auditory attention detection database

We build a database for AAD research, which consists of competing speech stimuli and associated human neural responses, i.e, electroencephalography (EEG) recordings, namely EEG-Speech AAD (ESAA) database.  This is the first AAD database with speech stimuli in a tonal language (Mandarin). Moreover, we develop an AAD baseline as a reference model for decoding which speech stream a listening subject is attending to (speaker attention detection), and a baseline for decoding which spatial locus a listening subject is attending to (speaker locus attention detection) on the ESAA database. We release the source code and the database for use in research purpose. This database consists of response data for 17 normal-hearing subjects (S1-S17). It includes: - 64-channel EEG data: responses to two-speaker speech stimuli - Auditory stimuli data (clean): Chinese short stories narrated by a female and a male professional story teller.  - Auditory stimuli data (hrtf): Auditory stimuli after head-related transfer function (HRTF) filtering (simulating sound coming from ± 90 deg). - Preprocessing code - AAD baseline (CNN model)

opencc-by-4.0Apr 2018View details →

Auditory stream segregation and selective attention for cochlear implant listeners: Evidence from behavioral measures and event-related potentials

Data set generated for the study "Auditory stream segregation and selective attention for cochlear implant listeners: Evidence from behavioral measures and event-related potentials"  <ol> <li>behavioral.txt: d' scores obtained by the listeners on the deviant detection task. <ul> <li>subject: listener ID</li> <li>distractor: Electrode separation condition</li> <li>deviant: Deviant triplet</li> <li>d: d' scores</li> <li>exp: experimental session (BEH / ERP)</li> </ul> </li> <li>ERP_by_condition.txt: <ul> <li>Subject: listener ID</li> <li>Type: Sound type (Target / Distractor)</li> <li>Dev: Deviant condition. Early = deviant triplets 1 or 2. Late = deviant triplet 3 or none.</li> <li>rep: Triplet number</li> <li>sound: sound number within the triplet</li> <li>amplitude: amplitude difference between the active and the passive listening conditions.</li> </ul> </li> </ol>

opencc-by-4.0Nov 2018View details →

Stimuli examples for Object-based attention in complex, naturalistic auditory streams

This is an example collection of stimuli used to study Object-based attention in complex, naturalistic auditory streams. The stimuli consist of single auditory streams of speech signals and environmental sounds signals that can contain a repetition of 750ms (starting name "target")or not (starting name "clean"). Two sounds stimuli, one with repetition and one without were randomly overlapped to form an auditory scene of two speech streams (starting name "sceneSS") or one speech and one environmental stream (starting name "sceneSE").

opencc-by-4.0Oct 2022View details →

Auditory Attention Detection Dataset Nanjing University

Auditory Attention Detection Dataset The experiment is conducted by Yuanming, Zhang and Ziyan, Yuan, supervised by Jing, Lu.   Two zip files are included in this dataset. The script.zip file contains several scripts to divide each trial into several parts. These scripts are useful if you want to split the dataset into training, validation and test set. The other zip file contains the preprocessed EEG and audio signals. (Not the raw EEG!) Each mat file contains EEG and audio signals from one subject. Their are two variables named expinfo and data. The expinfo contains the attended direction information and audio stimulus playback information of each trial of the experiments. The azimuth column in expinfo indicates the directions of the two competing speakers. The negative directions indicated the speaker coming from the left, and positive coming from the right. The data contains several essential fields relative to the EEG and audio signals. The data.eeg field is a cell array with 32 cell elements. Each cell contains the EEG signals in a single trial. And the index of the cell is the number of trial. The data.leftWav and data.rightWav field are original audio stimuli, and the data.leftEnv and data.rightEnv are the envelopes of the data.leftWav and data.rightWav respectively. The data.dim field tells you the physical meaning of each dimension of each signal, and the data.dim.chan.eeg contains the name of each channel of the EEG signal.

opencc-by-4.0Sep 2022View details →

EEG Dataset for 'Decoding of selective attention to continuous speech from the human auditory brainstem response' and 'Neural Speech Tracking in the Theta and in the Delta Frequency Band Differentially Encode Clarity and Comprehension of Speech in Noise'.

The repository contains the unprocessed EEG data recorded for the publications [1, 2]. For convenience, the onsets of the EEG data provided here are time-aligned with the onsets of the audio books in the 'audiobooks' folder, and the EEG data are provided in HDF5 format. Please refer to the original version of this dataset for more details. More details, as well as the original data files, are available at the original repository <a href="https://doi.org/10.5281/zenodo.7086209">here</a>. Examples of using these data (preprocessing, fitting linear models) can be found <a href="https://github.com/Mike-boop/trf-examples">here</a>. The English conditions (clean, lb, mb, hb, fM, fW) comprised a single recording session. The Dutch conditions (cleanDutch, lbDutch, mbDutch, hbDutch) comprised a separate recording session. You see which participants took part in each session in session_info.json. Please note some details about the stimulus presentation for the various listening conditions: <ul> <li>English speech-in-babble-noise (lb, mb, hb): babble noise was played by itself for one second before the audiobook track began. The babble noise was also played for one second after the audiobook track ended. Therefore, you should discard the first second and the last second from these trial during your analysis.</li> <li>Dutch speech-in-babble-noise (lbDutch, mbDutch, hbDutch): the story (narrated in Dutch) was played by itself for one second before the babble noise track began. Then, the babble noise was increased linearly in amplitude for one second. Therefore, you should discard the first two seconds from these trials during your analysis.</li> <li>Dutch in quiet, and Dutch-in-babble-noise (cleanDutch, lbDutch, mbDutch, hbDutch): some English sentences were embedded in the Dutch narratives in order to encourage attention. You should crop these from your analysis. The onsets and offsets of the English sentences (in samples, at 44100Hz) are provided in the audiobooks/*Dutch/english_onsets_info.json files.</li> <li>Competing-speakers conditions (fM, fW): sometimes the attended track is longer than the unattended track, or vice-versa. The onsets of both tracks are aligned. You should crop the trial to the length of the shortest track for your analysis.</li> </ul> If you use this data, please cite the original publications, as well as this repository [1,2,3]. [1] Etard O, Kegler M, Braiman C, Forte A E and Reichenbach T. “Decoding of selective attention to continuous speech from the human auditory brainstem response” 2019. NeuroImage 200 1–11 [2] Etard O and Reichenbach T. “Neural speech tracking in the theta and in the delta frequency band differentially encode clarity and comprehension of speech in noise” 2019. J. Neurosci. 39 5750–9 [3] Etard O and Reichenbach T. "EEG Dataset for 'Decoding of selective attention to continuous speech from the human auditory brainstem response' and 'Neural Speech Tracking in the Theta and in the Delta Frequency Band Differentially Encode Clarity and Comprehension of Speech in Noise". Doi: 10.5281/zenodo.7086208

zenodo36/100

Selective auditory attention in normal-hearing and hearing-impaired listeners

This repository contains the EEG and behavioral data described in: Fuglsang, S A, Märcher-Rørsted, J, Dau, T, Hjortkjær, J (2020). Effects of sensorineural hearing loss on cortical synchronization to competing speech during selective attention. Journal of Neuroscience, 40(12):2562–2572, <a href="https://doi.org/10.1523/JNEUROSCI.1936-19.2020">https://doi.org/10.1523/JNEUROSCI.1936-19.2020</a>  Please cite this paper when using the data The data set consists of response data for 22 hearing-impaired and 22 normal-hearing participants. It includes: - EEG data: responses to two-talker and single-talker speech stimuli - Envelopes of the corresponding speech audio - EEG data: responses to 1 kHz tone beeps for ERPs - EEG data: responses to periodic tone sequences for Envelope-following responses (EFRs) - EEG resting-state data recorded with eyes-open and eyes-closed - inEar EEG data for 19 of the 44 subjects (EEG recorded inside the ear canals) - Behavioral data: speech comprehension scores, task difficulty ratings, speech-in-noise scores (SRTs), tone-in-noise scores, digit span working memory scores, SSQ questionnaire ratings - Pure-tone audiograms For more information, see the README and 'dataset_description.json' file. Format ------ The dataset is formatted according to BIDS version 1.3.0 and the BIDS standard extension for EEG (BEP006) that has been merged in the main body of the specification. For more details, see https://bids-specification.readthedocs.io/en/latest/06-extensions.html Behavioural data are stored in the 'participants.tsv' file. Task-difficulty ratings and multiple choice questionnaire data from the selective attention experiment are stored in the events files (see 'task-selectiveattention_events.json').    Code ------ Code for analyzing the data is available at: https://gitlab.com/sfugl/snhl   Audio ------ Envelopes of the audio signals are included in the data set. For inquiries regarding the raw audio data, please send an email to jensh@drcmr.dk with the subject line "ds-eeg-snhl audio".   Acknowledgments ---------- This work was supported by the EU H2020-ICT grant number 644732 (COCOHA: Cognitive Control of a Hearing Aid) and by the Novo Nordisk Foundation synergy grant NNF17OC0027872 (UHeal). The EarEEG were kindly provided by Eriksholm Research Centre.

opencc-by-4.0Jan 2020View details →

zenodo36/100

Investigating the Auditory Selective Attention Switch using Matrix Sentences in VR

<h2>General</h2> The audio-visual Auditory Selective Attention VR - Long Stimuli (avASAlongStimuli) project serves to investigate the extensions by matrix sentences of a paradigm on the auditory selective attention switch in a close-to-real-life classroom setting. This dataset consists of a Unity project and Matlab code used to collect data on the voluntary switching of auditory selective attention in a virtual reality classroom scenario. The dataset contains:     Unity project for audiovisual display and the experiment structure     Matlab code for experiment preparation and HpFT measurement     Data collected in the experiment (experiment performance, head tracking) <h2>Experiment preparation using Matlab</h2> The code and software used to prepare the experiment is provided in the folder "matlab_avASAlongStimuli". The Matlab code used to prepare the trials for each participant as well as to measure the HpTFs. For the HpTF measurements, the  ITA Toolbox for Matlab was used and is provided (https://git.rwth-aachen.de/ita/toolbox). A developmental version of Virtual acoustics (VA) 2022a (https://www.virtualacoustics.org/VA/overview/) is provided. Software requirements:     Matlab 2020a or higher     ITA Toolbox for Matlab installed <h2> Experiment conduction in Unity</h2> The Unity project is provided in the folder "unity_avASAlongStimuli". Therefore, a virtual classroom with some basic furniture is provided. The used models, prefabs and plugins can be found in the Assets folder. The acoustic stimuli for the task were created using Voicemaker (https://voicemaker.in/). This Unity project was intended for the use in virtual reality using an HMD and respective controllers for input. However, it can also be used on a desktop pc. The mode can be changed using the "VRMode" toggle as described below. The audio reproduction is realized using the Unity plugin for Virtual Acoustics (VA, http://www.virtualacoustics.org/ and https://git.rwth-aachen.de/ita/vaunity_package). Software requirements:     Unity 2019.4.21.f1.     SteamVR 1.19.7     Virtual Acoustics v2022a, VAUnity: https://git.rwth-aachen.de/ita/VAUnity <h2> Data evaluation</h2> The collected data is provided in the folder "dataEvaluation_avASAlongStimuli". This folder contains two types of data: the raw data collected in the experiment (reaction times and error rates), the head tracking data.

opencc-by-4.0Sep 2024View details →

zenodo36/100

Investigating the Impact of Visual Stimuli on the Auditory Selective Attention in a VR Classroom

<h2>General</h2> The audio-visual Auditory Selective Attention - visual Priming (avASAvisPrim) project serves to investigate the impact of visual stimuli on auditory selective attention switch in a close-to-real-life classroom setting. This dataset consists of a Unity project and Matlab code used to collect data as well as the collected data and R scripts used for evaluation. The dataset contains:     Unity project for audiovisual display and the experiment structure     Matlab code for experiment preparation and HpFT measurement     Data collected in the experiment (experiment performance)     R code for data evaluation <h2>Experiment preparation using Matlab</h2> The code and software used to prepare the experiment is provided in the folder "matlab_avASAvisPrim". The Matlab code used to prepare the trials for each participant as well as to measure the HpTFs. For the HpTF measurements, the  ITA Toolbox for Matlab was used and is provided (https://git.rwth-aachen.de/ita/toolbox). A developmental version of Virtual acoustics (VA) 2021a (https://www.virtualacoustics.org/VA/overview/) is provided. Software requirements:     Matlab 2020a or higher     ITA Toolbox for Matlab installed <h2> Experiment conduction in Unity</h2> The Unity project is provided in the folder "unity_avASAvisPrim". Therefore, a virtual classroom with some basic furniture is provided. The used models, prefabs and plugins can be found in the Assets folder. The acoustic stimuli for the task are taken from Loh and Fels 2023 "ChildASA dataset: Speech and Noise Material fpr Child-appropriate Paradigms on Auditory Selective Attention" https://doi.org/10.18154/RWTH-2023-00740.  This Unity project was intended for the use in virtual reality using an HMD and respective controllers for input. However, it can also be used on a desktop pc. The mode can be changed using the "VRMode" toggle as described below. The audio reproduction is realized using the Unity plugin for Virtual Acoustics (VA, http://www.virtualacoustics.org/ and https://git.rwth-aachen.de/ita/vaunity_package). Software requirements:     Unity 2019.4.21.f1.     SteamVR 1.19.7     Virtual Acoustics v2021a, VAUnity: https://git.rwth-aachen.de/ita/VAUnity <h2> Data evaluation</h2> The collected data is provided in the folder "dataEvaluation_avASAvisPrim". This folder contains three types of data: the raw data collected in the experiment (reaction times and error rates). R code for the evaluation of the head tracking data and the questionnaires is provided.

opencc-by-4.0Aug 2024View details →

opencc-zeroJul 2020View details →

Data from: Vestigial auriculomotor activity indicates the direction of auditory attention in humans

Unlike dogs and cats, people do not point their ears as they focus attention on novel, salient, or task-relevant stimuli. Our species may nevertheless have retained a vestigial pinna-orienting system that has persisted as a "neural fossil" within in the brain for about 25 million years. Consistent with this hypothesis, we demonstrate that the direction of auditory attention is reflected in sustained electrical activity of muscles within the vestigial auriculomotor system. Surface electromyograms (EMGs) were taken from muscles that either move the pinna or alter its shape. To assess reflexive, stimulus-driven attention we presented novel sounds from speakers at four different lateral locations while the participants silently read a boring text in front of them. To test voluntary, goal-directed attention we instructed participants to listen to a short story coming from one of these speakers, while ignoring a competing story from the corresponding speaker on the opposite side. In both experiments, EMG recordings showed larger activity at the ear on the side of the attended stimulus, but with slightly different patterns. Upward movement (perking) differed according to the lateral focus of attention only during voluntary orienting; rearward folding of the pinna's upper-lateral edge exhibited such differences only during reflexive orienting. The existence of a pinna-orienting system in humans, one that is experimentally accessible, offers opportunities for basic as well as applied science.

opencc-zeroNov 2019View details →

Causal links between parietal alpha activity and spatial auditory attention

Both visual and auditory spatial selective attention result in lateralized alpha (8-14 Hz) oscillatory power in parietal cortex: alpha increases in the hemisphere ipsilateral to attentional focus. Brain stimulation studies suggest a causal relationship between parietal alpha and suppression of the representation of contralateral visual space. However, there is no evidence that parietal alpha controls auditory spatial attention. Here, we performed high definition transcranial alternating current stimulation (HD-tACS) on human subjects performing an auditory task in which they directed attention based on either spatial or nonspatial features. Alpha (10 Hz) but not theta (6 Hz) HD-tACS of right parietal cortex interfered with attending left but not right auditory space. Parietal stimulation had no effect for nonspatial auditory attention. Moreover, performance in post-stimulation trials returned rapidly to baseline. These results demonstrate a causal, frequency-, hemispheric-, and task-specific effect of parietal alpha brain stimulation on top-down control of auditory spatial attention.

ClinicalTrials.gov32/100

Brain Imaging, Attention, and Auditory Processing in Schizophrenia

ClinicalTrials.gov study NCT03068806. IPD Sharing: NO. Countries: 1. Publications: 41.

closedIPD-NOFeb 2026View details →

publicNov 2019View details →

Causal links between parietal alpha activity and spatial auditory attention

Open the record for dataset details and reuse information.