Find research datasets worth reusing

opencc-by-sa-4.0Apr 2024View details →

Audiovisual, Gaze-controlled Auditory Attention Decoding Dataset KU Leuven (AV-GC-AAD)

This dataset is described in detail in the following journal paper [1]: Rotaru, I., Geirnaert, S., Heintz, N., Van de Ryck, I., Bertrand, A., & Francart, T. (2024). What are we really decoding? Unveiling biases in EEG-based decoding of the spatial focus of auditory attention. Journal of Neural Engineering, 21(1), 016017. <a href="https://iopscience.iop.org/article/10.1088/1741-2552/ad2214/meta">https://iopscience.iop.org/article/10.1088/1741-2552/ad2214/meta</a> If using this dataset, please cite the original paper above and the current Zenodo repository. Note from the authors: Recent evaluations reveal that various published AAD (Auditory Attention Decoding) algorithms do not achieve significant above-chance performance on this AV-GC-AAD dataset, and in particular on the two gaze-incongruent conditions 'MovingVideo' and 'MovingTargetNoise'). This suggests that previously reported successes may have been largely influenced by eye gaze confounds present in other datasets, which can be exploited as shortcuts by machine learning algorithms. Despite these findings, poor performance on the AV-GC-AAD dataset is often dismissed, with reasons cited such as insufficient training data, high heterogeneity in audiovisual conditions, or the claim that participants were unable to focus their auditory attention due to the complexity of the instructions. To address these concerns, we provide a supplementary technical report (and accompanying code), showcasing results from a simple linear stimulus reconstruction AAD algorithm applied to this dataset. Our findings demonstrate that high AAD accuracy can be achieved within individual conditions, and that the model generalizes across conditions, new subjects, and even across different datasets. <a title="https://doi.org/10.48550/arxiv.2412.01401" href="https://doi.org/10.48550/arXiv.2412.01401" target="_blank" rel="noreferrer noopener">Report</a> | <a title="https://github.com/alexanderbertrandlab/linear-stimulus-reconstruction-aad-av-gc-aad-dataset" href="https://github.com/AlexanderBertrandLab/linear-stimulus-reconstruction-AAD-AV-GC-AAD-dataset" target="_blank" rel="noreferrer noopener">Matlab Code</a> Through this report, we aim to remove any doubts that the AV-GC-AAD dataset's limitations are the primary cause of AAD algorithms failing to exceed chance-level performance. Additionally, this report and its accompanying code offer a simple baseline evaluation procedure, which can serve as a minimal benchmark for testing more advanced AAD algorithms on this dataset. When reporting results on this data set, it is good practice to show performance for each condition separately, since 2 of the 4 conditions still contain gaze shortcuts, which could be exploited by machine learning algorithms.  ________________________________________________________________________________ Dataset description This work was performed at ExpORL, Dept. Neurosciences, KU Leuven and Dept. Electrical Engineering (ESAT), KU Leuven (Belgium), with the goal of investigating and controlling for the effect of gaze during a competing listening task. The full dataset contains EEG and EOG data collected from 16 normal-hearing subjects, during a competing listening task, where the subjects were instructed to focus on one of two competing speech signals. However, subjects 2, 5 and 6 were excluded from the online repository due to not consenting to sharing their data in a public database (cf. signed informed consents approved by KU Leuven Ethical Committee). EEG recordings were conducted in a soundproof, electromagnetically shielded room at ExpORL, KU Leuven. The BioSemi ActiveTwo system was used to record 64-channel EEG signals at 8196 Hz sample rate. Additionally, the participants' gaze movements were measured via 4 EOG (electrooculography) electrodes placed symmetrically around the eyes.  The audio signals were administered to each subject at 65 dB SPL through a pair of insert phones (Etymotic ER10). In some experimental trials, the video depicting the attended talker was also presented on the screen. The original presented speech and video stimuli (.wav and .mp4 files) are excluded from the dataset due to copyrights. However, the acoustic envelopes of the attended and unattended audio stimuli are calculated and included in the dataset (see below).  The experiments were conducted using custom-made Python scripts. The experimental trials were split into 2 blocks. Each block consisted of the following sequence of conditions: MovingVideo, MovingTargetNoise, NoVisuals, StaticVideo. The auditory task was the same for all conditions: the subjects had to attend to one of the two presented talkers, as indicated by an arrow on the screen. The visual task differed across conditions: <ul> <li>MovingVideo: the subjects had to follow the moving video of the to-be-attended speaker presented on a randomized horizontal trajectory on the screen.</li> <li>MovingTargetNoise: the subjects had to follow a moving cross-hair presented on a randomized horizontal trajectory on the screen.</li> <li>NoVisuals: a black screen was presented and the subjects had to fixate on an imaginary point in the center of the screen while minimizing the eye movements.</li> <li>StaticVideo: the subjects had to fixate the static video of the to-be-attended speaker presented on the same side with the audio stimulus of the attended speaker.</li> </ul> The full description of all experimental conditions can be consulted in [1]. Each trial/condition lasted for 10 minutes, with a spatial switch in attention after 5 minutes (i.e., the  presented speech stimuli  were programmed to swap sides - from L to R or vice versa, such that after the switch the subjects kept listening to the same speaker, but coming from the opposite spatial location). This means that the participant kept attending to the same speaker throughout an entire trial. To keep the subjects motivated, they had to answer one comprehension question related to the attended acoustic stimulus after each trial. For each subject, there is a .mat file containing the following variables: conditionID: the condition ID for each trial  data: the preprocessed EEG and EOG data for each trial (first 64 channels are EEG, last 4 are EOG) fs: the sampling rate of the EEG, EOG and stimuli envelopes initAttention: the initial spatial location of the attended stimulus for each trial metadata: the original metadata (e.g. channel names, triggers) saved in the raw .bdf files for each trial params: the filtering parameters used for each trial randomization: the randomization parameters (e.g. presented stimuli, attention switch times etc.) for each trial stimulus: the precalculated envelopes for the attended and unattended stimuli for each trial subjID: the anonymised ID of the current subject Preprocessing EEG and EOG All the following preprocessing steps were applied per trial. The EEG was initially downsampled using an antialiasing filter from 8192 Hz to 256 Hz. The data was then filtered between 1–40 Hz using a zero-phase Chebyshev filter (type II, with 80 dB attenuation at 10% outside the passband). Finally, downsampling to 128 Hz was performed to speed up computation. Speech envelopes extraction The original speech signals at 44100 Hz were downsampled to 8192 Hz (to match the EEG sampling rate). They were then passed through a gammatone filterbank, which roughly approximates the spectral decomposition as performed by the human auditory system. Per subband, the audio envelopes were extracted, and their dynamic range was compressed using a power-law operation with exponent 0.6 (as proposed in [2]). Each subband was then bandpass-filtered with the same filter used for the EEG data. The resulting subband envelopes were then summed to construct a single broadband envelope. Finally, the envelope signals were downsampled to 128 Hz to match the sampling rate of the preprocessed EEG. Notes <ol> <li>For subjects 1-3, 6 trials corresponding to 3 conditions (MovingVideo, NoVisuals, StaticVideo) were measured.</li> <li>For subjects 4-16, 8 trials corresponding to 4 conditions (MovingVideo, MovingTargetNoise, NoVisuals, StaticVideo) were measured.</li> <li>For subject 14, trial 2 from the StaticVideo condition was not recorded due to some technical problems.</li> <li>In the dataset, 'FixedVideo' is the alias name for the 'StaticVideo' condition described in [1].</li> <li>The EEG/EOG data was not referenced. Before further analysis, rereferencing the data (e.g., to an arbitrary EEG channel, or the common-average of all channels) is necessary to achieve a better common-mode rejection and thus increase the SNR of recorded data. (for details, see https://www.biosemi.com/faq/cms&drl.htm)</li> </ol> References [1] Rotaru, Iustina, et al. "What are we really decoding? Unveiling biases in EEG-based decoding of the spatial focus of auditory attention." Journal of Neural Engineering 21.1 (2024): 016017. [2] Biesmans, Wouter, et al. "Auditory-inspired speech envelope extraction methods for improved EEG-based auditory attention detection in a cocktail party scenario." IEEE Transactions on neural systems and rehabilitation engineering 25.5 (2016): 402-412.

opencc-by-4.0May 2018View details →

Audiovisual Vignettes of Sea Ice Ridging in the Beaufort Sea in 2007

This presents footage demonstrating the scales of sea ice motion involved in creating ridges under varied degrees of compression and shear.  Sound heard in these vignettes is associated with frictional dissipation of kinetic energy during vertical ice displacement. Images shown were recorded during April 2-15, 2007 UTC, as part of the field campaign: Sea Ice Experiment - Dynamic Nature of the Arctic (SEDNA). Footage and photographs presented in this vignette were taken by Andrew Roberts with the assistance of Jennifer Hutchings and Cathleen Geiger.  Funding for SEDNA was provided by  the National Science Foundation, grant number OPP ARC 0612527.  An overview of the SEDNA field campaign is given in: Hutchings, J. K. et al. (2008), Role of Ice Dynamics in the Sea Ice Mass Balance, Eos Trans. AGU, 89(50), doi:10.1029/2008EO500003.   [Version 2 includes minor corrections and additions to text in Version 1]

Repositório audiovisual para curta-metragem e animação no DSpace

Prints das telas do repositório audiovisual construído no software DSpace, versão 5.5, para gestão de curtas-metragens e animações produzidas como TCC na Casa Amarela Eusélio Oliveira, equipamento cultural da Universidade Federal do Ceará. As imagens fazem parte dos resultados de pesquisa da dissertação intitulada Gestão de acervos audiovisuais em repositórios, sob orientação da Prof.ª Dra. Maria Giovanna Guedes Farias e coorientação do Prof. Dr. Luiz Tadeu Feitosa.

Repositório audiovisual para documentário no DSpace

Prints das telas do repositório audiovisual construído no software DSpace, versão 5.5, para gestão de documentários produzidos como TCC no Curso de Jornalismo da Universidade Federal do Ceará. As imagens fazem parte dos resultados de pesquisa da dissertação intitulada Gestão de acervos audiovisuais em repositórios, sob orientação da Prof.ª Dra. Maria Giovanna Guedes Farias e coorientação do Prof. Dr. Luiz Tadeu Feitosa.

Repositório audiovisual para espetáculo musical no DSpace

Prints das telas do repositório audiovisual construído no software DSpace, versão 5.5, para gestão dos espetáculos cênicos e musicais produzidos pelo Coral da Universidade Federal do Ceará. As imagens fazem parte dos resultados de pesquisa da dissertação intitulada Gestão de acervos audiovisuais em repositórios, sob orientação da Prof.ª Dra. Maria Giovanna Guedes Farias e coorientação do Prof. Dr. Luiz Tadeu Feitosa.

opencc-by-4.0Sep 2019View details →

Dataset about Diada de Sant Jordi from public TV3 audiovisual archive - FIAT/IFTA Media Studies Grant

[eng] This a dataset related to the Diada de Sant Jordi through TV3 public audiovisual archive. This dataset contains for a total of 572 videos with title, link, length in time and the program that have published information about this day. Most of these videos belong to the news program. This dataset is published on the occasion FIAT-IFTA Media Studies Grant 2019 Additionally, three more files are included. These files include the final sample, the queries on Tv3 archive and recommended books that appear on the footage. The final sample has 337 videos and queries are useful to look by year   [cat] Aquest és un dataset vinculat a la Diada de Sant Jordi dels arxius audiovisuals públics de Tv3. Aquest dataset conté un total de 572 videos amb el títol, dreçera, longitud en temps i el programa que ha publicat informació relativa aquest dia. La major part d'aquests videos corresponent corresponen als programes de notícies. Aquestes dades es publiquen amb motiu de FIAT-IFTA Media Studies Grant 2019 A més a més s'inclouen 3 fitxers. Aquests fitxes inclouen la mostra final, les cadenes de cerca a l'arxiu de Tv3 i els llibres recomanats que apareixen en els videos. La mostra final conté 339 videos i les cadenes són útils per fer cerques per any.

opencc-by-4.0Nov 2022View details →

AVbook, a high-frame-rate corpus of narrative audiovisual speech for investigating multimodal speech perception

Please cite Varano E, Guilleminot P, Reichenbach T. AVbook, a high-frame-rate corpus of narrative audiovisual speech for investigating multimodal speech perception. J Acoust Soc Am. 2023 May 1;153(5):3130. doi: 10.1121/10.0019460. PMID: 37249407. Seeing a speaker's face can help substantially in understanding them, in particular in challenging listening conditions. Research into the neurobiological mechanisms behind the audiovisual integration has recently begun to employ continuous natural speech. However, these efforts are impeded by a lack of high-quality audiovisual recordings of a speaker narrating a longer text. Here we seek to close this gap by developing AVbook, an audiovisual speech corpus designed for cognitive neuroscience studies and audiovisual speech recognition. The corpus consists of 3.6 hours of audiovisual recordings of two speakers, one male and one female, reading 59 passages from a narrative English text. The recordings were acquired at a high frame rate of 119.88 frames per second. The corpus includes a sets of multiple-choice questions to test attention to the different passages. We verified the efficacy of these questions in a pilot study. A short written summary is also provided for each recording. To enable audiovisual synchronization when presenting the stimuli, four videos of an electronic clapperboard were recorded with the corpus. The corpus is available for download to support research into the neurobiology of audiovisual speech processing as well as the development of computer algorithms for audiovisual speech recognition.

opencc-by-nc-sa-4.0Apr 2023View details →

Virtual audiovisual scenes for hearing device fine-tuning

The virtual audiovisual scenes in this upload were developed to be used during hearing device fine-tuning. By letting patients try out the settings of their hearing devices in a number of different daily-life situations, they can experience what it sounds like in different situations. The audiologist can then ask specific questions and make further adjustments to the settings. This may result in a better fit. The development of  the virtual audiovisual scenes is documented in a paper presented at Forum Acusticum (Hendrikse, Dingemanse, Grimm, Hohmann, & Goedegebure, 2023). The development and evaluation of the fine-tuning procedure using these virtual audiovisual scenes will be the focus of future work, and the publication will be added to the related identifiers. This document provides a description of the virtual audiovisual scenes, software & hardware requirements, installation and usage instructions, and information about the licensing. For further information, contact Maartje Hendrikse (<a href="mailto:m.hendrikse@erasmusmc.nl">research.audiologie@erasmusmc.nl</a>).

opencc-byAug 2023View details →

Audiovisual Moments in Time: A Large-Scale Annotated Dataset of Audiovisual Actions

We present Audiovisual Moments in Time (AVMIT), a large-scale dataset of audiovisual action events. In an extensive annotation task 11 participants labelled a subset of 3-second audiovisual videos from the Moments in Time dataset (MIT). For each trial, participants assessed whether the labelled audiovisual action event was present and whether it was the most prominent feature of the video. The dataset includes the annotation of 57,177 audiovisual videos, each independently evaluated by 3 of 11 trained participants. From this initial collection, we created a curated test set of 16 distinct action classes, with 60 videos each (960 videos). We also offer 2 sets of pre-computed audiovisual feature embeddings, using VGGish/YamNet for audio data and VGG16/EfficientNetB0 for visual data, thereby lowering the barrier to entry for audiovisual DNN research. We further carried out an experiment to explore the utility of the AVMIT annotations and feature embeddings. A series of 6 Recurrent Neural Networks (RNNs) were trained on either AVMIT-filtered audiovisual events or modality-agnostic events from MIT, and then tested on our audiovisual test set. In all RNNs, top 1 accuracy was increased by 2.71-5.94\% by training exclusively on audiovisual events, even outweighing a three-fold increase in training data. We anticipate that the newly annotated AVMIT dataset will serve as a valuable resource for research and comparative experiments involving computational models and human participants, specifically when addressing research questions where audiovisual correspondence is of critical importance.

opencc-by-4.0May 2020View details →

Audiovisual crowd counting dataset

This dataset contains 1,935 annotated images, each image has one-second audio and a density map. For more details, please refer to our paper <a href="https://arxiv.org/abs/2005.07097">Ambient Sound Helps: Audiovisual Crowd Counting in Extreme Conditions</a> and <a href="https://github.com/qingzwang/AudioVisualCrowdCounting">code</a>.

Repositório audiovisual para videoaula e palestra no DSpace

Prints das telas do repositório audiovisual construído no software DSpace, versão 5.5, para gestão de videoaulas e palestras promovidas pela Biblioteca de Ciências Humanas da Universidade Federal do Ceará. As imagens fazem parte dos resultados de pesquisa da dissertação intitulada Gestão de acervos audiovisuais em repositórios, sob orientação da Prof.ª Dra. Maria Giovanna Guedes Farias e coorientação do Prof. Dr. Luiz Tadeu Feitosa.

Repositório audiovisual para programa de televisão, programa de rádio e podcast no DSpace

Prints das telas do repositório audiovisual construído no software DSpace, versão 5.5, para gestão de acervos televisivos e sonoros produzidos pelo Programa UFCTV e pela Rádio Universitária FM, veículos de comunicação da Universidade Federal do Ceará. As imagens fazem parte dos resultados de pesquisa da dissertação intitulada Gestão de acervos audiovisuais em repositórios, sob orientação da Prof.ª Dra. Maria Giovanna Guedes Farias e coorientação do Prof. Dr. Luiz Tadeu Feitosa.

opencc-by-nc-sa-4.0Oct 2018View details →

Database of movement behavior and EEG in virtual audiovisual everyday-life environments for hearing aid research

This database contains movement behavior (head, eye, torso) and EEG signals of 21 young normal-hearing (11 male, 11 female, mean age 25 +/- 3.6 years) and 19 elderly normal-hearing subjects (9 male, 12 female, mean age 69 +/- 5.4 years) measured in virtual audiovisual listening environments in the laboratory. The virtual audiovisual environments that were used are: a living room, a lecture hall, a cafeteria, a street and a train station. The video and audio material for the environments is also available (see Related identifiers). The methods and an analysis of the movement behavior are described in Hendrikse et al. (2019). The supplementary materials to this paper that are published here include plots of the gaze trajectories of the subjects in all environments, plotted separately for the young and elderly subjects so that they can be compared, and histograms of the head-, eye- and torso-rotation for the environments that were not included in the paper.

opencc-by-4.0Jun 2023View details →

Audiovisual Materials in Research and Teaching – an Overview of Copyright Aspects

Das Rechtsgutachten (Neuauflage der Fassung von 2015) erläutert, wie audiovisuelle Materialien in Forschung und Lehre genutzt werden dürfen. Es stellt dar, in welchen Fällen Forschende eine Erlaubnis einholen müssen und in welchen die Nutzung gesetzlich erlaubt ist.Dies ist die Datenpublikation (Ausgangsdateien, Abbildungen, Materialien zur Nachnutzung) für die NFDI4Culture Handreichung "Audiovisuelle Materialien in Forschung und Lehre – eine Übersicht zu urheberrechtlichen Aspekte".Die Online-Handreichung ist verfügbar unter <a href="https://nfdi4culture.de/go/E4105">https://nfdi4culture.de/go/E4105</a>.

dryad40/100

Data and code from: Multisensory integration enhances audiovisual responses in the Mauthner cell

Open the record for dataset details and reuse information.

publicDec 2024View details →

opencc-by-4.0Aug 2020View details →

Burst timing determines perceived consonant order in the McGurk combination effect (Audiovisual stimuli)

The file “AV_stimuli.zip” contains the audiovisual speech stimuli generated by a native French talker for the study “Burst timing determines perceived consonant order in the McGurk combination effect.” The stimuli were created by splitting the acoustic component of McGurk combination stimuli into two sequences. One sequence contained only the vowels /i_i/ (where the underscore represents a 300 ms intersyllabic pause). The other sequence contained the release burst and aspiration extracted from a natural articulation of /iki/. An auditory (A) continuum was then created by adding the burst and aspiration to the vowels at nine different temporal alignments (step size of 50 ms). At 0 ms, the midpoint of the burst coincided with the midpoint of the intersyllabic pause. At the extremes, -200 and 200 ms, the burst and aspiration almost completely overlapped with the initial or final vowel, respectively. Two audiovisual (AV) continua were then created by pairing the acoustic continuum with either visual (V) /ip_i/ or /i_pi/. Filename list: s5:congruentAV/i_i/ s6-s14: AV continuum, V/ip_i/ +A/i_i/ &burst paired at different asynchronies, -200, -150, -100, -50, 0, 50, 100, 150 and 200 ms, respectively. s15-s23: AV continuum, V/i_pi/ +A/i_i/ &burst paired at different asynchronies, -200, -150, -100, -50, 0, 50, 100, 150 and 200 ms, respectively. S27-s35: A continuum, A/i_i/ &burst paired at different asynchronies, -200, -150, -100, -50, 0, 50, 100, 150 and 200 ms, respectively.

opencc-by-4.0Jan 2021View details →

Perceiving tempo in incongruent audiovisual presentation of human motion: Evidence for a visual driving effect

The video set includes 81 audiovisual stimuli which served in the bisection experiment of the study "Perceiving tempo in incongruent audiovisual presentation of human motion: Evidence for a visual driving effect". The auditory sound tracks (bass drum) and visual stimuli (point-light displays of biological motions) each cover the tempo range from 60 to 180BPM, 15BPM per step. Coupling of the stimuli from both modalities results in 81 stimuli in total.

opencc-by-4.0Nov 2024View details →

Catàleg audiovisual de desdelsofa.cat

Aquest dataset recull informació detallada sobre pel·lícules, sèries i documentals disponibles a diverses plataformes de streaming catalogades a la web <a href="https://desdelsofa.cat" target="_blank" rel="noopener noreferrer">desdelsofa.cat. </a>Inclou dades com títols, versions, gèneres, durada, any, sinopsi, qualificacions, plataformes de visualització i enllaços a les imatges dels cartells. El període de temps que abasten les dades és des del maig de 2021 fins a l’actualitat.