Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
14
datasets available to search
ShareScore release 0.9.0
Dataset results
14 results for “Speech to Text”
WOLOF TTS(Text To Speech) Data
<p>This contains a WOLOF Text To Speech(TTS) dataset, it contains recordings from two natif Wolof actos (a male and female voice).<br> Each actor recored more than 20 000 sentences.<br> The notebook accompanying the dataset contains a brief analysis of the dataset and the code creating the appropriate train/validation and test set.<br> The file [male, female]train, [male, female]validation and [male, female]test are also present to extract the corespondig audios inside the data-commonvoice.zip</p> <p>The text dataset come from news website, Wikipedia and self curated text. We made sure with the help of our Wolof expert that the text dataset cover the different phonemes in the Wolof language.</p>
Ressources for End-to-End French Text-to-Speech Blizzard challenge
<p>Here are 289 chapters of 5 audiobooks from Librivox (51:12) read by Nadine Eckert-Boulet (NEB):</p> <ol> <li>Madame Bovary (MB) by Gustave Flaubert (FL) - 3 volumes, 35 chapters<br>(original <a href="https://librivox.org/madame-bovary-french-by-gustave-flaubert">wavs</a>; <a href="https://www.gutenberg.org/cache/epub/14155/pg14155.txt">text</a>)</li> <li>Les mystères de Paris (LMP) by Eugene Sue (ES) - 4 volumes, 83 chapters (original <a href="https://librivox.org/les-mysteres-de-paris-tome-1-by-eugene-sue">wavs1</a>, <a href="https://librivox.org/les-mysteres-de-paris-tome-2-by-eugene-sue/">wavs2</a>,<a href="https://librivox.org/les-mysteres-de-paris-tome-3-by-eugene-sue"> wavs3</a>; <a href="https://www.gutenberg.org/cache/epub/18921/pg18921.txt">text1</a>, <a href="https://www.gutenberg.org/cache/epub/18922/pg18922.txt">text2</a>, <a href="https://www.gutenberg.org/cache/epub/18923/pg18923.txt">text3</a>)</li> <li>Les tribulations d'un chinois en Chine (TCC) by Jules Verne (JV) - 1 volume, 22 chapters (original <a href="https://librivox.org/les-tribulations-dun-chinois-en-chine-by-jules-verne">wavs</a>; <a href="https://www.gutenberg.org/cache/epub/14162/pg14162.txt">text</a>)</li> <li>La fille du pirate (LFDP) by Henri Émile Chevalier (EC) - 7 volumes, 121 chapters (original <a href="https://librivox.org/la-fille-du-pirate-by-henri-emile-chevalier">wavs</a>, <a href="https://www.gutenberg.org/cache/epub/18403/pg18403.txt">text)</a></li> <li>La vampire (VAMP) by Paul Féval (PF) - 1 volume, 28 chapters (original <a href="https://librivox.org/la-vampire-by-feval-paul-henry-corentin">wavs</a>, <a href="https://www.gutenberg.org/cache/epub/10053/pg10053.txt">text</a>)</li> </ol> <p>and</p> <p>2515 utterances (2:03) read by another female French speaker Aurélie Derbier (AD):</p> <ol> <li>1608 utterances extracted from various books (DIVERS_BOOK_AD*)</li> <li>907 transcripts of the sessions of the French parliament (DIVERS_PARL_01*)</li> </ol> <p>We recently added three speakers from Librivox/Litteratureaudio:</p> <ol> <li>Ezwa (EZWA): L'épouvante by Maurice Level (original <a href="https://librivox.org/lepouvante-by-maurice-level-1010/">wavs</a>; <a href="https://www.gutenberg.org/cache/epub/17794/pg17794.txt">text</a>) - 11 chapters - 4869 utterances> 03:16</li> <li>Pauline Latournerie (PL): Le pédagogue n'aime pas les enfants by Henri Roorda (<a href="https://librivox.org/le-pedagogue-naime-pas-les-enfants-by-henri-roorda/">original wavs</a>; <a href="https://ebooks-bnr.com/ebooks/pdf4/roorda_le_pedagogue_n_aime_pas_les_enfants.pdf">text</a>) - 6 chapters - 1320 utterances> 01:17</li> <li>Jean-Luc Fischer (JLF): L’Affaire Charles Dexter Ward by Howard Phillips Lovecraft (<a href="https://www.litteratureaudio.com/livre-audio-gratuit-mp3/howard-phillips-lovecraft-laffaire-charles-dexter-ward.html">original wavs</a>; <a href="https://www.litteratureaudio.com/textes/H_P_Lovecraft_L_Affaire_CDW.pdf">text</a>) - 16 chapters - 1823 utterances> 02:37</li> </ol> <p>Each .wav file (sampled at 22050Hz) corresponds to one entire chapter. The format of the filenames is:<br>{author's acronym}_{book's acronym}_{reader's acronym}_{volume's number}_{chapter's number}</p> <p>The NEB_train.csv file gives text and phonetic alignments (essentially for MB and LMP) for utterances in 4 fields separated by '|':<br>{filename}|{start_ms}|{end_ms}|{text or phonetic content}. Most utterances are separated by at least a pause of 400ms. The intervals [start_ms:end_ms] comprise leading and trailing silences of 130ms (since wavs are entire chapters, these silences are "true" ambient silences). Same for AD_train.csv.</p> <p>When phonetic alignment has been performed, 2 additional fields have been added: {aligned phones}|{durations in ms}. Each input character or phone has a corresponding aligned phone and a duration. Note that all aligned utterances start and end with an aligned phone of 130ms. The set of aligned phones comprises:</p> <ul> <li>The set of input phones</li> <li>The silence: '__'</li> <li>The symbol '_' for silent characters, e.g. "chat" is aligned with 's^ _ a _'</li> <li>29 combined aligned phones ('a&i', 'a&j', 'b&q', 'd&q','d&z', 'd&z^', 'f&q', 'g&q', 'g&z', 'j&i', 'j&u', 'j&q', 'i&j', 'k&q', 'k&s', 'k&s&q', 'l&q', 'm&q', 'n&q', 'r&w', 'r&q', 's&q', 't&q', 't&s', 't&s^', 'w&a', 'z&q', 'p&q') that align to only one character, e.g. "expatrier" is aligned with 'e^ k&s p a t r i&j e _'</li> </ul> <p>Text is in UTF8. '«»','¬', '~','""','()','[]' are respectively used for speaking quotes, turn switches, three dots, quoted expression, aside quotes, notes. Because of rare occurrences, 'ö' has been transcribed as 'oe'. Paragraphs (two consecutive carriage returns in the original text) are cued by a special character '§'. It usually ends an utterance but could be used within an utterance if its associated pause is too short.</p> <p>When available, phonetic content is given per word in curly brackets '{}'. We use 39 phonetic symbols:</p> <ul> <li><strong>oral vowels</strong>: a (f<strong><em>a</em></strong>), e (f<em><strong>ée</strong></em>), e^ (f<em><strong>ait</strong></em>), x (f<em><strong>eu</strong></em>), x^ (c<em><strong>oeu</strong></em>r), i (r<em><strong>iz</strong></em>), y (f<em><strong>ut</strong></em>), u (f<em><strong>ou</strong></em>), o (f<em><strong>aux</strong></em>), o^ (p<strong><em>o</em></strong>rc)</li> <li><strong>schwa</strong>: q (gag<strong><em>e</em></strong>)</li> <li><strong>nasal vowels</strong>: a~ (r<strong><em>an</em></strong>g), e~ (f<em><strong>in</strong></em>), x~ (<strong><em>un</em></strong>), o~ (r<em><strong>on</strong></em>d)</li> <li><strong>semi-vowels</strong>: h (h<em><strong>u</strong></em>it), w (<strong><em>ou</em></strong>ate), j (h<em><strong>i</strong></em>er)</li> <li><strong>consonants</strong>: p (<em><strong>p</strong></em>as), t (<strong><em>t</em></strong>as), k (<em><strong>c</strong></em>as), b (<strong><em>b</em></strong>as), d (<em><strong>d</strong></em>os), g (<em><strong>g</strong></em>ars), f (<em><strong>f</strong></em>aux), s (<strong><em>s</em></strong>ot) , s^ (<strong><em>ch</em></strong>at), v (<strong><em>v</em></strong>u), z (<strong><em>z</em></strong>ut), z^ (<em><strong>j</strong></em>us), r (<strong><em>r</em></strong>iz), l (<em><strong>l</strong></em>a), m (<strong><em>m</em></strong>a), n (<strong><em>n</em></strong>on), n~ (oi<strong><em>gn</em></strong>on), ng (campi<em><strong>ng</strong></em>)</li> </ul> <p> </p>
Hate Speech and Bias against Asians, Blacks, Jews, Latines, and Muslims: A Dataset for Machine Learning and Text Analytics
<h1>Institute for the Study of Contemporary Antisemitism (ISCA) at Indiana University Dataset on bias against Asians, Blacks, Jews, Latines, and Muslims </h1> <div> <h2> </h2> <h2>Description </h2> </div> <div> <p>The dataset is a product of a research project at Indiana University on biased messages on Twitter against ethnic and religious minorities. We scraped all live messages with the keywords "Asians, Blacks, Jews, Latinos, and Muslims" from the Twitter archive in 2020, 2021, and 2022.</p> <p>Random samples of 600 tweets were created for each keyword and year, including retweets. The samples were annotated in subsamples of 100 tweets by undergraduate students in Professor Gunther Jikeli's class 'Researching White Supremacism and Antisemitism on Social Media' in the fall of 2022 and 2023. A total of 120 students participated in 2022. They annotated datasets from 2020 and 2021. 134 students participated in 2023. They annotated datasets from the years 2021 and 2022. The annotation was done using the <a href="https://annotationportal.com/" target="_blank" rel="noreferrer noopener">Annotation Portal</a> (Jikeli, Soemer and Karali, 2024). The updated version of our portal, <a href="https://portal2.annotationportal.com/" target="_blank" rel="noreferrer noopener">AnnotHate</a>, is now publicly available. Each subsample was annotated by an average of 5.65 students per sample in 2022 and 8.32 students per sample in 2023, with a range of three to ten and three to thirteen students, respectively. Annotation included questions about bias and calling out bias. </p> </div> <div> <p>Annotators used a scale from 1 to 5 on the bias scale (confident not biased, probably not biased, don't know, probably biased, confident biased), using definitions of bias against each ethnic or religious group that can be found in the research reports from <a href="https://isca.indiana.edu/publication-research/social-media-project/Research-Report-BIAS-on-Twitter-against-Asians--Blacks-Jews-Latinos-Muslims-final-002.pdf" target="_blank" rel="noreferrer noopener">2022</a> and <a href="https://isca.indiana.edu/documents/BIAS%20Against%20Asian-Black-Hispanic-Jewish-and-%20Muslim-People%20on%20X-Twitter%20in%202021%20and%202022.pdf" target="_blank" rel="noreferrer noopener">2023</a>. If the annotators interpreted a message as biased according to the definition, they were instructed to choose the specific stereotype from the definition that was most applicable. Tweets that denounced bias against a minority were labeled as "calling out bias". </p> </div> <div> <p>The label was determined by a 75% majority vote. We classified “probably biased” and “confident biased” as biased, and “confident not biased,” “probably not biased,” and “don't know” as not biased. </p> </div> <div> <p>The stereotypes about the different minorities varied. About a third of all biased tweets were classified as general 'hate' towards the minority. The nature of specific stereotypes varied by group. Asians were blamed for the Covid-19 pandemic, alongside positive but harmful stereotypes about their perceived excessive privilege. Black people were associated with criminal activity and were subjected to views that portrayed them as inferior. Jews were depicted as wielding undue power and were collectively held accountable for the actions of the Israeli government. In addition, some tweets denied the Holocaust. Hispanic people/Latines faced accusations of being undocumented immigrants and "invaders," along with persistent stereotypes of them as lazy, unintelligent, or having too many children. Muslims were often collectively blamed for acts of terrorism and violence, particularly in discussions about Muslims in India. </p> </div> <div> <p>The annotation results from both cohorts (Class of 2022 and Class of 2023) will not be merged. They can be identified by the "cohort" column. While both cohorts (Class of 2022 and Class of 2023) annotated the same data from 2021,* their annotation results differ. The class of 2022 identified more tweets as biased for the keywords "Asians, Latinos, and Muslims" than the class of 2023, but nearly all of the tweets identified by the class of 2023 were also identified as biased by the class of 2022. The percentage of biased tweets with the keyword 'Blacks' remained nearly the same. </p> </div> <div> <p>*Due to a sampling error for the keyword "Jews" in 2021, the data are not identical between the two cohorts. The 2022 cohort annotated two samples for the keyword Jews, one from 2020 and the other from 2021, while the 2023 cohort annotated samples from 2021 and 2022.The 2021 sample for the keyword "Jews" that the 2022 cohort annotated was not representative. It has only 453 tweets from 2021 and 147 from the first eight months of 2022, and it includes some tweets from the query with the keyword "Israel". The 2021 sample for the keyword "Jews" that the 2023 cohort annotated was drawn proportionally for each trimester of 2021 for the keyword "Jews". </p> </div> <div> <h2> </h2> <h2>Content</h2> <h3>Cohort 2022 </h3> </div> <div> <p>This dataset contains 5880 tweets that cover a wide range of topics common in conversations about Asians, Blacks, Jews, Latines, and Muslims. 357 tweets (6.1 %) are labeled as biased and 5523 (93.9 %) are labeled as not biased. 1365 tweets (23.2 %) are labeled as calling out or denouncing bias. </p> </div> <div> <p>1180 out of 5880 tweets (20.1 %) contain the keyword "Asians," 590 were posted in 2020 and 590 in 2021. 39 tweets (3.3 %) are biased against Asian people. 370 tweets (31,4 %) call out bias against Asians. </p> </div> <div> <p>1160 out of 5880 tweets (19.7%) contain the keyword "Blacks," 578 were posted in 2020 and 582 in 2021. 101 tweets (8.7 %) are biased against Black people. 334 tweets (28.8 %) call out bias against Blacks. </p> </div> <div> <p>1189 out of 5880 tweets (20.2 %) contain the keyword "Jews," 592 were posted in 2020, 451 in 2021, and ––as mentioned above––146 tweets from 2022. 83 tweets (7 %) are biased against Jewish people. 220 tweets (18.5 %) call out bias against Jews. </p> </div> <div> <p>1169 out of 5880 tweets (19.9 %) contain the keyword "Latinos," 584 were posted in 2020 and 585 in 2021. 29 tweets (2.5 %) are biased against Latines. 181 tweets (15.5 %) call out bias against Latines. </p> </div> <div> <p>1182 out of 5880 tweets (20.1 %) contain the keyword "Muslims," 593 were posted in 2020 and 589 in 2021. 105 tweets (8.9 %) are biased against Muslims. 260 tweets (22 %) call out bias against Muslims. </p> </div> <div> <h3>Cohort 2023 </h3> </div> <div> <p>The dataset contains 5363 tweets with the keywords “Asians, Blacks, Jews, Latinos and Muslims” from 2021 and 2022. 261 tweets (4.9 %) are labeled as biased, and 5102 tweets (95.1 %) were labeled as not biased. 975 tweets (18.1 %) were labeled as calling out or denouncing bias. </p> </div> <div> <p>1068 out of 5363 tweets (19.9 %) contain the keyword "Asians," 559 were posted in 2021 and 509 in 2022. 42 tweets (3.9 %) are biased against Asian people. 280 tweets (26.2 %) call out bias against Asians. </p> </div> <div> <p>1130 out of 5363 tweets (21.1 %) contain the keyword "Blacks," 586 were posted in 2021 and 544 in 2022. 76 tweets (6.7 %) are biased against Black people. 146 tweets (12.9 %) call out bias against Blacks. </p> </div> <div> <p>971 out of 5363 tweets (18.1 %) contain the keyword "Jews," 460 were posted in 2021 and 511 in 2022. 49 tweets (5 %) are biased against Jewish people. 201 tweets (20.7 %) call out bias against Jews. </p> </div> <div> <p>1072 out of 5363 tweets (19.9 %) contain the keyword "Latinos," 583 were posted in 2021 and 489 in 2022. 32 tweets (2.9 %) are biased against Latines. 108 tweets (10.1 %) call out bias against Latines. </p> </div> <div> <p>1122 out of 5363 tweets (20.9 %) contain the keyword "Muslims," 576 were posted in 2021 and 546 in 2022. 62 tweets (5.5 %) are biased against Muslims. 240 tweets (21.3 %) call out bias against Muslims. </p> </div> <div> <h2> </h2> <h2>File Description</h2> </div> <div> <p>The dataset is provided in a csv file format, with each row representing a single message, including replies, quotes, and retweets. The file contains the following columns: </p> <p>'TweetID': Represents the tweet ID. </p> </div> <div> <p>'Username': Represents the username who published the tweet (if it is a retweet, it will be the user who retweetet the original tweet. </p> </div> <div> <p>'Text': Represents the full text of the tweet (not pre-processed). </p> </div> <div> <p>'CreateDate': Represents the date the tweet was created. </p> </div> <div> <p>'Biased': Represents the labeled by our annotators if the tweet is biased (1) or not (0). </p> </div> <div> <p>'Calling_Out': Represents the label by our annotators if the tweet is calling out bias against minority groups (1) or not (0). </p> </div> <div> <p>'Keyword': Represents the keyword that was used in the query. The keyword can be in the text, including mentioned names, or the username. </p> </div> <div> <p> ‘Cohort’: Represents the year the data was annotated (class of 2022 or class of 2023) </p> </div> <div> <h2> </h2> <h2>Acknowledgements </h2> </div> <div> <p>We are grateful for the technical collaboration with Indiana University's Observatory on Social Media (OSoMe). We thank all class participants for the annotations and contributions, including Kate Baba, Eleni Ballis, Garrett Banuelos, Savannah Benjamin, Luke Bianco, Zoe Bogan, Elisha S. Breton, Aidan Calderaro, Anaye Caldron, Olivia Cozzi, Daj Crisler, Jenna Eidson, Ella Fanning, Victoria Ford, Jess Gruettner, Ronan Hancock, Isabel Hawes, Brennan Hensler, Kyra Horton, Maxwell Idczak, Sanjana Iyer, Jacob Joffe, Katie Johnson, Allison Jones, Kassidy Keltner, Sophia Knoll, Jillian Kolesky, Emily Lowrey, Rachael Morara, Benjamin Nadolne, Rachel Neglia, Seungmin Oh, Kirsten Pecsenye, Sophia Perkovich, Joey Philpott, Katelin Ray, Kaleb Samuels, Chloe Sherman, Rachel Weber, Molly Winkeljohn, Ally Wolfgang, Rowan Wolke, Michael Wong, Jane Woods, Kaleb Woodworth, Aurora Young, Sydney Allen, Hundre Askie, Norah Bardol, Olivia Baren, Samuel Barth, Emma Bender, Noam Biron, Kendyl Bond, Graham Brumley, Kennedi Bruns, Leah Burger, Hannah Busche, Morgan Butrum-Griffith, Zoe Catlin, Angeli Cauley, Nathalya Chavez Medrano, Mia Cooper, Suhani Desai, Isabella Flick, Samantha Garcez, Isabella Grady, Macy Hutchinson, Sarah Kirkman, Ella Leitner, Elle Marquardt, Madison Moss, Ethan Nixdorf, Reya Patel, Mickey Racenstein, Kennedy Rehklau, Grace Roggeman, Jack Rossell, Madeline Rubin, Fernando Sanchez, Hayden Sawyer, Diego Scheker, Lily Schwecke, Brooke Scott, Megan Scott, Samantha Secchi, Jolie Segal, Katherine Smith, Constantine Stefanidis, Cami Stetler, Madisyn West, Alivia Yusefzadeh, Tayssir Aminou, Karen Fecht, Luciana Orrego-Hoyos, Hannah Pickett, and Sophia Tracy. </p> </div> <div> <p>This work used Jetstream2 at Indiana University through allocation HUM200003 from the Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS) program, which is supported by National Science Foundation grants #2138259, #2138286, #2138307, #2137603, and #2138296. </p> </div> <div> <p> </p> </div>
On the Effectiveness of Text and Image Embeddings in Multimodal Hate Speech Detection
<p>Additional resources for the paper:</p> <h3><strong><a href="https://ieeexplore.ieee.org/abstract/document/10826088">On the Effectiveness of Text and Image Embeddings in Multimodal Hate Speech Detection.</a></strong></h3> <p>Lewis, N., Cavalcante, C. C., Boukouvalas, Z., & Corizzo, R.</p> <p><em>2024 IEEE International Conference on Big Data (BigData)</em> (pp. 3277-3281). IEEE.</p> <pre> </pre> <p> </p> <p>MMHS150K [1] is a manually labeled multimodal dataset that contains $150000$ tweets with two modalities: text, and corresponding image. Tweets are collected from September 2018 until February 2019 and are labeled according to different types of hate speech: no attacks to any community, racist, sexist, homophobic, religion-based attacks, or attacks to other communities. </p> <p>We extract vector embeddings leveraging different text (BERT, OpenAI) and image (ResNet, PVT, ViT) modele backbones and assess their effectiveness in the hate speech detection task.</p> <p> </p> <h2>Citation:</h2> <pre>@inproceedings{lewis2024effectiveness, title={On the Effectiveness of Text and Image Embeddings in Multimodal Hate Speech Detection}, author={Lewis, Nora and Cavalcante, Charles C and Boukouvalas, Zois and Corizzo, Roberto}, booktitle={2024 IEEE International Conference on Big Data (BigData)}, pages={3277--3281}, year={2024}, organization={IEEE} }</pre>
TIMIT-TTS: a Text-to-Speech Dataset for Synthetic Speech Detection
<p>With the rapid development of deep learning techniques, the generation and counterfeiting of multimedia material are becoming increasingly straightforward to perform. At the same time, sharing fake content on the web has become so simple that malicious users can create unpleasant situations with minimal effort. Also, forged media are getting more and more complex, with manipulated videos (e.g., deepfakes where both the visual and audio contents can be counterfeited) that are taking the scene over still images.<br> The multimedia forensic community has addressed the possible threats that this situation could imply by developing detectors that verify the authenticity of multimedia objects. However, the vast majority of these tools only analyze one modality at a time.<br> This was not a problem as long as still images were considered the most widely edited media, but now, since manipulated videos are becoming customary, performing monomodal analyses could be reductive. Nonetheless, there is a lack in the literature regarding multimodal detectors (systems that consider both audio and video components). This is due to the difficulty of developing them but also to the scarsity of datasets containing forged multimodal data to train and test the designed algorithms.</p> <p>In this paper we focus on the generation of an audio-visual deepfake dataset.<br> First, we present a general pipeline for synthesizing speech deepfake content from a given real or fake video, facilitating the creation of counterfeit multimodal material. The proposed method uses Text-to-Speech (TTS) and Dynamic Time Warping (DTW) techniques to achieve realistic speech tracks. Then, we use the pipeline to generate and release TIMIT-TTS, a synthetic speech dataset containing the most cutting-edge methods in the TTS field. This can be used as a standalone audio dataset, or combined with DeepfakeTIMIT and VidTIMIT video datasets to perform multimodal research. Finally, we present numerous experiments to benchmark the proposed dataset in both monomodal (i.e., audio) and multimodal (i.e., audio and video) conditions.<br> This highlights the need for multimodal forensic detectors and more multimodal deepfake data.</p> <ul> <li>For the initial version of TIMIT-TTS <strong>v1.0</strong> <ul> <li>Arxiv: https://arxiv.org/abs/2209.08000</li> <li>TIMIT-TTS Database v1.0: https://zenodo.org/record/6560159</li> </ul> </li> </ul>
Lada: Ukrainian High-Quality Female Text-to-Speech Dataset
<p>The dataset has high-quality data recorded in a professional studio. </p> <p>Archives with a <strong>trimmed</strong> tag are having removed silence (aligned) using <a href="https://github.com/proger/uk">https://github.com/proger/uk</a> </p> <p><strong>Features</strong></p> <ul> <li>Quality: high</li> <li>Duration: 10h37m</li> <li>Audio formats: OPUS/WAV</li> <li>Text format: JSONL (a <code>metadata.jsonl</code> file)</li> <li>Frequency: 16000/22050/48000 Hz</li> </ul>
Preprocessed Data and Pretrained Models for Zero-Shot Multi-Speaker Text-To-Speech with State-of-the-art Neural Speaker Embeddings
<p>This is preprocessed data and pretrained models from two of our papers:</p> <p>"Zero-Shot Multi-Speaker Text-To-Speech with State-of-the-art Neural Speaker Embeddings," by Erica Cooper, Cheng-I Lai, Yusuke Yasuda, Fuming Fang, Xin Wang, Nanxin Chen, and Junichi Yamagishi. (ICASSP 2020)<br> <a href="https://arxiv.org/abs/1910.10838">https://arxiv.org/abs/1910.10838</a></p> <p> "Pretraining Strategies, Waveform Model Choice, and Acoustic Configurations for Multi-Speaker End-to-End Speech Synthesis," by Erica Cooper, Xin Wang, Yi Zhao, Yusuke Yasuda, and Junichi Yamagishi. (arXiv) <a href="https://arxiv.org/abs/2011.04839">https://arxiv.org/abs/2011.04839</a></p> <p>This data is meant to be used with our open-source implementation, which can be found here: https://github.com/nii-yamagishilab/multi-speaker-tacotron</p> <p>More information about the directory structure and how to use the data can be found in the READMEs on GitHub.</p>
Synthetic Göttingen Sentence Test material created with a text-to-speech system
<p>The speech material of the German Göttingen Sentence Test [1] was synthesized using a commercial text-to-speech system (Acapela Cloud Service). More details will be found in [2].</p> <p>Files:</p> <p>goesa_synth_female.zip<br> contains all 200 sentences with a synthetic female voice and the corresponding speech adjusted noise, which was generated by superimposing the speech material 30 times according to [3].</p> <p>goesa_synth_male.zip<br> contains all 200 sentences with a synthetic male voice and the corresponding speech adjusted noise, which was generated by superimposing the speech material 30 times according to [3].</p> <p> </p>
Training a Text-to-Speech System for Dialectal Arabic with a Focus on the Iraqi Dialect
<p>This research introduces a novel approach to Text-to-Speech (TTS) synthesis, focusing on the phonetic complexities of Arabic dialects, with particular emphasis on the Iraqi dialect. While existing Arabic speech corpora provide substantial coverage of Modern Standard Arabic (MSA), they fall short in capturing the phonetic richness of regional dialects. To address this gap, we utilized Nawar Halabi's Arabic Speech Corpus as a base dataset and enriched it with custom-recorded samples of the Iraqi dialect, incorporating distinctive phonemes such as گ ,ڤ ,پ ,چ ,ۆ ,ڵ ,ێ, and using the Tatweel character (ـ) as a vowel. Our approach, powered by the FastPitch model and a customized phonetiser, successfully synthesized the Iraqi dialect while also demonstrating adaptability to other Arabic dialects, including Egyptian, Khaliji, Syrian, and more. The results of this research signify a promising advancement in Arabic TTS technology, expanding its scope to authentically represent the diverse linguistic landscape of the Arabic-speaking world.</p>
SOMOS: The Samsung Open MOS Dataset for the Evaluation of Neural Text-to-Speech Synthesis
<p>This is the public release of the Samsung Open Mean Opinion Scores (SOMOS) dataset for the evaluation of neural text-to-speech (TTS) synthesis, which consists of audio files generated with a public domain voice from trained TTS models based on bibliography, and numbers assigned to each audio as quality (naturalness) evaluations by several crowdsourced listeners.<br><br><strong>Description</strong><br><br>The SOMOS dataset contains 20,000 synthetic utterances (wavs), 100 natural utterances and 374,955 naturalness evaluations (human-assigned scores in the range 1-5). The synthetic utterances are single-speaker, generated by training several Tacotron-like acoustic models and an LPCNet vocoder on the LJ Speech voice public dataset. 2,000 text sentences were synthesized, selected from Blizzard Challenge texts of years 2007-2016, the LJ Speech corpus as well as Wikipedia and general domain data from the Internet.<br>Naturalness evaluations were collected via crowdsourcing a listening test on Amazon Mechanical Turk in the US, GB and CA locales. The records of listening test participants (workers) are fully anonymized. Statistics on the reliability of the scores assigned by the workers are also included, generated through processing the scores and validation controls per submission page.</p> <p>To listen to audio samples of the dataset, please see <a href="https://innoetics.github.io/publications/somos-dataset/index.html">our Github page</a>.</p> <p>The dataset release comes with a carefully designed train-validation-test split (70%-15%-15%) with unseen systems, listeners and texts, which can be used for experimentation on MOS prediction.</p> <p><em>This version also contains the necessary resources to obtain the transcripts corresponding to all dataset audios.</em></p> <p><strong>Terms of use</strong></p> <ul> <li>The dataset may be used for <strong>research</strong> purposes only, for <strong>non-commercial</strong> purposes only, and may be distributed with the same terms.</li> <li>Every time you produce research that has used this dataset, please <strong>cite </strong>the dataset appropriately.</li> </ul> <p>Cite as:</p> <pre><code>@inproceedings{maniati22_interspeech, author={Georgia Maniati and Alexandra Vioni and Nikolaos Ellinas and Karolos Nikitaras and Konstantinos Klapsas and June Sig Sung and Gunu Jho and Aimilios Chalamandaris and Pirros Tsiakoulis}, title={{SOMOS: The Samsung Open MOS Dataset for the Evaluation of Neural Text-to-Speech Synthesis}}, year=2022, booktitle={Proc. Interspeech 2022}, pages={2388--2392}, doi={10.21437/Interspeech.2022-10922} } </code></pre> <p><br><strong>References of resources & models used</strong></p> <p>Voice & synthesized texts:<br>K. Ito and L. Johnson, “The LJ Speech Dataset,” https://keithito.com/LJ-Speech-Dataset/, 2017.</p> <p>Vocoder:<br>J.-M. Valin and J. Skoglund, “LPCNet: Improving neural speech synthesis through linear prediction,” in Proc. ICASSP, 2019.<br>R. Vipperla, S. Park, K. Choo, S. Ishtiaq, K. Min, S. Bhattacharya, A. Mehrotra, A. G. C. P. Ramos, and N. D. Lane, “Bunched lpcnet: Vocoder for low-cost neural text-to-speech systems,” in Proc. Interspeech, 2020.</p> <p>Acoustic models:<br>N. Ellinas, G. Vamvoukakis, K. Markopoulos, A. Chalamandaris, G. Maniati, P. Kakoulidis, S. Raptis, J. S. Sung, H. Park, and P. Tsiakoulis, “High quality streaming speech synthesis with low, sentence-length-independent latency,” in Proc. Interspeech, 2020.<br>Y. Wang, R. Skerry-Ryan, D. Stanton, Y. Wu, R. J. Weiss, N. Jaitly, Z. Yang, Y. Xiao, Z. Chen, S. Bengio et al., “Tacotron: Towards End-to-End Speech Synthesis,” in Proc. Interspeech, 2017.<br>J. Shen, R. Pang, R. J. Weiss, M. Schuster, N. Jaitly, Z. Yang, Z. Chen, Y. Zhang, Y. Wang, R. Skerrv-Ryan et al., “Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions,” in Proc. ICASSP, 2018.<br>J. Shen, Y. Jia, M. Chrzanowski, Y. Zhang, I. Elias, H. Zen, and Y. Wu, “Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modeling,” arXiv preprint arXiv:2010.04301, 2020.<br>M. Honnibal and M. Johnson, “An Improved Non-monotonic Transition System for Dependency Parsing,” in Proc. EMNLP, 2015.<br>M. Dominguez, P. L. Rohrer, and J. Soler-Company, “PyToBI: A Toolkit for ToBI Labeling Under Python,” in Proc. Interspeech, 2019.<br>Y. Zou, S. Liu, X. Yin, H. Lin, C. Wang, H. Zhang, and Z. Ma, “Fine-grained prosody modeling in neural speech synthesis using ToBI representation,” in Proc. Interspeech, 2021.<br>K. Klapsas, N. Ellinas, J. S. Sung, H. Park, and S. Raptis, “WordLevel Style Control for Expressive, Non-attentive Speech Synthesis,” in Proc. SPECOM, 2021.<br>T. Raitio, R. Rasipuram, and D. Castellani, “Controllable neural text-to-speech synthesis using intuitive prosodic features,” in Proc. Interspeech, 2020.</p> <p>Synthesized texts from the Blizzard Challenges 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2016:<br>M. Fraser and S. King, "The Blizzard Challenge 2007," in Proc. SSW6, 2007.<br>V. Karaiskos, S. King, R. A. Clark, and C. Mayo, "The Blizzard Challenge 2008," in Proc. Blizzard Challenge Workshop, 2008.<br>A. W. Black, S. King, and K. Tokuda, "The Blizzard Challenge 2009," in Proc. Blizzard Challenge, 2009.<br>S. King and V. Karaiskos, "The Blizzard Challenge 2010," 2010.<br>S. King and V. Karaiskos, "The Blizzard Challenge 2011," 2011.<br>S. King and V. Karaiskos, "The Blizzard Challenge 2012," 2012.<br>S. King and V. Karaiskos, "The Blizzard Challenge 2013," 2013.<br>S. King and V. Karaiskos, "The Blizzard Challenge 2016," 2016.</p> <p><strong>Contact</strong></p> <p>Alexandra Vioni - a.vioni@samsung.com</p> <ul> <li>If you have any questions or comments about the dataset, please feel free to write to us.</li> <li>We are interested in knowing if you find our dataset useful! If you use our dataset, please email us and tell us about your research.</li> </ul>
Personalization of Visual Characters and Text-to-Speech (TTS) to Support Learning in Students With Reading Difficulties
ClinicalTrials.gov study NCT06947512. IPD Sharing: YES. Countries: 1. Publications: 0.
Benchmarking speech-to-text robustness in noisy emergency medical dialogues: An evaluation of stt-models under realistic acoustic conditions
Open the record for dataset details and reuse information.
Speech-to-text Tool for Heart Failure Patient-Reported Data
ClinicalTrials.gov study NCT07217366. IPD Sharing: YES. Countries: 1. Publications: 0.
Synthetic phrase-based test material created with a text-to-speech system
<p>New speech material consisting of phrases was synthesized using a commercial text-to-speech system (Acapela Cloud Service). More details will be found in [1].</p> <p>Files:</p> <p>syntheticPhrases.zip<br>contains all 772 phrases with a synthetic female voice and the corresponding speech adjusted noise, which was generated by superimposing the speech material 30 times according to [2].</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.