CaSET-catalan-stance-emotions-twitter

<h3>Dataset Summary</h3> <p>The CaSET dataset is a Catalan corpus of Tweets annotated with Emotions, Static Stance, and Dynamic Stance. The dataset contains 11k unique sentences on five controversial topics, grouped in 6k pairs of sentences, paired as parent messages and replies to these messages.</p> <p>We provide the following files and folders:</p> <ul> <li>The dataset folder contains the final dataset (an aggregation of the dynamic stance, static stance, and emotion identification tasks), as well as the dataloader and README file as provided in Hugging Face.</li> <li>The annotations folders with one file for each of the three annotation tasks.</li> <li>The annotation guidelines folders with a file for each task.</li> </ul> <h3>Supported Tasks and Leaderboards</h3> <p>This dataset can be used to train models for emotion detection, static stance detection, and dynamic stance detection.</p> <h3>Languages</h3> <p>The dataset is in Catalan (ca-ES).</p> <h3>Dataset Structure</h3> <p>Each instance in the dataset is a pair of parent-reply messages, annotated with the relation between the two messages (the dynamic stance) and the topic of the messages. For each message there is the id to retrieve it with the Twitter API, the emotions identified in the message, and the relation between the message and the topic (static stance). The text fields have to be retrieved using the Twitter API.</p> <h3>Data Instances</h3> <p>{<br>"id_parent": "1413960970066710533",&nbsp;<br>"id_reply": "1413968453690658816",&nbsp;<br>&nbsp; "parent_text": "",&nbsp;<br>&nbsp; "reply_text": "",&nbsp;<br>&nbsp; &nbsp; "topic": "vaccines",&nbsp;<br>&nbsp; &nbsp; "dynamic_stance": "Disagree",&nbsp;<br>&nbsp; "parent_stance": "FAVOUR",&nbsp;<br>&nbsp; "reply_stance": "AGAINST",&nbsp;<br>&nbsp; "parent_emotion": ["distrust", "joy", "disgust"],&nbsp;<br>&nbsp; "reply_emotion": ["distrust"]<br>}</p> <h3>Data Splits</h3> <p>The dataset does not contain splits.</p> <h3>Dataset Creation</h3> <p>We created this corpus to contribute to the development of language models in Catalan, a low-resource language.<br>Source Data</p> <p>The data was collected using the Twitter API by the Barcelona Supercomputing Center.</p> <h3>Initial Data Collection and Normalization</h3> <p>The data was collected based on a list of keywords related to the five topics included in the dataset: vaccines, rent regulation, surrogate pregnancy, airport expansion, and a TV show rigging. Specific periods in which the topic was under discussion were also selected.</p> <h3>Who are the source language producers?</h3> <p>The source language producers are users of Twitter.</p> <h3>Annotations</h3> <ul> <li>Emotions are annotated in a multi-label fashion. The labels can be: Anger, Anticipation, Disgust, Fear, Joy, Sadness, Surprise, Distrust, and No emotion. CA</li> <li>Static stance is annotated per message. The labels can be: FAVOUR, AGAINST, NEUTRAL, NA.&nbsp; &nbsp;</li> <li>Dynamic stance is annotated per pair. The labels can be: Agree, Disagree, Elaborate, Query, Neutral, Unrelated, NA.</li> </ul> <h3>Annotation process</h3> <ul> <li>For emotions there were 3 annotators. The gold labels are an aggregation of all the labels annotated by the 3. The IAA calculated with Fleiss' Kappa per label was, on average, 45.38.</li> <li>For static stance there were 2 annotators, in the cases of disagreement a third annotated chose the gold label. The overall Fleiss' Kappa between the 2 annotators is 82.71.</li> <li>For dynamic stance there were 4 annotators. If at least 3 of the annotators disagreed, a fifth annotator chose the gold label. The overall Fleiss' Kappa between the 4 annotators was 56.51, and the average Fleiss' Kappa of the annotators with the gold labels is 85.17.</li> </ul> <h3>Who are the annotators?</h3> <p>All the annotators are native speakers of Catalan.<br><br>Social Impact of Dataset</p> <p>We hope this corpus contributes to the development of language models in Catalan, a low-resource language.</p> <h3>Discussion of Biases</h3> <p>We are aware that, since the data comes from social media, this will contain biases, hate speech and toxic content. We have not applied any steps to reduce their impact.</p> <h3>Other Known Limitations</h3> <p>The dataset has to be downloaded using the Twitter API, therefore some instances might be lost.</p> <h3>Dataset Curators</h3> <p>Language Technologies Unit (LangTech) at the Barcelona Supercomputing Center.</p> <p>This work has been promoted and financed by the Generalitat de Catalunya through the Aina project.</p> <h3>Licensing Information</h3> <p>Creative Commons Attribution 4.0.</p> <h3>Citation Information</h3> <p>@inproceedings{figueras-etal-2023-dynamic,<br>&nbsp; &nbsp; title = "Dynamic Stance: Modeling Discussions by Labeling the Interactions",<br>&nbsp; &nbsp; author = "Figueras, Blanca &nbsp;and<br>&nbsp; &nbsp; &nbsp; Baucells, Irene &nbsp;and<br>&nbsp; &nbsp; &nbsp; Caselli, Tommaso",<br>&nbsp; &nbsp; editor = "Bouamor, Houda &nbsp;and<br>&nbsp; &nbsp; &nbsp; Pino, Juan &nbsp;and<br>&nbsp; &nbsp; &nbsp; Bali, Kalika",<br>&nbsp; &nbsp; booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2023",<br>&nbsp; &nbsp; month = dec,<br>&nbsp; &nbsp; year = "2023",<br>&nbsp; &nbsp; address = "Singapore",<br>&nbsp; &nbsp; publisher = "Association for Computational Linguistics",<br>&nbsp; &nbsp; url = "https://aclanthology.org/2023.findings-emnlp.432",<br>&nbsp; &nbsp; doi = "10.18653/v1/2023.findings-emnlp.432",<br>&nbsp; &nbsp; pages = "6503--6515",<br>}</p> <p>@inproceedings{gonzalez-agirre-etal-2024-building-data,<br>&nbsp; &nbsp; title = "Building a Data Infrastructure for a Mid-Resource Language: The Case of {C}atalan",<br>&nbsp; &nbsp; author = "Gonzalez-Agirre, Aitor &nbsp;and<br>&nbsp; &nbsp; &nbsp; Marimon, Montserrat &nbsp;and<br>&nbsp; &nbsp; &nbsp; Rodriguez-Penagos, Carlos &nbsp;and<br>&nbsp; &nbsp; &nbsp; Aula-Blasco, Javier &nbsp;and<br>&nbsp; &nbsp; &nbsp; Baucells, Irene &nbsp;and<br>&nbsp; &nbsp; &nbsp; Armentano-Oller, Carme &nbsp;and<br>&nbsp; &nbsp; &nbsp; Palomar-Giner, Jorge &nbsp;and<br>&nbsp; &nbsp; &nbsp; Kulebi, Baybars &nbsp;and<br>&nbsp; &nbsp; &nbsp; Villegas, Marta",<br>&nbsp; &nbsp; editor = "Calzolari, Nicoletta &nbsp;and<br>&nbsp; &nbsp; &nbsp; Kan, Min-Yen &nbsp;and<br>&nbsp; &nbsp; &nbsp; Hoste, Veronique &nbsp;and<br>&nbsp; &nbsp; &nbsp; Lenci, Alessandro &nbsp;and<br>&nbsp; &nbsp; &nbsp; Sakti, Sakriani &nbsp;and<br>&nbsp; &nbsp; &nbsp; Xue, Nianwen",<br>&nbsp; &nbsp; booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",<br>&nbsp; &nbsp; month = may,<br>&nbsp; &nbsp; year = "2024",<br>&nbsp; &nbsp; address = "Torino, Italia",<br>&nbsp; &nbsp; publisher = "ELRA and ICCL",<br>&nbsp; &nbsp; url = "https://aclanthology.org/2024.lrec-main.231",<br>&nbsp; &nbsp; pages = "2556--2566",<br>}</p> <h3>Contact information</h3> <p>For further information, please send an email to langtech@bsc.es.</p>