Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
4
datasets available to search
ShareScore release 0.9.0
Dataset results
4 results for “Semeval-2024”
SemEval-2024 Task 6: SHROOM, a Shared-task on Hallucinations and Related Observable Overgeneration Mistakes
<p><strong>Task description:</strong> SHROOM participants will need to detect grammatically sound output that contains incorrect semantic information (i.e. unsupported or inconsistent with the source input), with or without having access to the model that produced the output.</p> <p><strong>Overview of the task:</strong> The modern NLG landscape is plagued by two interlinked problems: On the one hand, our current neural models have a propensity to produce inaccurate but fluent outputs; on the other hand, our metrics are most apt at describing fluency, rather than correctness. This leads neural networks to “hallucinate”, i.e., produce fluent but incorrect outputs that we currently struggle to detect automatically. For many NLG applications, the correctness of an output is however mission critical. For instance, producing a plausible-sounding translation that is inconsistent with the source text puts in jeopardy the usefulness of a machine translation pipeline. With our shared task, we hope to foster the growing interest in this topic in the community.</p> <p>With SHROOM we adopt a post hoc setting, where models have already been trained and outputs already produced: participants will be asked to perform binary classification to identify cases of fluent overgeneration hallucinations in two different setups: model-aware and model-agnostic tracks. That is, participants must detect grammatically sound outputs which contain incorrect or unsupported semantic information, inconsistent with the source input, with or without having access to the model that produced the output. To that end, we will provide participants with a collection of checkpoints, inputs, references and outputs of systems covering three different NLG tasks: definition modeling (DM), machine translation (MT) and paraphrase generation (PG), trained with varying degrees of accuracy. The development set will provide binary annotations from at least five different annotators and a majority vote gold label.</p>
SemEval-2024 Task 9: BRAINTEASER: A Novel Task Defying Common Sense
<h3>Data for the SemEval 2024 paper "SemEval-2024 Task 9: BRAINTEASER: A Novel Task Defying Common Sense"</h3> <p> </p> <p>The data of the two subtasks is saved in the <strong>data</strong> folder, <em>BTDATA.zip</em>, which contains the data for the sentence puzzle and word puzzle.</p> <p>The data contained in <em>BTDATA.zip</em> are as follows:</p> <ul> <li>Semeval Competition <ul> <li>Training Data <ul> <li><code>SP_train.npy</code> (Semeval training data)</li> <li><code>WP_train.npy</code> (Semeval training data)</li> </ul> </li> <li>Test Data <ul> <li><code>SP_test.npy</code> (Semeval test data)</li> <li><code>WP_test.npy</code> (Semeval test data)</li> <li><code>SP_test_answer.npy</code> (Semeval test data answer)</li> <li><code>WP_test_answer.npy</code> (Semeval test data answer)</li> </ul> </li> </ul> </li> </ul> <div> <h3><strong>Relation to EMNLP 2023 Paper</strong></h3> <a href="https://github.com/1171-jpg/BrainTeaser#2-relation-to-semeval2024-task9"></a></div> <p>The relationship between EMNLP and SemEval involves using the same dataset but with different data splitting and utilization methods. In EMNLP, the entire dataset is employed for testing, while in SemEval, the dataset is divided into training and testing sets, with the training set comprising a significant majority.</p> <p>Our EMNLP paper results on GitHub are tested on the entire data in a <strong>zero-shot manner</strong>. In the SemEval2024-Task9, although the whole dataset is the same as our EMNLP paper, we allow people to <strong>train on 80% of the whole dataset</strong>, and we <strong>evaluate the system on the 20% left</strong>.</p> <ul> <li>EMNLP Zero-Shot Experiment <ul> <li><code>sentence_puzzle.npy</code> (on all sentence puzzle data)</li> <li><code>word_puzzle.npy</code> (on all word puzzle data)</li> </ul> </li> </ul> <p><strong>Note:</strong> To prevent automatic data crawlers, <em>BTDATA.zip</em> needs a password: <strong>brainteaser</strong></p>
SemEval-2024 Task 1: Semantic Textual Relatedness for African and Asian Languages
<p>This is the GitHub repository hosting data and code for SemEval-2024 Task 1: Semantic Textual Relatedness for African and Asian Languages. For additional information about the task, please consult the official website.</p>
SemEval-2024 Task 8: Multidomain, Multimodel and Multilingual Black-Box Machine-Generated Text Detection
<p>Large language models (LLMs) are becoming mainstream and easily accessible, ushering in an explosion of machine-generated content over various channels, such as news, social media, question-answering forums, educational, and even academic contexts. Recent LLMs, such as ChatGPT and GPT-4, generate remarkably fluent responses to a wide variety of user queries. The articulate nature of such generated texts makes LLMs attractive for replacing human labor in many scenarios. However, this has also resulted in concerns regarding their potential misuse, such as spreading misinformation and causing disruptions in the education system. Since humans perform only slightly better than chance when classifying machine-generated vs. human-written text, there is a need to develop automatic systems to identify machine-generated text with the goal of mitigating its potential misuse.</p> <p>We offer three subtasks over two paradigms of text generation: (1) <strong>full text</strong> when a considered text is entirely written by a human or generated by a machine; and (2) <strong>mixed text</strong> when a machine-generated text is refined by a human or a human-written text paraphrased by a machine.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.