Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
11
datasets available to search
ShareScore release 0.9.0
Dataset results
11 results for “word sense”
Word-in-Context Target Sense Verification
<pre>Formally, WiC is framed as a <strong>binary classification</strong> task. Each instance in WiC-TSV consists of a target word <em>w</em> with a corresponding target sense <em>s</em> represented by either its definition (subtask 1) or its hypernym/s (subtask 2), and a context <em>c</em> containing the target word <em>w</em>. The task aims to determine whether the meaning of the word <em>w</em> used in the context <em>c</em> matches the target sense <em>s</em>. In the following table there are some examples from the dataset. </pre> <p> </p> <p>Subtasks</p> <p> WiC-TSV has <strong>three subtasks</strong> - participants can submit results in any of the subtasks:</p> <p>Subtask 1: Definitions</p> <p>In Subtask 1 systems make use of <strong>definitions</strong> for deciding whether the target word in context corresponds to the given definition or not.</p> <p>Subtask 2: Hypernyms</p> <p>In Subtask 2 systems make use of <strong>hypernymy</strong> information for deciding whether the target word in context is a hyponym of the given hypernym or not.</p> <p>Subtask 3: Definitions + Hypernyms</p> <p>In subtask 3 systems can make use of <strong>both</strong> sources of information, i.e., definitions and hypernyms.</p>
RUSSE'2018: Human-Annotated Sense-Disambiguated Word Contexts for Russian
<p>This dataset contains human-annotated sense identifiers for 2562 contexts of 20 words used in the <a href="https://russe.nlpub.org/2018/wsi/">RUSSE'2018</a> shared task on Word Sense Induction and Disambiguation for the Russian language; part of the <em>bts-rnc</em> evaluation dataset. These sense identifiers are disambiguated as according to the sense inventory of the <a href="http://gramota.ru/slovari/info/bts/">Large Explanatory Dictionary of Russian</a>.</p> <p>The annotation is done on December 1, 2017, on the <a href="https://tolokanyandex.com/">Yandex.Toloka</a> crowdsourcing platform. In particular, 80 pre-annotated contexts are used for training the human annotators, 2562 contexts are annotated by humans such that each context was annotated by 9 different annotators. The annotation reliability is indicated by a high value of Krippendorff's α = 0.83. After the annotation, every context was additionally inspected (“curated”) by the organizers of the shared task.</p> <p>The following words are represented: <em>акция</em> (action / stock), <em>байка</em> (yarn / tale), <em>гвоздика</em> (carnation / nail), <em>гипербола</em> (hyperbole), <em>град</em> (avalanche), <em>гусеница</em> (grub), <em>домино</em> (domino), <em>кабачок</em> (marrow / pub), <em>капот</em> (hood), <em>карьер</em> (mine / career), <em>кок</em> (cook), <em>крона</em> (top / crown), <em>круп</em> (croup), <em>мандарин</em> (mandarine), <em>рок</em> (fate / rock), <em>слог</em> (syllable), <em>стопка</em> (glass, stack), <em>таз</em> (bowl), <em>такса</em> (rate / badger-dog), <em>шах</em> (shah / check).</p> <p>The following files are included in this dataset:</p> <ul> <li>Toloka assignments (training: <em>tasks-train.tsv</em>, annotation: <em>tasks-test.tsv</em>)</li> <li>Toloka output (non-aggregated: <em>assignments_01-12-2017.tsv.xz</em>, aggregated: <em>aggregated_results_pool_1036853__2017_12_01.tsv</em>)</li> <li>annotator agreement report (<em>agreement.txt</em>)</li> <li>curated report (<em>report-curated.tsv.xz</em> and a supplementary file <em>tasks-eval.tsv.xz</em>)</li> <li>the final aggregated dataset (<em>bts-rnc-crowd.tsv</em>)</li> </ul> <p>The <em>bts-rnc-crowd.tsv</em> file has the following format: <em>id</em>, <em>lemma</em>, <em>sense_id</em>, <em>left</em> hand side context, <em>word</em> form, <em>right</em> hand side context, list of <em>senses</em>. The encoding is UTF-8 and the line breaks are LF (UNIX).</p>
French Word Sense Disambiguation with Princeton WordNet Identifiers
<p>This is a dataset for the Word Sense Disambiguation of French using Princeton WordNet identifiers. It contains two training corpora : the SemCor and the WordNet Gloss Corpus, both automatically translated from their original English version, and with sense tags automatically aligned. It contains also a test corpus : the task 12 of SemEval 2013, originally sense annotated with BabelNet identifiers, converted into Princeton WordNet 3.0.</p>
Word Sense Change Testset
<p><strong>Overview</strong></p> <p>This testset consists of 23 terms which have experienced word sense change during the past centuries. The main changes for each term were found using Wikipedia, dictionary.com and the Oxford English Dictionary. We consider major changes in usage as well as changes to sense. In cases where multiple (fine-grained) senses were available, we opted to accept the widest sense. E.g. for the term <em>rock</em> we consider a music sense without any distinction between different types of <em>rock music</em>, because our dataset is unlikely to have fine-grained sense differentiations. If a clear time point cannot be pinpointed, we choose the earliest possible. For comparison purposes we also chose a set of 11 terms that have experienced minimal change during the investigated period, i.e., stable terms.</p> <p> </p> <p><strong>Supplementary material</strong></p> <p><em>1. testset.txt </em></p> <p>Contains a list of all terms and the different change types for each term with a short description of the sense and change.</p> <p> </p> <p><em>2. Files of the kind "TERM.txt"</em></p> <p>The header tells us the term, which clustering coefficient was used, which similarity threshold and which similarity measure.</p> <p>A path starts with "Path:". </p> <p>A unit starts with "UNIT:"</p> <p>and the numbers following indicate 1. the number of years that the unit spans, and then a list of all years that the internal clusters stem from.</p> <p>E.g., UNIT: 83 1785, 1787, 1790, 1793, 1798, 1801, 1823, 1867, spanns 83 years and consists of clusters from year 1785, 1787, 1790 etc.</p> <p>Indentation shows the tree structure, more indentation means lower level branch in the tree.</p> <p>As an example, in AEROPLANE.txt unit UNIT: 23 1908, 1909, 1910, 1911, 1914, 1918, 1930, 1908 is the root node and the unit is related to UNIT: 27 1916, 1919, 1924, 1932, 1942, 1916.</p> <p> </p> <p><strong>Interesting findings</strong></p> <p>The longest units and paths are found for stable terms, e.g., <em>newspaper</em>. These are statistically significantly longer than the average units and paths for terms that later evolve.</p> <p><em>Newspaper</em> has a unit that spans 145 years and the first path spans from 1852 - 2007.</p> <p> </p> <p><em>FLIGHT.txt</em></p> <p>For the term <em>flight</em> we find that the first unit captures a name, <em>Flight & Robson</em> who were organ builders.</p> <p>The second unit (it its own path) represents the <em>flight</em> over a hurdle: UNIT: 28 1868, 1869, 1870, 1877, 1885, 1889, 1890, 1892, 1893, 1894, 1895</p> <p>There is a unit (it its own path) that represents the <em>flight</em> of a cricket ball: UNIT: 29 1938, 1957, 1966</p> <p>Finally, the last path represents <em>flight</em> as in a means of transportation, in particular for holidays, starting with UNIT: 19 1962, 1970, 1973, 1980</p> <p> </p> <p><em>TAPE.txt</em></p> <p>The first path for tape is a path related to <em>sowing tape</em>.</p> <p>Then there is a second path starting with UNIT: 38 1970, 1974, 2007 that takes up the <em>musical tape</em>.</p> <p>The last path end in the same units that the second path ends in, also related to the <em>musical tape</em>.</p> <p>The <em>music tape</em> and the <em>sowing tape</em> should be related because of their shape, but we cannot find any relation as there are few or no overlapping terms.</p> <p> </p>
Unsupervised Does Not Mean Uninterpretable: The Case for Word Sense Induction and Disambiguation
<p>This dataset contains the models for interpretable Word Sense Disambiguation (WSD) that were employed in Panchenko et al. (2017; the paper can be accessed at https://www.lt.informatik.tu-darmstadt.de/fileadmin/user_upload/Group_LangTech/publications/EACL_Interpretability___FINAL__1_.pdf).</p> <p>The files were computed on a 2015 dump from the English Wikipedia. Their contents:</p> <ul> <li>Induced Sense Inventories: <strong>wp_stanford_sense_inventories.tar.gz</strong><br> This file contains 3 inventories (coarse, medium fine)</li> <li>Language Model (3-gram): <strong>wiki_text.3.arpa.gz</strong><br> This file contains all n-grams up to n=3 and can be loaded into an index</li> <li>Weighted Dependency Features: <strong>wp_stanford_lemma_LMI_s0.0_w2_f2_wf2_wpfmax1000_wpfmin2_p1000.gz</strong><br> This file contains weighted word--context-feature combinations and includes their count and an LMI significance score</li> <li>Distributional Thesaurus (DT) of Dependency Features: <strong>wp_stanford_lemma_BIM_LMI_s0.0_w2_f2_wf2_wpfmax1000_wpfmin2_p1000_simsortlimit200_feature expansion.gz</strong><br> This file contains a DT of context features. The context feature similarities can be used for context expansion</li> </ul> <p>For further information, consult the paper and the companion page: http://jobimtext.org/wsd/</p> <p>Panchenko A., Ruppert E., Faralli S., Ponzetto S. P., and Biemann C. (2017): Unsupervised Does Not Mean Uninterpretable: The Case for Word Sense Induction and Disambiguation. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL'2017). Valencia, Spain. Association for Computational Linguistics.</p> <p> </p> <p> </p>
DWUG DE Sense: A data set of historical word sense annotations in German
<p>This data collection contains a subset of <a href="https://zenodo.org/record/5543723">DWUG DE</a> word usage data annotated with classical word sense definitions (<em>DWUG DE Sense</em>, see <code>data/*/judgments_senses.csv</code>). From these annotations aggregated and cleaned sense labels were derived (<code>labels/*/labels_senses.csv</code>). From these labels we derived additional binary semantic proximity labels between use pairs ('0' for different sense, '1' for same sense, <code>labels/*/labels_proximity.csv</code>) and change labels reflecting sense changes between the two time periods from which word usages were sampled (<code>stats/*/stats_groupings.csv</code>).</p> <p>The sense labels were derived from the sense annotation by removing instances where not at least 2/3 annotators agree on the label (<code>maj_2</code>/<code>maj_3</code>). Note that the binary proximity labels were <em>derived</em> from the sense annotation, and not directly judged by humans (in contrast to other <a href="https://www.ims.uni-stuttgart.de/data/wugs">WUG data sets</a>). Note that consequently also the change scores EARLIER, LATER and COMPARE were not calculated directly from human judgments, but from the inferred binary proximity labels. Please find the code aggregating and cleaning the data, deriving proximity labels and deriving change labels in the <a href="https://github.com/Garrafao/WUGs">WUG repository</a>.</p> <p>Please find more information on the provided data in the paper referenced below.</p> <p>Version: 1.0.1, 01.11.2024. Correct or remove some normalization and lemmatization errors in the uses. Updated references.</p> <h3>Reference</h3> <p>Dominik Schlechtweg, Frank D. Zamora-Reina, Felipe Bravo-Marquez, Nikolay Arefyev. 2024. <a href="https://doi.org/10.1007/s10579-024-09771-7">Sense Through Time: Diachronic Word Sense Annotations for Word Sense Induction and Lexical Semantic Change Detection</a>. Language Resources and Evaluation.</p> <p>Dominik Schlechtweg. 2023. <a href="http://dx.doi.org/10.18419/opus-12833">Human and Computational Measurement of Lexical Semantic Change</a>. PhD thesis. University of Stuttgart.</p>
SemEval-2013 Task 13: Word Sense Induction for Graded and Non-Graded Senses
<p>Data originally provided by the SemEval-2013 Task 13 organizers and which was available through the following link:</p> <p>https://www.cs.york.ac.uk/semeval-2013/task13/data/uploads/semeval-2013-task-13-test-data.zip</p>
SemEval-2010 Task 14: Word Sense Induction & Disambiguation
<p>Data originally provided by the SemEval-2010 Task 14 organizers and which was available through the following links:</p> <p>1. https://www.cs.york.ac.uk/semeval2010_WSI/files/evaluation.zip<br> 2. https://www.cs.york.ac.uk/semeval2010_WSI/files/training_data.tar.gz<br> 3. https://www.cs.york.ac.uk/semeval2010_WSI/files/test_data.tar.gz</p>
Word associations for 20 verb senses in Spanish
<p>Collection of word associations collected for 20 verb senses in Spanish. For more details about the reccollection methodology consult:</p> <p>Gil-Vallejo, Lara, Marta Coll-Florit, Irene Castellón, et al. 2017. Verb similarity: Comparing corpus and psycholinguistic data . <em>Corpus Linguistics and Linguistic Theory</em>. 14(2): 275-30</p> <p>degruyter.com/view/j/cllt.2018.14.issue-2/cllt-2016-0045/cllt-2016-0045.xml</p>
Sense-Tagged corpora for Word Sense Disambiguation in several languages
<p>Copies of Sense-Tagged corpora for Word Sense Disambiguation in several languages</p> <p>English, French, Spanish, Russian</p> <p> </p>
Amharic WSD Dataset: Advancing Word Sense Disambiguation in Amharic
<p>This dataset is specifically designed for the Word Sense Disambiguation (WSD) task in the Amharic language, consisting of 50,415 annotated sentences. Each sentence includes the correct sense for one of 200 ambiguous words chosen based on homonymy relations, where a single word may have multiple meanings depending on its context.</p> <p>The ambiguous words were selected to capture the nuances of Amharic vocabulary, drawing from diverse textual sources such as news articles, literature, and social media. This ensures a broad and representative range of usage across various contexts, making the dataset particularly valuable for advancing Amharic NLP research. Potential applications include improvements in machine translation, sentiment analysis, and other semantic processing tasks in Amharic.</p> <p>The dataset is organized in a structured format, with each entry containing fields for sentence, ambiguous word, sense, gloss, and sense label, facilitating ease of use for machine learning models.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.