Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
4
datasets available to search
ShareScore release 0.9.0
Dataset results
4 results for “East Slavic”
A hybrid approach to the small unannotated corpus-based language comparison and its application to the Old East Slavic charters - Supplementary material 1 (Old East Slavic)
<h1>Old East Slavic charters (XII - XIV century)</h1> <h2>General description</h2> <p>A set of nine historical Old East Slavic legal texts from Smolensk, Polack and Novgorod from the end of the XII century to the first half of the XIV century. The source for Smolensk charters is Avanesov (1963), which contains original texts as well as their initial deciphering (not machine-readable). The source for Polack charters is Horoshkevich (2015), containing trascribed machine-readable texts that required only an additional check and preparation. The source for Novgorod charters is Napierskij (1857), carrying original texts and their initial non-machine-readable deciphering. All the texts underwent an additional preprocessing of reconstructed and contracted parts deletion in order to better represent the actual texts under consideration and exclude as many research biases as it is possible. The last step was a manual tokenisation and the joining of each text into a single string.</p> <p>The data statement is available among the downloadable files.</p> <h2>How-to</h2> <p>This section contains the tutorials that allow to use this data with the intended pipelines.</p> <h3>Corpus-based distance measurement package</h3> <p>The source code for package is available <a href="https://doi.org/10.5281/zenodo.13958502" target="_blank" rel="noopener">here</a>, the manual is available in the <a href="https://github.com/The-One-Who-Speaks-and-Depicts/corpus_distance/blob/dev/README.md" target="_blank" rel="noopener">README</a> section of the repository.</p> <p>To use this dataset for the measurement of distance between Smolensk, Polack and Novgorod lects, and their subsequent clusterisation, following steps should be completed:</p> <ol> <li>Download the <a href="https://github.com/The-One-Who-Speaks-and-Depicts/corpus_distance/blob/dev/example/Corpus_distance_tutorial.ipynb" target="_blank" rel="noopener">Jupyter notebook</a> that streamlines the package use.</li> <li>Download the dataset.</li> <li>Put the dataset into a selected folder on your computer (make sure there are no other files within this folder).</li> <li>Insert the path to the directory into <code>CONTENT_DIR</code> variable in the Jupyter notebook.</li> <li>Run the notebook, adjusting the parameters, if necessary.</li> </ol> <h2> </h2>
A hybrid approach to the small unannotated corpus-based language comparison and its application to the Old East Slavic charters - Supplementary material 3 (Modern standard Slavic lects)
<h1>Modern standard Slavic lects (Croatian, Slovak, Slovenian)</h1> <h2>General description</h2> <p>The dataset consists of texts, written in three modern stanard Slavic lects: Croatian, Slovak, and Slovenian. The texts are parallel in order to compensate for the possible genre influences. The text is John’s Gospel in each of the given languages.</p> <h3>Sources</h3> <p>Croatian original text is from the <a href="https://www.wordproject.org/bibles/cr/index.htm">Ivan Šarić’s translation</a> of New Testament. Slovenian text is from the <a href="https://www.bible.com/bible/2319/JHN.1.SSP">standard Slovenian translation</a> of the New Testament. Slovak text is from the modern <a href="https://svatepismo.sk/evanjelium-podla-jana-1">Catholic translation</a> of the New Testament.</p> <p>The data statement is available among the downloadable files.</p> <h2>How-to</h2> <p>This section contains the tutorials that allow to use this data with the intended pipelines.</p> <h3>Corpus-based distance measurement package</h3> <p>The source code for package is available <a href="https://doi.org/10.5281/zenodo.13958502" target="_blank" rel="noopener">here</a>, the manual is available in the <a href="https://github.com/The-One-Who-Speaks-and-Depicts/corpus_distance/blob/dev/README.md" target="_blank" rel="noopener">README</a> section of the repository.</p> <p>To use this dataset for the measurement of distance between Slovak, Croatian and Slovenian lects, and their subsequent clusterisation, following steps should be completed:</p> <ol> <li>Download the <a href="https://github.com/The-One-Who-Speaks-and-Depicts/corpus_distance/blob/dev/example/Corpus_distance_tutorial.ipynb" target="_blank" rel="noopener">Jupyter notebook</a> that streamlines the package use.</li> <li>Download the dataset.</li> <li>Put the dataset into a selected folder on your computer (make sure there are no other files within this folder).</li> <li>Insert the path to the directory into <code>CONTENT_DIR</code> variable in the Jupyter notebook.</li> <li>Run the notebook, adjusting the parameters, if necessary.</li> </ol> <h2> </h2>
A hybrid approach to the small unannotated corpus-based language comparison and its application to the Old East Slavic charters - Supplementary material 5 (Corpus-based language distance measurement results)
<h1>General description</h1> <p>These are the results of the experiments with the use of <a href="https://doi.org/10.5281/zenodo.11395683" target="_blank" rel="noopener">corpus-based language distance measurement package</a> on the material of <a href="https://doi.org/10.5281/zenodo.14057668" target="_blank" rel="noopener">Old East Slavic</a>, <a href="https://doi.org/10.5281/zenodo.14148179" target="_blank" rel="noopener">modern East Slavic</a>, and <a href="https://doi.org/10.5281/zenodo.14148561">modern standard Slavic</a> lects. There are 40 possible experiments for each data set, divided by the usage of:</p> <ul> <li>topic antimodelling heuristic,</li> <li>Soerensen-Dice coefficient-based normalisation,</li> <li>the presence of hybridisation of frequency-based metric for coinciding units and combined frequency-based metric and string similarity measure for non-coinciding units,</li> <li>hybridisation type,</li> <li>exact type of string similarity measure used for combination,</li> <li>alphabet entropy-based normalisation for vector-based string similarity measures.</li> </ul> <p>In addition, modern standard Slavic dataset undergoes experiments 4 times that differ by the share of its size, used for measurements (0.1, 0.3, 0.6 and 1).</p> <p>For further information on each of the experiment parameters, refer to the documentation of the package.</p> <h1>Data set structure</h1> <h2>Executive summary</h2> <p>Data set consists of 240 folders that represent information on the experiments and 1 <code>.csv</code>-file that aggregates the resulting values into a single table.</p> <p>Folders with indices 1-20 and 121-140 contain experiments with 0.1 share of the modern standard Slavic dataset; the first sequence applies topic antimodelling heuristic, the second sequence does not employ it.</p> <p>Folders with indices 21-40 and 141-160 contain experiments with 0.3 share of the modern standard Slavic dataset; the first sequence applies topic antimodelling heuristic, the second sequence does not employ it.</p> <p>Folders with indices 41-60 and 161-180 contain experiments with 0.6 share of the modern standard Slavic dataset; the first sequence applies topic antimodelling heuristic, the second sequence does not employ it.</p> <p>Folders with indices 61-80 and 181-200 contain experiments with the full share of the modern standard Slavic dataset; the first sequence applies topic antimodelling heuristic, the second sequence does not employ it.</p> <p>Folders with indices 81-100 and 201-220 contain experiments with the full share of the modern East Slavic dataset; the first sequence applies topic antimodelling heuristic, the second sequence does not employ it.</p> <p>Folders with indices 101-120 and 221-240 contain experiments with the full share of the Old East Slavic dataset; the first sequence applies topic antimodelling heuristic, the second sequence does not employ it.</p> <h2>.csv-file</h2> <p>Named <code>aggregated_results.csv</code>, lies in the root of the dataset. Separator is <strong>comma</strong> (<strong>,</strong>). Contains <strong>13 columns</strong> and <strong>241 row</strong>. The first row is <strong>header</strong>, the other 240 rows contain description for each conducted experiment and its resulting values, according to the columns. The columns are the following (in <strong>rtl</strong> order):</p> <ul> <li><strong>X.</strong> (<em>int</em>) - experiment ID; column is used as index.</li> <li><strong>Material </strong>(<em>string</em>) - the data set used for language distance measurement. The possible values are: <ul> <li>Slavic standard - Croatian, Slovenian, Slovak standard lects.</li> <li>Modern East Slavic - Northern Russian lect Megra, Central Russian lect Belogornoje, and Northern Belarusian lect Zialionka.</li> <li>Old East Slavic - Novgorod, Polack and Smolensk parts of the Old East Slavic continuum.</li> </ul> </li> <li><strong>Gensim </strong>(<em>int</em>) - the binary numeric indicator (0 or 1) of using the heuristic of <em>topic antimodelling</em>, namely, cleaning the words that were defined as a topic words by <em>gensim</em> Latent Dirichlet Association implementation (Rehurek & Sojka, 2010). The intention of using this heuristic is to remove the tokens that are characteristic for the genre of the texts presented in the corpus for the sake of increasing the presence of the tokens that are characteristic of the lects themselves.</li> <li><strong>Split</strong> (<em>float</em>) - the used share of the data set (from 0 to 1); required to check the influence of the data set size on the metric efficiency.</li> <li><strong>Hybridisation</strong> (<em>string</em>) - the indicator of implementation of the hybridisation between the frequency-based metric between the 3-shingles (character 3-grams) that coincide for the compared lect pair, and the combination of frequency-based metric and string similarity measure between the 3-shingles that do not coincide for the compared lect pair. The possible values are: <ul> <li>TRUE: the experiment utilises hybridisation</li> <li>FALSE: the experiment does not utilise hybridisation.</li> </ul> </li> <li><strong>Hybridisation_type</strong> (<em>string</em>) - the indicator of how the frequency metric between coinciding 3-shingles and the combined metric between non-coinciding 3-shingles undergo the hybridisation process. The values are:<br> <ul> <li>JOINED - the approach is to multiply the means of the two.</li> <li>ARRAY - the approach is to join all the values into a single list, and then to score the mean.</li> <li>NOT_USED - experiment does not employ hybridisation. (<strong>Hybridisation </strong>is FALSE).</li> </ul> </li> <li><strong>Soerensen_normalisation</strong> (<em>string</em>) - the indicator of whether the frequency-based metric value undergoes normalisation with the division by Soerensen-Dice coefficient (a measure of number of coincidences between two lists) (Soerensen, 1948), in order to compensate the skewing between the coinciding and non-coinciding 3-shingles of the lects. The values are: <ul> <li>NOT_USED - <strong>Hybridisation_type </strong>is ARRAY, so there are no values to use Soerensen-Dice coefficient on.</li> <li>TRUE - the frequency-based metric undergoes division by the Soerensen-Dice coefficient.</li> <li>FALSE - the algorithm does not apply the normalisation by the Soerensen-Dice coefficient.</li> </ul> </li> <li><strong>Alphabet_normalisation</strong> (<em>string</em>) - indicator of whether the algorithm applies normalisation with the alphabet entropy (Shannon, 1948), the measure of differences in the skewings of symbols distribution in the texts, between the given lects. The values are:<br> <ul> <li>NOT_USED - a heuristic may not be implemented; present either in the cases, when <strong>Hybridisation </strong>is FALSE, or when the next parameter, <strong>Auxiliary_metrics</strong> is not VDND or VWJDND.</li> <li>TRUE<strong> </strong>- the experiment employs the heuristic.</li> <li>FALSE - the experiment does not employ the heuristic.</li> </ul> </li> <li><strong>Auxiliary_metrics </strong>(<em>string</em>) - the string similarity measure, used for the combination with the frequency-based metric for non-coinciding 3-shingles between analysed lects. There are five possible values: <ul> <li>LDND (Levenshtein distance normalised between analysed 3-shingles) (Holman et al., 2008).</li> <li>WJWDND (weighted Jaro-Winkler distance normalised between analysed 3-shingles) (Gueddah et al., 2015).</li> <li>VDND (Euclidean distance between the sums of symbol vector values between 3-shingles).</li> <li>VWJDND (VDND multiplied by scoring Jaro (Jaro, 1989) distance between analysed 3-shingles).</li> </ul> </li> <li><strong>Outgroup.identification</strong> (<em>string</em>) - the indicator of whether the outgroup detected in the given experiment coincides with the lect that preliminary manual classification supposes to be the outgroup. There are two possible values:<br> <ul> <li>CORRECT - the detected outgroup coincides with the supposed one.</li> <li>INCORRECT - the detected outgroup does not coincide with the supposed one.</li> </ul> </li> <li><strong>Outer.distance.split </strong>(<em>float</em>) - the length of the outgroup branch.</li> <li><strong>Inner.distance.split</strong> (<em>float</em>) - the distance between the split between the outgroup and the ingroup, and the split between the two ingroup lects.</li> <li><strong>Split.difference</strong> (<em>float</em>) - the division of <strong>Outer.distance.split</strong> by <strong>Inner.distance.split</strong>.</li> </ul> <h2>Folders</h2> <p>Each folder contains 6 files, each named according to the used experiment setup:</p> <ul> <li>3 <code>.csv</code>-files that contain unit-by-unit comparison between each pair of the analysed lects. Each <code>.csv</code>-file is <strong>semi-colon</strong>-separated, and has <strong>4 </strong>columns, <strong>header row</strong>, and rows that describes each unit-to-unit comparison. The columns contain the following information (in <strong>rtl</strong> order):<br> <ul> <li><code>[Name of the first compared lect]</code> : unit (character 3-shingle, or just 3-shingle) of the [name of the first compared lect] that undergoes comparison with units of the [name of the second compared lect]; datatype: string.</li> <li><code>[Name of the second compared lect] </code>: unit (character 3-shingle, or just 3-shingle) of the [name of the second compared lect] that undergoes comparison with units of the [name of the first compared lect]; if units coincide, contains value <code>id.</code>; datatype: string.</li> <li><code>[Experiment setup]</code>: name of the metric, a combination of the [experiment setup](concatenated through <code><em>-</em></code> parameters) and its exact part, which compares the two units; datatype: string. The possible values are: <ul> <li><code>[experiment setup] - DistRank</code> - the frequency-based metric that compares identical units</li> <li><code>[experiment setup] - hybrid</code> - the string similarity measure for non-identical units, combined with the frequency-based metric </li> </ul> </li> <li><code>Distance</code>: value of the metric; datatype: float</li> </ul> </li> <li><code>.info</code>-file that contains data on branch lengths along with coincidence/non-coincidence of the detected outgroup with the manually defined one. The file is a <strong>tabular-separated plain text</strong> that always contains three values: coincidence (CORRECT)/non-coincidence (INCORRECT) of the yielded classification with the supposed one; outer distance split (the length of the outgroup branch; datatype: float) and inner distance split (the length of the ingroup branch before split of its lects; datatype: float).</li> <li><code>.newick</code> -file that contains the result of an experiment, the phylogenetic tree built by UPGMA classifier. One can read it with <a href="https://cran.r-project.org/web/packages/TreeTools/vignettes/load-trees.html">ape::read.tree</a> (R), or <a href="https://biopython.org/wiki/Phylo">Phylo.read</a> (Python).</li> <li><code>.png</code> -file that contains the phylogenetic tree visualisation.</li> </ul> <p> </p> <h1>How-to</h1> <p>For the analysis of the results, download and unpack the archive, and further refer to the <a href="https://doi.org/10.5281/zenodo.14169792" target="_blank" rel="noopener">companion R notebook</a>. </p>
A hybrid approach to the small unannotated corpus-based language comparison and its application to the Old East Slavic charters - Supplementary material 2 (Modern East Slavic)
<h1>Modern East Slavic dialects (Belogornoje, Megra, Zialionka)</h1> <h2>General description</h2> <p>A set of modern East Slavic Belogornoje, Megra and Zialionka small territorial lects subcorpora. Megra is an autochthonous (Barannikova, 2005) Northern Russian small territorial lect (Kryuchkova and Goldin, 2011). Belogornoje is a late settlement (Barannikova, 2005) Central Russian small territorial lect (Kryuchkova and Goldin, 2011). Zialionka is an autochthonous Northern Belarusian lect, radically different from Belogornoje and Megra by most of the key isoglosses within the Eastern part of the Slavic continuum.</p> <h3>Sources</h3> <p>Both Megra and Belogornoje texts originate from the Saratov dialectological corpus (Kryuchkova and Goldin, 2011). These are manually transcribed interviews with dialect speakers, mostly on the slice-of-life, rarely touching the topic of religion, recorded during the field trips of Saratov State University from 1980 to 2019. They possess some tagging, but for the purpose of clear cross-evaluation, the experiments do not use this information. The transcription is phonemic, faithful to the dialect features, and remains untouched in the experiments.</p> <p><br>Zialionka texts are also phonemically transcribed and untouched in experiments, they come from the Polack ethnographic collection (Lobač, 2011). The main genre is folklore tales, collected by transcribing interviews with small territorial lects speakers during the field trips of Polack State University (Belarus) from 1992 to 2010 years. There are no traces of notable phonetic irregularities within the texts. Unfortunately, there is no way to reliably establish it, as there are no available original recordings.</p> <p>The data statement is available among the downloadable files.</p> <h2>How-to</h2> <p>This section contains the tutorials that allow to use this data with the intended pipelines.</p> <h3>Corpus-based distance measurement package</h3> <p>The source code for package is available <a href="https://doi.org/10.5281/zenodo.13958502" target="_blank" rel="noopener">here</a>, the manual is available in the <a href="https://github.com/The-One-Who-Speaks-and-Depicts/corpus_distance/blob/dev/README.md" target="_blank" rel="noopener">README</a> section of the repository.</p> <p>To use this dataset for the measurement of distance between Belogornoje, Megra and Zialionka lects, and their subsequent clusterisation, following steps should be completed:</p> <ol> <li>Download the <a href="https://github.com/The-One-Who-Speaks-and-Depicts/corpus_distance/blob/dev/example/Corpus_distance_tutorial.ipynb" target="_blank" rel="noopener">Jupyter notebook</a> that streamlines the package use.</li> <li>Download the dataset.</li> <li>Put the dataset into a selected folder on your computer (make sure there are no other files within this folder).</li> <li>Insert the path to the directory into <code>CONTENT_DIR</code> variable in the Jupyter notebook.</li> <li>Run the notebook, adjusting the parameters, if necessary.</li> </ol> <h2> </h2>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.