Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
99
datasets available to search
ShareScore release 0.9.0
Dataset results
99 results for “tokenization”
Exploring Data-Driven Chemical SMILES Tokenization Approaches to Identify Key Protein-Ligand Binding Moieties
<p>This repository contains materials for the paper, "Exploring Data-Driven Chemical SMILES Tokenization Approaches to Identify Key Protein-Ligand Binding Moieties", published in <a href="https://onlinelibrary.wiley.com/doi/10.1002/minf.202300249">Molecular Informatics.</a></p> <p>`data.zip` contains vocabulary and dataset files for identifying chemical vocabularies and key chemical words associated with protein ligand binding. </p> <p>`results.zip` comprises outputs specific to vocabularies and datasets, as well as various related statistics.</p> <p> </p> <p> </p>
Replication Package for "Benefits and pitfalls of token-level SZZ: An empirical study on OSS projects"
<p>Replication Package for "Benefits and pitfalls of token-level SZZ: An empirical study on OSS projects"</p><p>All materials are licensed under the MIT License (see LICENSE file). </p>
Data set for "Token-Level Multilingual Epidemic Dataset for Event Extraction"
<p>This is the data for the TPDL 2021 paper "<a href="https://zenodo.org/record/5780020">Token-Level Multilingual Epidemic Dataset for Event Extraction</a>". If you use this resource, please cite the paper:</p> <pre><code>@inproceedings{mutuvi2021dataset, title = "Token-level Multilingual Epidemic Dataset for Event Extraction", author = {Mutuvi, Stephen and Boros, Emanuela and Doucet, Antoine, and Lejeune, Gaël and Jatowt, Adam and Odeo, Moses}, booktitle = "Proceedings of the 25th International Conference on Theory and Practice of Digital Libraries, September 13–17, 2021, TPDL 2021", year = "2021", location = "Online" }</code></pre> <p> </p> <p>This work has been supported by the European Union Horizon 2020 research and innovation programme under grants 825153 (Embeddia) and 770299 (NewsEye).</p>
OpenNLP tokenization model for Picard
<p>OpenNLP tokenization model for Picard, trained on the Restaure corpus.</p> <p>The apostrophes must be standardized in the input file: l’bas -> l'bas</p> <p>To tokenize a file: <input-file.txt opennlp TokenizerME pcd-token.bin</p>
Robot swarms neutralize harmful Byzantine robots using a blockchain-based token economy
Through cooperation, robot swarms can perform tasks or solve problems that a single robot from the swarm could not perform/solve by itself. However, it has been shown that a single Byzantine robot (e.g., a malfunctioning or malicious robot) can disrupt the coordination strategy of the entire swarm. Therefore, a versatile swarm robotics framework that addresses security issues in inter-robot communication and coordination is urgently needed. In this paper, we show that security issues can be addressed by setting up a token economy between the robots. To create and maintain the token economy, we use blockchain technology, originally developed for the digital currency Bitcoin. The robots are given crypto tokens that allow them to participate in the swarm's security-critical activities. The token economy is regulated via a smart contract that decides how to distribute crypto tokens among the robots depending on their contributions. We design the smart contract so that Byzantine robots soon run out of crypto tokens and can therefore no longer influence the rest of the swarm. In experiments with up to 24 physical robots, we demonstrate that our smart contract approach indeed works: the robots can maintain blockchain networks and a blockchain-based token economy can be used to neutralize the destructive actions of Byzantine robots in a collective-sensing scenario. In experiments with more than 100 simulated robots, we study the scalability and long-term behavior of our approach. The obtained results demonstrate the feasibility and viability of blockchain-based swarm robotics.
GROVER tokenized Human Genome hg19
<p>Data of the Human Genome Hg19, tokenised with byte-pair tokenisation of 600 cycles. Required for the DNA language model GROVER. More information can be found at https://www.biorxiv.org/content/10.1101/2023.07.19.549677v1.</p>
Urdu Text Normalization and Tokenization Dataset
<p>This dataset is made public so researchers can perform NLP tasks.</p>
Robot swarms neutralize harmful Byzantine robots using a blockchain-based token economy
Open the record for dataset details and reuse information.
Replication Package for "Did You Remember To Test Your Tokens?"
<p>Supplementary data to "Did You Remember To Test Your Tokens?", accepted for publication at MSR 2020. We include several artifacts that were generated during the analysis of JUnit tests for token authentication, as well as the original dataset.</p>
Acquisition of object-robbing and object/food-bartering behaviors: A culturally maintained token economy in free-ranging long-tailed macaques
<p>The token exchange paradigm shows that monkeys and great apes are able to use objects as symbolic tools to request specific food rewards. Such studies provide insights into the cognitive underpinnings of economic behavior in non-human primates. However, the ecological validity of these lab-based experimental situations tends to be limited. Our field research aims to address the need for a more ecologically valid primate model of trading systems in humans. Around the Uluwatu Temple in Bali, Indonesia, a large free-ranging population of long-tailed macaques spontaneously and routinely engage in token-mediated bartering interactions with humans. These interactions occur in two phases: after stealing inedible and more or less valuable objects from humans, the macaques appear to use them as tokens, by returning them to humans in exchange for food. Our field observational and experimental data showed (1) age differences in robbing/bartering success, indicative of experiential learning, and (2) clear behavioral associations between value-based token possession and quantity or quality of food rewards rejected and accepted by subadult and adult monkeys, suggestive of robbing/bartering payoff maximization and economic decision-making. This population-specific, prevalent, cross-generational, learned and socially influenced practice may be the first example of a culturally maintained token economy in free-ranging animals.</p>
Supplementary Material for "Detecting Automatic Software Plagiarism via Token Sequence Normalization"
<p>This repository contains additional material supporting the paper titled "Detecting Automatic Software Plagiarism via Token Sequence Normalization", presented at ICSE 2024 (research track).</p> <div> <div> <div> <p>The paper presents a defense mechanism against automated plagiarism generators utilizing program dependence graphs and demonstrates its effectiveness in countering insertion-based and reordering-based obfuscation attacks.</p> </div> </div> </div> <p>The defense mechanism was also integrated into the software plagiarism detector <a title="JPlag Repository on GitHub" href="https://github.com/jplag/JPlag">JPlag</a>, thus providing a widely accessible solution.</p> <p><strong>Contents Overview:</strong></p> <ul> <li> <p><strong>Datasets:</strong> Two datasets from the <a title="PROGpedia Repository" href="../record/7449056">PROGpedia</a> collection and two internal datasets. For the latter, only the metadata is available due to the sensitive nature of the data.</p> </li> <li> <p><strong>Plagiarized Submissions:</strong> Generated plagiarism instances illustrating various obfuscation methods such as insertion, reordering, and insert-after-reordering.</p> </li> <li> <p><strong>Evaluation Data:</strong> JSON files detailing calculated similarities and runtime measurements for all datasets.</p> </li> <li> <p><strong>Source Code:</strong> The implementation of our defense mechanism based on the software plagiarism detector <a title="JPlag Repository on GitHub" href="https://github.com/jplag/JPlag">JPlag</a> (v4.0.0). Note that JPlag is licensed under the GPL-3.0 license.</p> </li> <li> <p><strong>Evaluation Code:</strong> Python code for runtime measurements.</p> </li> <li> <p><strong>Interactive Plots:</strong> HTML visualizations of the paper's plots, offering dynamic insights into the research findings. particularly focusing on the detection of automatic software plagiarism through token sequence normalization.</p> </li> <li><strong>Demo:</strong> A packaged JAR of our implementation alongside an instruction on how to execute it.</li> </ul>
NY Transit Token
NEW York Transit Token Source: Objaverse 1.0 / Sketchfab
Dataset for BPM2024, Educators Forum: Comprehension of (business) process models via tokens: an eye-tracking approach
Open the record for dataset details and reuse information.
Deprecated - VAMDC extraction with query token = tipbase:ccfb5ac9-82d6-4ab2-9ec0-3aa8051a32c1:head
<p>Deprecated because the file upload was corrupted.</p> <p>This dataset comes from the VAMDC(Virtual Atomic and Molecular Data Center) node named http://tipbase.obspm.fr/12.07/vamdc/tap/ by doing these queries: (query=select * where ( atomsymbol = 'fe' ); ) . The corresponding dataset is versioned on 2017-01-30 with the XSAMS 12.07 format. This data is associated with a unique identifier uuid=a9a38253-7788-496f-b12c-b5c44751535f. Bibliographic references are: N.N. (2018). Storey, P. J. and Zeippen, C. J. and Le Dourneuf, M. (2002). Atomic data from the IRON Project. LI. Electron impact excitation of Fe IX. Astronomy and Astrophysics. Butler, K. and Zeippen, C. J. (2001). Atomic data from the IRON Project. L. Electron impact excitation of Fe xix. Astronomy and Astrophysics. Butler, K. and Zeippen, C. J. (2001). Atomic data from the IRON Project. XLIX. Electron impact excitation of Fexx. Astronomy and Astrophysics. Pelan, J. C. and Berrington, K. A. (2001). Atomic data from the IRON project. XLVI. Electron excitation of 3s3p6 and 3s23p43d fine-structure transitions in Fe x. Astronomy and Astrophysics. Chen, Guo Xin and Pradhan, Anil K. (2000). Atomic data from the Iron Project. XLIV. Transition probabilities and line ratios for Fe VI with fluorescent excitation in planetary nebulae. Astronomy and Astrophysics Supplement Series. Chidichimo, M. C. and Zeman, V. and Tully, J. A. and Berrington, K. A. (2000). Erratum: Atomic Data from the IRON Project. Astronomy and Astrophysics Supplement Series. Storey, P. J. and Mason, H. E. and Young, P. R. (2000). Atomic data from the IRON Project. XL. Electron impact excitation of the Fe XIV EUV transitions. Astronomy and Astrophysics Supplement Series. Chidichimo, M. C. and Zeman, V. and Tully, J. A. and Berrington, K. A. (1999). Atomic data from the IRON Project. XXXVI. Electron excitation of Be-like Fe XXIII between 1s(2 2l_1 2l_2 SLJ) and 1s(2 2l_3 2l_4 S'L'J'). Astronomy and Astrophysics Supplement Series. Eissner, W. and Galavis, M. E. and Mendoza, C. and Zeippen, C. J. (1999). Atomic data from the IRON Project. XXXVIII. Electron impact excitation of the fine-structure transitions in the n=3 complex of Fe XV. Astronomy and Astrophysics Supplement Series. Chen, G. X. and Pradhan, A. K. (1999). Atomic data from the Iron Project. XXXVII. Electron impact excitation collision strengths and rate coefficients for Fe VI. Astronomy and Astrophysics Supplement Series. Eissner, W. and Galavis, M. E. and Mendoza, C. and Zeippen, C. J. (1999). Atomic data from the IRON Project. XXXIV. Electron impact excitation of Fe XVI. Astronomy and Astrophysics Supplement Series. Nahar, S. N. and Pradhan, A. K. (1999). Atomic data from the Iron Project. XXXV. Relativistic fine structure oscillator strengths for Fe XXIV and Fe XXV. Astronomy and Astrophysics Supplement Series. Galavis, M. E. and Mendoza, C. and Zeippen, C. J. (1998). Atomic data from the IRON Project. XXXII. On the accuracy of the effective collision strength for the electron impact excitation of the quadrupole transition in AR III. Astronomy and Astrophysics Supplement Series. Binello, A. M. and Mason, H. E. and Storey, P. J. (1998). Atomic data from the IRON project. XXXI. Electron impact excitation of optically allowed and intercombination electric dipole transitions in Fe XII. Astronomy and Astrophysics Supplement Series. Berrington, K. A. and Saraph, H. E. and Tully, J. A. (1998). Atomic data from the IRON Project. XXVIII. Electron excitation of the ^2P_(3/2) deg -> ^2P_(1/2) deg fine structure transition in fluorine-like ions at higher temperatures. Astronomy and Astrophysics Supplement Series. Binello, A. M. and Mason, H. E. and Storey, P. J. (1998). Atomic data from the IRON Project. XXV. Electron impact excitation of fine-structure transitions in the ground configuration of Fe XII. Astronomy and Astrophysics Supplement Series. Zhang, H. L. and Pradhan, A. K. (1997). Atomic data from the Iron Project. XXVII. Electron impact excitation collision strengths and rate coefficients for Fe IV. Astronomy and Astrophysics Supplement Series. Berrington, K. A. and Tully, J. A. (1997). Atomic data from the IRON Project. XXIV. Electron excitation of Li-like Fe XXIV between the N = 2 and n' = 2, 3, 4 fine-structure levels. Astronomy and Astrophysics Supplement Series. Zhang, H. L. and Pradhan, A. K. (1997). Atomic data from the IRON Project. XXIII. Relativistic excitation rate coefficients for Fe XXII with inclusion of radiation damping. Astronomy and Astrophysics Supplement Series. Zhang, H. (1996). Atomic data from the Iron Project. XVIII. Electron impact excitation collision strengths and rate coefficients for Fe III.. Astronomy and Astrophysics Supplement Series. Berrington, K. A. and Pelan, J. (1996). Erratum: "Atomic data from the IRON Project. XII. Electron excitation of forbidden transitions in V-like ions Mn III, Fe IV, Co V and Ni VI" [Astron. Astrophys., Suppl. Ser., Vol. 114, No. 2, p. 367 - 371 (Dec 1995)].. Astronomy and Astrophysics Supplement Series. Storey, P. J. and Mason, H. E. and Saraph, H. E. (1996). Atomic data from the IRON Project. XIV. Electron impact excitation of the Fe XIV fine-structure transition ^2^P^o^_1/2_-^2^P^o^_3/2_. Astronomy and Astrophysics. Bautista, M. A. and Pradhan, A. K. (1996). Atomic data from the Iron project. XIII. Electron excitation rates and emissivity ratios for forbidden transitions in NI II and Fe II.. Astronomy and Astrophysics Supplement Series. Berrington, K. A. and Pelan, J. C. (1995). Atomic data from the IRON Project. XII. Electron excitation of forbidden transitions in V-like ions MN III, Fe IV, CO V and NI VI.. Astronomy and Astrophysics Supplement Series. Galavis, M. E. and Mendoza, C. and Zeippen, C. J. (1995). Atomic data from the IRON Project. X. Effective collision strengths for infrared transitions in silicon- and sulphur-like ions.. Astronomy and Astrophysics Supplement Series. Pelan, J. and Berrington, K. A. (1995). Atomic data from the IRON Project. IX. Electron excitation of the ^2^P^0^_3/2_-^2^P^0^_1/2_ fine-structure transition in chlorine-like ions, from AR II to NI XII.. Astronomy and Astrophysics Supplement Series. Berrington, K. A. (1995). Atomic data from the IRON Project. VIII. Electron excitation of the 3d^4^ ^5^D_J_ ground state fine-structure transitions in Ti-like ions V II, CR III, MN IV, Fe V, CO VI and NI VII.. Astronomy and Astrophysics Supplement Series. Zhang, H. L. and Pradhan, A. K. (1995). Atomic data from the Iron Project. VI. Collision strengths and rate coefficients for Fe II.. Astronomy and Astrophysics. Edlen, Bengt (1983). Comparison of Theoretical and Experimental Level Values of the n = 2 Configurations in the Boron Isoelectronic Sequence. Physica Scripta. Jupen, C. and Isler, R. C. and Trabert, E. (1993). Solar Identifications of Fex-Fexiv Based on Comparison with Beam-Foil Tokamak and Laser-Produced Plasma Spectra. Monthly Notices of the Royal Astronomical Society. Shirai T. and Sugar J. and Musgrove (2000). Spectral Data for Highly Ionized Atoms: TI, V, Cr, Mn, Fe, Ni, Cu, Kr, and Mo. Journal of Physical and Chemical Reference Data monograph 8. Shirai T. and Sugar J. and Musgrove (1985). Atomic Energy Levels of the Iron Period Elements: Potassium through Nickel. J. Phys. Chem. Ref. Data 14, Supplement No. 2. Reader, Joseph and Sugar, Jack and Acquista, Nicolo and Bahr, Raymond (1994). Laser-produced and tokamak spectra of lithiumlike iron, Fe23 +. Journal of the Optical Society of America B: Optical Physics. Ballance, C. P. and Badnell, N. R. and Berrington, K. A. (2002). Electron-impact excitation of H-like Fe at high temperatures. Journal of Physics B Atomic Molecular Physics. Ekberg, J. O. (1981). Term Analysis of Fe VII. Physica Scripta. Churilov, S. S. and Levashov, V. E. (1993). The 3p23d and 3s3d2 configurations in aluminium-like KV VII-Ni XVI. Physica Scripta. Churilov, S. S. and Levashov, V. E. and Wyart, J. F. (1989). Extended analysis of the 3d2-3p3d transitions in the sequence K VIII-Cu XVIII and isoelectronic trends in Mg-like ions through Kr XXV. Physica Scripta. Churilov, S. S. and Kononov, E. Ya and Ryabtsev, A. N. and Zayikin, Yu F. (1985). A Detailed Analysis of the n=3 - n'=3 Transitions in the Mg-like Ions FeXV, CoXVI and NiXVII. Physica Scripta. Whiteford, A. D. and Badnell, N. R. and Ballance, C. P. and O'Mullane, M. G. and Summers, H. P. and Thomas, A. L. (2001). A radiation-damped <strong>R</strong>-matrix approach to the electron-impact excitation of helium-like ions for diagnostic application to fusion and astrophysical plasmas. Journal of Physics B Atomic Molecular Physics. Redfors, Andreas and Litzèn, Ulf (1989). Extended analysis of spectra and term systems in aluminumlike Ca viii-Ni xvi. Journal of the Optical Society of America B Optical Physics. Redfors, Andreas (1988). The 3d2 configuration in Ca IX-Zn XIX. Physica Scripta. Litzèn, Ulf and Redfors, Andreas (1987). Revised and extended analysis of transitions and energy levels in the n = 3 complex of Mg-like Ca IX-Ge XXI. Physica Scripta.</p>
TOKENOMICS MODELING: DETERMINING THE INFLUENCE OF TOKENOMICS PARAMETERS ON TOKEN PRICES IN DECENTRALIZED EXCHANGES
<p>Data is presented in an Excel file, which contains three pages. Page "Attributes" shows the data sets of project community members and their attributes, that are used for the 12 main simulations. Page "DEX simulations" calculations of 12 main simulations. Page "Bonus DEX simulations" shows calculations and data sets of project community members of six bonus DEX simulations.</p>
Acquisition of object-robbing and object/food-bartering behaviors: A culturally maintained token economy in free-ranging long-tailed macaques
Open the record for dataset details and reuse information.
Transcripts for the "What is Token Engineering? A stakeholder study" publication
Open the record for dataset details and reuse information.
A Code Token Type Taxonomy-enhanced dataset with pre-computed token types for Python150k
<p>Code Token Type Taxonomy (CT3) is a methodology for refined evaluation of ML-based code completion approaches.</p> <p>We published the CT3-enhanced dataset with pre-computed token types for each token in the <a href="https://www.sri.inf.ethz.ch/py150">Python150k dataset</a>.</p> <p>The dataset was obtained from an empirical study of the below paper:</p> <p>Kim Tuyen Le, Gabriel Rashidi, and Artur Andrzejak. A Methodology for Refined Evaluation of ML-based Code Completion Approaches. In <em>Special Issue on Programming Language Processing, Data Mining and Knowledge Discovery</em>.</p> <p>Please read the README.txt file for detailed information of structuring the enhanced dataset.</p>
Interface-based Classification of Token Contracts on Ethereum
<p>Dataset for the bachelor thesis "Interface-based Classification of Token Contracts on Ethereum".</p> <p><strong>Contents:</strong></p> <p><em>type_<x>_rules.json</em>: Rules for Type <x>.</p> <p><em>type_1_rules_extended.json</em>: Type I rules extended by functions within Levenshtein distance of 1 of original Type I rules.</p> <p><em>word_occurences.json</em>: Words sorted by how many unique functions they occur in.</p> <p><em>word_coverage_potential.json</em>: Words sorted by how many function occurrences functions including the word make up.</p> <p><em>categorized_functions_with_origins.json</em>: each categorized function, including key statistics such as the number of skeletons implementing them. Each function includes a list of Type I, II, III rules that captured it.</p> <p><em>top_rules_per_category.json</em>: The rules for each category sorted by how many occurrences they captured.</p> <p><em>skeletons_with_compliance.json</em>: Each skeleton's known functions and events and information about what standards they comply with.</p>
MatText tokenizers
<p>Vocabularies for the tokenizers of the MatText package </p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.