Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
1,549
datasets available to search
ShareScore release 0.9.0
Dataset results
1,549 results for “benchmarks”
Artifact Description/Artifact Evaluation/Computational Artifact for "SPEChpc 2021 Benchmarks on Ice Lake and Sapphire Rapids Infiniband Clusters: A Performance and Energy Case Study"
<p>We provide reproducibility initiative dependencies (Artifact Description or Artifact Evaluation or Computational Results Analysis) appendix at https://github.com/RRZE-HPC/PMBS23-AD. To allow a third party to duplicate the findings, this article provides our extensive performance data artifact and describes further details regarding the software environments, experimental design, and methodology employed for the results shown in the paper, entitled "SPEChpc 2021 Benchmarks on Ice Lake and Sapphire Rapids Infiniband Clusters: A Performance and Energy Case Study". The computational artifacts will enable experienced performance engineers to reproduce and interpret the data shown in the paper in the appropriate way and to follow the conclusions we draw from it.</p>
Automated benchmarking of combined protein structure and ligand conformation prediction
<p>The prediction of protein-ligand complexes (PLC), using both experimental and predicted structures, is an active and important area of research, underscored by the inclusion of the Protein-Ligand Interaction category in the latest round of the Critical Assessment of Protein Structure Prediction experiment CASP15. The prediction task in CASP15 consisted of predicting both the three-dimensional structure of the receptor protein as well as the position and conformation of the ligand. This paper addresses the challenges and proposed solutions for devising automated benchmarking techniques for PLC prediction. The reliability of experimentally solved PLC as ground truth reference structures is assessed using various validation criteria. Similarity of PLC to previously released complexes are employed to judge PLC diversity and the difficulty of a PLC as a prediction target. We show that the commonly used PDBBind time-split test-set is inappropriate for comprehensive PLC evaluation, with state-of-the-art tools showing conflicting results on a more representative and high quality dataset constructed for benchmarking purposes. We also show that redocking on crystal structures is a much simpler task than docking into predicted protein models, demonstrated by the two PLC-prediction-specific scoring metrics created. Finally, we introduce a fully automated pipeline that predicts PLC and evaluates the accuracy of the protein structure, ligand pose, and protein-ligand interactions.</p> <p>This repository contains:</p> <ol> <li> <p>all_validation_clustering_data.tsv - X-ray validation data and MMSeqs cluster identifiers at different sequence identities for over a million small molecule and ion-binding pockets in the PDB. </p> </li> <li> <p>hqr_dataset.tsv - PDB IDs and ligand information for the high quality representative (HQR) dataset described in the manuscript</p> </li> <li> <p>score_files.tar.gz - Full docking results for all detected pockets for the PDBBind time-split test-set, the HQR dataset, and the subsets of AF models created for both datasets. One file per tool benchmarked with the following columns: Tool, Complex, Pocket, Rank, lDDT-PLI, lDDT-LP, BiSyRMSD, Reference_Ligand, Tool-generated Score</p> </li> <li> <p>errors_all_sets.csv - Report of failures running the pipeline with the following columns: Process, Complex/Ligand/Receptor, Problem</p> </li> </ol>
Virtual sensors benchmark study test timeseries tier 0 - part 1
<p>A dataset with time series for testing virtual sensor models for wind turbine aeroelastic loads. Data are in zipped format, with 100 time series available.</p> <p>The test sets in the study are organized with several levels of "difficulty" according to added uncertainties and noise with respect to the training data. This particular dataset (tier 0) is with exactly the same distribution as the training data.</p>
Virtual benchmarking study training time series - binary set 1
<p>A dataset with time series for training virtual sensor models for wind turbine aeroelastic loads. Data are in zipped format, each zip file contains 1000 individual time series. For the purpose of space preservation, each time series is stored in parquet binary format with "snappy" encoding.</p> <p>Reading a parquet file in Python can be done with the pandas library, with the following command sequence:</p> <p>import pandas as pd<br> Data = pd.read_parquet(filename)</p> <p> </p>
Benchmark Data Repositories: Lessons and Recommendations
<p>Our dataset "repository_survey" summarizes a comprehensive survey of over 150 data repositories, characterizing their metadata documentation and standardization, data curation and validation, and tracking of dataset use in the literature. In addition, "survey_model_evaluation" includes our findings on model evaluation for five benchmark repositories. Column descriptions and further details can be found in "README.pdf." The data are associated with our paper "Benchmark Data Repositories: Lessons and Recommendations." </p>
Benchmarking Micro2Micro Transformation: An Approach with GNN and VAE
<p>This research ventures into the emerging domain of microservice-to-microservice transformation, a novel concept focused on optimizing existing cloud-native systems. We experiment with a machine learning methodology initially designed for monolith-to-microservices migration, adapting it to the complex microservices landscape, with a specific focus on the train-ticket application.</p>
Evaluation results of the xMEN entity linking toolkit for multiple benchmark datasets
Open the record for dataset details and reuse information.
Tassie BRUV: A benchmark data set for computer vision and movement quantification algorithms
Open the record for dataset details and reuse information.
A database for benchmarking organ dose estimates and uncertainties in CT
<p>This database includes patient images and associated verified Monte Carlo based estimates of organ doses that may be used for benchmarking different organ dose estimation techniques against a reference standard.</p>
NA12878 WES Benchmark dataset
<p>This dataset makes available the UCSC Genome Browser (genome.ucsc.edu) GRCh37 genome build public session <strong>NA12878 WES Benchmark </strong>files in a single dataset so that these files can be used in other applications or genome browsers such as IGV. </p> <p>The <a href="https://usegalaxy.org/u/erinija/p/omim-genes-in-na12878-wes-benchmark">"Procedure and datasets to cross-reference OMIM genes with the genomic regions of interest"</a> Galaxy page on <strong>usegalaxy.org</strong> server's <em><strong>Shared Data Pages</strong></em> describes practical procedure and several possible use cases for this data set. This page can be accessed freely by users logged into their accounts on usegalaxy.org. Please register if you don't have an account on usegalaxy.org Galaxy server. </p> <p>All genomic variant calls in all VCF files of this data set were decomposed and normalized with vt. This dataset contains: </p> <ol> <li>Genome in a bottle (GIAB) version 3.3.2 high confidence (HC) variant calls and genomic regions for HapMap individual NA12878 : <ol> <li>GIAB_v3.3.2_NA12878-decomposed-normalized.vcf.gz</li> <li>GIAB_v3.3.2_NA12878-decomposed-normalized.vcf.gz.tbi</li> <li>GIAB_v3.3.2_NA12878_HC_regions.bed</li> </ol> </li> <li>HapMap individual NA12878 WES variant calls (VCF) and capture regions (BED) from diagnostic laboratories : <ul> <li>ARUP whole exome sequencing data (HiSeq 2000) publically available from NCBI GeT-RM Browser <ol> <li>converted_ARUP_NA12878_Exome-decomposed-normalized.vcf.gz</li> <li>converted_ARUP_NA12878_Exome-decomposed-normalized.vcf.gz.tbi</li> <li> ARUP_SeqCap_EZ_Exome.bed</li> </ol> </li> <li>UCSF whole exome sequencing data (HiSeq 2500) publically available from NCBI GeT-RM Browser <ol> <li>converted_UCSF_NA12878_WES_Agilent_V4_Custom-decomposed-normalized.vcf.gz</li> <li>converted_UCSF_NA12878_WES_Agilent_V4_Custom-decomposed-normalized.vcf.gz.tbi</li> <li>UCSF_WES_Agilent_V4_Custom.bed</li> </ol> </li> <li>Whole exome data (NextSeq 500) sequenced in CHEO diagnostic laboratory <ol> <li>CHEO_NA12878_WES_S1dataset.vcf.gz</li> <li>CHEO_NA12878_WES_S1dataset.vcf.gz.tbi</li> <li>Agilent_CRE_v2.bed</li> </ol> </li> </ul> </li> <li>Genomic coordinates (BED) of OMIM genes for which a molecular basis of the associated disease is known (as of September 2019) : <ul> <li>Omim_Genes.bed </li> </ul> </li> </ol>
Benchmarking data collected on CloudLab
<p>This dataset includes over 6.8M performance measurements collected on the CloudLab testbed. For more information about *how* and *why* it is collected, please refer to the "Taming Performance Variability" paper (<a href="https://www.usenix.org/system/files/osdi18-maricq.pdf">https://www.usenix.org/system/files/osdi18-maricq.pdf</a>). For additional information about CPU measurements, you can refer to: https://www.flux.utah.edu/download?uid=288.</p> <p>The dataset is split into 3 files, for <strong>CPU</strong> (2.0M records), <strong>memory</strong> (4.3M records), and <strong>disk</strong> (664K) results. All these records were exported from the live database dedicated to this benchmarking project on 02/23/2020. </p> <p>--------------</p> <p>Contributors:</p> <ul> <li>Dmitry Duplyakin (<a href="mailto:dmdu@cs.utah.edu">dmdu@cs.utah.edu</a>)</li> <li>Alexandru Uta (<a href="mailto:a.uta@vu.nl">a.uta@vu.nl</a>)</li> <li>Aleksander Maricq (<a href="mailto:amaricq@cs.utah.edu">amaricq@cs.utah.edu</a>)</li> <li>Robert Ricci (<a href="mailto:ricci@cs.utah.edu">ricci@cs.utah.edu</a>)</li> </ul>
Data for the paper "Insights gained from a comprehensive all-against-all transcription factor binding motif benchmarking study".
<p>Data for the paper "Insights gained from a comprehensive all-against-all transcription factor binding motif benchmarking study".</p>
ProGene - A Large-scale, High-Quality Protein-Gene Annotated Benchmark Corpus
<p>The Pro(tein)/Gene corpus was developed at the JULIE Lab Jena under supervision of Prof. Udo Hahn.</p> <p>The goals of the annotation project were</p> <ul> <li>to construct a consistent and (as far as possible) subdomain-independent/-comprehensive protein-annotated corpus</li> <li>to differentiate between protein families and groups, protein complexes, protein molecules, protein variants (e.g. alleles) and elliptic enumerations of proteins.</li> </ul> <p>The corpus has the following annotation levels / entity types:</p> <ul> <li>protein</li> <li>protein_familiy_or_group</li> <li>protein_complex</li> <li>protein_variant</li> <li>protein_enum</li> </ul> <p>For definitions of the annotation levels, please refer to the Proteins-guidelines-final.doc file that is found in the download package.</p> <p>To achieve a large coverage of biological subdomains, document from multiple other protein / gene corpora were reannotated. For further coverage, new document sets were created. All documents are abstracts from PubMed/MEDLINE. The corpus is made up of the union of all the documents in the different subcorpora.<br> All document are delivered as MMAX2 (http://mmax2.net/) annotation projects.</p>
Experimental Data Sets for the study "Benchmarking a $(\mu+\lambda)$ Genetic Algorithm with Configurable Crossover Probability"
<p>This is the experimental result of the study "Benchmarking a (μ+λ) Genetic Algorithm with Configurable Crossover Probability". A novel (μ+λ) GA is proposed and benchmarked, in which we stochastically determine whether to apply the crossover operator either for each individual or generation with a crossover probability <span class="math-tex">\(p_c\)</span>. This data set consists of two parts:</p> <ol> <li>The results of (μ+λ) GA on 25 pseudo-Boolean problems defined in <em>IOHprofiler </em>(<a href="https://iohprofiler.github.io/">https://iohprofiler.github.io/</a>) with the following setup: <span class="math-tex">\(\mu \in \{10, 50, 100\}, \lambda \in \{1, \lceil\mu/2\rceil, \mu\}, p_c\in\{0, 0.5\}.\)</span> <ul> <li>'IOHprofiler_Problems_standard_bit_mutation.csv' --> the (μ+λ) GA with standard bit mutation.</li> <li>'IOHprofiler_Problems_fast_mutation.csv' --> the (μ+λ) GA with fast mutation.</li> </ul> </li> <li>The results of (μ+λ) GA on OneMax and LeadingOnes problems with the following setup: <span class="math-tex">\(n \in \{64,100,150,200,250,500\}, \mu \in \{2,3,5,8,10,20,30,...,100\}, \\ \lambda \in \{1, \lceil \mu/2 \rceil, \mu\}, \text{and }p_c \in \{0.1 k \mid k \in [0..9]\}\cup\{0.95\}.\)</span> <ul> <li>'OneMax_raw.csv' --> the fixed-target running time/first hitting time from 100 independent runs for target values in <span class="math-tex">\([1..n]\)</span>.</li> <li>'OneMax_summary.csv' --> the mean, median, standard deviation, some quantiles, expected running time (ERT), the number of successful runs, and the success rate from 100 independent runs for target values in <span class="math-tex">\([1..n]\)</span>.</li> <li>'LeadingOnes_raw.csv' --> the same with 'OneMax_raw.csv' for LeadingOnes.</li> <li>'LeadingOnes_summary.csv' --> the same with 'OneMax_summary.csv' for LeadingOnes.</li> </ul> </li> </ol> <p><strong>Contact</strong>: if you have any questions or suggestions, please feel free to contact <a href="https://www.universiteitleiden.nl/en/staffmembers/furong-ye#tab-1">Furong Ye</a> or <a href="http://www-ia.lip6.fr/~doerr/">Carola Doerr</a>.</p>
Facies Classification Benchmark
<p> The recent interest in using deep learning for seismic interpretation tasks, such as facies classification, has been facing a significant obstacle, namely the absence of large publicly available annotated datasets for training and testing models. As a result, researchers have often resorted to annotating their own training and testing data. However, different researchers may annotate different classes, or use different train and test splits. In addition, it is common for papers that apply machine learning for facies classification to not contain quantitative results, and rather rely solely on visual inspection of the results. All of these practices have lead to subjective results and have greatly hindered the ability to compare different machine learning models against each other and understand the advantages and disadvantages of each approach.</p> <p><br> To address these issues, we open-source a fully-annotated 3D geological model of the Netherlands F3 Block. This model is based on the study of the 3D seismic data in addition to 26 well logs, and is grounded on the careful study of the geology of the region. Furthermore, we propose two baseline models for facies classification based on a deconvolution network architecture and make their codes publicly available. Finally, we propose a scheme for evaluating different models on this dataset, and we share the results of our baseline models. In addition to making the dataset and the code publicly available, this work helps advance research in this area by creating an objective benchmark for comparing the results of different machine learning approaches for facies classification.</p>
MarkusThill/MGAB: The Mackey-Glass Anomaly Benchmark
<p>This repository contains the Mackey-Glass anomaly benchmark (MGAB), which is composed of synthetic Mackey-Glass time series with non-trivial anomalies. Mackey-Glass time series are known to exhibit chaotic behavior under certain conditions. MGAB contains 10 MG time series of length 100k. Into each time series 10 anomalies are inserted with a procedure as described below. In contrast to other synthetic benchmarks, it is very hard for the human eye to distinguish the introduced anomalies from the normal (chaotic) behavior.</p>
JCrashPack: A Java Crash Reproduction Benchmark
<p>A Java Crash reproduction benchmark</p>
Erroneous Reagent Checking (ERC) benchmark
<p>The <strong>Erroneous Reagent Checking (ERC) benchmark</strong> assesses the accuracy of fact-checkers screening biomedical publications for dubious mentions of nucleotide sequence reagents. It comes with a test collection comprised of 1,679 nucleotide sequence reagents that were curated by biomedical experts.</p>
Pangeo Cloud Storage Read Throughput Benchmarks
<p>These are the results of benchmarking distributed reads from various cloud storage services and data formats. All tests were performed from the Google cloud US-CENTRAL1 region.</p>
ClaimBuster: A Benchmark Dataset of Check-worthy Factual Claims
<p>The ClaimBuster dataset consists of statements extracted from all U.S. general election presidential debates (1960-2016) along with human-annotated check-worthiness labels where each sentence is categorized into one of the three categories: non-factual statement, unimportant factual statement, and check-worthy factual statement.</p> <p>If you use this dataset, please cite the following paper:</p> <p>@inproceedings{arslan2020claimbuster,<br> title={{A Benchmark Dataset of Check-worthy Factual Claims}},<br> author={Arslan, Fatma and Hassan, Naeemul and Li, Chengkai and Tremayne, Mark },<br> booktitle={14th International AAAI Conference on Web and Social Media},<br> year={2020},<br> organization={AAAI}<br> }</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.