Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
17
datasets available to search
ShareScore release 0.9.0
Dataset results
17 results for “Computer benchmarking”
Bio-logger Ethogram Benchmark: A benchmark for computational analysis of animal behavior, using animal-borne tags
<p>This repository contains the datasets and experiment results presented in our <a href="https://arxiv.org/abs/2305.10740">arxiv paper</a>:</p> <blockquote> <p>B. Hoffman, M. Cusimano, V. Baglione, D. Canestrari, D. Chevallier, D. DeSantis, L. Jeantet, M. Ladds, T. Maekawa, V. Mata-Silva, V. Moreno-González, A. Pagano, E. Trapote, O. Vainio, A. Vehkaoja, K. Yoda, K. Zacarian, A. Friedlaender, "A benchmark for computational analysis of animal behavior, using animal-borne tags," 2023.</p> </blockquote> <p>Standardized code to implement, train, and evaluate models can be found at <a href="https://github.com/earthspecies/BEBE/">https://github.com/earthspecies/BEBE/</a>. </p> <p>Please note the licenses in each dataset folder.</p> <p><strong>Zip folders beginning with "formatted":</strong> These are the datasets we used to run the experiments reported in the benchmark paper. </p> <p><strong>Zip folders beginning with "raw": </strong>These are the unprocessed datasets used in BEBE. Code to process these raw datasets into the formatted ones used by BEBE can be found at <a href="https://github.com/earthspecies/BEBE-datasets/">https://github.com/earthspecies/BEBE-datasets/</a>.</p> <p><strong>Zip folders beginning with "experiments": </strong>Results of the cross-validation experiments reported in the paper, as well as hyperparameter optimization. Confusion matrices for all experiments can also be found here. Note that dt, rf, and svm refer to the feature set from Nathan et al., 2012.</p> <p><em>Results used in Fig. 4 of <a href="https://arxiv.org/abs/2305.10740">arxiv paper</a> (deep neural networks vs. classical models)</em><br>{dataset}_ harnet_nogyr<br>{dataset}_CRNN<br>{dataset}_CNN<br>{dataset}_dt<br>{dataset}_rf<br>{dataset}_svm<br>{dataset}_wavelet_dt<br>{dataset}_wavelet_rf<br>{dataset}_wavelet_svm</p> <p><em>Results used in Fig. 5D of <a href="https://arxiv.org/abs/2305.10740">arxiv paper</a> (full data setting)<br></em>If dataset contains gyroscope (HAR, jeantet_turtles, vehkaoja_dogs):<br>{dataset}_harnet_nogyr<br>{dataset}_harnet_random_nogyr<br>{dataset}_harnet_unfrozen_nogyr<br>{dataset}_RNN_nogyr<br>{dataset}_CRNN_nogyr<br>{dataset}_rf_nogyr<br><br>Otherwise:<br>{dataset}_harnet_nogyr<br>{dataset}_harnet_unfrozen_nogyr<br>{dataset}_harnet_random_nogyr<br>{dataset}_RNN_nogyr<br>{dataset}_CRNN<br>{dataset}_rf</p> <p><em>Results used in Fig. 5E of <a href="https://arxiv.org/abs/2305.10740">arxiv paper</a> (reduced data setting)<br></em>If dataset contains gyroscope (HAR, jeantet_turtles, vehkaoja_dogs):<br>{dataset}_harnet_low_data_nogyr<br>{dataset}_harnet_random_low_data_nogyr<br>{dataset}_harnet_unfrozen_low_data_nogyr<br>{dataset}_RNN_low_data_nogyr<br>{dataset}_wavelet_RNN_low_data_nogyr<br>{dataset}_CRNN_low_data_nogyr<br>{dataset}_rf_low_data_nogyr</p> <p>Otherwise:<br>{dataset}_harnet_low_data_nogyr<br>{dataset}_harnet_random_low_data_nogyr<br>{dataset}_harnet_unfrozen_low_data_nogyr<br>{dataset}_RNN_low_data_nogyr<br>{dataset}_wavelet_RNN_low_data_nogyr<br>{dataset}_CRNN_low_data<br>{dataset}_rf_low_data<br><br></p> <p><strong>CSV files</strong>: we also include summaries of the experimental results in experiments_summary.csv, experiments_by_fold_individual.csv, experiments_by_fold_behavior.csv. </p> <p><em>experiments_summary.csv - results averaged over individuals and behavior classes<br></em>dataset (str): name of dataset<br>experiment (str): name of model with experiment setting <br>fig4 (bool): True if dataset+experiment was used in figure 4 of <a href="https://arxiv.org/abs/2305.10740">arxiv paper</a><br>fig5d (bool): True if dataset+experiment was used in figure 5d of <a href="https://arxiv.org/abs/2305.10740">arxiv paper</a><br>fig5e (bool): True if dataset+experiment was used in figure 5e of <a href="https://arxiv.org/abs/2305.10740">arxiv paper</a><br>f1_mean (float): mean of macro-averaged F1 score, averaged over individuals in test folds<br>f1_std (float): standard deviation of macro-averaged F1 score, computed over individuals in test folds<br>prec_mean, prec_std (float): analogous for precision<br>rec_mean, rec_std (float): analogous for recall<em><br><br>experiments_by_fold_individual.csv - results per individual in the test folds<br></em>dataset (str): name of dataset<br>experiment (str): name of model with experiment setting <br>fig4 (bool): True if dataset+experiment was used in figure 4 of <a href="https://arxiv.org/abs/2305.10740">arxiv paper</a><br>fig5d (bool): True if dataset+experiment was used in figure 5d of <a href="https://arxiv.org/abs/2305.10740">arxiv paper</a><br>fig5e (bool): True if dataset+experiment was used in figure 5e of <a href="https://arxiv.org/abs/2305.10740">arxiv paper</a><br>fold (int): test fold index<br>individual (int): individuals are numbered zero-indexed, starting from fold 1<br>f1 (float): macro-averaged f1 score for this individual<br>precision (float): macro-averaged precision for this individual<br>recall (float): macro-averaged recall for this individual<em><br></em></p> <p><em>experiments_by_fold_behavior.csv - results per behavior class, for each test fold<br></em>dataset (str): name of dataset<br>experiment (str): name of model with experiment setting <br>fig4 (bool): True if dataset+experiment was used in figure 4 of <a href="https://arxiv.org/abs/2305.10740">arxiv paper</a><br>fig5d (bool): True if dataset+experiment was used in figure 5d of <a href="https://arxiv.org/abs/2305.10740">arxiv paper</a><br>fig5e (bool): True if dataset+experiment was used in figure 5e of <a href="https://arxiv.org/abs/2305.10740">arxiv paper</a><br>fold (int): test fold index<br>behavior_class (str): name of behavior class<br>f1 (float): f1 score for this behavior, averaged over individuals in the test fold<br>precision (float): precision for this behavior, averaged over individuals in the test fold<br>recall (float): recall for this behavior, averaged over individuals in the test fold<br>train_ground_truth_label_counts (int): number of timepoints labeled with this behavior class, in the training set<em><br></em></p>
Transprecision Computing Benchmarks
<p>The datasets have been collected by benchmarking three algorithms for Transprecision Computing (Correlation, Convolution, Saxpy), on three different hardware platforms (pc, vm, g100).</p> <p>Transprecision Computing<sup>1</sup> is a paradigm that allows users to trade the energy associated with computation in exchange for a reduction in the quality of the computation results. In this complex domain, a typical target are Floating-point (FP) operations: transprecision techniques allow to specify the number of bits used to represent FP variables, and using a smaller number of bits decreases the precision, thus saving energy. To analytically calculate the impact of varying the number of bits on the computation results for programs with more than a couple of instructions is a crucial point. However, this relationship can be learned from data.</p> <p>The provided benchmarks have been used for training several machine learning models, to predict the performance (time, error, memory) of a given algorithm, when running with a particular configuration (the precision assigned to each variable) on a certain hardware architecture. Afterward, the produced models have been embedded into HADA, an optimization engine for hardware dimensioning and algorithm configuration, developed by the AI research group at the University of Bologna, as partner of the EU Horizon 2020 Project StairwAI (g.a. 101017142).</p> <p> </p> <p><strong>Bibliography</strong></p> <p>1. Andrea Borghesi, Giuseppe Tagliavini, Michele Lombardi, Luca Benini and Michela Milano. 2020. Combining learning and optimization for transprecision computing. In <em>Proceedings of the 17th ACM International Conference on Computing Frontiers </em>(<em>CF '20</em>). Association for Computing Machinery, New York, NY, USA, 10–18. https://doi.org/10.1145/3387902.3392615</p>
Application-Motivated, Holistic Benchmarking of a Full Quantum Computing Stack: Experimental Data
<p>Full experimental dataset for the publication "Application-Motivated, Holistic Benchmarking of a Full Quantum Computing Stack". The archive `application_motivated_benchmarks.zip` contains the following files and directories:</p> <p>- uncompiled_log.csv</p> <p>Gives IDs for the uncompiled circuits initially generated for use in our<br> experiments, along with the properties of the circuits.</p> <p>- properties_log.csv</p> <p>Gives IDs for device property files, along with the device and the time at which<br> they were collected.</p> <p>- compiled_log.csv</p> <p>Gives the calculated figures of merits for the compiled and run circuits.<br> Compiled circuits are identified by the ID of the uncompiled circuit, the<br> compilation strategy used, and the device compiled onto. Device property IDs at<br> the time of compilation and run are given.</p> <p>- circuits/</p> <p>Contains a subdirectory for each uncompiled circuit. Each subdirectory has files<br> of 2 forms.<br> <br> - uncompiled.qasm is the uncompiled circuit.<br> - files of the form 'strategy'_'device'.qasm are the compiled circuits.</p> <p>- data/</p> <p>Contains a subdirectory for each uncompiled circuit. Each subdirectory has files<br> of 3 forms.</p> <p> - prob_vector.csv contains the ideal output probability distribution.<br> - files of the form 'strategy'_'device'.csv contain the shot counts for<br> each compiled circuit when run on the real device.<br> - files of the form 'strategy'_'device'_simulated.csv contain the shot<br> counts for each compiled circuit when run using a classical simulator<br> with noise model build from device properties at the time of the real<br> run.</p> <p>- device_properties/</p> <p>Contains json files detailing device properties for each device property ID.</p> <p> </p>
Application-Oriented Performance Benchmarks for Quantum Computing
<p>Complete dataset and Jupyter Notebook used to produce image files for the paper at</p> <p> https://arxiv.org/abs/2110.03137.</p> <p>To execute the notebook, copy the .ipynb file and the _data directory to the top level of the repository at:</p> <p> https://github.com/SRI-International/QC-App-Oriented-Benchmarks</p> <p> </p>
GTSRB - German Traffic Sign Recognition Benchmark by Real-Time Computer Vision at Ruhr-Universität Bochum
<div> <div>The German Traffic Sign Benchmark is a multi-class, single-image classification challenge held at the International Joint Conference on Neural Networks (IJCNN) 2011. <br>Our benchmark has the following properties: <br>- Single-image, multi-class classification problem <br>- More than 40 classes<br>- More than 50,000 images in total <br>- Large, lifelike database<br><br>Acknowledgements: [INI Benchmark Website][1]<br>[1]: http://benchmark.ini.rub.de/</div> </div>
Tassie BRUV: A benchmark data set for computer vision and movement quantification algorithms
Open the record for dataset details and reuse information.
Real-time benchmark dynamics of the Ohmic Spin-Boson Model computed with Time-Dependent Variational Matrix Product States. (TDVMPS) coupling strength and temperature parameter space
<p>Data describing the complete propagators (maps) for the evolution of the Ohmic Spin-Boson Model are made available, here. Using a time-dependent variotnal matrix product states (TDVMPS) respresentation of the complete spin-environment wave function, non -perturbative results are presented over a wide range of coupling strengths, temperatures and initial conditions. The results in this repository are associated with the article: </p> <p>https://www.preprints.org/manuscript/202012.0016/v1 </p> <p>A mathematica notebook that allows the data to be visualised and manipulated is also provided. </p>
Code, benchmarks and experiment data for the ICAPS 2024 paper "Merging or Computing Saturated Cost Partitionings? A Merge Strategy for the Merge-and-Shrink Framework"
<p>This bundle contains code, scripts and benchmarks for reproducing all experiments reported in the paper. It also contains the data generated for the paper. Finally, it contains an appendix with some more detailed results for the paper.</p> <p>appendix.pdf: document with more detailed results ommitted in the paper.</p> <p>sievers-et-al-icaps2024-fast-downward.zip contains the implementation based on Fast Downward. It also contains the experiment scripts compatible with Lab 7.1 for reproducing all experiments of the paper, under experiments/scp-ms. The scripts 2024-03-* contain configurations for running the experiments and the script paper-crc.py gathers the data and produces plots and tables. (Note that some adjustments to the scripts would need to be done because, e.g., the entire tree is not a repository anymore.)</p> <p>sievers-et-al-icaps2024-ipc-benchmarks.zip contains the IPC benchmarks. It consists of the STRIPS IPC benchmarks used in all optimal sequential tracks of IPCs up to 2023 (suite optimal_strips from https://github.com/aibasel/downward-benchmarks).</p> <p>sievers-et-al-icaps2024-lab.tar.gz contains a copy of Lab 7.1 (https://github.com/aibasel/lab).</p> <p>sievers-et-al-icaps2024-raw-data.zip and sievers-et-al-icaps2024-processed-data.zip contain the experimental data. Directories without the "-eval" ending (sievers-et-al-icaps2024-raw-data.zip) contain raw data, distributed over a subdirectory for each experiment. Each of these contain a subdirectory tree structure "runs-*" where each planner run has its own directory. For each run, there are symbolic links to the input PDDL files domain.pddl and problem.pddl (can be resolved by putting the benchmarks directory to the right place), the run log file "run.log" (stdout), possibly also a run error file "run.err" (stderr), the run script "run" used to start the experiment, and a "properties" file that contains data parsed from the log file(s). Directories with the "-eval" suffix (sievers-et-al-icaps2024-processed-data.zip) contain a "properties" file, which contains a JSON directory with combined data of all runs of the corresponding experiment. In essence, the properties file is the union over all properties files generated for each individual planner run.</p> <p>Note on license: we chose GPL v3.0 or later mainly because we consider our implementation based on Fast Downward the main contribution of this package, and Fast Downward comes with GPL v3.0. We only include a copy of Lab and the benchmarks for convenience.</p>
Transprecision Computing (Micro-benchmarks)
<p>The micro-benchmarks involve multiple Floating-Point variables whose precision, that is the number of bits used for the mantissa and the exponent, can be tuned using the FlexFloat SW library (see "A Transprecision Floating-Point Platform for Ultra-Low Power Computing", by G. Tagliavini et al., in 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE) 2018 Mar 19 (pp. 1051-1056). IEEE). </p> <p>The data set comprises 7 different micro-benchmarks run under different precision configurations; this data set was created during the 2020 FET project OPRECOMP (g.a. 732631).</p> <p>The data set is fully described in the research paper: "Combining learning and optimization for transprecision computing", by authors Borghesi A, Tagliavini G, Lombardi M, Benini L, Milano M. In Proceedings of the 17th ACM International Conference on Computing Frontiers 2020 May 11 (pp. 10-18). See here the full paper: https://arxiv.org/pdf/2002.10890.pdf</p>
Systematic benchmarking of computational methods to identify spatially variable genes: Part 1
<div> <div> <div> <p>Spatially resolved transcriptomics offers unprecedented insight by enabling the profiling of gene expression within the intact spatial context of cells, effectively adding a new and essential dimension to data interpretation. To efficiently detect spatial structure of interest, an essential step in analyzing such data involves identifying spatially variable genes. Despite researchers having developed several computational methods to accomplish this task, the lack of a comprehensive benchmark evaluating their performance remains a considerable gap in the field. Here, we present a systematic evaluation of 14 methods using 60 simulated datasets generated by four different simulation strategies, 12 real-world transcriptomics, and three spatial ATAC-seq datasets. We find that spatialDE2 consistently outperforms the other benchmarked methods, and Moran’s I achieves competitive performance in different experimental settings. Moreover, our results reveal that more specialized algorithms are needed to identify spatially variable peaks. </p> <p> </p> </div> </div> </div>
Scalable Randomized Benchmarking of Quantum Computers using Mirror Circuits
<p>This is supplemental data and code for: T. Proctor et al., <em><a href="http://https://doi.org/10.48550/arXiv.2112.09853">Scalable randomized benchmarking of quantum computers using mirror circuits</a>, </em>arXiv 2112.09853 (2021).</p> <p>This folder contains all the data and the analysis code to generate the results presented in that paper. The core data analysis routines use PyGSTi, which can be found at <a href="https://github.com/pyGSTio/pyGSTi">https://github.com/pyGSTio/pyGSTi</a>.</p> <p>Please direct any questions to Timothy Proctor (tjproct@sandia.gov).</p>
SeBS: A Serverless Benchmark Suite for Function-as-a-Service Computing
<p>This upload contains the software prototype, data, analysis scripts, and replication scripts for the paper "SeBS: A Serverless Benchmark Suite for Function-as-a-Service Computing" (ACM/IFIP Middleware 2021).</p> <p>With our artifact we provide the following components:</p> <ul> <li> <p><code>serverless-benchmarks</code> - source code of the benchmark suite</p> </li> <li> <p><code>data</code> - benchmarking results obtained for the paper</p> </li> <li> <p><code>analysis</code> - Python plotting and analysis scripts used for data analysis</p> </li> <li> <p><code>experiments</code> - scripts helping to reproduce the experiments</p> </li> <li> <p><code>docker</code> - compressed Docker images that were used for our experiments.</p> </li> </ul> <p>Our data has been obtained in January 2020, July and August 2020, and November 2020.</p>
Analyzed Benchmarks on Experiments for a Complications for Computational Experiments from Modern Processors
<p>For details see: </p> <p>Johannes K. Fichte, Markus Hecher, Ciaran McCreesh, Anas Shahab: Complications for Computational Experiments from Modern Processors, Proceedings of the 27th International Conference on Principles and Practice of Constraint Programming (CP'2021).</p> <p>For the benchmark set, we refer to https://www.cs.uni-potsdam.de/wv/projects/sets/set-industrial-09-12.tar.xz or https://www.cs.uni-potsdam.de/wv/projects/sets. The instances are also available on Zenodo at: https://doi.org/10.5281/zenodo.3989071</p> <p>The tested solver is available at: https://github.com/arminbiere/cadical </p>
Data for "A benchmark of computational pipelines for single-cell histone modification data"
<p>The `time_features.txt` and `time_correct.txt` files contain the runtimes for the various embedding methods.</p> <p>The .csv files contain the computed scores (both supervised and unsupervised) for all the methods.</p>
Benchmarking of Computational Demultiplexing Methods for Single-Nucleus RNA Sequencing Data [dataset 2]
GEO Series GSE298266. Homo sapiens; Ovis aries. 10 samples. Type: Expression profiling by high throughput sequencing.
Benchmarking Recent Computational Tools Available for DNA Binding Protein Identification
<p><strong>Train_Test_PSSM.pickle:</strong></p> <p>We have provided a train and a test fasta file containing DNA binding and non-binding protein sequences. The PSSM matrices for these sequences are available inside this .pickle file. You will need to provide the path of this .pickle file when running predictions on the provided train or test fasta file sequences using the two provided tools (LocalDPP and LSTM-CNN_Fusion)</p> <p><strong>db.zip:</strong></p> <p>You will need to unzip this database file and provide the unzipped folder as path to the PSSM generation method. The PSSM creation method provided in github uses this database to generate the .pickle file (full of PSSM matrices) from user provided fasta file sequences.</p>
Benchmarking of Computational Demultiplexing Methods for Single-Nucleus RNA Sequencing Data [dataset 1]
GEO Series GSE298265. Homo sapiens. 6 samples. Type: Expression profiling by high throughput sequencing.
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.