Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
1,549
datasets available to search
ShareScore release 0.9.0
Dataset results
1,549 results for “benchmarks”
RSW gun fault prediction benchmark data set (demo)
<p>The resistance spot welding (RSW) welding gun fault prediction benchmark data set has 72 multivariate time series in the training set and 8 in the testing set. Each time series length 604800 sampled at 1 Hz with missing values and has 20 dimensions (c1-c19 and the error code). We retain the missing value and the outliers of the welding gun time series for the potential of imputation research in the future.<br> This data set supports an academic paper named 'benchmark study for welding gun fault prediction'.</p> <p><strong>Feature name and explanation:</strong></p> <p>c1 : Electrode cap offset;</p> <p>c2 : Electrode force;</p> <p>c3 : Electrode position;</p> <p>c4 : Force build-up;</p> <p>c5 : Balance pressure;</p> <p>c6 : Friction;</p> <p>c7 : Maximum aperture;</p> <p>c8 : Maximum electrode force;</p> <p>c9 : Mtart friction;</p> <p>c10 : US2;</p> <p>c11 : Welding point count;</p> <p>c12 : Position count;</p> <p>c13: Setpoints of counterbalance pressure;</p> <p>c14: Setpoints of electrode force;</p> <p>c15 : Setpoints of electrode position;</p> <p>c16: Setpoints of sheet thickness;</p> <p>c17 : Setpoints of velocity;<br> c18: Setpoints of force build-up;<br> c19 : Offset value in robot.</p> <p><strong>Machine Learning Task:</strong><br> This dataset is suitable for a time series forecasting task, where machine learning models can be trained to predict future welding parameters based on the provided welding parameters time series in history. </p> <p><strong>Code for quick start:</strong></p> <p><a href="https://zenodo.org/record/7655025">https://zenodo.org/record/7655025</a></p> <p>If you want to have an overview of the data before downloading all of it, you can download only the files with the word "Damo" in the file name.</p> <p>For any question, please contact 1910633@stu.neu.edu.cn</p>
Synthesis of large scale 3D microscopic images of 3D cell cultures for training and benchmarking
<p>Accompaning data to the paper:</p> <p>Synthesis of large scale 3D microscopic images of 3D cell cultures for training and benchmarking</p>
Materials Science Optimization Benchmark Dataset for High-dimensional, Multi-objective, Multi-fidelity Optimization of CrabNet Hyperparameters
Benchmarks are an essential driver of progress in scientific disciplines. Ideal benchmarks mimic real-world tasks as closely as possible, where insufficient difficulty or applicability can stunt growth in the field. Benchmarks should also have sufficiently low computational overhead to promote accessibility and repeatability. The goal is then to win a "Turing test" of sorts by creating a surrogate model that is indistinguishable from the ground truth observation (at least within the dataset bounds that were explored), necessitating a large amount of data. In materials science and chemistry, industry-relevant optimization tasks are often hierarchical, noisy, multi-fidelity, multi-objective, high-dimensional, and non-linearly correlated while exhibiting mixed numerical and categorical variables subject to linear and non-linear constraints. To complicate matters, unexpected, failed simulation or experimental regions may be present in the search space. In this study, 173219 quasi-random hyperparameter combinations were generated across 23 hyperparameters and used to train CrabNet on the Matbench experimental band gap dataset. The results were logged to a free-tier shared MongoDB Atlas dataset. This study resulted in a regression dataset mapping hyperparameter combinations (including repeats) to MAE, RMSE, computational runtime, and model size for CrabNet model trained on the Matbench experimental band gap benchmark task1. This dataset is used to create a surrogate model as close as possible to running the actual simulations by incorporating heteroskedastic noise. Failure cases for bad hyperparameter combinations were excluded via careful construction of the hyperparameter search space, and so were not considered as was done in prior work. For the regression dataset, percentile ranks were computed within each of the groups of identical parameter sets to enable capturing heteroskedastic noise. This contrasts with a more traditional approach that imposes a-priori assumptions such as Gaussian noise, e.g., by providing a mean and standard deviation. A similar approach can be applied to other benchmark datasets to bridge the gap between optimization benchmarks with low computational overhead and realistically complex, real-world optimization scenarios.
Supramolecular cages benchmark datasets
<p>Two benchmark datasets, comprising 22 well-known supramolecular cages, have been selected from the supramolecular chemistry literature to evaluate cavity detection and volume characterization.</p> <p>All supramolecular cage structures were prepared in the following way: the original CIF files were downloaded from the Cambridge Structural Database (CSD) and atoms and molecular fragments, that are not part of the cage-framework, were removed from the structures, using Diamond Crystal and Molecular Structure Visualization software and saved in a PDB format.</p> <ul> <li><strong>Benchmark dataset 1:</strong> host-guest pairs</li> </ul> <p>For benchmark dataset 1, the guest structures were obtained, from the corresponding CIF files, by removing the supramolecular cage and other molecular fragments that are not part of the guest. </p> <table> <tbody> <tr> <td> <p><strong>Supramolecular Cage Identifier </strong></p> </td> <td> <p><strong>Guest </strong></p> </td> <td> <p><strong>CSD Identifier </strong></p> </td> <td> <p><strong>DOI </strong></p> </td> </tr> <tr> <td> <p><strong>B1</strong> </p> </td> <td> <p>Et4N+ </p> </td> <td> <p>718468 </p> </td> <td> <p><a href="https://doi.org/10.1021/ic8012848">10.1021/ic8012848</a></p> </td> </tr> <tr> <td> <p><strong>B2</strong> </p> </td> <td> <p>BnNMe3+ </p> </td> <td> <p>718469 </p> </td> <td> <p><a href="https://doi.org/10.1021/ic8012848">10.1021/ic8012848</a></p> </td> </tr> <tr> <td> <p><strong>B3</strong> </p> </td> <td> <p>(Cp)2Co+ </p> </td> <td> <p>718470 </p> </td> <td> <p><a href="https://doi.org/10.1021/ic8012848">10.1021/ic8012848</a></p> </td> </tr> <tr> <td> <p><strong>B4</strong> </p> </td> <td> <p>(Cp*)2Co+ </p> </td> <td> <p>718471 </p> </td> <td> <p><a href="https://doi.org/10.1021/ic8012848">10.1021/ic8012848</a></p> </td> </tr> <tr> <td> <p><strong>B5</strong> </p> </td> <td> <p>BF4- </p> </td> <td> <p>1862753 </p> </td> <td> <p><a href="https://doi.org/10.1002/asia.201801262">10.1002/asia.201801262</a></p> </td> </tr> <tr> <td> <p><strong>B6</strong> </p> </td> <td> <p>ClO4- </p> </td> <td> <p>1862752 </p> </td> <td> <p><a href="https://doi.org/10.1002/asia.201801262">10.1002/asia.201801262</a></p> </td> </tr> <tr> <td> <p><strong>B7</strong> </p> </td> <td> <p>C60 </p> </td> <td> <p>942782 </p> </td> <td> <p><a href="https://doi.org/10.1021/ja4110446">10.1021/ja4110446</a></p> </td> </tr> <tr> <td> <p><strong>B8</strong> </p> </td> <td> <p>C60 </p> </td> <td> <p>1872778 </p> </td> <td> <p><a href="https://doi.org/10.1002/chem.201805353">10.1002/chem.201805353</a></p> </td> </tr> <tr> <td> <p><strong>B9</strong> </p> </td> <td> <p>Adamantane-2,6-dione </p> </td> <td> <p>183906 </p> </td> <td> <p><a href="https://doi.org/10.1002/1521-3757(20021018)114:20<3947::AID-ANGE3947>3.0.CO;2-X">10.1002/1521-3757(20021018)114:20<3947::AID-ANGE3947>3.0.CO;2-X</a></p> </td> </tr> <tr> <td> <p><strong>B10</strong> </p> </td> <td> <p>Et4N+ </p> </td> <td> <p>100947 </p> </td> <td> <p><a href="https://doi.org/10.1002/(SICI)1521-3773(19980803)37:13/14<1840::AID-ANIE1840>3.0.CO;2-D">10.1002/(SICI)1521-3773(19980803)37:13/14<1840::AID-ANIE1840>3.0.CO;2-D</a></p> </td> </tr> <tr> <td> <p><strong>B11</strong> </p> </td> <td> <p>Corannulene and Cyclohexane </p> </td> <td> <p>2068665 </p> </td> <td> <p><a href="https://doi.org/10.1038/s41467-021-24344-w">10.1038/s41467-021-24344-w</a></p> </td> </tr> <tr> <td> <p><strong>B12</strong> </p> </td> <td> <p>C60 </p> </td> <td> <p>2068666 </p> </td> <td> <p><a href="https://doi.org/10.1038/s41467-021-24344-w">10.1038/s41467-021-24344-w</a></p> </td> </tr> <tr> <td> <p><strong>B13</strong> </p> </td> <td> <p>C70 </p> </td> <td> <p>2068667 </p> </td> <td> <p><a href="https://doi.org/10.1038/s41467-021-24344-w">10.1038/s41467-021-24344-w</a></p> </td> </tr> </tbody> </table> <p> </p> <p><strong>Benchmark dataset 2: </strong>topologically and morphologically distinct supramolecular cages</p> <p>The single resorcin[4]arene ligand (O2) was prepared from the corresponding resorcin[4]arene-cage (A1) by removing 5 ligands. In the cyclotricatechylene cage (C1), the outward-pointing phenyl groups (that do not affect the cavity calculations) were removed.</p> <table> <tbody> <tr> <td> <p><strong>Supramolecular Cage Identifier </strong></p> </td> <td> <p><strong>CSD Identifier </strong></p> </td> <td> <p><strong>DOI </strong></p> </td> </tr> <tr> <td> <p><strong>A1</strong> </p> </td> <td> <p>1207879 </p> </td> <td> <p><a href="https://doi.org/10.1038/38985">10.1038/38985</a></p> </td> </tr> <tr> <td> <p><strong>C1</strong> </p> </td> <td> <p>1892128 </p> </td> <td> <p><a href="https://doi.org/10.1039/C9CC02103E">10.1039/C9CC02103E</a></p> </td> </tr> <tr> <td> <p><strong>F1</strong> </p> </td> <td> <p>293777 </p> </td> <td> <p><a href="https://doi.org/10.1126/science.1124985">10.1126/science.1124985</a></p> </td> </tr> <tr> <td> <p><strong>F2</strong> </p> </td> <td> <p>1831430 </p> </td> <td> <p><a href="https://doi.org/10.1038/nature20771">10.1038/nature20771</a></p> </td> </tr> <tr> <td> <p><strong>H1</strong> </p> </td> <td> <p>768969 </p> </td> <td> <p><a href="https://doi.org/10.1039/C0CC00234H">10.1039/C0CC00234H</a></p> </td> </tr> <tr> <td> <p><strong>N1</strong> </p> </td> <td> <p>1541839 </p> </td> <td> <p><a href="https://doi.org/10.1021/jacs.7b05202">10.1021/jacs.7b05202</a></p> </td> </tr> <tr> <td> <p><strong>O1</strong> </p> </td> <td> <p>2074472 </p> </td> <td> <p><a href="https://doi.org/10.1021/acs.joc.1c00794">10.1021/acs.joc.1c00794</a></p> </td> </tr> <tr> <td> <p><strong>O2</strong> </p> </td> <td> <p>1207879 </p> </td> <td> <p><a href="https://doi.org/10.1038/38985">10.1038/38985</a></p> </td> </tr> <tr> <td> <p><strong>W1</strong> </p> </td> <td> <p>1416694 </p> </td> <td> <p><a href="https://doi.org/10.1038/nchem.2452">10.1038/nchem.2452</a></p> </td> </tr> </tbody> </table>
SciQA benchmark: Dataset and RDF dump
<p>SciQA benchmark of questions and queries.</p> <p>The data dump is in NTriples format (RDF NT) taken from the ORKG system on 14.02.2023 at 02:04PM.<br> The dump can be imported into a virtuoso endpoint or any RDF engine so it can be queried.</p> <p>The questions/queries are provided as JSON files, also train and test files are provided for each of the sets. </p> <p><strong>Types</strong> of questions and queries:</p> <ul> <li>Handcrafted set of 100 questions</li> <li>Auto-generated set of 2465 questions</li> </ul> <p><strong>More details</strong> on certain columns:<br> "Classification rationale" It may contain the following values:</p> <ul> <li>Nested facts in the question</li> <li>Sorting, sum, average, minimum, maximum or count calculation required</li> <li>Filter used</li> <li>Mappings of Asking Point in the question to the ORKG ontology</li> </ul> <p><strong>Explanation of Rationale for Non-factoid</strong>:</p> <ul> <li>Nested facts in the question. An entity (e.g., a system or a paper) or predicate is requested that is not explicitly stated in the question text and must be inferred while searching for an answer. </li> <li>Sorting, sum, average, minimum, maximum or count calculation required. To get the answer to the question it is necessary to make an aggregation of the query results. </li> <li>Filter used. To get the answer to the question it is necessary to use filtering of the query results by some conditions.<br> </li> </ul>
CWFETB-China: Gridded dataset of consumptive water footprints, evaporation, transpiration, and associate benchmarks of crop production in China (2000-2018)
<p>The CWFETB-China is a 5-arcmin gridded dataset of monthly green and blue water footprint of crop production (WFCP), evaporation (E), transpiration (Tr), and associated unit WFCP benchmarks for 21 crops grown in China during 2000-2018. As compared to the existing gridded WFCP datasets, the CWFETB-China has four improvements: (i) It evaluated the effects of different water supply modes (irrigated or rain-fed) and irrigation practices (furrow, sprinkler, and micro-irrigation) on water consumption throughout the crop growth period. (ii) It distinguished between monthly blue and green water consumption via soil evaporation and crop transpiration. (iii) The dataset encompassed both the WFCP in m<sup>3 </sup>yr<sup>-1</sup> and the uWFCP in m<sup>3 </sup>ton<sup>-1</sup>. (iv) It identified uWFCP benchmarks that differentiated between various climatic zones and irrigation practices. The dataset is able to support for precise crop water productivity assessments, agricultural water-saving evaluations, the development of sustainable irrigation techniques, cropping structure optimisation, and crop-related interregional virtual water trade analysis.</p> <p> </p> <p>Format: NetCDF-4 (5 arcmin) or .xlsx files (benchmark data).</p> <p>Projected coordinate system: WGS 84</p>
MetroPT2: A Benchmark dataset for predictive maintenance
<p><strong>Abstract</strong></p> <p>The MetroPT2 data set is an outcome of a eXplainable Predictive Maintenance (XPM) project with an urban metro public transportation service in Porto, Portugal. The data was collected in 2022 that aimed to evaluate machine learning methods for online anomaly detection and failure prediction. By capturing several analogic sensor signals (pressure, temperature, current consumption), digital signals (control signals, discrete signals), and GPS information (latitude, longitude, and speed), we provide a dataset that can be easily used to evaluate online machine learning methods. This dataset contains some interesting characteristics and can be a good benchmark for predictive maintenance models.</p> <table> <tbody> <tr> <td> <p>Data Set Characteristics:</p> </td> <td> <p>Multivariate Time series</p> </td> <td> <p>Number of Instances:</p> </td> <td> <p>7116940</p> </td> </tr> <tr> <td> <p>Attribute Characteristics:</p> </td> <td> <p>Real</p> </td> <td> <p>Number of Attributes</p> </td> <td> <p>21</p> </td> </tr> <tr> <td> <p>Associated Tracks:</p> </td> <td> <p>Classification, Regression</p> </td> <td> <p>Missing Values</p> </td> <td> <p>N/A</p> </td> </tr> </tbody> </table> <p><strong>Data Set Information:</strong></p> <p>The dataset was collected to support the development of predictive maintenance, anomaly detection, and remaining useful life (RUL) prediction models for compressors using deep learning and machine learning methods.</p> <p>It consists of multivariate time series data obtained from several analogue and digital sensors installed on the compressor of a train. The data span between 2022-04-28 and 2022-07-28 and includes 16 signals, such as pressures, motor current, oil temperature, flowmeter and electrical signals of air intake valves. The monitoring and logging of industrial equipment events, such as temporal behaviour and fault events, were obtained from records generated by the sensors. The data were logged at 1Hz by an onboard embedded device. You can find a schematic diagram of the air production unit of the compressor system in Figure 4 of the accompanying paper [1]. Also, the paper [2] provides a detailed examination of data collection and specifications of various types of potential failures in an air compressor system. </p> <p><strong>Relevant Papers:</strong></p> <p>[1]- Davari, N., Veloso, B., Ribeiro, R.P., Pereira, P.M., Gama, J.: Predictive maintenance based on anomaly detection using deep learning for air production unit in the railway industry. In: 2021 IEEE 8th International Conference on Data Science and Advanced Analytics (DSAA). pp. 1–10. IEEE (2021) (DOI: <a href="https://doi.org/10.1109/DSAA53316.2021.9564181">10.1109/DSAA53316.2021.9564181</a>)</p> <p>[2] Veloso, B., Ribeiro, R.P., Pereira, P.M., Gama, J.: The MetroPT dataset for predictive maintenance. Scientific Data 9, no. 1 (2022): 764. (DOI: 10.1038/s41597-022-01877-3)</p> <p>[3]-Barros, M., Veloso, B., Pereira, P.M., Ribeiro, R.P., Gama, J.: Failure detection of an air production unit in the operational context. In: IoT Streams for Data-Driven Predictive Maintenance and IoT, Edge, and Mobile for Embedded Machine Learning, pp. 61–74. Springer (2020) (DOI: 10.1007/978-3-030-66770-2_5)</p> <p><strong>Failure Information:</strong></p> <p>The dataset is unlabeled, but the failure reports provided by the company are available in the following table. This allows for evaluating the effectiveness of anomaly detection, failure prediction, and RUL estimation algorithms.</p> <p> </p> <table> <tbody> <tr> <td> <p>Nr.</p> </td> <td> <p>Start Time</p> </td> <td> <p>End Time</p> </td> <td>Failure</td> </tr> <tr> <td> <p>1</p> </td> <td> <p>2022-06-04 10:19:24.300</p> </td> <td> <p>2022-06-04 14:22:39.188</p> </td> <td>Air Leak</td> </tr> <tr> <td> <p>2</p> </td> <td> <p>2022-07-11 10:10:18.948</p> </td> <td> <p>2022-07-14 10:22:08.046</p> </td> <td>Oil Leak</td> </tr> </tbody> </table> <p> </p>
ArabicSL-Net: A Benchmark Video Dataset for Arabic Words Sign Language
<p>The data was captured by mobile camera in four main organization namely Bank , Cafe , Hospital , and Train station. The ArabicSL-Net initially consists of 307 words recorded in approximately 30,000 videos. For each organization, we capture the most representative words that are used in those places. For Bank data, we have a total of 76 of words, while Cafe data contains 54 words. For Hospital, we collects videos for 102 words, and collects videos for 71 words in Train station.</p>
SMT-Solving Induction Proofs of Inequalities Benchmarking Repository
<p>This repository contains the full list of files and the benchmarking results that were used in the benchmarking processes described in the paper:<br> A.K. Uncu, J.H. Davenport and M. England. "SMT-Solving Induction Proofs of Inequalities". Proceedings of the 7th International Workshop on Satisfiability Checking and Symbolic Computation (SC^2 2022). </p> <p>The files are split in three branches. The Mathematica and Maple files include the calls that were made to the respective computer algebra systems, and the smt2 files are the ones used by the considered SMT solvers: Z3, CVC5 and Yices.</p> <p>The Benchmarking Results cvc has the results. The columns record the file names, the satisfiability outcome of the calls, then the times (in seconds) of the respective programmes. Any empty box (which the Maple:-RegularChains column has) would mean that the implementation does not accept that sort of input (this is due to rational functions - see the paper for details). Any time over 1200 seconds would mean that the program times out and the outcome of the question was not found in the given time.</p> <p> </p> <p> </p>
A benchmark of gene expression tissue-specificity metrics
<p>Supplementary figures and data to the paper "A benchmark of gene expression tissue-specificity metrics"</p> <p><em>Briefings in Bioinformatics</em>, Volume 18, Issue 2, March 2017, Pages 205–214, <a href="https://doi.org/10.1093/bib/bbw008">https://doi.org/10.1093/bib/bbw008</a></p> <p>Previously published at FigShare, republishing because of access problems for some researchers.</p>
A Benchmark Dataset with Knowledge Graph Generation for Industry 4.0 Production Lines
<p>A benchmark dataset for knowledge graph generation in Industry 4.0 production lines and to show the benefits of using ontologies and semantic annotations of data to showcase how I4.0 industry can benefit from KGs and semantic datasets. This work is a result of collaborations with the production line managers, supervisors, and engineers of a football industry to acquire realistic production line data. Knowledge Graphs (KGs) or a Knowledge Graph (KG) emerged as a significant technology to store the semantics of the domain entities. The data is mapped and populated with RGOM classes and relations using an automated solution based on JenaAPI, producing an I4.0 KG. <br> <br> Usage:<br> <br> Recently, we use this dataset to analyze the performance of the five state-of-the-art KG embedding models, namely ComplEx, DistMult,TransE, ConvKB, and ConvE. We evaluated the models using two key metrics: Mean Reciprocal Rank (MRR), and Hits@N (Hits@10, Hits@3, and Hits@1). We observed that the TransE model outperforms other models, followed by ComplEx and DistMult, with ConvE demonstrating the lowest performance. Similarly, the dataset can be used alternatively in other potential scenarios.</p>
Virtual sensors for wind energy applications benchmark study data - preliminary version
<p>Test version of the time series data for the wind energy virtual sensing benchmark study data.</p>
scMARK an 'MNIST' like benchmark to evaluate and optimize models for unifying scRNA data
<p>Here we present a novel benchmark dataset (scMARK.v2), that consists of 11 published cancer scRNA-seq studies, for which we standardized cell-type author labels and gene identifiers. scMARK.v2 can be used to ask how well models integrate data from different scRNA studies. We also provide a 12th standardized study (Wu et al 2021) that we held-out for evaluation of alignment of data "never seen" before, and a 13th study of newly generated in-vitro scRNA-seq data from cancer and fibroblast cells.</p> <ul> <li>Data is provided as aData *h5ad files that can be read with Python's library <a href="https://scanpy.readthedocs.io/en/stable/">Scanpy</a>.</li> <li>Studies inclided in scMARK.v2 were downsampled to 10,000 cells per study.</li> <li>The difference between <a href="https://zenodo.org/record/5765804">scMARK.v1</a> and scMARK.v2, is that in v2, we provide at least two studies for each cancer type and each cell type; whereas in v1 a handfull of cell types were present only in one study.</li> </ul>
Phables v 0.2.0 benchmarking data and results
<p>This record contains all the benchmarking datasets and results for the Phables manuscript. The datasets used are,</p> <ol> <li>Water samples from Nansi Lake and Dongping Lake in n Shandong Province, China (NCBI BioProject number PRJNA756429), referred to as <strong>Lake Water</strong></li> <li>Soil samples from flooded paddy fields from Hunan Province, China (NCBI BioProject number PRJNA866269), referred to as <strong>Paddy Soil</strong></li> <li>Wastewater virome (NCBI BioProject number PRJNA434744), referred to as <strong>Wastewater</strong></li> <li>Stool samples from patients with IBD and their healthy household controls (NCBI BioProject number PRJEB7772), referred to as <strong>IBD</strong></li> </ol> <p>Each dataset was preprocessed using Hecatomb. The Lake Water dataset was also assembled using MEGAHIT (referred to as <strong>Lake Water - MEGAHIT</strong>) and metaSPAdes (referred to as <strong>Lake Water - metaSPAdes</strong>). All the datasets were run using PHAMB and Phables and the genomes were evaluated using CheckV.</p>
UniProt Human Proteome Benchmarking data for "aaHash: recursive amino acid hashing"
<p>aaHash is a rolling hash algorithm tailed for amino acids. Here, we provide the human proteome benchmarking data used in the aaHash paper "aaHash: recursive amino acid sequence hashing".</p>
SCA-2023: A two-part dataset for benchmarking the methods of image precompensation for users with refractive errors
<p>The recent practices of demonstrating various static and video images to users by means of digital, processor-controlled, often self-luminous devices (computer monitors, smartphone and tablet screens, etc.) have spurred the development of various methods for improving the perception of such images through their computer processing. In particular, this applies to the task of precompensating images shown to users with various anomalies of refraction of the eyes (e.g. myopia or astigmatism) in situations where they are not equipped with glasses or other corrective devices. Researchers have proposed a considerable number of such precompensation methods, but to this day there has been no way to accurately compare their quality. We propose an original dataset, which we called “SCA-2023”, of images specially designed for this purpose. Its most important feature is the fact that it includes not only a set of ground-truth images for implementing the precompensation transform, but also a separate set of images characterizing specific types and degrees of manifestation of the refractive errors. The benchmarking procedure itself includes applying the precompensation transformation to a certain image from the first part of the dataset, computer simulation of the so-called retinal image (distribution of light on the retina of an imaginary observer) based on the selection of the “distorting eye” from the second part of the dataset, and evaluating the similarity of this image to the ground-truth image, using any of the commonly used similarity metrics for this purpose.</p>
Text2KGBench: A Benchmark for Ontology-Driven Knowledge Graph Generation from Text
<p>This is the repository for ISWC 2023 Resource Track submission for <code>Text2KGBench: Benchmark for Ontology-Driven Knowledge Graph Generation from Text</code>. Text2KGBench is a benchmark to evaluate the capabilities of language models to generate KGs from natural language text guided by an ontology. Given an input ontology and a set of sentences, the task is to extract facts from the text while complying with the given ontology (concepts, relations, domain/range constraints) and being faithful to the input sentences.</p> <p>It contains two datasets (i) Wikidata-TekGen with 10 ontologies and 13,474 sentences and (ii) DBpedia-WebNLG with 19 ontologies and 4,860 sentences.</p> <p><strong>An example</strong></p> <p>An example test sentence:</p> <pre><code>Test Sentence: {"id": "ont_music_test_n", "sent": "\"The Loco-Motion\" is a 1962 pop song written by American songwriters Gerry Goffin and Carole King."} </code></pre> <p>An example of ontology:</p> <p>Ontology: <a href="https://github.com/cenguix/Text2KGBench/blob/main/data/wikidata_tekgen/ontologies/owl/ont_2_music.ttl">Music Ontology</a></p> <p>Expected Output:</p> <pre><code>{ "id": "ont_k_music_test_n", "sent": "\"The Loco-Motion\" is a 1962 pop song written by American songwriters Gerry Goffin and Carole King.", "triples": [ { "sub": "The Loco-Motion", "rel": "publication date", "obj": "01 January 1962" },{ "sub": "The Loco-Motion", "rel": "lyrics by", "obj": "Gerry Goffin" },{ "sub": "The Loco-Motion", "rel": "lyrics by", "obj": "Carole King" },] } </code></pre> <p>The data is released under a Creative Commons Attribution-ShareAlike 4.0 International (CC BY 4.0) License.</p> <p>The structure of the repo is as the following.</p> <ul> <li>Text2KGBench <ul> <li>src: the source code used for generation and evaluation, and baseline <ul> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/src/benchmark"><code>benchmark</code></a> the code used to generate the benchmark</li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/src/evaluation"><code>evaluation</code></a> evaluation scripts for calculating the results</li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/src/evaluation">baseline</a> code for generating the baselines including prompts, sentence similarities, and LLM client.</li> </ul> </li> <li>data: the benchmark datasets and baseline data. There are two datasets: wikidata_tekgen and dbpedia_webnlg. <ul> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/wikidata_tekgen">wikidata_tekgen</a> Wikidata-TekGen Dataset <ul> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/wikidata_tekgen/ontologies">ontologies</a> 10 ontologies used by this dataset</li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/wikidata_tekgen/train">train</a> training data</li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/wikidata_tekgen/test">test</a> test data</li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/wikidata_tekgen/manually_verified_sentences">manually_verified_sentences</a> ids of a subset of test cases manually validated</li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/wikidata_tekgen/unseen_sentences">unseen_sentences</a> new sentences that are added by the authors which are not part of Wikipedia <ul> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/wikidata_tekgen/unseen_sentences/test">test unseen</a> test unseen test sentences</li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/wikidata_tekgen/unseen_sentences/ground_truth">ground_truth</a> ground truth for unseen test sentences.</li> </ul> </li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/wikidata_tekgen/ground_truth">ground_truth</a> ground truth for the test data</li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/wikidata_tekgen/baselines">baselines</a> data related to running the baselines. <ul> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/wikidata_tekgen/baselines/test_train_sent_similarity">test_train_sent_similarity</a> for each test case, 5 most similar train sentences generated using SBERT T5-XXL model.</li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/wikidata_tekgen/baselines/prompts">prompts</a> prompts corresponding to each test file <ul> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/wikidata_tekgen/baselines/prompts/unseen">unseen prompts</a> unseen prompts for the unseen test cases</li> </ul> </li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/wikidata_tekgen/baselines/Alpaca-LoRA-13B">Alpaca-LoRA-13B</a> data related to the Alpaca-LoRA model <ul> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/wikidata_tekgen/baselines/Alpaca-LoRA-13B/llm_responses">llm_responses</a> raw LLM responses and extracted triples</li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/wikidata_tekgen/baselines/Alpaca-LoRA-13B/eval_metrics">eval_metrics</a> ontology-level and aggregated evaluation results</li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/wikidata_tekgen/baselines/Alpaca-LoRA-13B/unseen">unseen results</a> results for the unseen test cases <ul> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/wikidata_tekgen/baselines/Alpaca-LoRA-13B/unseen/llm_responses">llm_responses</a> raw LLM responses and extracted triples</li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/wikidata_tekgen/baselines/Alpaca-LoRA-13B/unseen/eval_metrics">eval_metrics</a> ontology-level and aggregated evaluation results</li> </ul> </li> </ul> </li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/wikidata_tekgen/baselines/Vicuna-13B">Vicuna-13B</a> data related to the Vicuna-13B model <ul> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/wikidata_tekgen/baselines/Vicuna-13B/llm_responses">llm_responses</a> raw LLM responses and extracted triples</li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/wikidata_tekgen/baselines/Vicuna-13B/eval_metrics">eval_metrics</a> ontology-level and aggregated evaluation results</li> </ul> </li> </ul> </li> </ul> </li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/dbpedia_webnlg">dbpedia_webnlg</a> DBpedia Dataset <ul> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/dbpedia_webnlg/ontologies">ontologies</a> 19 ontologies used by this dataset</li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/dbpedia_webnlg/train">train</a> training data</li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/dbpedia_webnlg/test">test</a> test data</li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/dbpedia_webnlg/ground_truth">ground_truth</a> ground truth for the test data</li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/dbpedia_webnlg/baselines">baselines</a> data related to running the baselines. <ul> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/dbpedia_webnlg/baselines/test_train_sent_similarity">test_train_sent_similarity</a> for each test case, 5 most similar train sentences generated using SBERT T5-XXL model.</li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/dbpedia_webnlg/baselines/prompts">prompts</a> prompts corresponding to each test file</li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/dbpedia_webnlg/baselines/Alpaca-LoRA-13B">Alpaca-LoRA-13B</a> data related to the Alpaca-LoRA model <ul> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/dbpedia_webnlg/baselines/Alpaca-LoRA-13B/llm_responses">llm_responses</a> raw LLM responses and extracted triples</li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/dbpedia_webnlg/baselines/Alpaca-LoRA-13B/eval_metrics">eval_metrics</a> ontology-level and aggregated evaluation results</li> </ul> </li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/dbpedia_webnlg/baselines/Vicuna-13B">Vicuna-13B</a> data related to the Vicuna-13B model <ul> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/dbpedia_webnlg/baselines/Vicuna-13B/llm_responses">llm_responses</a> raw LLM responses and extracted triples</li> <li><a href="https://github.com/cenguix/Text2KGBench/blob/main/data/dbpedia_webnlg/baselines/Vicuna-13B/eval_metrics">eval_metrics</a> ontology-level and aggregated evaluation results</li> </ul> </li> </ul> </li> </ul> </li> </ul> </li> </ul> </li> </ul> <p>This benchmark contains data derived from the TekGen corpus (part of the KELM corpus) [1] released under CC BY-SA 2.0 license and WebNLG 3.0 corpus [2] released under CC BY-NC-SA 4.0 license.</p> <p>[1] Oshin Agarwal, Heming Ge, Siamak Shakeri, and Rami Al-Rfou. 2021. Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3554–3565, Online. Association for Computational Linguistics.</p> <p>[2] Claire Gardent, Anastasia Shimorina, Shashi Narayan, and Laura Perez-Beltrachini. 2017. Creating Training Corpora for NLG Micro-Planners. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 179–188, Vancouver, Canada. Association for Computational Linguistics.</p>
Benchmark Instances for Robust Combinatorial Optimization with Budgeted Uncertainty
<p>We provide test instances for robust combinatorial optimization with budget uncertainty in the objective function.<br> The set contains nominal problems from the MIPLIB 2017 that have been converted into robust problems and instances of the robust knapsack problem. Both problem sets have been described and used for benchmarking in the paper "A Branch & Bound Algorithm for Robust Binary Optimization with Budget Uncertainty", published in Mathematical Programming Computation by Christina Büsing, Timo Gersing and Arie Koster.<br> Furthermore, we provide instances for robust weighted matching on bipartite graphs and robust weighted independent set. The latter are based on graphs for the clique problem of the second DIMACS implementation challenge (1993). Both problem sets have been described and used for benchmarking in the paper "Recycling Inequalities for Robust Combinatorial Optimization with Budget Uncertainty", presented at IPCO 2023 by the same authors.</p> <p> </p> <p>Paper "A Branch & Bound Algorithm for Robust Binary Optimization with Budget Uncertainty": <a href="https://doi.org/10.1007/s12532-022-00232-2"> https://doi.org/10.1007/s12532-022-00232-2</a><br> Paper "Recycling Inequalities for Robust Combinatorial Optimization with Budget Uncertainty": <a href="https://doi.org/10.1007/978-3-031-32726-1_5">https://doi.org/10.1007/978-3-031-32726-1_5</a><br> For algorithms solving these problems see: <a href="https://doi.org/10.5281/zenodo.7463371">https://doi.org/10.5281/zenodo.7463371</a></p>
Deep Reinforcement Learning Enables Better Bias Control in Benchmark for Virtual Screening
<p>This compressed file contains all datasets made for the validation of MUBDsyn.</p><ul><li>datasets_int_val: 17 cases in this folder are derived from <a href="https://github.com/jwxia2014/ULS-UDS">MUBD for GPCRs</a>. MUBDreal was made by <a href="https://github.com/jwxia2014/MUBD-DecoyMaker2.0">MUBD-DecoyMaker2.0</a> and MUBDsyn was made by <a href="https://github.com/taoshen99/MUBDsyn">MUBD-DecoyMakersyn</a>.</li><li>datasets_ext_val_classical_VS: Five cases in this folder are derived from the shared cases of MUV and DUD-E. The active sets of MUV were taken as the input to make corresponding MUBD datasets. Files in SBVS are raw molecular docking results by smina.</li><li>datasets_ext_val_SI_classical_VS: DeepCoy and TocoDecoy were used to make the datasets corresponding to the same five cases above. The data of DeepCoy was directly retrieved from <a href="https://opig.stats.ox.ac.uk/resources">DeepCoy resources at OPIG</a> while topology decoys of TocoDecoy_9W were made based on the scripts provided at <a href="https://github.com/5AGE-zhang/TocoDecoy">TocoDecoy GitHub Repository</a>. Files in SBVS are raw molecular docking results by smina.</li><li>datasets_ext_val_ML_VS: Ten cases in this folder are derived from <a href="http://nrlist.drugdesign.fr/">NRLiSt-BDB</a>. Corresponding MUBD datasets were made as described above.</li></ul><p>All these datasets can be used for the reproduction of validation performed in the manuscript or to benchmark various virtual screening methods.</p>
Hang-Time HAR: A Benchmark Dataset for Basketball Activity Recognition using Wrist-worn Inertial Sensors
<p>In this paper we present a benchmark dataset for evaluation of physical human activity recognition from wrist-worn sensors, for the specific setting of basketball training, drills, and games.<br> Basketball activities lend themselves well for measurement by wrist-worn inertial sensors, and systems that are able to detect such sport-relevant activities could be used in applications toward game analysis, guided training, and personal physical activity tracking.<br> The dataset was recorded for two teams from separate countries (USA and Germany) with a total of 24 players who wore an inertial sensor on their wrist and spanned both repetitive basketball training sessions and full games.<br> Particular features of this dataset include an inherent variance through cultural differences in game rules and styles as the data was recorded in two countries, as well as different sport skill levels, since the participants were heterogeneous in terms of prior basketball experience.<br> We illustrate the datasets' features in several time-series analyses and report on a baseline classification performance study with a state-of-the-art deep learning architecture.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.