Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
1,549
datasets available to search
ShareScore release 0.9.0
Dataset results
1,549 results for “benchmarks”
HADDOCK file I/O benchmark
<p>The benchmark is composed of 1,000,000 structures in PDB format extracted from the results of applying HADDOCK to the cases of the Protein-Protein Docking Benchmark v5 (BM5), which is a well-accepted benchmark in the protein docking community. Due to the nature of the BM5 benchmark, the PDB files are different in length as they represent different molecular structures. Files are placed in a single directory and named according to the BM5 they originated from, docking procedure used, docking stage and model number, example; <em>1BVN_ti5-it1_167.pdb </em>and<em> 1AK4_cm-it0_1601.pdb</em>. The HADDOCK file I/O benchmark is related to our BioExcel project on <em><a href="https://bioexcel.eu/research/projects/high-throughput-modelling-of-interactomes/">high throughput modelling of interactomes</a></em>, this task will generate a gargantuan number of files and will only be possible by taking advantage of exascale computing resources both for processing and I/O handling.</p>
Bug Injector Benchmarks for IEEE SCAM 2019
<p>We present the benchmark data-set accompanying our paper "Automated Customized Bug-Benchmark Generation", published in the 19th IEEE International Working Conference on Source Code Analysis and Manipulation. </p>
Raw data for "Benchmarking the in vitro toxicity and chemical composition of plastic consumer products"
<p>These files are associated with the publication "Benchmarking the in vitro toxicity and chemical composition of plastic consumer products" published in Environmental Science & Technology at <a href="https://doi.org/10.1021/acs.est.9b02293">https://doi.org/10.1021/acs.est.9b02293</a>.</p> <p>Raw data files for the non-target chemical analysis performed on consumer plastics using GC-QTOF-MS (Agilent .D format).</p> <p>FTIR spectra of the plastic products (pdf format).</p>
Reads and truth variant set for benchmarking variant calling/genotyping
<p><em>downsampled.fasta</em> is created by converting this file (ftp://<a href="http://ftp-trace.ncbi.nlm.nih.gov/giab/ftp/data/AshkenazimTrio/HG002_NA24385_son/NIST_HiSeq_HG002_Homogeneity-10953946/HG002Run01-11419412/HG002run1_S1.bam">ftp-trace.ncbi.nlm.nih.gov/giab/ftp/data/AshkenazimTrio/HG002_NA24385_son/NIST_HiSeq_HG002_Homogeneity-10953946/HG002Run01-11419412/HG002run1_S1.bam</a>) to fasta and picking every second read (to get half the coverage and half the number of reads).</p> <p><em>The HG002_GRCh37_GIAB_highconf_CG</em>... file is created by picking variants on chromosome from this file (ftp://<a href="http://ftp-trace.ncbi.nlm.nih.gov/giab/ftp/release/AshkenazimTrio/HG002_NA24385_son/NISTv3.3.2/GRCh37/HG002_GRCh37_GIAB_highconf_CG-IllFB-IllGATKHC-Ion-10X-SOLID_CHROM1-22_v.3.3.2_highconf_triophased.vcf.gz">ftp-trace.ncbi.nlm.nih.gov/giab/ftp/release/AshkenazimTrio/HG002_NA24385_son/NISTv3.3.2/GRCh37/HG002_GRCh37_GIAB_highconf_CG-IllFB-IllGATKHC-Ion-10X-SOLID_CHROM1-22_v.3.3.2_highconf_triophased.vcf.gz</a>).</p> <p> </p> <p><strong>These two files can be used in benchmarking variant calling/genotyping.</strong></p>
Benchmarking data and outputs for CLASSIC v. 1.0
<p><strong>CLASSIC v. 1.0 model inputs and outputs for benchmarking </strong></p> <p>This dataset is used by scripts in the <a href="https://zenodo.org/record/3522407">CLASSIC codebase</a> along with the <a href="https://zenodo.org/record/3525249">CLASSIC Singularity software container</a>.<strong> Please ensure you obtain them prior to using this dataset. Instructions are provided on the CLASSIC <a href="https://cccma.gitlab.io/classic_pages/info/get_started/">Quick Start Guide</a>.</strong></p> <p>This dataset contains <a href="https://fluxnet.fluxdata.org/data/fluxnet2015-dataset/">FLUXNET2015</a> data that is used to benchmark the <a href="https://cccma.gitlab.io/classic_pages/">Canadian Land Surface Scheme including Biogeochemical Cycles </a>(CLASSIC) v. 1.0. All model inputs required for the (31 for version 1.0) <a href="https://cccma.gitlab.io/classic_pages/benchmarking/">FLUXNET sites</a> are provided along with example outputs that benchmark CLASSIC v. 1.0. The model outputs include raw model outputs, plots of select variables and benchmarking results from the Automated Model Benchmarking (<a href="https://cran.r-project.org/package=amber">AMBER</a>) package. Following the the CLASSIC <a href="https://cccma.gitlab.io/classic_pages/info/get_started/">Quick Start Guide</a> will generate all outputs on the user's own machine.</p> <p>This work used eddy covariance data acquired and shared by the FLUXNET community, including these networks: AmeriFlux, AfriFlux, AsiaFlux, CarboAfrica, CarboEuropeIP, CarboItaly, CarboMont, ChinaFlux, Fluxnet-Canada, GreenGrass, ICOS, KoFlux, LBA, NECC, OzFlux-TERN, TCOS-Siberia, and USCCC. The ERA-Interim reanalysis data are provided by ECMWF and processed by LSCE. The FLUXNET eddy covariance data processing and harmonization was carried out by the European Fluxes Database Cluster, AmeriFlux Management Project, and Fluxdata project of FLUXNET, with the support of CDIAC and ICOS Ecosystem Thematic Center, and the OzFlux, ChinaFlux and AsiaFlux offices.</p> <p>We thank C. Le Quéré for allowing us to distribute her CO<sub>2</sub> record that was originally made for the TRENDY project.</p> <p> </p> <p> </p> <p> </p> <p> </p>
A small dataset for demonstrating the benchmarking of spot-detection/spot-counting workflows with BIAFLOWS
<p>The images were generated by <a href="http://www.cs.tut.fi/sgn/csb/simcep/tool.html">SIMCEP</a>, a widefield fluorescence microscopy biological images simulator.</p> <p>The dataset contains 5 input images and 5 ground-truth images with the suffix _lbl.</p>
MINBLoG Benchmarks: Boolean Logic Functions for Logic Minimization
<h3>Introduction:</h3> <p>The dataset contains synthetically generated boolean functions in PLA file format that can be used for testing runtime and QoR of logic minimization approaches (e.g Espresso, BOOM, MinBLoG etc.) The size of the functions ranges from 100 to 400 variables and 200 to 1000 cubes. The dataset is divided into three different types of boolean functions:</p> <blockquote> <p>F-Type: Functions specified using their ON-Set. Contained in synth_bench_f.zip. </p> <p>FR-Type: Functions specified using their ON-Set and OFF-Set. Contained in synth_bench_fr.zip. </p> <p>FD-Type: Functions specified using their ON-Set and DontCare-Set (DC-Set). Contained in synth_bench_fd.zip.</p> </blockquote> <h3>File Naming Convention:</h3> <p>The filenames of the PLA files indicate the size of the function it contain. </p> <p>Example:</p> <blockquote> <p>fd_200_400-0.pla = Contains an FD type function with 200 variables and 400 cubes. The suffix 0 at the end indicates it is the first of the 5 random functions generated with 200 variables and 400 cubes.</p> </blockquote> <h3>Reference:</h3> <p>Please cite the work below when using this benchmark. The dataset was generated for testing the boolean logic minimization tool MinBLoG published in this work. </p> <p>Prianka Sengupta, Aakash Tyagi, Jiang Hu, Vivek K Rajan, Hesham Mostafa, and Somdeb Majumdar. 2024. MinBLoG: Minimization of Boolean Logic Functions using Graph Attention Network. In 2024 ACM/IEEE International Symposium on Machine Learning for CAD (MLCAD ’24), September 9–11, 2024, Salt Lake City, UT, USA. ACM, New York, NY, USA, 8 pages. https: //doi.org/10.1145/3670474.3685962</p> <p>For the most up-to-date version of the dataset and accompanying tools, please check the GitHub repository below:</p> <p><a href="https://github.com/puprianka/minblog" target="_blank" rel="noopener">https://github.com/puprianka/minblog</a></p>
An in-situ daily dataset for benchmarking temporal variability of groundwater recharge
<p>A newly developed benchmark dataset of groundwater recharge per unit specific yield (RpSy, n meters) at daily temporal resolution is presented. The data has been obtained through the application of the Water table Fluctuation (WTF) method at groundwater wells within the continental US. To ensure high-fidelity estimates, only wells that meet a set of stringent criteria have been considered. The RpSy dataset may serve as a benchmark for validating the temporal consistency of recharge products and daily simulation results from land surface and integrated hydrologic models.</p> <p>The resulting product is a continuous daily RpSy (n meters) time series data for 485 groundwater wells. The data files are provided in the .csv format and consist of three columns for each observation well. The first column lists the local time, while the second and third columns provide the RpSy and RpSyu (considering a groundwater depth-dependent specific yield) time series in meters per day. Additionally, a file containing site information for all the selected wells is included. It contains four columns that detail the USGS ID of the groundwater well, its latitude (Lat), longitude (Long), and screen depth (depth, in meters). The data file can be accessed in most text editors and spreadsheets.</p>
Benchmark EEG data set for trust assessment for interactions with social robots
<p>The data collection consisted of a game interaction with a small humanoid EZ-robot. The robot explains a word to the participant either through movements depicting the concept or by verbal description. Depending on their performance, participants could "earn" or loose candy as remuneration for their participation.</p> <p>The dataset comprises EEG (Electroencephalography) recordings from 21 participants, gathered using Emotiv headsets. Each participant's EEG data includes timestamps and measurements from 14 sensors placed across different regions of the scalp. The sensor labels in the header are as follows: EEG.AF3, EEG.F7, EEG.F3, EEG.FC5, EEG.T7, EEG.P7, EEG.O1, EEG.O2, EEG.P8, EEG.T8, EEG.FC6, EEG.F4, EEG.F8, EEG.AF4, and Time.</p> <p>The EEG data provides insights into the electrical activity of the brain, offering a window into cognitive processes and emotional responses during various activities or stimuli in the form of microvolt and with a frame rate of 128 Hz. The whole data set consists of 3651124 data points for each sensor, i.e. 173863 on average for each participant (min. 128505, max. 249631). </p> <p>Files are named after participant numbers starting with ID01. The data has to be pre-processed making use of the information given in the details.xlsx file that contains annotations corresponding to the EEG recordings. These annotations denote the timing of different phases related to trust across the participants' interactions. Each phase is delineated by a start time and an end time, representing distinct stages of the trust-building process. All the other data (timestamps) which are outside the start and end of each phase should be considered as breaks, e.g. filling out the questionnaires. The last element is the trust score for the given phase, which is calculated on the answers in an MDMT questionnaire.</p> <p>The following phases have been annotated:</p> <ol> <li>Trust Building: This phase involves friendly initial interactions for establishing trust between participants and the robot.</li> <li>Situational Awareness: This phase continues to build up trust by showing situation awareness of the robot, e.g. by complimenting on the participant's fashion choice.</li> <li>Transparency: Trust is maintained by increased openness and clarity in communicating about the robot's abilities.</li> <li>Trust Violation: Trust is compromised during this phase by deliberately misleading the participant and making it impossible to answer correctly. </li> <li>Trust Repair: The robot shows efforts to repair trust by apologizing for the behavior in the previous stage.</li> </ol> <p>If you work with the data, please cite one of the article given below.</p>
GTSRB - German Traffic Sign Recognition Benchmark by Real-Time Computer Vision at Ruhr-Universität Bochum
<div> <div>The German Traffic Sign Benchmark is a multi-class, single-image classification challenge held at the International Joint Conference on Neural Networks (IJCNN) 2011. <br>Our benchmark has the following properties: <br>- Single-image, multi-class classification problem <br>- More than 40 classes<br>- More than 50,000 images in total <br>- Large, lifelike database<br><br>Acknowledgements: [INI Benchmark Website][1]<br>[1]: http://benchmark.ini.rub.de/</div> </div>
Supporting Data for: The V30 Benchmark Set for Anharmonic Vibrational Frequencies of Molecular Dimers
<p>Intermolecular vibrations are extremely challenging to describe but are the most crucial part for determining entropy and hence free energies, and enable for instance the distinction between different crystal-packing arrangements of the same molecule via THz spectroscopy. Herein, we introduce a benchmark data set - V30 - containing 30 small molecular dimers with intermolecular interactions ranging from exclusively van-der-Waals dispersion to systems with hydrogen bonds. All calculations are performed with the gold standard of Quantum Chemistry CCSD(T). We discuss vibrational frequencies obtained via different models starting with the harmonic approximation over independent Morse oscillators up to second-order vibrational perturbation theory (VPT2), which allows a proper anharmonic treatment including coupling of vibrational modes. However, large amplitude motions present in many low-frequency intermolecular modes are problematic for VPT2. In analogy to the often used treatment for internal rotations, we replace such problematic modes by a simple one-dimensional hindered rotor model. We compare selected dimers with available experimental data or high-level calculations of potential energy surfaces and show that VPT2 in combination with hindered rotors can yield a very good description of fundamental frequencies for the discussed subset of dimers involving small and semi-rigid molecules.<br><br>This supporting dataset includes the calculated force constants, harmonic frequencies, Morse frequencies, VPT2 frequencies, and the optimized structures for the V30 dataset. See the included README.md file for more details. The related preprint can be found at <a href="https://doi.org/10.48550/arXiv.2209.04392">https://doi.org/10.48550/arXiv.2209.04392</a>.</p>
Dataset: Benchmark Problems for Simulating Hyperloop Aerodynamics
<p>Dataset for the results contained within 'Benchmarks problems for Simulating Hyperloop Aerodynamics', Lang et al., Phys. Fluids 36 (2024). doi.org/10.1063/5.0229914</p> <p>In this study, 3 benchmark problems for simulating the aerodynamics of a Hyperloop system are proposed. This dataset gives the raw data used to generate the figures and also coordinates of the geometries used in the simulations.</p>
SUPER-G Benchmarking performance on commercial farms (task 3.1)
<p><span>The SUPER-<em>G </em>project aimed to co-develop sustainable Permanent Grassland (PG) systems and policies with farmers and policy makers to effectively optimise productivity whist supporting biodiversity and delivering other Ecosystem Services (ES). This was achieved through the completion of a series of Work Packages (WP). Work Package 3 aimed </span><span>to benchmark performance in terms of profitability and sustainability and the delivery of ES. This was achieved in part through </span><span>deliverable (D) D3.2, where an “</span><span>Overview of data, key gaps, and trends in PG management in the different biogeographic regions” was compiled. This overview included a survey of 352 farms across six biogeographic regions conducted in 2019, which explored on-farm management and differences between regions. A second, subsequent, survey was undertaken in 2023, which included 203 of the original farms from five biogeographic areas. Deliverable report 3.3. </span>describes the distribution of key indicators from farms surveyed in the first survey and describes any changes which were captured in the second survey. <span>A</span><span>gri-environment indicators and key sustainability indicators were identified in WP2 and include forage efficiency, grass utilisation (the proportion of grass dry matter grown that is consumed by livestock) and grass/clover dry matter (DM) productivity. Due to the limitations of how many questions can be asked in a survey and the time required to perform a survey, it was not possible to calculate all these metrics. This report examines the sustainability indicators of stocking density, grass production and output per hectare and environmental indicators of areas reserved for nature and biodiversity.</span></p> <p> </p> <p><span>The data files include the survey questions, responses from 2021 survey, responses from 2023 survey and a blank survey form. </span></p>
FRGADB - FIRST Radio Galaxy Anomaly Detection Benchmark
<p>This dataset is a combination of samples from the MiraBest, FRGMRC and LRG catalogues. It is intended to serve as a benchmark for models' performance with respect to anomalous source detection in radio astronomy.</p>
Dataset for Benchmarking the Sim-to-Real Gap in Cloth Manipulation
<p>This dataset is supplemental to the paper "Benchmarking the Sim-to-Real Gap in Cloth Manipulation".</p> <p>D. Blanco-Mulero, O. Barbany, G. Alcan, A. Colomé, C. Torras and V. Kyrki, "Benchmarking the Sim-to-Real Gap in Cloth Manipulation," in IEEE Robotics and Automation Letters, vol. 9, no. 3, pp. 2981-2988, March 2024, doi: 10.1109/LRA.2024.3360814</p>
Surrogate Modeling Benchmark - Ishigami function
<h1>Surrogate Modeling Benchmark - Ishigami function</h1> <p>This dataset is related to the Ishigami function benchmark case. A detailed description of the benchmark case can be found on the public online community website UQWorld: <a href="https://uqworld.org/t/benchmark-case-ishigami-function/" target="_blank" rel="noopener">https://uqworld.org/t/benchmark-case-ishigami-function/</a>.</p> <p>The experimental designs include datasets with 40, 80, 120, 160, and 200 samples, each generated using optimized maximin distance Latin Hypercube Sampling (LHS) with 1000 iterations. Each dataset is replicated 20 times. The validation set contains 100,000 samples generated by Monte Carlo simulation. Each dataset contains input samples and the corresponding computational model responses.</p> <h2>Description of the dataset file</h2> <p>The dataset file includes two variables:</p> <ul> <li><em>ExpDesigns</em>, and</li> <li><em>ValidationSet</em>.</li> </ul> <p>Both variables are Matlab structures with fields <em>X</em>, <em>Y</em>, and <em>nSamples</em>. Variable <em>ExpDesigns</em> is a non-scalar structure sized according to the number of experimental design groups. Each field of <em>X</em> for the i-th element of the struct array contains replicated datasets, forming a matrix of size [number of samples] x [dimensionality] x [number of replications]. Similarly, each field of Y for the i-th element contains replicated computational model responses that correspond to the experimental design of the same replication, sized [number of samples] x [number of model outputs] x [number of replications]. The same structure logic applies to the <em>ValidationSet</em> variable, except it contains only one dataset per benchmark case.</p> <p>The structure can be summarized as follows:</p> <ul> <li>ExpDesigns(i).X(j,k,l) <ul> <li>i: dataset group,</li> <li>j: sample index,</li> <li>k: variable index, and</li> <li>l: replication index.</li> </ul> </li> </ul> <ul> <li>ExpDesigns(i).Y(j,m,l) <ul> <li>i, j, l: same as above,</li> <li>m: computational model output index.</li> </ul> </li> </ul> <ul> <li>ValidationSet.X(j,k) <ul> <li>j, k: same as above.</li> </ul> </li> </ul> <ul> <li>ValidationSet.Y(j,m) <ul> <li>j, m: same as above.</li> </ul> </li> </ul> <h2>Description of benchmarked metamodel competitors</h2> <p>The selection of competitors was based on our experience with meta-modeling and includes various metamodel types: Polynomial Chaos Expansions (PCE), Polynomial Chaos Kriging (PCK), and Kriging. Given that each metamodel has many hyperparameters, we chose the most general settings to address different benchmark case difficulties, including dimensionality, nonlinearity, and non-monotonicity.</p> <p>For <strong>Polynomial Chaos Expansions (PCE)</strong>, we used a polynomial degree and q-norm adaptivity approach. This approach adaptively increases the maximum polynomial degree and truncation q-norm until the estimated leave-one-out error starts increasing. Maximum polynomial interaction terms were limited to 2 due to the memory requirements for large model dimensionality and large experimental designs. We tested three different solvers to calculate the PCE coefficients: Least Angle Regression (LARS), Orthogonal Matching Pursuit (OMP), and Subspace Pursuit (SP).</p> <p><strong>Polynomial Chaos Kriging (PCK)</strong> employs a sequential combination strategy of PCE and Kriging. PCE uses degree adaptivity with a fixed q-norm. The maximum number of interactions is again set to 2 with the LARS solver. Ordinary Kriging is applied using the Matérn-5/2 correlation family, ellipsoidal, and anisotropic correlation function. We used a hybrid genetic algorithm to optimize the hyperparameters.</p> <p>We benchmarked both linear and ordinary <strong>Kriging</strong>, including Matérn-5/2 and Gaussian correlation families and separable and ellipsoidal correlation, resulting in eight different Kriging competitors. The hyperparameters were calculated using a hybrid covariance matrix adaptation-evolution strategy optimization.</p> <p>For further details on the settings, please refer to the competitors.m file and UQLab user manuals:</p> <ul> <li>S. Marelli, N. Luethen, B. Sudret, <a href="https://www.uqlab.com/pce-user-manual">UQLab User Manual – Polynomial Chaos Expansions</a>, Report UQLab-V2.1-104, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland, 2024.</li> <li>C. Lataniotis, D. Wicaksono, S. Marelli, B. Sudret, <a href="https://www.uqlab.com/kriging-user-manual">UQLab User Manual – Kriging (Gaussian Process Modeling)</a>, Report UQLab-V2.1-105, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland, 2024.</li> <li>R. Schoebi, S. Marelli, B. Sudret, <a href="https://www.uqlab.com/pck-user-manual">UQLab User Manual – Polynomial Chaos Kriging</a>, Report UQLab-V2.0-109, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland, 2022.</li> </ul> <h2>Description of the results file</h2> <p>The results file contains one variable: <em>Metrics</em>. It is a Matlab structure with fields corresponding to each competitor (currently 12). Each competitor field contains data of type non-scalar struct array. The performance metrics included are RelMSE, RelRMSE, RelMAE, MAPE, Q2, and RelCVErr. Each field of Metrics.(CompetitorName) for the i-th element of the struct array contains metrics corresponding to the replicated dataset and the competitor, structured as follows:</p> <ul> <li>Metrics.(CompetitorName)(i).(MetricName)(l)<br> <ul> <li>i: dataset group,</li> <li>l: replication index.</li> </ul> </li> </ul> <p>The description of the performance measures (metrics) can be found here: <a href="https://uqworld.org/t/metamodel-performance-measures/" target="_blank" rel="noopener">https://uqworld.org/t/metamodel-performance-measures/</a>.</p> <h2>Additional files</h2> <p>We provide files in three languages (MATLAB, Python, and Julia) to showcase how to work with datasets, results, and their visualization. The files are called <em>working_with_datafiles.*</em> (the extension depends on the selected language).</p> <h2>Acknowledgment</h2> <p>This project was supported by the Open Research Data Program of the ETH Board under Grant number EPFL SCR0902285. The calculations were run on the Euler cluster of ETH Zürich using the MATLAB-based UQLab software developed at the Chair of Risk, Safety and Uncertainty Quantification of ETH Zürich.</p>
Surrogate Modeling Benchmark - Two-dimensional heat diffusion model
<p>This dataset is related to the Two-dimensional heat diffusion model benchmark case. A detailed description of the benchmark case can be found on the public online community website UQWorld: <a href="https://uqworld.org/t/benchmark-case-two-dimensional-heat-diffusion-model/" target="_blank" rel="noopener">https://uqworld.org/t/benchmark-case-two-dimensional-heat-diffusion-model/</a>.</p> <p>The experimental designs include datasets with 400, 800, 1200, 1600, and 2000 samples, each generated using optimized maximin distance Latin Hypercube Sampling (LHS) with 1000 iterations. Each dataset is replicated 20 times. The validation set contains 100,000 samples generated by Monte Carlo simulation. Each dataset contains input samples and the corresponding computational model responses.</p> <h2>Description of the dataset file</h2> <p>The dataset file includes two variables:</p> <ul> <li><em>ExpDesigns</em>, and</li> <li><em>ValidationSet</em>.</li> </ul> <p>Both variables are Matlab structures with fields <em>X</em>, <em>Y</em>, and <em>nSamples</em>. Variable <em>ExpDesigns</em> is a non-scalar structure sized according to the number of experimental design groups. Each field of <em>X</em> for the i-th element of the struct array contains replicated datasets, forming a matrix of size [number of samples] x [dimensionality] x [number of replications]. Similarly, each field of Y for the i-th element contains replicated computational model responses that correspond to the experimental design of the same replication, sized [number of samples] x [number of model outputs] x [number of replications]. The same structure logic applies to the <em>ValidationSet</em> variable, except it contains only one dataset per benchmark case.</p> <p>The structure can be summarized as follows:</p> <ul> <li>ExpDesigns(i).X(j,k,l) <ul> <li>i: dataset group,</li> <li>j: sample index,</li> <li>k: variable index, and</li> <li>l: replication index.</li> </ul> </li> </ul> <ul> <li>ExpDesigns(i).Y(j,m,l) <ul> <li>i, j, l: same as above,</li> <li>m: computational model output index.</li> </ul> </li> </ul> <ul> <li>ValidationSet.X(j,k) <ul> <li>j, k: same as above.</li> </ul> </li> </ul> <ul> <li>ValidationSet.Y(j,m) <ul> <li>j, m: same as above.</li> </ul> </li> </ul> <h2>Description of benchmarked metamodel competitors</h2> <p>The selection of competitors was based on our experience with meta-modeling and includes various metamodel types: Polynomial Chaos Expansions (PCE), Polynomial Chaos Kriging (PCK), and Kriging. Given that each metamodel has many hyperparameters, we chose the most general settings to address different benchmark case difficulties, including dimensionality, nonlinearity, and non-monotonicity.</p> <p>For <strong>Polynomial Chaos Expansions (PCE)</strong>, we used a polynomial degree and q-norm adaptivity approach. This approach adaptively increases the maximum polynomial degree and truncation q-norm until the estimated leave-one-out error starts increasing. Maximum polynomial interaction terms were limited to 2 due to the memory requirements for large model dimensionality and large experimental designs. We tested three different solvers to calculate the PCE coefficients: Least Angle Regression (LARS), Orthogonal Matching Pursuit (OMP), and Subspace Pursuit (SP).</p> <p><strong>Polynomial Chaos Kriging (PCK)</strong> employs a sequential combination strategy of PCE and Kriging. PCE uses degree adaptivity with a fixed q-norm. The maximum number of interactions is again set to 2 with the LARS solver. Ordinary Kriging is applied using the Matérn-5/2 correlation family, ellipsoidal, and anisotropic correlation function. We used a hybrid genetic algorithm to optimize the hyperparameters.</p> <p>We benchmarked both linear and ordinary <strong>Kriging</strong>, including Matérn-5/2 and Gaussian correlation families and separable and ellipsoidal correlation, resulting in eight different Kriging competitors. The hyperparameters were calculated using a hybrid covariance matrix adaptation-evolution strategy optimization.</p> <p>For further details on the settings, please refer to the competitors.m file and UQLab user manuals:</p> <ul> <li>S. Marelli, N. Luethen, B. Sudret, <a href="https://www.uqlab.com/pce-user-manual">UQLab User Manual – Polynomial Chaos Expansions</a>, Report UQLab-V2.1-104, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland, 2024.</li> <li>C. Lataniotis, D. Wicaksono, S. Marelli, B. Sudret, <a href="https://www.uqlab.com/kriging-user-manual">UQLab User Manual – Kriging (Gaussian Process Modeling)</a>, Report UQLab-V2.1-105, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland, 2024.</li> <li>R. Schoebi, S. Marelli, B. Sudret, <a href="https://www.uqlab.com/pck-user-manual">UQLab User Manual – Polynomial Chaos Kriging</a>, Report UQLab-V2.0-109, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland, 2022.</li> </ul> <h2>Description of the results file</h2> <p>The results file contains one variable: <em>Metrics</em>. It is a Matlab structure with fields corresponding to each competitor (currently 12). Each competitor field contains data of type non-scalar struct array. The performance metrics included are RelMSE, RelRMSE, RelMAE, MAPE, Q2, and RelCVErr. Each field of Metrics.(CompetitorName) for the i-th element of the struct array contains metrics corresponding to the replicated dataset and the competitor, structured as follows:</p> <ul> <li>Metrics.(CompetitorName)(i).(MetricName)(l)<br> <ul> <li>i: dataset group,</li> <li>l: replication index.</li> </ul> </li> </ul> <p>The description of the performance measures (metrics) can be found here: <a href="https://uqworld.org/t/metamodel-performance-measures/" target="_blank" rel="noopener">https://uqworld.org/t/metamodel-performance-measures/</a>.</p> <h2>Additional files</h2> <p>We provide files in three languages (MATLAB, Python, and Julia) to showcase how to work with datasets, results, and their visualization. The files are called <em>working_with_datafiles.*</em> (the extension depends on the selected language).</p> <h2>Acknowledgment</h2> <p>This project was supported by the Open Research Data Program of the ETH Board under Grant number EPFL SCR0902285. The calculations were run on the Euler cluster of ETH Zürich using the MATLAB-based UQLab software developed at the Chair of Risk, Safety and Uncertainty Quantification of ETH Zürich.</p>
Surrogate Modeling Benchmark - 100D function
<p>This dataset is related to the 100D function benchmark case. A detailed description of the benchmark case can be found on the public online community website UQWorld: <a href="https://uqworld.org/t/benchmark-case-100d-function/" target="_blank" rel="noopener">https://uqworld.org/t/benchmark-case-100d-function/</a>.</p> <p>The experimental designs include datasets with 400, 800, 1200, 1600, and 2000 samples, each generated using optimized maximin distance Latin Hypercube Sampling (LHS) with 1000 iterations. Each dataset is replicated 20 times. The validation set contains 100,000 samples generated by Monte Carlo simulation. Each dataset contains input samples and the corresponding computational model responses.</p> <h2>Description of the dataset file</h2> <p>The dataset file includes two variables:</p> <ul> <li><em>ExpDesigns</em>, and</li> <li><em>ValidationSet</em>.</li> </ul> <p>Both variables are Matlab structures with fields <em>X</em>, <em>Y</em>, and <em>nSamples</em>. Variable <em>ExpDesigns</em> is a non-scalar structure sized according to the number of experimental design groups. Each field of <em>X</em> for the i-th element of the struct array contains replicated datasets, forming a matrix of size [number of samples] x [dimensionality] x [number of replications]. Similarly, each field of Y for the i-th element contains replicated computational model responses that correspond to the experimental design of the same replication, sized [number of samples] x [number of model outputs] x [number of replications]. The same structure logic applies to the <em>ValidationSet</em> variable, except it contains only one dataset per benchmark case.</p> <p>The structure can be summarized as follows:</p> <ul> <li>ExpDesigns(i).X(j,k,l) <ul> <li>i: dataset group,</li> <li>j: sample index,</li> <li>k: variable index, and</li> <li>l: replication index.</li> </ul> </li> </ul> <ul> <li>ExpDesigns(i).Y(j,m,l) <ul> <li>i, j, l: same as above,</li> <li>m: computational model output index.</li> </ul> </li> </ul> <ul> <li>ValidationSet.X(j,k) <ul> <li>j, k: same as above.</li> </ul> </li> </ul> <ul> <li>ValidationSet.Y(j,m) <ul> <li>j, m: same as above.</li> </ul> </li> </ul> <h2>Description of benchmarked metamodel competitors</h2> <p>The selection of competitors was based on our experience with meta-modeling and includes various metamodel types: Polynomial Chaos Expansions (PCE), Polynomial Chaos Kriging (PCK), and Kriging. Given that each metamodel has many hyperparameters, we chose the most general settings to address different benchmark case difficulties, including dimensionality, nonlinearity, and non-monotonicity.</p> <p>For <strong>Polynomial Chaos Expansions (PCE)</strong>, we used a polynomial degree and q-norm adaptivity approach. This approach adaptively increases the maximum polynomial degree and truncation q-norm until the estimated leave-one-out error starts increasing. Maximum polynomial interaction terms were limited to 2 due to the memory requirements for large model dimensionality and large experimental designs. We tested three different solvers to calculate the PCE coefficients: Least Angle Regression (LARS), Orthogonal Matching Pursuit (OMP), and Subspace Pursuit (SP).</p> <p><strong>Polynomial Chaos Kriging (PCK)</strong> employs a sequential combination strategy of PCE and Kriging. PCE uses degree adaptivity with a fixed q-norm. The maximum number of interactions is again set to 2 with the LARS solver. Ordinary Kriging is applied using the Matérn-5/2 correlation family, ellipsoidal, and anisotropic correlation function. We used a hybrid genetic algorithm to optimize the hyperparameters.</p> <p>We benchmarked both linear and ordinary <strong>Kriging</strong>, including Matérn-5/2 and Gaussian correlation families and separable and ellipsoidal correlation, resulting in eight different Kriging competitors. The hyperparameters were calculated using a hybrid covariance matrix adaptation-evolution strategy optimization.</p> <p>For further details on the settings, please refer to the competitors.m file and UQLab user manuals:</p> <ul> <li>S. Marelli, N. Luethen, B. Sudret, <a href="https://www.uqlab.com/pce-user-manual">UQLab User Manual – Polynomial Chaos Expansions</a>, Report UQLab-V2.1-104, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland, 2024.</li> <li>C. Lataniotis, D. Wicaksono, S. Marelli, B. Sudret, <a href="https://www.uqlab.com/kriging-user-manual">UQLab User Manual – Kriging (Gaussian Process Modeling)</a>, Report UQLab-V2.1-105, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland, 2024.</li> <li>R. Schoebi, S. Marelli, B. Sudret, <a href="https://www.uqlab.com/pck-user-manual">UQLab User Manual – Polynomial Chaos Kriging</a>, Report UQLab-V2.0-109, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland, 2022.</li> </ul> <h2>Description of the results file</h2> <p>The results file contains one variable: <em>Metrics</em>. It is a Matlab structure with fields corresponding to each competitor (currently 12). Each competitor field contains data of type non-scalar struct array. The performance metrics included are RelMSE, RelRMSE, RelMAE, MAPE, Q2, and RelCVErr. Each field of Metrics.(CompetitorName) for the i-th element of the struct array contains metrics corresponding to the replicated dataset and the competitor, structured as follows:</p> <ul> <li>Metrics.(CompetitorName)(i).(MetricName)(l)<br> <ul> <li>i: dataset group,</li> <li>l: replication index.</li> </ul> </li> </ul> <p>The description of the performance measures (metrics) can be found here: <a href="https://uqworld.org/t/metamodel-performance-measures/" target="_blank" rel="noopener">https://uqworld.org/t/metamodel-performance-measures/</a>.</p> <h2>Additional files</h2> <p>We provide files in three languages (MATLAB, Python, and Julia) to showcase how to work with datasets, results, and their visualization. The files are called <em>working_with_datafiles.*</em> (the extension depends on the selected language).</p> <h2>Acknowledgment</h2> <p>This project was supported by the Open Research Data Program of the ETH Board under Grant number EPFL SCR0902285. The calculations were run on the Euler cluster of ETH Zürich using the MATLAB-based UQLab software developed at the Chair of Risk, Safety and Uncertainty Quantification of ETH Zürich.</p>
Surrogate Modeling Benchmark - Morris function
<p>This dataset is related to the Morris function benchmark case. A detailed description of the benchmark case can be found on the public online community website UQWorld: <a href="https://uqworld.org/t/benchmark-case-morris-function" target="_blank" rel="noopener">https://uqworld.org/t/benchmark-case-morris-function</a>.</p> <p>The experimental designs include datasets with 400, 800, 1200, 1600, and 2000 samples, each generated using optimized maximin distance Latin Hypercube Sampling (LHS) with 1000 iterations. Each dataset is replicated 20 times. The validation set contains 100,000 samples generated by Monte Carlo simulation. Each dataset contains input samples and the corresponding computational model responses.</p> <h2>Description of the dataset file</h2> <p>The dataset file includes two variables:</p> <ul> <li><em>ExpDesigns</em>, and</li> <li><em>ValidationSet</em>.</li> </ul> <p>Both variables are Matlab structures with fields <em>X</em>, <em>Y</em>, and <em>nSamples</em>. Variable <em>ExpDesigns</em> is a non-scalar structure sized according to the number of experimental design groups. Each field of <em>X</em> for the i-th element of the struct array contains replicated datasets, forming a matrix of size [number of samples] x [dimensionality] x [number of replications]. Similarly, each field of Y for the i-th element contains replicated computational model responses that correspond to the experimental design of the same replication, sized [number of samples] x [number of model outputs] x [number of replications]. The same structure logic applies to the <em>ValidationSet</em> variable, except it contains only one dataset per benchmark case.</p> <p>The structure can be summarized as follows:</p> <ul> <li>ExpDesigns(i).X(j,k,l) <ul> <li>i: dataset group,</li> <li>j: sample index,</li> <li>k: variable index, and</li> <li>l: replication index.</li> </ul> </li> </ul> <ul> <li>ExpDesigns(i).Y(j,m,l) <ul> <li>i, j, l: same as above,</li> <li>m: computational model output index.</li> </ul> </li> </ul> <ul> <li>ValidationSet.X(j,k) <ul> <li>j, k: same as above.</li> </ul> </li> </ul> <ul> <li>ValidationSet.Y(j,m) <ul> <li>j, m: same as above.</li> </ul> </li> </ul> <h2>Description of benchmarked metamodel competitors</h2> <p>The selection of competitors was based on our experience with meta-modeling and includes various metamodel types: Polynomial Chaos Expansions (PCE), Polynomial Chaos Kriging (PCK), and Kriging. Given that each metamodel has many hyperparameters, we chose the most general settings to address different benchmark case difficulties, including dimensionality, nonlinearity, and non-monotonicity.</p> <p>For <strong>Polynomial Chaos Expansions (PCE)</strong>, we used a polynomial degree and q-norm adaptivity approach. This approach adaptively increases the maximum polynomial degree and truncation q-norm until the estimated leave-one-out error starts increasing. Maximum polynomial interaction terms were limited to 2 due to the memory requirements for large model dimensionality and large experimental designs. We tested three different solvers to calculate the PCE coefficients: Least Angle Regression (LARS), Orthogonal Matching Pursuit (OMP), and Subspace Pursuit (SP).</p> <p><strong>Polynomial Chaos Kriging (PCK)</strong> employs a sequential combination strategy of PCE and Kriging. PCE uses degree adaptivity with a fixed q-norm. The maximum number of interactions is again set to 2 with the LARS solver. Ordinary Kriging is applied using the Matérn-5/2 correlation family, ellipsoidal, and anisotropic correlation function. We used a hybrid genetic algorithm to optimize the hyperparameters.</p> <p>We benchmarked both linear and ordinary <strong>Kriging</strong>, including Matérn-5/2 and Gaussian correlation families and separable and ellipsoidal correlation, resulting in eight different Kriging competitors. The hyperparameters were calculated using a hybrid covariance matrix adaptation-evolution strategy optimization.</p> <p>For further details on the settings, please refer to the competitors.m file and UQLab user manuals:</p> <ul> <li>S. Marelli, N. Luethen, B. Sudret, <a href="https://www.uqlab.com/pce-user-manual">UQLab User Manual – Polynomial Chaos Expansions</a>, Report UQLab-V2.1-104, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland, 2024.</li> <li>C. Lataniotis, D. Wicaksono, S. Marelli, B. Sudret, <a href="https://www.uqlab.com/kriging-user-manual">UQLab User Manual – Kriging (Gaussian Process Modeling)</a>, Report UQLab-V2.1-105, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland, 2024.</li> <li>R. Schoebi, S. Marelli, B. Sudret, <a href="https://www.uqlab.com/pck-user-manual">UQLab User Manual – Polynomial Chaos Kriging</a>, Report UQLab-V2.0-109, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland, 2022.</li> </ul> <h2>Description of the results file</h2> <p>The results file contains one variable: <em>Metrics</em>. It is a Matlab structure with fields corresponding to each competitor (currently 12). Each competitor field contains data of type non-scalar struct array. The performance metrics included are RelMSE, RelRMSE, RelMAE, MAPE, Q2, and RelCVErr. Each field of Metrics.(CompetitorName) for the i-th element of the struct array contains metrics corresponding to the replicated dataset and the competitor, structured as follows:</p> <ul> <li>Metrics.(CompetitorName)(i).(MetricName)(l)<br> <ul> <li>i: dataset group,</li> <li>l: replication index.</li> </ul> </li> </ul> <p>The description of the performance measures (metrics) can be found here: <a href="https://uqworld.org/t/metamodel-performance-measures/" target="_blank" rel="noopener">https://uqworld.org/t/metamodel-performance-measures/</a>.</p> <h2>Additional files</h2> <p>We provide files in three languages (MATLAB, Python, and Julia) to showcase how to work with datasets, results, and their visualization. The files are called <em>working_with_datafiles.*</em> (the extension depends on the selected language).</p> <h2>Acknowledgment</h2> <p>This project was supported by the Open Research Data Program of the ETH Board under Grant number EPFL SCR0902285. The calculations were run on the Euler cluster of ETH Zürich using the MATLAB-based UQLab software developed at the Chair of Risk, Safety and Uncertainty Quantification of ETH Zürich.</p>
Surrogate Modeling Benchmark - One-dimensional diffusion model
<p>This dataset is related to the One-dimensional diffusion model benchmark case. A detailed description of the benchmark case can be found on the public online community website UQWorld: <a href="https://uqworld.org/t/benchmark-case-one-dimensional-diffusion-model" target="_blank" rel="noopener">https://uqworld.org/t/benchmark-case-one-dimensional-diffusion-model</a>.</p> <p>The experimental designs include datasets with 200, 400, 600, 800, and 1000 samples, each generated using optimized maximin distance Latin Hypercube Sampling (LHS) with 1000 iterations. Each dataset is replicated 20 times. The validation set contains 100,000 samples generated by Monte Carlo simulation. Each dataset contains input samples and the corresponding computational model responses.</p> <h2>Description of the dataset file</h2> <p>The dataset file includes two variables:</p> <ul> <li><em>ExpDesigns</em>, and</li> <li><em>ValidationSet</em>.</li> </ul> <p>Both variables are Matlab structures with fields <em>X</em>, <em>Y</em>, and <em>nSamples</em>. Variable <em>ExpDesigns</em> is a non-scalar structure sized according to the number of experimental design groups. Each field of <em>X</em> for the i-th element of the struct array contains replicated datasets, forming a matrix of size [number of samples] x [dimensionality] x [number of replications]. Similarly, each field of Y for the i-th element contains replicated computational model responses that correspond to the experimental design of the same replication, sized [number of samples] x [number of model outputs] x [number of replications]. The same structure logic applies to the <em>ValidationSet</em> variable, except it contains only one dataset per benchmark case.</p> <p>The structure can be summarized as follows:</p> <ul> <li>ExpDesigns(i).X(j,k,l) <ul> <li>i: dataset group,</li> <li>j: sample index,</li> <li>k: variable index, and</li> <li>l: replication index.</li> </ul> </li> </ul> <ul> <li>ExpDesigns(i).Y(j,m,l) <ul> <li>i, j, l: same as above,</li> <li>m: computational model output index.</li> </ul> </li> </ul> <ul> <li>ValidationSet.X(j,k) <ul> <li>j, k: same as above.</li> </ul> </li> </ul> <ul> <li>ValidationSet.Y(j,m) <ul> <li>j, m: same as above.</li> </ul> </li> </ul> <h2>Description of benchmarked metamodel competitors</h2> <p>The selection of competitors was based on our experience with meta-modeling and includes various metamodel types: Polynomial Chaos Expansions (PCE), Polynomial Chaos Kriging (PCK), and Kriging. Given that each metamodel has many hyperparameters, we chose the most general settings to address different benchmark case difficulties, including dimensionality, nonlinearity, and non-monotonicity.</p> <p>For <strong>Polynomial Chaos Expansions (PCE)</strong>, we used a polynomial degree and q-norm adaptivity approach. This approach adaptively increases the maximum polynomial degree and truncation q-norm until the estimated leave-one-out error starts increasing. Maximum polynomial interaction terms were limited to 2 due to the memory requirements for large model dimensionality and large experimental designs. We tested three different solvers to calculate the PCE coefficients: Least Angle Regression (LARS), Orthogonal Matching Pursuit (OMP), and Subspace Pursuit (SP).</p> <p><strong>Polynomial Chaos Kriging (PCK)</strong> employs a sequential combination strategy of PCE and Kriging. PCE uses degree adaptivity with a fixed q-norm. The maximum number of interactions is again set to 2 with the LARS solver. Ordinary Kriging is applied using the Matérn-5/2 correlation family, ellipsoidal, and anisotropic correlation function. We used a hybrid genetic algorithm to optimize the hyperparameters.</p> <p>We benchmarked both linear and ordinary <strong>Kriging</strong>, including Matérn-5/2 and Gaussian correlation families and separable and ellipsoidal correlation, resulting in eight different Kriging competitors. The hyperparameters were calculated using a hybrid covariance matrix adaptation-evolution strategy optimization.</p> <p>For further details on the settings, please refer to the competitors.m file and UQLab user manuals:</p> <ul> <li>S. Marelli, N. Luethen, B. Sudret, <a href="https://www.uqlab.com/pce-user-manual">UQLab User Manual – Polynomial Chaos Expansions</a>, Report UQLab-V2.1-104, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland, 2024.</li> <li>C. Lataniotis, D. Wicaksono, S. Marelli, B. Sudret, <a href="https://www.uqlab.com/kriging-user-manual">UQLab User Manual – Kriging (Gaussian Process Modeling)</a>, Report UQLab-V2.1-105, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland, 2024.</li> <li>R. Schoebi, S. Marelli, B. Sudret, <a href="https://www.uqlab.com/pck-user-manual">UQLab User Manual – Polynomial Chaos Kriging</a>, Report UQLab-V2.0-109, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland, 2022.</li> </ul> <h2>Description of the results file</h2> <p>The results file contains one variable: <em>Metrics</em>. It is a Matlab structure with fields corresponding to each competitor (currently 12). Each competitor field contains data of type non-scalar struct array. The performance metrics included are RelMSE, RelRMSE, RelMAE, MAPE, Q2, and RelCVErr. Each field of Metrics.(CompetitorName) for the i-th element of the struct array contains metrics corresponding to the replicated dataset and the competitor, structured as follows:</p> <ul> <li>Metrics.(CompetitorName)(i).(MetricName)(l)<br> <ul> <li>i: dataset group,</li> <li>l: replication index.</li> </ul> </li> </ul> <p>The description of the performance measures (metrics) can be found here: <a href="https://uqworld.org/t/metamodel-performance-measures/" target="_blank" rel="noopener">https://uqworld.org/t/metamodel-performance-measures/</a>.</p> <h2>Additional files</h2> <p>We provide files in three languages (MATLAB, Python, and Julia) to showcase how to work with datasets, results, and their visualization. The files are called <em>working_with_datafiles.*</em> (the extension depends on the selected language).</p> <h2>Acknowledgment</h2> <p>This project was supported by the Open Research Data Program of the ETH Board under Grant number EPFL SCR0902285. The calculations were run on the Euler cluster of ETH Zürich using the MATLAB-based UQLab software developed at the Chair of Risk, Safety and Uncertainty Quantification of ETH Zürich.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.