Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
1,549
datasets available to search
ShareScore release 0.7.1
Dataset results
1,549 results for “benchmark”
Soil moisture and soil temperature from Benchmark Stations at the HJ Andrews Experimental Forest, 1987 to present
A three-level hydro-climatological network for data monitoring was established in 1994. The networks at each level are nested to form a coordinated program of data acquisition and measurement. A future vision of linking the benchmark meteorological stations with regional weather stations to expand the future scope of studies was also considered in designing this network. The first-level in this top-down approach consists of Benchmark Meteorological Stations (BMS) and Benchmark Stream Stations. The BMS are designed to represent the environment across the Andrews. These stations are intended to provide complete, long-term, high temporal resolution, meso-scale hydroclimatological data. The location of the BMS network is based on factors such as elevation, aspect, vegetation gradients, and accessibility. Collected meteorological parameters are generally standardized across the BMS as well as methods and instrumentation. Secondary Meteorological Stations also follow standardized methods and serve similar purposes but are somewhat limited in meteorological parameters collected. The Primary Meteorological Station (PRIMET), Central Meteorological Station (CENMET), Upper Lookout Meteorological Station (UPLMET), and Vanilla Leaf Meteorological Station (VANMET) are the four Benchmark Stations, Climatic Station at Watershed 2 (CS2MET) and the Hi-15 Meteorological Station (H15MET) are Secondary Stations. These soil parameters were previously part of database code MS001, but were separated out into their own database in 2024.
Wind speed and direction data from benchmark stations at the HJ Andrews Experimental Forest, 1973 to present
A three-level hydro-climatological network for data monitoring was established in 1994. The networks at each level are nested to form a coordinated program of data acquisition and measurement. A future vision of linking the benchmark meteorological stations with regional weather stations to expand the future scope of studies was also considered in designing this network. The first-level in this top-down approach consists of Benchmark Meteorological Stations (BMS) and Benchmark Stream Stations. The BMS are designed to represent the environment across the Andrews. These stations are intended to provide complete, long-term, high temporal resolution, meso-scale hydroclimatological data. The location of the BMS network is based on factors such as elevation, aspect, vegetation gradients, and accessibility. Collected meteorological parameters are generally standardized across the BMS as well as methods and instrumentation. Secondary Meteorological Stations also follow standardized methods and serve similar purposes but are somewhat limited in meteorological parameters collected. The Primary Meteorological Station (PRIMET), Central Meteorological Station (CENMET), Upper Lookout Meteorological Station (UPLMET), and Vanilla Leaf Meteorological Station (VANMET) are the four Benchmark Stations, Climatic Station at Watershed 2 (CS2MET) and the Hi-15 Meteorological Station (H15MET) are Secondary Stations. These wind parameters were previously part of database code MS001, but were separated out into their own database in 2024.
Solar radiation data from benchmark stations at the HJ Andrews Experimental Forest, 1973 to present
A three-level hydro-climatological network for data monitoring was established in 1994. The networks at each level are nested to form a coordinated program of data acquisition and measurement. A future vision of linking the benchmark meteorological stations with regional weather stations to expand the future scope of studies was also considered in designing this network. The first-level in this top-down approach consists of Benchmark Meteorological Stations (BMS) and Benchmark Stream Stations. The BMS are designed to represent the environment across the Andrews. These stations are intended to provide complete, long-term, high temporal resolution, meso-scale hydroclimatological data. The location of the BMS network is based on factors such as elevation, aspect, vegetation gradients, and accessibility. Collected meteorological parameters are generally standardized across the BMS as well as methods and instrumentation. Secondary Meteorological Stations also follow standardized methods and serve similar purposes but are somewhat limited in meteorological parameters collected. The Primary Meteorological Station (PRIMET), Central Meteorological Station (CENMET), Upper Lookout Meteorological Station (UPLMET), and Vanilla Leaf Meteorological Station (VANMET) are the four Benchmark Stations, Climatic Station at Watershed 2 (CS2MET), and the Hi-15 Meteorological Station (H15MET) are Secondary Stations. In 2006, an additional Secondary Station was added at Watershed 7 (WS7MET) These solar radiation parameters were previously part of database code MS001, but were separated out into their own database in 2024 and the entities and attributes were reorganized in 2025 into shortwave, longwave, net radiation and PAR. The shortwave and longwave entities include both incoming and outgoing radiation where measured (see probe_code and method_code).
Meteorological data from benchmark stations at the HJ Andrews Experimental Forest, 1957 to present
A three-level hydro-climatological network for data monitoring was established in 1994. The networks at each level are nested to form a coordinated program of data acquisition and measurement. A future vision of linking the benchmark meteorological stations with regional weather stations to expand the future scope of studies was also considered in designing this network. The first-level in this top-down approach consists of Benchmark Meteorological Stations (BMS) and Benchmark Stream Stations. The BMS are designed to represent the environment across the Andrews. These stations are intended to provide complete, long-term, high temporal resolution, meso-scale hydroclimatological data. The location of the BMS network is based on factors such as elevation, aspect, vegetation gradients, and accessibility. Collected meteorological parameters are generally standardized across the BMS as well as methods and instrumentation. Secondary Meteorological Stations also follow standardized methods and serve similar purposes but are somewhat limited in meteorological parameters collected. The Primary Meteorological Station (PRIMET), Central Meteorological Station (CENMET), Upper Lookout Meteorological Station (UPLMET), and Vanilla Leaf Meteorological Station (VANMET) are the four Benchmark Stations, Climatic Station at Watershed 2 (CS2MET) and the Hi-15 Meteorological Station (H15MET) are Secondary Stations. Watershed 7 Meteorological Station (WS7MET) was instrumented in 2006 and serves as an additional Secondary Station. In 2024, entities/measurement parameters (precipitation, wind, solar, soil, and snow) from MS001 database were partitioned out into separate databases.
Dataset: Environmental benchmarks for European Cement Industry
<p>This dataset contains the information relative to the article "Environemntal benchmarks for European cement industry".</p> <p><a title="Persistent link using digital object identifier" href="https://doi.org/10.1016/j.spc.2024.01.020" target="_blank" rel="noopener">Reference paper</a></p> <p><a href="https://www.researchgate.net/publication/377796848_Environmental_benchmarks_for_the_European_cement_industry" target="_blank" rel="noopener">ResearchGate link</a></p>
Benchmarking on Microservices Configurations and the Impact on the Performance in Cloud Native Environments
<p><strong>The peer reviewed publication for this dataset has been published in LCN 2022, 47th Annual IEEE Conference on Local Computer Networks. Please cite this paper when referring to the dataset: https://www.eurecom.fr/publication/6971.</strong></p> <p>Cloud-native and containerization have changed the way to develop and deploy applications. Cloud-native rethinks the application architecture by embracing a microservice approach, where each microservice is packaged into containers to run in a centralized or an edge cloud. When deploying the container running the micro-service, the tenant has to specify the needed computing resources to run their workload in terms of the amount of CPU and memory limit. However, it is not straightforward for a tenant to know in advance the computing amount that allows running the microservice optimally. This will have an impact not only on the service performances but also on the infrastructure provider, particularly if the resource overprovisioning approach is used. To overcome this issue, we conduct an experimental study aiming to detect if a tenant's configuration allows running its service optimally. We run several experiments on a cloud-native platform, using different types of applications under different resource configurations. The obtained results are presented in the accepted IEEE LCN paper (https://www.eurecom.fr/publication/6971) and are shared in this dataset.</p> <p>The datasets are collected for 3 types of applications: Web servers written in python and Golang, RabbitMQ data broker and the OpenAirInterface 5G Core network function AMF (Access and Mobility Management Function).</p> <p><br> </p> <p><strong>Web Servers:</strong></p> <p><strong>files: </strong>golang-web-server-performance.csv, python-web-server-performance.csv</p> <p>We used Golang and Python-based web servers for the test. Each request to the web server returns a video of a size 43 MB. For testing we used ApacheBench, a command-line program used for benchmarking HTTP web servers. ApacheBench allows parallel requests from multiple clients. For each web server instance we send a number of requests ranging from 100 to 1000 and a concurrency level between 1 and 100, representing the number of parallel clients performing the requests.</p> <p>The information available in the dataset are as follows:</p> <p><strong>time:</strong> timestamp of collection of metrics.</p> <p><strong>ram_limit:</strong> the memory allocated to the container in megabytes.</p> <p><strong>cpu_limit:</strong> the CPU allocated to the container.</p> <p><strong>ram_usage:</strong> the amount of memory used by the container at the time of the metrics collection in byte.</p> <p><strong>cpu_usage:</strong> the amount of CPU used by the container at the time of the metrics collection.</p> <p><strong>n:</strong> the number of requests sent to the container.</p> <p><strong>c:</strong> the concurrency level in the requests.</p> <p><strong>lat50:</strong> the least response time for the best 50% requests in microseconds.</p> <p><strong>lat66:</strong> the least response time for the best 66% requests in microseconds.</p> <p><strong>lat75:</strong> the least response time for the best 75% requests in microseconds.</p> <p><strong>lat80:</strong> the least response time for the best 80% requests in microseconds.</p> <p><strong>lat90:</strong> the least response time for the best 90% requests in microseconds.</p> <p><strong>lat95:</strong> the least response time for the best 95% requests in microseconds.</p> <p><strong>lat98:</strong> the least response time for the best 98% requests in microseconds.</p> <p><strong>lat99:</strong> the least response time for the best 99% requests in microseconds.</p> <p><strong>lat100:</strong> the least response time in microseconds.</p> <p> </p> <p><strong>5G Core network’s AMF:</strong></p> <p><strong>file: </strong>amf-performance.csv</p> <p>For testing we use my5G-RANTester, a tool for emulating control and data planes of the UE and gNB (5G base station). The number of simultaneous registration requests that are sent to each instance of the AMF varies between 10 and 400.</p> <p>The information available in the dataset are as follows:</p> <p><strong>time:</strong> timestamp of collection of metrics.</p> <p><strong>ram_limit:</strong> the memory allocated to the container in megabytes.</p> <p><strong>cpu_limit:</strong> the CPU allocated to the container.</p> <p><strong>ram_usage:</strong> the amount of memory used by the container at the time of the metrics collection in byte.</p> <p><strong>cpu_usage:</strong> the amount of CPU used by the container at the time of the metrics collection.</p> <p><strong>n:</strong> the number of parallel registration requests sent to the AMF.</p> <p><strong>mean:</strong> the mean registration time for all the registration requests in microseconds.</p> <p><strong>lat50:</strong> the median registration time for registration requests in microseconds.</p> <p><strong>lat75: </strong>the least registration time for the best 75% registration requests in microseconds.</p> <p><strong>lat80:</strong> the least registration time for the best 80% registration requests in microseconds.</p> <p><strong>lat90:</strong> the least registration time for the best 90% registration requests in microseconds.</p> <p><strong>lat95:</strong> the least registration time for the best 95% registration requests in microseconds.</p> <p><strong>lat98:</strong> the least registration time for the best 98% registration requests in microseconds.</p> <p><strong>lat99:</strong> the least registration time for the best 99% registration requests in microseconds.</p> <p><strong>lat100:</strong> the least registration time in microseconds.</p> <p> </p> <p><strong>RabbitMQ data broker:</strong></p> <p><strong>file: </strong>rabbitmq-performance.csv</p> <p>For testing we used RabbitMQ PerfTest which is a throughput testing tool that simulates basic workloads and provides the throughput and the time that a message takes to be consumed by a consumer. For each deployed RabbitMQ server we used a number of producers and consumers that ranges from 50 to 500. Each producer sends messages to the broker with a rate of 100 messages per second for a period of time of 90 seconds.</p> <p>The information available in the dataset are as follows:</p> <p><strong>time:</strong> timestamp of collection of metrics.</p> <p><strong>ram_limit:</strong> the memory allocated to the container in megabytes.</p> <p><strong>cpu_limit:</strong> the CPU allocated to the container.</p> <p><strong>ram_usage:</strong> the amount of memory used by the container at the time of the metrics collection in byte.</p> <p><strong>cpu_usage:</strong> the amount of CPU used by the container at the time of the metrics collection.</p> <p><strong>n:</strong> the number of producers sending messages to the RabbitMQ server.</p> <p><strong>Min:</strong> the minimum consumption time for the producer messages.</p> <p><strong>lat50:</strong> the median consumption time for the producer messages.</p> <p><strong>lat75:</strong> the least consumption time for the best 75% messages in microseconds.</p> <p><strong>lat95:</strong> the least consumption time for the best 95% messages in microseconds.</p> <p><strong>lat99:</strong> the least consumption time for the best 99% messages in microseconds.</p>
MUHAI Benchmark : Task 1 (Short story generation with Knowledge Graphs)
<p><strong>Meaning and Understanding in Human-Centric AI (MUHAI) Benchmark<br> Task 1 (Short story generation with Knowledge Graphs and Language Models)</strong> </p> <p>The dataset can be used to test understandability of text generated through the combination of knowledge graphs and language models without using knowledge graph embeddings.<br> <br> The task here is to generate 5-sentence stories from a set of <em>subject-predicate-object</em> triples that are extracted from a knowledge graph. Two steps need to be performed:</p> <p>1. Language model fine-tuning (SVO triple extraction + model fine-tuning)<br> 2. Story generation (knowledge enrichment + text generation) <br> <br> The submission includes the following data:</p> <ol> <li>Original ROC stories corpus (100 stories)</li> <li>ROC stories encoded with relevant triples (extracted through SpaCy, 2 versions, with and without coreference resolution)</li> <li>Stories generated by the pre-trained model (GPT2-simple)</li> <li>Stories generated by the fine-tuned model (DICE + ConceptNet + DBpedia )</li> <li>Stories generated by the fine-tuned model (DICE + ConceptNet + DBpedia + WordNet )</li> <li>Stories generated by the GPT-2-keyword-generation (an open-source software that uses GPT-2 to generate text pertaining to the specified keywords)</li> <li>Model results</li> <li>Evaluation metrics description</li> <li>User-evaluation questionnaire </li> </ol> <p>Code : https://github.com/kmitd/muhai-dice_story</p>
A Benchmark dataset on Semantic Change in Scholarly Publications on Disability
<p>This is a benchmark dataset for semantic shift detection in disability-related corpora, including collected title and abstract text from PubMed and ArXiv, annotation sets based on domain experts and LLMs, and extracted KGs (Wikidata entity claims). The corpus from PubMed covers the period from the 1900s to 2023, while the corpus from ArXiv covers the period from the 1990s to 2023. The corpus was filtered based on 16 disability-related target words. In the annotation sets, '1' indicates that a semantic shift occurred for a target word, while '0' indicates the opposite. In particular, the LLM-based annotation sets include their generated text, and we used the Llama2 and GPT-4 models. '7b' refers to the parameter size of the Llama2 model. Graph_data.zip contains Wikidata entity claims.</p>
Vibration-based Monitoring of a Small-scale Wind Turbine Blade Under Varying Climate Conditions. Part I: An Experimental Benchmark
<p>This repository contains all publicly available data related to the experimental part of <a href="https://onlinelibrary.wiley.com/doi/epdf/10.1002/stc.2660">Sonkyo-Benchmark</a>. The data of each experimental case (R, A, B, C, D, E, F, G, H, I, J, K, L) and temperature point (-15, -10, -5, 0, 5, 10, 15, 20, 25, 30, 35, 40) are stored in a zip file named "Case_<em>X</em>_(<em>T</em>)", where <em>X</em> denotes the case label and <em>T</em> refers to the temperature value. Each file "Case_<em>X</em>_(<em>T</em>).zip" contains two folders "Case_<em>X</em>_(<em>T</em>)_1" and "Case_<em>X</em>_(<em>T</em>)_2", wherein the test results from the two sensor layouts are stored. </p>
A blind test on wind turbine wake modelling based on wind tunnel experiments: Phase I – The benchmark case
<p>This data set ("Data files.zip") contains the wind tunnel measurement data from Phase I of the Blind test on wind turbine wake modelling based on wind tunnel experiments organised during the TWEET-IE project (www.tweet-ie.eu).</p> <p>This updated version <strong>replaces</strong> the older versions 1.0.0 (https://doi.org/10.5281/zenodo.10566401), 1.1.0 (https://doi.org/10.5281/zenodo.11370112), 2.0 (https://doi.org/10.5281/zenodo.12188194) and 2.1 (https://doi.org/ 10.5281/zenodo.13918935). In comparison to the previous version 2.1 the data documentation has been updated to follow the template of the TWEET-IE project documents, indicating the Grant Agreement Number with the European Union and the Call Topic of the project.</p> <p>All tests were conducted in the closed-loop, low-speed boundary layer wind tunnel of the Chair of Aerodynamics and Fluid Mechanics at Technische Universität München (TUM). The experiments concerned two wind turbines, aligned with the flow, one downstream of the other, at a distance of 5 diameters. For Phase I, no control was applied to the wind turbine models, which were operating at constant RPM. The turbine models, designed and manufactured by TUM, were instrumented with multiple sensors and actuators and had a diameter of 1.1M. Measurements include velocity, power and loads on the turbines. A detailed description of the experimental set up can be found in the accompanying document ("Data documentation.pdf"). </p> <p>File "Submission procedure.zip" includes the format description and the templates of the output data that should be submitted by the participants in the blind test comparison.</p>
Benchmark Data for AI Safety for High Energy Physics
<p><strong>Datasets for the paper "AI Safety for High Energy Physics" by Ben Nachman and Chase Shimmin (<a href="https://arxiv.org/abs/1910.08606">arXiv:1910.08606</a>)</strong></p> <p>This record contains two files: particles_jj.npz and particles_yz.npz, which contain simulated events of dijet and Z+photon production, respectively, from proton-proton collisions at sqrt(s)=13 TeV.</p> <p>The parton-level events are generated with MadGraph5 aMC@NLO, which are then passed to Pythia 8 for parton showering and hardonization, and then finally to Delphes3 for ATLAS-like detector simulation. Reconstructed calorimeter towers are clustered using the anti-kT algorithm with radius parameter R=1.0. The highest-pT jet from each event is selected, and only events with jet pT > 300 GeV are saved.</p> <p>The Npz files contain three dictionary keys:</p> <ul> <li><strong>jets</strong><strong>:</strong> (N, 4)-shape array containing the pT, eta, phi, and mass of the leading R=1.0 jet for each event</li> <li><strong>constituents:</strong> (N, 128, 3)-shape array containing the pT, eta, phi of up to 128 highest-pT constituent momenta from the leading jet cluster. Jets with fewer than 128 constituents are padded with zero values.</li> <li><strong>photons:</strong> (N, 3)-shape array containing the pT, eta, phi of the leading reconstructed photon (if any) of the event. Events with no photon are filled with zeros.</li> </ul> <p>pT and mass values are stored in units of TeV.</p>
Data of Benchmarking Elasticity of FaaS Platforms
<p>This data is part of the publication "Benchmarking Elasticity of FaaS Platforms as a Foundation for Objective-driven Design of Serverless Applications", it contains all plots and data used for the assessment of FaaS platform quality under volatile workloads from a client-side perspective. The paper is part of SAC'20, Brno, Czech Republic.</p>
Data set supplementing "Benchmarking triage capability of symptom checkers against that of medical laypersons: Survey study"
<p>This is the de-identified data set used to conduct the analyses in the study published as Original Research in the JMIR under the title "Benchmarking triage capability of symptom checkers against that of medical laypersons: Survey study" (https://doi.org/10.2196/24475)</p> <p>The data set contains the assessments of the urgency of symptoms to 45 fictitious clinical case vignettes by 91 US participants, and the participants' age, gender and level of education. Data for the symptom checker apps is needed to fully reproduce our study and can be found in the appendix of the paper "Evaluation of symptom checkers for self diagnosis and triage: audit study" by Semigran et al. (2015) (https://doi.org/10.1136/bmj.h3480).</p>
FISBe: A real-world benchmark dataset for instance segmentation of long-range thin filamentous structures
<h2>General</h2> <p>For more details and the most up-to-date information please consult our project page: <a href="https://kainmueller-lab.github.io/fisbe" target="_blank" rel="noopener">https://kainmueller-lab.github.io/fisbe</a>.</p> <h2>Summary</h2> <ul> <li>A new dataset for neuron instance segmentation in 3d multicolor light microscopy data of fruit fly brains <ul> <li>30 completely labeled (segmented) images</li> <li>71 partly labeled images</li> <li>altogether comprising ∼600 expert-labeled neuron instances (labeling a single neuron takes between 30-60 min on average, yet a difficult one can take up to 4 hours)</li> </ul> </li> <li>To the best of our knowledge, the first real-world benchmark dataset for instance segmentation of long thin filamentous objects</li> <li>A set of metrics and a novel ranking score for respective meaningful method benchmarking</li> <li>An evaluation of three baseline methods in terms of the above metrics and score</li> </ul> <h2>Abstract</h2> <p>Instance segmentation of neurons in volumetric light microscopy images of nervous systems enables groundbreaking research in neuroscience by facilitating joint functional and morphological analyses of neural circuits at cellular resolution. Yet said multi-neuron light microscopy data exhibits extremely challenging properties for the task of instance segmentation: Individual neurons have long-ranging, thin filamentous and widely branching morphologies, multiple neurons are tightly inter-weaved, and partial volume effects, uneven illumination and noise inherent to light microscopy severely impede local disentangling as well as long-range tracing of individual neurons. These properties reflect a current key challenge in machine learning research, namely to effectively capture long-range dependencies in the data. While respective methodological research is buzzing, to date methods are typically benchmarked on synthetic datasets. To address this gap, we release the FlyLight Instance Segmentation Benchmark (FISBe) dataset, the first publicly available multi-neuron light microscopy dataset with pixel-wise annotations. In addition, we define a set of instance segmentation metrics for benchmarking that we designed to be meaningful with regard to downstream analyses. Lastly, we provide three baselines to kick off a competition that we envision to both advance the field of machine learning regarding methodology for capturing long-range data dependencies, and facilitate scientific discovery in basic neuroscience.</p> <h2>Dataset documentation:</h2> <p>We provide a detailed documentation of our dataset, following the <a href="https://arxiv.org/abs/1803.09010" target="_blank" rel="noopener">Datasheet for Datasets</a> questionnaire:</p> <p><em>>> <a href="https://kainmueller-lab.github.io/fisbe/datasheet" target="_blank" rel="noopener">FISBe Datasheet</a></em></p> <p>Our dataset originates from the <a href="https://www.janelia.org/project-team/flylight" target="_blank" rel="noopener">FlyLight project</a>, where the authors released a large image collection of nervous systems of ~74,000 flies, <a href="https://gen1mcfo.janelia.org/cgi-bin/gen1mcfo.cgi" target="_blank" rel="noopener">available for download</a> under CC BY 4.0 license.</p> <h2>Files</h2> <ul> <li>fisbe_v1.0_{completely,partly}.zip <ul> <li>contains the image and ground truth segmentation data; there is one <em>zarr</em> file per sample, see below for more information on how to access <em>zarr</em> files.</li> </ul> </li> <li>fisbe_v1.0_mips.zip <ul> <li>maximum intensity projections of all samples, for convenience.</li> </ul> </li> <li>sample_list_per_split.txt <ul> <li>a simple list of all samples and the subset they are in, for convenience.</li> </ul> </li> <li>view_data.py <ul> <li>a simple python script to visualize samples, see below for more information on how to use it.</li> </ul> </li> <li>dim_neurons_val_and_test_sets.json <ul> <li>a list of instance ids per sample that are considered to be of low intensity/dim; can be used for extended evaluation.</li> </ul> </li> <li>Readme.md <ul> <li>general information</li> </ul> </li> </ul> <h2>How to work with the image files</h2> <p>Each sample consists of a single 3d MCFO image of neurons of the fruit fly.<br>For each image, we provide a pixel-wise instance segmentation for all separable neurons.<br>Each sample is stored as a separate <em>zarr</em> file (<a href="https://zarr.readthedocs.io" target="_blank" rel="noopener">zarr</a> is a file storage format for chunked, compressed, N-dimensional arrays based on an open-source specification.").<br>The image data ("raw") and the segmentation ("gt_instances") are stored as two arrays within a single zarr file.<br>The segmentation mask for each neuron is stored in a separate channel.<br>The order of dimensions is CZYX.</p> <p>We recommend to work in a virtual environment, e.g., by using conda:</p> <p><code>conda create -y -n flylight-env -c conda-forge python=3.9</code><br><code>conda activate flylight-env</code></p> <h3>How to open <em>zarr</em> files</h3> <ol> <li>Install the python zarr package: <pre><code>pip install zarr</code></pre> </li> <li>Opened a zarr file with:<br> <p><code>import zarr</code><br><code>raw = zarr.open(<path_to_zarr>, mode='r', path="volumes/raw")</code><br><code>seg = zarr.open(<path_to_zarr>, mode='r', path="volumes/gt_instances")</code></p> <p><code># optional:</code><br><code>import numpy as np</code><br><code>raw_np = np.array(raw)</code></p> </li> </ol> <p>Zarr arrays are read lazily on-demand.<br>Many functions that expect numpy arrays also work with zarr arrays.<br>Optionally, the arrays can also explicitly be converted to numpy arrays.</p> <h3>How to view <em>zarr</em> image files</h3> <p>We recommend to use <a href="https://napari.org" target="_blank" rel="noopener">napari</a> to view the image data.</p> <ol> <li>Install napari: <pre><code>pip install "napari[all]"</code></pre> </li> <li>Save the following Python script: <br> <p><code>import zarr, sys, napari</code></p> <p><code>raw = zarr.load(sys.argv[1], mode='r', path="volumes/raw")</code><br><code>gts = zarr.load(sys.argv[1], mode='r', path="volumes/gt_instances")</code></p> <p><code>viewer = napari.Viewer(ndisplay=3)</code><br><code>for idx, gt in enumerate(gts):</code><br><code> viewer.add_labels(</code><br><code> gt, rendering='translucent', blending='additive', name=f'gt_{idx}')</code><br><code>viewer.add_image(raw[0], colormap="red", name='raw_r', blending='additive')</code><br><code>viewer.add_image(raw[1], colormap="green", name='raw_g', blending='additive')</code><br><code>viewer.add_image(raw[2], colormap="blue", name='raw_b', blending='additive')</code><br><code>napari.run()</code></p> </li> <li>Execute: <pre><code>python view_data.py <path-to-file>/R9F03-20181030_62_B5.zarr</code></pre> </li> </ol> <h2>Metrics</h2> <ul> <li>S: Average of avF1 and C</li> <li>avF1: Average F1 Score</li> <li>C: Average ground truth coverage</li> <li>clDice_TP: Average true positives clDice</li> <li>FS: Number of false splits</li> <li>FM: Number of false merges</li> <li>tp: Relative number of true positives</li> </ul> <p>For more information on our selected metrics and formal definitions please see <a href="https://arxiv.org/abs/2404.00130" target="_blank" rel="noopener">our paper</a>.</p> <h2>Baseline</h2> <p>To showcase the FISBe dataset together with our selection of metrics, we provide evaluation results for three baseline methods, namely <a href="https://github.com/Kainmueller-Lab/PatchPerPix" target="_blank" rel="noopener">PatchPerPix (ppp)</a>, <a href="https://github.com/google/ffn" target="_blank" rel="noopener">Flood Filling Networks (FFN)</a> and a non-learnt application-specific <a href="https://www.biorxiv.org/content/10.1101/2020.06.07.138941v1" target="_blank" rel="noopener">color clustering from Duan et al.</a>.<br>For detailed information on the methods and the quantitative results please see <a href="https://arxiv.org/abs/2404.00130" target="_blank" rel="noopener">our paper</a>.</p> <h2>License</h2> <p>The FlyLight Instance Segmentation Benchmark (FISBe) dataset is licensed under the <a href="https://creativecommons.org/licenses/by/4.0" target="_blank" rel="noopener">Creative Commons Attribution 4.0 International (CC BY 4.0) license</a>.</p> <h2>Citation</h2> <p>If you use <em>FISBe</em> in your research, please use the following BibTeX entry: </p> <pre><code>@misc{mais2024fisbe, title = {FISBe: A real-world benchmark dataset for instance segmentation of long-range thin filamentous structures}, author = {Lisa Mais and Peter Hirsch and Claire Managan and Ramya Kandarpa and Josef Lorenz Rumberger and Annika Reinke and Lena Maier-Hein and Gudrun Ihrke and Dagmar Kainmueller}, year = 2024, eprint = {2404.00130}, archivePrefix ={arXiv}, primaryClass = {cs.CV} }</code></pre> <h2>Acknowledgments</h2> <p>We thank Aljoscha Nern for providing unpublished MCFO images as well as Geoffrey W. Meissner and the entire FlyLight Project Team for valuable<br>discussions.<br>P.H., L.M. and D.K. were supported by the HHMI Janelia Visiting Scientist Program.<br>This work was co-funded by Helmholtz Imaging.</p> <h2>Changelog</h2> <p>There have been no changes to the dataset so far.<br>All future change will be listed <a href="https://kainmueller-lab.github.io/fisbe/changelog" target="_blank" rel="noopener">on the changelog page</a>.</p> <h2>Contributing</h2> <p>If you would like to contribute, have encountered any issues or have any suggestions, please <a href="https://github.com/Kainmueller-Lab/fisbe/issues" target="_blank" rel="noopener">open an issue</a> for the FISBe dataset in the accompanying github repository.</p> <p>All contributions are welcome!</p>
Lehigh University Benchmark (LUBM): Evolving Graph Simulation
<p>The Lehigh University Benchmark (LUBM) generates benchmark datasets containing people working at universities [1]. We use the Data Generator v1.7 to generate 10 versions of a graph containing 100 universities [2].<br> Thus, all versions are of similar size, but we emulate modifications by generating different vertex identifiers, i.e., each version is considered a timestamped graph. Each graph contains about 2.1 M vertices and 13 M edges.<br> Over all versions, the mean degree is 6.7 (+- 0.1), the mean in-degree is 6.8 (+- 0.1), and the mean out-degree is 5.1 (+- 0.1).</p> <p>1. <a href="https://dblp.uni-trier.de/pid/80/5390.html">Yuanbo Guo</a>, <a href="https://dblp.uni-trier.de/pid/48/6834.html">Zhengxiang Pan</a>, <a href="https://dblp.uni-trier.de/pid/94/1154.html">Jeff Heflin</a>: LUBM: A benchmark for OWL knowledge base systems. <a href="https://dblp.uni-trier.de/db/journals/ws/ws3.html#GuoPH05">J. Web Semant. 3(2-3)</a>: 158-182 (2005)</p> <p>2. <a href="https://dblp.uni-trier.de/pid/222/6353.html">Till Blume</a>, <a href="https://dblp.uni-trier.de/pid/r/DavidRicherby.html">David Richerby</a>, <a href="https://dblp.uni-trier.de/pid/06/2380.html">Ansgar Scherp</a>: Incremental and Parallel Computation of Structural Graph Summaries for Evolving Graphs. <a href="https://dblp.uni-trier.de/db/conf/cikm/cikm2020.html#BlumeRS20">CIKM 2020</a>: 75-84</p>
Berlin Sparql Benchmark (BSBM): Evolving Graph Simulation
<p>The Berlin SPARQL Benchmark (BSBM) is a suite of benchmarks built around an e-commerce use case [1]. We generated 21 versions of the dataset with different scale factors. The first dataset, with a scale factor of 100, contains about 7,000 vertices and 75,000 edges. We generated versions with scale factors between 2,000 and 40,000 in steps of 2,000. The largest dataset contains about 1.3 M vertices and 13 M edges. For our experiments in [2], we first use the different versions ordered from smallest to largest (version 0 to 20) to simulate a growing graph database. Subsequently, we reverse the order to emulate a shrinking graph database. Over all versions, the mean degree is 8.1 (+- 0.5), the mean in-degree is 4.6 (+- 0.3), and the mean out-degree is 9.8 (+- 0.2).</p> <p>1. <a href="https://dblp.uni-trier.de/pid/b/ChristianBizer.html">Christian Bizer</a>, <a href="https://dblp.uni-trier.de/pid/47/7466.html">Andreas Schultz</a>: The Berlin SPARQL Benchmark. <a href="https://dblp.uni-trier.de/db/journals/ijswis/ijswis5.html#BizerS09">Int. J. Semantic Web Inf. Syst. 5(2)</a>: 1-24 (2009)</p> <p>2. <a href="https://dblp.uni-trier.de/pid/222/6353.html">Till Blume</a>, <a href="https://dblp.uni-trier.de/pid/r/DavidRicherby.html">David Richerby</a>, <a href="https://dblp.uni-trier.de/pid/06/2380.html">Ansgar Scherp</a>: Incremental and Parallel Computation of Structural Graph Summaries for Evolving Graphs. <a href="https://dblp.uni-trier.de/db/conf/cikm/cikm2020.html#BlumeRS20">CIKM 2020</a>: 75-84</p>
A living catalogue of artificial intelligence datasets and benchmarks for medical decision making
<p>We provide a comprehensive curated catalogue of <strong>artificial intelligence datasets</strong> and <strong>benchmarks for medical decision making</strong>. At the time of first release (April 2021), the dataset contains more than 400 biomedical and clinical datasets of which 252 are publicly available or available upon request.</p> <p>The dataset was compiled based on a systematic literature review covering both biomedical and computer science literature and grey literature data sources. All datasets were manually systematized and annotated for meta-information, such as:</p> <ul> <li>Availability and licensing information</li> <li>Type of source data</li> <li>Links to source publications, main references or dataset repositories</li> </ul> <p>Benchmark dataset were additionally annotated for the following information:</p> <ul> <li>Associated task</li> <li>Performance metrics commonly used for evaluation</li> <li>Clinical relevance</li> <li>The availability of data splits</li> </ul> <p>In addition to the versioned TSV file on Zenodo, the dataset can also be explored live via <a href="https://docs.google.com/spreadsheets/d/1QjUxxnZ3tuyW5dj6nkt_o5yJcWUZec4ttfJxO8Zlty4/edit?usp=sharing">this Google Spreadsheet</a>. The dataset is intended as a living, extendable resource. Edit suggestions and additions are encouraged and can be submitted via the comment function of the Google sheet.</p> <p> </p> <p><strong>File descriptions</strong></p> <p><em>annotated-datasets.tsv</em> -- contains the annotated datasets</p> <p><em>arXiv-literature-export.tsv</em> -- contains the original literature record export from arXiv</p> <p><em>pubmed-literature-export.tsv</em> -- contains the original literature record export from PubMed</p> <p><em>README.md</em> -- contains a detailed description of all annotation fields</p>
Benchmark dataset for preprint: "EDEN: A high-performance, general-purpose, NeuroML-based neural simulator"
<p>The benchmark files and scripts to reproduce the figures of the preprint "EDEN: A high-performance, general-purpose, NeuroML-based neural simulator" ( https://arxiv.org/abs/2106.06752 )</p> <p>The benchmarks require a computer running Linux with Docker installed.</p> <p>Unpack the paper_experiments.zip file and follow the instructions in the README.md file to run the benchmarks and reproduce the figures.</p> <p> </p>
UPWARDS - Høgjaeren noise prediction benchmark
<p>The benchmark consists in a reduced layout of nine wind turbines located in the Høgjaeren wind farm in Norway for two wind conditions of similar wind speed amplitude but having opposite wind directions. The necessary information for the user to produce the noise footprint on an observer grid are detailed in the Upwards_D4_6_v1.pdf document.</p>
MUHAI Benchmark : Task 2 (Credibility of knowledge-based generated gossip stories)
<p><strong>Meaning and Understanding in Human-Centric AI (MUHAI) Benchmark<br> Task 2 (Credibility of knowledge-based generated gossip stories)</strong></p> <p> </p> <p>This dataset aims at investigating whether the use of Knowledge Graphs has an impact on the credibililty of automatically-generated stories.</p> <p><br> The submission includes the following data:</p> <ol> <li>Generated stories (.txt)</li> <li>Story generation template </li> <li>A tsv file with entities and triples (to be used for generating stories)</li> <li>Evaluation description : Questions and metrics submitted to the users</li> </ol> <p>The "gossip stories" are generated with the T5 languge model fine-tuned on the WebNLG challenge. The model takes the triples (file 3) as input and generates one sentence each. A link prediction algorithm based on Jaccard's similarity learns the likelihood of two entities to be related (3). Then, the narrative continues with automatically generated celebrity background descriptions.</p> <p>The credibility of the story is evaluated using a questionnaire based on Gaziano et. al. The questionnaire was filled in by the test subjects after reading each generated article. One for a KG-generated text where links were predicted using the link prediction and one for text that was generated using triples of random entities (celebrities). </p> <p>Full code available at : https://github.com/kmitd/muhai-credibility-KR</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.