Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
29
datasets available to search
ShareScore release 0.9.0
Dataset results
29 results for “Robustness evaluation”
Expansion Cones Dataset for Robust Tetrahedral Mapping Evaluation
<p>This record stores the dataset used by the <a href="https://www.algohex.eu/publications/expansion-cones/">"Expansion Cones"</a> paper.<br>This paper presents a volumetric mapping method called <em>Shrink-And-Expand</em> (SAE for short).<br>Its robustness was evaluated by mapping ~3'000 ball-topology meshes to 4 different target boundaries (spherical, tetrahedral, "stiff" tetrahedral and random star-shaped). Other tetrahedral mapping methods can be evaluated by using the rest meshes and either of the target boundaries.</p> <p>The record is structured as follows:</p> <ul> <li>`ball_topology_tet_wild_ovm_format`: all input meshes in their original shapes, in <a href="https://www.graphics.rwth-aachen.de/software/openvolumemesh/">.ovm</a> format. These consists of the <a href="https://github.com/Yixin-Hu/TetWild">TetWild</a> dataset, restricted to ball-topology meshes. If you need to convert the meshes to another format, you can use Martin Heistermann's <a href="https://github.com/mheistermann/meshio">fork</a> of <a href="https://pypi.org/project/meshio/">meshio</a>.<br>NOTE: Those meshes were pre-processed by splitting all <strong>interior</strong> edges connecting two boundary vertices and all <strong>interior</strong> faces connecting three boundary vertices. This was done to satisfy the prerequisites of our method.</li> <li>`XXX_boundary_conditions.zip`: boundary conditions for the 4 boundary shapes: tetrahedral, "stiff" tetrahedral, ball and random star-shape. Please read the related paper for more details about the boundary types.<br>Those boundary conditions are stored as `.txt` files, each containing a list of boundary vertices indices, along with their prescribed positions.</li> </ul> <p>The other archives contain data used to compare SAE with two other volumetric mapping methods called <a href="https://github.com/duxingyi-charles/lifting_simplices_to_find_injectivity"><em>Total Lifted Conten</em>t (TLC)</a> and <a href="https://dl.acm.org/doi/abs/10.1145/3450626.3459847">FoldOver-Free (FOF)</a></p> <ul> <li> `SAE_results/expansion_data`: All `.json` files corresponding to executions of our method (SAE) on those meshes.</li> <li> `SAE_results/quality_stats`: conformal and volumetric distortion values for _all_ cells of _all_ meshes.</li> <li>`TLC_FOF_results`: For each mesh and each competitor method (and each boundary map), there are 1 to 3 files, depending on the results of running those methods. <ul> <li>`TLC/FOF_timing_s.txt` gives the number of seconds it took to run</li> <li>`TLC/FOF_stats.txt` stores the following data: `[mesh name, #vertices, #edges, #faces, #cells, #degenerate cells after running the method, #flipped cells after running the method]`. This file only exists if the method terminated within the 12hours time limit.</li> <li>`TLC/FOF_quality_stats.txt` lists the conformal and volumetric distortion of each individual cell in the resulting map produced by the competitor methods (one line per cell), This file only exists if the method produced a valid map.</li> </ul> </li> </ul> <p> </p> <p><strong>NOTE</strong>: If you want to download the codomain meshes, with prescribed boundary positions already set. you can find them in the v2.0 of this same dataset.</p>
Results and figures from "Evaluating the robustness of the ARIO model for a local disaster: 2021 Flooding in Germany"
<p>These files and notebook allow reproducing the figures from "Evaluating the robustness of the ARIO model for a local disaster: 2021 Flooding in Germany".</p> <p>The main result file is a pandas DataFrame saved in parquet format under "results/general-plot_df.parquet".</p> <p>Two notebooks allow post-processing these results and plot the figures.</p> <p>Reproduction of the raw results can be achieved with the Snakemake pipeline, available here: https://github.com/spjuhel/BoARIO-Sensitivity</p> <p> </p>
Understanding the robustness of spectral-temporal metrics across the global Landsat archive from 1984-2019 – a quantitative evaluation: extended material
<p>This dataset contains extended material for the paper:</p> <p>Frantz, D., Rufin, P., Janz, A., Ernst, S., Pflugmacher, D., Schug, F., Hostert, P.<strong>: Understanding the robustness of spectral-temporal metrics across the global Landsat archive from 1984-2019 – a quantitative evaluation. </strong><em>In revision.</em></p>
Data from: Comparing alternative harvest strategies to address robustness to recruitment variability and uncertainty: Implications for Alaska Sablefish tested with management strategy evaluation
Open the record for dataset details and reuse information.
Human annotation data for ACL 2020 paper "Designing Precise and Robust Dialogue Evaluators"
<p>Human annotation data for ACL 2020 paper "Designing Precise and Robust Dialogue Evaluators"</p>
Evaluating Robustness to Context-Sensitive Feature Perturbations of Different Granularities: Further Appendices
<p>Further appendices for <a href="https://arxiv.org/abs/2001.11055">Evaluating Robustness to Context-Sensitive Feature Perturbations of Different Granularities</a>.</p> <p>Includes:</p> <ol> <li>imagenet_examples_appendix.pdf, which contains many examples of context-sensitive ImageNet feature perturbations of different granularities.</li> <li>animations_appendix.zip, which contains animations showing the effect as the feature perturbations are gradually introduced to different images.</li> </ol> <p>Each appendix includes a much more detailed description of its contents.</p>
GERBIL evaluation data of Robust and Collective Entity Disambiguation through Semantic Embeddings
<p>A table containing the SIGIR 2016 experiments performed with GERBIL in context of the SIGIR 2016 work "Robust and Collective Entity Disambiguation through Semantic Embeddings" by Stefan Zwicklbauer, Christin Seifert and Michael Granitzer</p> <p>It also contains the original URL to the GERBIL website</p> <p>Corresponding GitHub Repository:</p> <p>https://github.com/quhfus/</p> <p> </p>
Evaluation of tracking performance and robustness for a hybrid locomotion controller
<p>Legged locomotion is a complex control problem that requires both accuracy and robustness to cope with real-world challenges. Legged systems have traditionally been controlled using trajectory optimization with inverse dynamics. Such hierarchical model-based methods are appealing due to intuitive cost function tuning, accurate planning, generalization, and most importantly, the insightful understanding gained from more than one decade of extensive research. However, model mismatch and violation of assumptions are common sources of faulty operation. Simulation-based reinforcement learning, on the other hand, results in locomotion policies with unprecedented robustness and recovery skills.<br>Yet, all learning algorithms struggle with sparse rewards emerging from environments where valid footholds are rare, such as gaps or stepping stones. In this work, we propose a hybrid control architecture that combines the advantages of both worlds to simultaneously achieve greater robustness, foot-placement accuracy, and terrain generalization. Our approach utilizes a model-based planner to roll out a reference motion during training. A deep neural network policy is trained in simulation, aiming to track the optimized footholds. We evaluate the accuracy of our locomotion pipeline on sparse terrains, where pure data-driven methods are prone to fail. Furthermore, we demonstrate superior robustness in the presence of slippery or deformable ground when compared to model-based counterparts. Finally, we show that our proposed tracking controller generalizes across different trajectory optimization methods not seen during training. In conclusion, our work unites the predictive capabilities and optimality guarantees of online planning with the inherent robustness attributed to offline learning.</p>
Data from: Clustering Deviation Index (CDI): A robust and accurate internal measure for evaluating scRNA-seq data clustering
<div> <div> <p>The clustering of cells has been widely used to explore the heterogeneity of cell populations in single-cell RNA-sequencing (scRNA-seq). We proposed a parametric model for monoclonal and polyclonal scRNA-seq data to evaluate clustering results. Based on the parametric model, we proposed a metric (CDI) to quantify the goodness-of-fit of cell clustering to the data. Here we presented CT26.WT and T-CELL as two datasets to examine the performance of our model and metric. CT26.WT contains wild-type CT26 cells from the murine colorectal carcinoma cell line, and cells in CT26.WT are highly homogeneous. T-CELL contains T-cells from tumor tissue of mice three weeks after 4T1 tumor injection. From these datasets and public datasets, we validated our model and benchmarked our metric.</p> </div> </div>
Freebase Datasets for Robust Evaluation of Knowledge Graph Link Prediction Models
<p><strong>Freebase</strong> is amongst the largest public cross-domain knowledge graphs. It possesses three main data modeling idiosyncrasies. It has a strong <strong>type system</strong>; its properties are purposefully represented in <strong>reverse pairs</strong>; and it uses <strong>mediator objects</strong> to represent multiary relationships. These design choices are important in modeling the real-world. But they also pose nontrivial challenges in research of embedding models for knowledge graph completion, especially when models are developed and evaluated agnostically of these idiosyncrasies. We make available several variants of the Freebase dataset by inclusion and exclusion of these data modeling idiosyncrasies. This is the first-ever publicly available <strong>full-scale</strong> Freebase dataset that has gone through <strong>proper preparation</strong>. </p><p> </p><p>Dataset Details</p><p>The dataset consists of the four variants of Freebase dataset as well as related mapping/support files. For each variant, we made three kinds of files available:</p><ul><li>Subject matter triples file<ul><li><i>fb+/-CVT+/-REV</i> One folder for each variant. In each folder there are 5 files: train.txt, valid.txt, test.txt, entity2id.txt, relation2id.txt Subject matter triples are the triples belong to subject matters domains—domains describing real-world facts.<ul><li>Example of a row in train.txt, valid.txt, and test.txt: <ul><li>2, 192, 0</li></ul></li><li>Example of a row in entity2id.txt:<ul><li>/g/112yfy2xr, 2</li></ul></li><li>Example of a row in relation2id.txt:<ul><li>/music/album/release_type, 192</li></ul></li><li>Explaination<ul><li>"/g/112yfy2xr" and "/m/02lx2r" are the MID of the subject entity and object entity, respectively. "/music/album/release_type" is the realtionship between the two entities. 2, 192, and 0 are the IDs assigned by the authors to the objects.</li></ul></li></ul></li></ul></li><li>Type system file<ul><li><i>freebase_endtypes</i>: Each row maps an edge type to its required subject type and object type.<ul><li>Example<ul><li>92, 47178872, 90</li></ul></li><li>Explanation<ul><li>"92" and "90" are the type id of the subject and object which has the relationship id "47178872".</li></ul></li></ul></li></ul></li><li>Metadata files<ul><li><i>object_types</i>: Each row maps the MID of a Freebase object to a type it belongs to.<ul><li>Example<ul><li>/g/11b41c22g, /type/object/type, /people/person</li></ul></li><li>Explanation<ul><li>The entity with MID "/g/11b41c22g" has a type "/people/person"</li></ul></li></ul></li><li><i>object_names</i>: Each row maps the MID of a Freebase object to its textual label.<ul><li>Example<ul><li>/g/11b78qtr5m, /type/object/name, "Viroliano Tries Jazz"@en</li></ul></li><li>Explanation<ul><li>The entity with MID "/g/11b78qtr5m" has name "Viroliano Tries Jazz" in English.</li></ul></li></ul></li><li><i>object_ids</i>: Each row maps the MID of a Freebase object to its user-friendly identifier.<ul><li>Example<ul><li>/m/05v3y9r, /type/object/id, "/music/live_album/concert"</li></ul></li><li>Explanation<ul><li>The entity with MID "/m/05v3y9r" can be interpreted by human as a music concert live album.</li></ul></li></ul></li><li><i>domains_id_label</i>: Each row maps the MID of a Freebase domain to its label.<ul><li>Example<ul><li>/m/05v4pmy, geology, 77</li></ul></li><li>Explanation<ul><li>The object with MID "/m/05v4pmy" in Freebase is the domain "geology", and has id "77" in our dataset.</li></ul></li></ul></li><li><i>types_id_label</i>: Each row maps the MID of a Freebase type to its label.<ul><li>Example<ul><li>/m/01xljxh, /government/political_party, 147</li></ul></li><li>Explanation<ul><li>The object with MID "/m/01xljxh" in Freebase is the type "/government/political_party", and has id "147" in our dataset.</li></ul></li></ul></li><li><i>entities_id_label</i>: Each row maps the MID of a Freebase entity to its label.<ul><li>Example<ul><li>/g/11b78qtr5m, Viroliano Tries Jazz, 2234</li></ul></li><li>Explanation<ul><li>The entity with MID "/g/11b78qtr5m" in Freebase is "Viroliano Tries Jazz", and has id "2234" in our dataset.</li></ul></li><li><i>properties_id_label</i>: Each row maps the MID of a Freebase property to its label.<ul><li>Example<ul><li>/m/010h8tp2, /comedy/comedy_group/members, 47178867</li></ul></li><li>Explanation<ul><li>The object with MID "/m/010h8tp2" in Freebase is a property(relation/edge), it has label "/comedy/comedy_group/members" and has id "47178867" in our dataset.</li></ul></li></ul></li><li><i>uri_original2simplified</i> and <i>uri_simplified2original</i>: The mapping between original URI and simplified URI and the mapping between simplified URI and original URI repectively.<ul><li>Example<ul><li><i>uri_original2simplified</i><ul><li>"<a href="http://rdf.freebase.com/ns/type.property.unique">http://rdf.freebase.com/ns/type.property.unique</a>": "/type/property/unique"</li></ul></li><li><i>uri_simplified2original</i><ul><li>"/type/property/unique": "<a href="http://rdf.freebase.com/ns/type.property.unique">http://rdf.freebase.com/ns/type.property.unique</a>"</li></ul></li></ul></li><li>Explanation<ul><li>The URI "<a href="http://rdf.freebase.com/ns/type.property.unique">http://rdf.freebase.com/ns/type.property.unique</a>" in the original Freebase RDF dataset is simplified into "/type/property/unique" in our dataset.</li><li>The identifier "/type/property/unique" in our dataset has URI <a href="http://rdf.freebase.com/ns/type.property.unique">http://rdf.freebase.com/ns/type.property.unique</a> in the original Freebase RDF dataset.</li></ul></li></ul></li></ul></li></ul></li></ul>
Data from: Clustering Deviation Index (CDI): A robust and accurate internal measure for evaluating scRNA-seq data clustering
Open the record for dataset details and reuse information.
Evaluation of tracking performance and robustness for a hybrid locomotion controller
Open the record for dataset details and reuse information.
SIDIRE: Synthetic Image Dataset for Illumination Robustness Evaluation
<p>SIDIRE is a freely available image dataset which provides synthetically generated images allowing to investigate the influence of illumination changes on object appearance. The images are renderings of 3D coin models with different material BRDFs and levels of texturedness. Thus, the dataset makes it possible to directly evaluate the influence of these conditions on the performance of image recognition without introducing a bias due to different objects used between image sets. The dataset has been used for evaluation in [1].</p> <p><strong>Usage</strong></p> <p>The dataset is freely available for non-commercial research use. Please cite our paper [1] when using the dataset for your research.</p> <p><strong>Technical Details</strong></p> <p>Full Image Dataset</p> <p>The full image dataset consists of images of 14 coin models which have been rendered using the open-source graphics software <a href="http://www.blender.org">Blender</a>. For each model, twelve sets of 500×500 images with 65 illumination directions were rendered where each set represents one out of four material BRDFs and one out of three texture density levels. Material BRDFs are intended to represent different levels of specularity starting from a Lambertian material with zero specularity up to specular intensity values of 0.25, 0.50 and 1.00. The first texture density level shows no texture and thus represents the set of textureless objects. For the remaining two levels synthetically generated textures were used. The camera image plane is placed parallel to the coin and light source positions are defined by their azimuth angle φ and elevation angle λ. We used eight levels of λ with eight levels of φ each to produce 64 images. The 65th image is rendered with the light placed at the camera position (i.e. λ=90°).<br> In the provided RAR-file, all the 65 images of a specific model, specularity level and texturedness level are contained in separate directories. For instance, the directory ‘texture_level0\Ref_level2\2874-back’ contains the images of the model ‘2874-back’ rendered without texture and a specularity of 0.50.</p> <p><strong>Patch Dataset</strong></p> <p>The patch dataset contains 50000 matching patch pairs for every of the 12 subsets of SIDIRE. It can be used to generate groups of feature distances by means of true and false patch pairs, in the same manner as, e.g., Matthew Brown’s <a href="http://phototour.cs.washington.edu/patches/default.htm">patch dataset</a>. Please see [1,2] for a detailed description of the evaluation scheme of patch pair databases.<br> The patches have a size of 64×64 and are arranged in images of size 3200×3200. Thus, every image contains 2500 patches where corresponding patches are placed side by side. The patches of the 12 subsets are contained in directories indicating their texture density and reflectance level, e.g. patches rendered without texture and a specularity of 0.50 are contained in the directory ‘tex0_ref2’.<br> </p> <p><strong>References</strong></p> <p>[1] Zambanini S., Kampel M. “Evaluation of Low-Level Image Representations for Illumination-Insensitive Recognition of Textureless Objects”, <em>International Conference on Image Analysis and Processing – ICIAP’13</em>, Naples, Italy, September 2013. (<a href="https://cvl.tuwien.ac.at/wp-content/uploads/2014/12/iciap13.pdf">pdf</a>, <a href="https://cvl.tuwien.ac.at/wp-content/uploads/2014/12/iciap13_supp1.pdf">supplementary material</a>)<br> [2] Brown, M., Gang Hua, Winder, S., “Discriminative Learning of Local Image Descriptors”, <em>Pattern Analysis and Machine Intelligence, </em> vol.33, no.1, pp.43-57, 2011.</p>
Data from: Experimental evaluation of the robustness of the growth-stress tolerance trade-off within the perennial grass Dactylis glomerata
1. A core tenet of functional ecology is that the vast phenotypic diversity observed in the plant kingdom could be partly generated by a trade- off between the ability of plants to grow quickly and acquire resources in rich environments vs. the ability to conserve resources and avoid mortality under stress. However, experimental demonstrations remain scarce and potentially blurred by phylogenetic constraints in cross-species analyses. Here we experimentally decoupled growth potential and stress survival by applying an off-season stress on contrasting populations of the perennial grass Dactylis glomerata exhibiting a range of seasonal dormancy. 2. Seventeen populations of D. glomerata, originating from a latitudinal gradient from Norway to Morocco, were subjected to three types of dehydration stress: winter frost in Norway, summer drought, and early spring (off-season) drought stress in the south of France. Growth rate, and two leaf traits (leaf width and leaf dry matter content) suspected to be involved in the adaptation to dehydration stress, were monitored under optimal conditions. We quantified plant dehydration survival as the amount of plant recovery after a severe stress. 3. Nordic populations were found to be winter dormant. Winter and summer dormant populations better survived frost and summer drought, respectively. However, no trade-off between growth potential and dehydration survival was detected in non-dormant plants in early spring when dehydration occurred unseasonably for all populations. Furthermore, Mediterranean populations better survived an early spring drought. 4. Our results highlight the importance of assessing plant growth potential as a response to seasonal environmental cues. They suggest that growth potential and stress survival trade-off when plants exhibit seasonal dormancy but can be functionally independent at other seasons. Consequently, the growth-stress survival relationship could be better described as a dynamic linkage rather than a constant and general trade-off. Moreover, leaf trait values, such as thinner and more lignified leaves reflecting drought adaptation, may have contributed to the improved drought-stress survival without resulting in a cost to growth. 5. Further exploration of the growth-stress survival relationship should permit deciphering the suite of plant traits and trait covariations involved in plants' responses to increasing stress.
Data from: Evaluating population receptive field estimation frameworks in terms of robustness and reproducibility
Within vision research retinotopic mapping and the more general receptive field estimation approach constitute not only an active field of research in itself but also underlie a plethora of interesting applications. This necessitates not only good estimation of population receptive fields (pRFs) but also that these receptive fields are consistent across time rather than dynamically changing. It is therefore of interest to maximize the accuracy with which population receptive fields can be estimated in a functional magnetic resonance imaging (fMRI) setting. This, in turn, requires an adequate estimation framework providing the data for population receptive field mapping. More specifically, adequate decisions with regard to stimulus choice and mode of presentation need to be made. Additionally, it needs to be evaluated whether the stimulation protocol should entail mean luminance periods and whether it is advantageous to average the blood oxygenation level dependent (BOLD) signal across stimulus cycles or not. By systematically studying the effects of these decisions on pRF estimates in an empirical as well as simulation setting we come to the conclusion that a bar stimulus presented at random positions and interspersed with mean luminance periods is generally most favorable. Finally, using this optimal estimation framework we furthermore tested the assumption of temporal consistency of population receptive fields. We show that the estimation of pRFs from two temporally separated sessions leads to highly similar pRF parameters.
Evaluating the Robustness of Deep-learning Algorithm-selection Models by Evolving Adversarial Instances - Code and Data
<p>This repository contains the code and data for reproducibility of the paper 'Evaluating the Robustness of Deep-learning Algorithm-selection Models by Evolving Adversarial Instances'. </p> <p>The following files are included:</p> <ul> <li>Data.zip : contains the original instances in the datasets;</li> <li>Models.zip : trained Deep Neural Networks models used in the paper;</li> <li>New_instances.zip : generated instances using the approach;</li> <li>Parsed_data.zip : results and statistics of the experiments;</li> <li>script_adversarial_v3.py : Python script used to generate the results</li> </ul>
Evaluation benchmark for natural robustness evaluation of retinal vessel segmentation models
<p>A dataset contains benchmark images for natural robustness evaluation of deep learning models for retinal vessel segmentation. The dataset consists of three mainstream retinal vessel segmentation datasets: DRIVE, STARE, and CHASE_DB1.</p> <p>For each dataset are provided:</p> <ul> <li><em>images </em>- directory containing fundus images augmented using <a href="https://github.com/goranagojic/AugOOD">AugOOD</a> tool for fast image augmentation for OOD robustness evaluation.</li> <li><em>labels</em> - directory with labels that correspond to the images.</li> <li><em>masks</em> - directory with FoV masks that correspond to the images.</li> </ul> <p>The benchmark is used in the paper <a href="https://onlinelibrary.wiley.com/doi/abs/10.1002/cpe.6809">Robustness of deep learning methods for ocular fundus segmentation: Evaluation of blur sensitivity</a> to evaluate natural robustness of a portfolio of deep learning models for retinal vessel segmentation from fundus images.</p>
Evaluation of the Accuracy, Safety and Robustness of a Single-input-single-output (SISO) Model-based Predictive Closed-loop System to Guide Patient-individualized ICU Sedation
ClinicalTrials.gov study NCT00735631. IPD Sharing: Not stated. Countries: 1. Publications: 1.
Robust Evaluation to Measure Improvements in Nonadherence From Low-cost Devices
ClinicalTrials.gov study NCT02015806. IPD Sharing: Not stated. Countries: 1. Publications: 2.
Benchmarking speech-to-text robustness in noisy emergency medical dialogues: An evaluation of stt-models under realistic acoustic conditions
Open the record for dataset details and reuse information.
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.