Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
1,167
datasets available to search
ShareScore release 0.9.0
Dataset results
1,167 results for “Real-World”
FISBe: A real-world benchmark dataset for instance segmentation of long-range thin filamentous structures
<h2>General</h2> <p>For more details and the most up-to-date information please consult our project page: <a href="https://kainmueller-lab.github.io/fisbe" target="_blank" rel="noopener">https://kainmueller-lab.github.io/fisbe</a>.</p> <h2>Summary</h2> <ul> <li>A new dataset for neuron instance segmentation in 3d multicolor light microscopy data of fruit fly brains <ul> <li>30 completely labeled (segmented) images</li> <li>71 partly labeled images</li> <li>altogether comprising ∼600 expert-labeled neuron instances (labeling a single neuron takes between 30-60 min on average, yet a difficult one can take up to 4 hours)</li> </ul> </li> <li>To the best of our knowledge, the first real-world benchmark dataset for instance segmentation of long thin filamentous objects</li> <li>A set of metrics and a novel ranking score for respective meaningful method benchmarking</li> <li>An evaluation of three baseline methods in terms of the above metrics and score</li> </ul> <h2>Abstract</h2> <p>Instance segmentation of neurons in volumetric light microscopy images of nervous systems enables groundbreaking research in neuroscience by facilitating joint functional and morphological analyses of neural circuits at cellular resolution. Yet said multi-neuron light microscopy data exhibits extremely challenging properties for the task of instance segmentation: Individual neurons have long-ranging, thin filamentous and widely branching morphologies, multiple neurons are tightly inter-weaved, and partial volume effects, uneven illumination and noise inherent to light microscopy severely impede local disentangling as well as long-range tracing of individual neurons. These properties reflect a current key challenge in machine learning research, namely to effectively capture long-range dependencies in the data. While respective methodological research is buzzing, to date methods are typically benchmarked on synthetic datasets. To address this gap, we release the FlyLight Instance Segmentation Benchmark (FISBe) dataset, the first publicly available multi-neuron light microscopy dataset with pixel-wise annotations. In addition, we define a set of instance segmentation metrics for benchmarking that we designed to be meaningful with regard to downstream analyses. Lastly, we provide three baselines to kick off a competition that we envision to both advance the field of machine learning regarding methodology for capturing long-range data dependencies, and facilitate scientific discovery in basic neuroscience.</p> <h2>Dataset documentation:</h2> <p>We provide a detailed documentation of our dataset, following the <a href="https://arxiv.org/abs/1803.09010" target="_blank" rel="noopener">Datasheet for Datasets</a> questionnaire:</p> <p><em>>> <a href="https://kainmueller-lab.github.io/fisbe/datasheet" target="_blank" rel="noopener">FISBe Datasheet</a></em></p> <p>Our dataset originates from the <a href="https://www.janelia.org/project-team/flylight" target="_blank" rel="noopener">FlyLight project</a>, where the authors released a large image collection of nervous systems of ~74,000 flies, <a href="https://gen1mcfo.janelia.org/cgi-bin/gen1mcfo.cgi" target="_blank" rel="noopener">available for download</a> under CC BY 4.0 license.</p> <h2>Files</h2> <ul> <li>fisbe_v1.0_{completely,partly}.zip <ul> <li>contains the image and ground truth segmentation data; there is one <em>zarr</em> file per sample, see below for more information on how to access <em>zarr</em> files.</li> </ul> </li> <li>fisbe_v1.0_mips.zip <ul> <li>maximum intensity projections of all samples, for convenience.</li> </ul> </li> <li>sample_list_per_split.txt <ul> <li>a simple list of all samples and the subset they are in, for convenience.</li> </ul> </li> <li>view_data.py <ul> <li>a simple python script to visualize samples, see below for more information on how to use it.</li> </ul> </li> <li>dim_neurons_val_and_test_sets.json <ul> <li>a list of instance ids per sample that are considered to be of low intensity/dim; can be used for extended evaluation.</li> </ul> </li> <li>Readme.md <ul> <li>general information</li> </ul> </li> </ul> <h2>How to work with the image files</h2> <p>Each sample consists of a single 3d MCFO image of neurons of the fruit fly.<br>For each image, we provide a pixel-wise instance segmentation for all separable neurons.<br>Each sample is stored as a separate <em>zarr</em> file (<a href="https://zarr.readthedocs.io" target="_blank" rel="noopener">zarr</a> is a file storage format for chunked, compressed, N-dimensional arrays based on an open-source specification.").<br>The image data ("raw") and the segmentation ("gt_instances") are stored as two arrays within a single zarr file.<br>The segmentation mask for each neuron is stored in a separate channel.<br>The order of dimensions is CZYX.</p> <p>We recommend to work in a virtual environment, e.g., by using conda:</p> <p><code>conda create -y -n flylight-env -c conda-forge python=3.9</code><br><code>conda activate flylight-env</code></p> <h3>How to open <em>zarr</em> files</h3> <ol> <li>Install the python zarr package: <pre><code>pip install zarr</code></pre> </li> <li>Opened a zarr file with:<br> <p><code>import zarr</code><br><code>raw = zarr.open(<path_to_zarr>, mode='r', path="volumes/raw")</code><br><code>seg = zarr.open(<path_to_zarr>, mode='r', path="volumes/gt_instances")</code></p> <p><code># optional:</code><br><code>import numpy as np</code><br><code>raw_np = np.array(raw)</code></p> </li> </ol> <p>Zarr arrays are read lazily on-demand.<br>Many functions that expect numpy arrays also work with zarr arrays.<br>Optionally, the arrays can also explicitly be converted to numpy arrays.</p> <h3>How to view <em>zarr</em> image files</h3> <p>We recommend to use <a href="https://napari.org" target="_blank" rel="noopener">napari</a> to view the image data.</p> <ol> <li>Install napari: <pre><code>pip install "napari[all]"</code></pre> </li> <li>Save the following Python script: <br> <p><code>import zarr, sys, napari</code></p> <p><code>raw = zarr.load(sys.argv[1], mode='r', path="volumes/raw")</code><br><code>gts = zarr.load(sys.argv[1], mode='r', path="volumes/gt_instances")</code></p> <p><code>viewer = napari.Viewer(ndisplay=3)</code><br><code>for idx, gt in enumerate(gts):</code><br><code> viewer.add_labels(</code><br><code> gt, rendering='translucent', blending='additive', name=f'gt_{idx}')</code><br><code>viewer.add_image(raw[0], colormap="red", name='raw_r', blending='additive')</code><br><code>viewer.add_image(raw[1], colormap="green", name='raw_g', blending='additive')</code><br><code>viewer.add_image(raw[2], colormap="blue", name='raw_b', blending='additive')</code><br><code>napari.run()</code></p> </li> <li>Execute: <pre><code>python view_data.py <path-to-file>/R9F03-20181030_62_B5.zarr</code></pre> </li> </ol> <h2>Metrics</h2> <ul> <li>S: Average of avF1 and C</li> <li>avF1: Average F1 Score</li> <li>C: Average ground truth coverage</li> <li>clDice_TP: Average true positives clDice</li> <li>FS: Number of false splits</li> <li>FM: Number of false merges</li> <li>tp: Relative number of true positives</li> </ul> <p>For more information on our selected metrics and formal definitions please see <a href="https://arxiv.org/abs/2404.00130" target="_blank" rel="noopener">our paper</a>.</p> <h2>Baseline</h2> <p>To showcase the FISBe dataset together with our selection of metrics, we provide evaluation results for three baseline methods, namely <a href="https://github.com/Kainmueller-Lab/PatchPerPix" target="_blank" rel="noopener">PatchPerPix (ppp)</a>, <a href="https://github.com/google/ffn" target="_blank" rel="noopener">Flood Filling Networks (FFN)</a> and a non-learnt application-specific <a href="https://www.biorxiv.org/content/10.1101/2020.06.07.138941v1" target="_blank" rel="noopener">color clustering from Duan et al.</a>.<br>For detailed information on the methods and the quantitative results please see <a href="https://arxiv.org/abs/2404.00130" target="_blank" rel="noopener">our paper</a>.</p> <h2>License</h2> <p>The FlyLight Instance Segmentation Benchmark (FISBe) dataset is licensed under the <a href="https://creativecommons.org/licenses/by/4.0" target="_blank" rel="noopener">Creative Commons Attribution 4.0 International (CC BY 4.0) license</a>.</p> <h2>Citation</h2> <p>If you use <em>FISBe</em> in your research, please use the following BibTeX entry: </p> <pre><code>@misc{mais2024fisbe, title = {FISBe: A real-world benchmark dataset for instance segmentation of long-range thin filamentous structures}, author = {Lisa Mais and Peter Hirsch and Claire Managan and Ramya Kandarpa and Josef Lorenz Rumberger and Annika Reinke and Lena Maier-Hein and Gudrun Ihrke and Dagmar Kainmueller}, year = 2024, eprint = {2404.00130}, archivePrefix ={arXiv}, primaryClass = {cs.CV} }</code></pre> <h2>Acknowledgments</h2> <p>We thank Aljoscha Nern for providing unpublished MCFO images as well as Geoffrey W. Meissner and the entire FlyLight Project Team for valuable<br>discussions.<br>P.H., L.M. and D.K. were supported by the HHMI Janelia Visiting Scientist Program.<br>This work was co-funded by Helmholtz Imaging.</p> <h2>Changelog</h2> <p>There have been no changes to the dataset so far.<br>All future change will be listed <a href="https://kainmueller-lab.github.io/fisbe/changelog" target="_blank" rel="noopener">on the changelog page</a>.</p> <h2>Contributing</h2> <p>If you would like to contribute, have encountered any issues or have any suggestions, please <a href="https://github.com/Kainmueller-Lab/fisbe/issues" target="_blank" rel="noopener">open an issue</a> for the FISBe dataset in the accompanying github repository.</p> <p>All contributions are welcome!</p>
Detection of Real-World Influence through Social Media
<p><strong>Description. </strong>This dataset corresponds to the resources produced for the following conference paper and its extended version:</p> <ol> <li>J.-V. Cossu, N. Dugué, and V. Labatut, “Detecting Real-World Influence Through Twitter,” in <em>2nd European Network Intelligence Conference (ENIC)</em>, 2015, pp. 83–90. ⟨<a href="https://hal.archives-ouvertes.fr/hal-01164453">hal-01164453</a>⟩ DOI: <a href="http://doi.org/10.1109/ENIC.2015.20">10.1109/ENIC.2015.20</a></li> <li>J.-V. Cossu, V. Labatut, and N. Dugué, “A Review of Features for the Discrimination of Twitter Users: Application to the Prediction of Offline Influence,” <em>Social Network Analysis and Mining </em>6:25, 2016. ⟨<a href="https://hal.archives-ouvertes.fr/hal-01203171">hal-01203171</a>⟩ DOI: <a href="http://doi.org/10.1007/s13278-016-0329-x">10.1007/s13278-016-0329-x</a></li> </ol> <p>Raw data are available through the official RepLab page: <a href="http://nlp.uned.es/replab2014/">http://nlp.uned.es/replab2014/</a> (follow <a href="http://nlp.uned.es/replab2014/replab2014-dataset.tar.gz">http://nlp.uned.es/replab2014/replab2014-dataset.tar.gz</a>)</p> <p><strong>Source code. </strong>The source code used to generate these output is available on GitHub: <a href="https://github.com/CompNet/Influence">https://github.com/CompNet/Influence</a></p> <p><strong>Funding. </strong>This work was partly funded by the French National Research Agency (ANR), through the project <a href="https://anr.fr/Project-ANR-12-CORD-0002">ImagiWeb ANR-12-CORD-0002</a>.</p> <p><strong>Contact. </strong>Jean-Valère Cossu <<a href="mailto:jean-valere.cossu@alumni.univ-avignon.fr">jean-valere.cossu@alumni.univ-avignon.fr</a>></p> <p><strong>Citation. </strong>If you use these data, please cite paper [1] above.</p> <p><br><code>@InProceedings{Cossu2015,</code><br><code> author = {Cossu, Jean-Valère and Dugué, Nicolas and Labatut, Vincent},</code><br><code> title = {Detecting Real-World Influence Through {Twitter}},</code><br><code> booktitle = {2\textsuperscript{nd} European Network Intelligence Conference},</code><br><code> year = {2015},</code><br><code> pages = {83-90},</code><br><code> address = {Karlskrona, SE},</code><br><code> publisher = {IEEE Publishing},</code><br><code> doi = {10.1109/ENIC.2015.20},</code><br><code>}</code></p> <p><strong>Details. </strong>This archive contains all ranking outputs formatted according to the TREC-EVAL tool format. These outputs consist for each domain in a ranked list of user from the most influential to the least influential. For a classification-type evaluation, just consider that users having a score higher than 0.5 are influential.</p> <p>File names correspond to the system (those starting with Cos*, indicate: the method BoT for Bag-of-Tweets, UaD for User-as-Document; the use of the Tweet-Selection strategy files denoted Artex; the learning process with Global or separated models which are noted Multi and last but not least the decision strategy for Bag-of-Tweets: Counting or Sum) or feature name. Files starting with out_* contain the results of logistic regression ranking outputs. Files matrix_auto.dat and matrix_bank.dat contain the data used to feed the PLS model (code: plspm4influence.R).</p> <p>RepLab 2014 uses Twitter data in English and Spanish. The balance between both languages depends on the availability of data for each of the profiles included in the dataset.</p> <p>The training dataset consists of 7,000 Twitter profiles (all with at least 1,000 followers) related to the automotive and banking domains, evaluation is performed separately. Each profile consists of (i) author name; (ii) profile URL and (iii) the last 600 tweets published by the author at crawling time and have been manually labelled by reputation experts either as “opinion maker” (i.e. authors with reputational influence) or “non-opinion maker”. The objective is to find out which authors have more reputational influence (who the opinion makers are) and which profiles are less influential or have no influence at all. </p> <p>Since Twitter ToS do not allow redistribution of tweets, only tweets ids and screen names are provided. Replab organizers provide details about how to download the tweets.</p>
Replication data for: Online Media Use and COVID-19 Vaccination in Real-World Personal Networks: Quantitative Study
<p>This is the replication data for the scientific paper titled "Online Media Use and COVID-19 Vaccination in Real-World Personal Networks: Quantitative Study" accepted for publication in the Journal of Medical Internet Research (JMIR). For details on how to use the data files, please consider the "supplementary_material.R" file or the "supplementary_material.pdf" where the variables of interest and R code are presented.</p> <p>For the code to run correctly, have the files "multilevel_labels.R" and "glm_labels.R" in the same working directory as the .R or .Rmd script. They are executed in the background, applying modifications to labels inside the regression tables. </p> <p> </p>
SIMUSAFE cyclist behavior in simulator and in real-world
<p>This dataset includes data collected by the SIMUSAFE H2020 EU project (2017-2021) during its first data acquisition cycle. Voluntary bicycle riders are the subjects in this dataset, and the dataset includes a combination of sensory and psychological characteristics data. Sensory data was recorded in one of two settings: driving around the city in reality (NDT) and driving in a simulator (NST) along routes that were designed to imitate similar in-city driving.</p> <p>Overall, the dataset consists of recordings from 7 subjects for the following durations:</p> <p><strong>Total time per user (hours):</strong></p> <p>User Overall recording duration</p> <p>USER0 0 days 17:28:06.284997888</p> <p>USER1 0 days 04:25:50.084999680</p> <p>USER2 0 days 00:11:50.677999616</p> <p>USER3 0 days 01:30:38.754000640</p> <p>USER4 0 days 02:44:50.625000192</p> <p>USER5 0 days 00:24:49.070000128</p> <p>USER6 0 days 01:15:46.702999040</p> <p><strong>Data measurements and computed attributes:</strong></p> <p><strong><em>GPS </em></strong>coordinates, acquired by a real GPS receiver in NDT and via a simulated receiver in NST. We extracted <em>velocity </em>from the GPS measurements, computed as the distance between every two subsequent coordinates divided by their corresponding timestamps. As a second derivative, <em>Acceleration</em> was then also derived from the difference of the above-mentioned velocity change between the two subsequent points divided by the time-delta.</p> <p><strong><em>Accelerometer </em></strong>data were used to compute the Euclidean norm of the acceleration (a.k.a l<sup>2</sup>-norm) over the acceleration coordinates vector (i.e., {a<sub>x</sub>; a<sub>y</sub>; a<sub>z</sub>}) at each point in time. This feature is sometimes also referred to as the <em>energy-expenditure</em> of the motion.</p> <p><strong>Additional features:</strong></p> <p><strong><em>De/Acceleration {high / low / none}</em></strong>, computed per user per scenario. For each user, acceleration measurements were partitioned by quartiles and were computed per scenario. <em>High-Acceleration </em>was defined as values above the 3<sup>rd</sup> quartile and <em>low-acceleration</em> as values below. <em>No-acceleration</em> was denoted for the case of acceleration is equal to zero. Respectively, decelerations were computed in an equivalent manner, computed from the partitioning of negative acceleration values.</p> <p><strong>Data preparation & preprocessing</strong></p> <ul> <li>GPS coordinates were de-duplicated w.r.t subsequent entries.</li> <li>To avoid issues originating from weak/loss of GPS signal, entries were partitioned into sessions. A session is defined as a sequence of entries with time-deltas no larger than 10 seconds. Velocity & acceleration were derived based on time-deltas within sessions.</li> <li>Rows with a velocity above or equal to 50km/h were filtered out based on the assumption that a regular bike rider won't reach such speeds.</li> </ul> <p>In addition to the data sources and processing procedures mentioned above, the data has been processed according to the following. Per each subject & scenario, the data was partitioned into windows of 30 seconds using a sliding window with overlap. On each window 5 statistics were computed I.e., entropy, mean, variance, skew, kurtosis on 3 different sources: GPS-based velocity, GPS-based acceleration, and accelerometer-based magnitude. Achieving a total of 15 features.</p> <p><strong>Data Schema</strong></p> <p>The data comprises measurement data, data computed after windowing as well as subject psychometric evaluation data. Window data is computed with a sliding window of size 40 (samples) with an overlap of 20 samples. Before computing windows, the measurement data is filtered from entries with missing ‘v_gps’ (velocity computed from GPS coordinates) values.</p> <p>The measurement dataset is in the attached bicycle_cycle_1_measurement_data.csv file.</p> <p>Dataset computed with windowing is in the attached bicycle_cycle_1_windowed_data_w_computed_features.csv file.</p> <p>The psychometric evaluation dataset is in the attached bicycle_cycle_1_subject_psychometric_evaluation.csv file.</p> <p>Description of all dataset attributes in all three datasets is detailed in the Data description.docx file.</p> <p>Preliminary correlations identified in the dataset is detailed in BICYCLE-DATA-CORRELATIONS.pptx file</p>
Annotated Benchmark of Real-World Data for Approximate Functional Dependency Discovery
<p><strong>Annotated Benchmark of Real-World Data for Approximate Functional Dependency Discovery</strong></p> <p>This collection consists of ten open access relations commonly used by the data management community. In addition to the relations themselves (please take note of the references to the original sources below), we added three lists in this collection that describe approximate functional dependencies found in the relations. These lists are the result of a manual annotation process performed by two independent individuals by consulting the respective schemas of the relations and identifying column combinations where one column implies another based on its semantics. As an example, in the <em>claims.csv</em> file, the <em>AirportCode</em> implies <em>AirportName</em>, as each code should be unique for a given airport.</p> <p>The file <em>ground_truth.csv</em> is a comma separated file containing approximate functional dependencies. <em>table</em> describes the relation we refer to, <em>lhs</em> and <em>rhs</em> reference two columns of those relations where semantically we found that <em>lhs</em> implies <em>rhs</em>.</p> <p>The file <em>excluded_candidates.csv</em> and <em>included_candidates.csv</em> list all column combinations that were excluded or included in the manual annotation, respectively. We excluded a candidate if there was no tuple where both attributes had a value or if the <em>g3_prime</em> value was too small.</p> <p><strong>Dataset References</strong></p> <ul> <li><em>adult.csv</em>: Dua, D. and Graff, C. (2019). <a href="http://archive.ics.uci.edu/ml">UCI Machine Learning Repository</a>. Irvine, CA: University of California, School of Information and Computer Science.</li> <li><em>claims.csv</em>: TSA Claims Data 2002 to 2006, <a href="https://www.dhs.gov/tsa-claims-data">published by the U.S. Department of Homeland Security</a>.</li> <li><em>dblp10k.csv</em>: Frequency-aware Similarity Measures. Lange, Dustin; Naumann, Felix (2011). 243–248. <a href="https://hpi.de/naumann/projects/repeatability/datasets/dblp-dataset.html">Made available as DBLP Dataset 2</a>.</li> <li><em>hospital.csv</em>: Hospital dataset used in Johann Birnick, Thomas Bläsius, Tobias Friedrich, Felix Naumann, Thorsten Papenbrock, and Martin Schirneck. 2020. Hitting set enumeration with partial information for unique column combination discovery. Proc. VLDB Endow. 13, 12 (August 2020), 2270–2283. https://doi.org/10.14778/3407790.3407824. <a href="https://owncloud.hpi.de/s/j6Z0yvXC0qhtGCk/download">Made available as part the dataset collection to that paper.</a></li> <li><em>t_biocase_...</em> files: t_bioc_... files used in Johann Birnick, Thomas Bläsius, Tobias Friedrich, Felix Naumann, Thorsten Papenbrock, and Martin Schirneck. 2020. Hitting set enumeration with partial information for unique column combination discovery. Proc. VLDB Endow. 13, 12 (August 2020), 2270–2283. https://doi.org/10.14778/3407790.3407824. <a href="https://owncloud.hpi.de/s/j6Z0yvXC0qhtGCk/download">Made available as part the dataset collection to that paper.</a></li> <li><em>tax.csv</em>: Tax dataset used in Johann Birnick, Thomas Bläsius, Tobias Friedrich, Felix Naumann, Thorsten Papenbrock, and Martin Schirneck. 2020. Hitting set enumeration with partial information for unique column combination discovery. Proc. VLDB Endow. 13, 12 (August 2020), 2270–2283. https://doi.org/10.14778/3407790.3407824. <a href="https://owncloud.hpi.de/s/j6Z0yvXC0qhtGCk/download">Made available as part the dataset collection to that paper.</a></li> </ul>
Advanced Non-Clear Cell Renal Cell Carcinoma Treatments and Survival: A Real-World Single-Centre Experience
<p>Dataset of the paper "Advanced Non-Clear Cell Renal Cell Carcinoma Treatments and Survival: A Real-World Single-Centre Experience"</p>
Dataset of vehicle emission measurements in real-world subfreezing winter conditions
<p>Dataset of vehicle emission measurements in real-world subfreezing winter conditions. Measured by chasing the measured vehicle. See Info.txt for description of the data.</p>
Student discourse in small-group collaborative contexts in real-world higher education
<p>This dataset contains anonymised student discourse in small-group collaborative contexts in real-world higher education settings.</p> <p>There are 28 sessions, composed of 10799 utterances.</p> <p>The columns consist of 13 columns:</p> <ul> <li>session: the session number</li> <li>start: the starting time of the utterance</li> <li>end: the ending time of the utterance</li> <li>speaker: anonymised speaker name</li> <li>content: the content of the utterance</li> <li>ep: the episode number of the utterance. An episode refers to a single topic of discussion with multiple utterances.</li> <li>C: a binary value indicating the presence or absence of a cognitive challenge, where 0 means no challenge, and 1 means there is a cognitive challenge.</li> <li>E: a binary value indicating the presence or absence of an emotional/motivational challenge, where 0 means no challenge, and 1 means there is an emotional/motivational challenge.</li> <li>M: a binary value indicating the presence or absence of a metacognitive challenge, where 0 means no challenge, and 1 means there is a metacognitive challenge.</li> <li>T: a binary value indicating the presence or absence of a technical/other challenge, where 0 means no challenge, and 1 means there is a technical/other challenge.</li> <li>TA: a binary value indicating the presence or absence of the regulatory process "task analysis," where 0 means no regulation and 1 means task analysis is present.</li> <li>MC: a binary value indicating the presence or absence of the regulatory process "monitoring/control," where 0 means no regulation and 1 means monitoring/control is present.</li> <li>RA: a binary value indicating the presence or absence of the regulatory process "reflection/adaptation," where 0 means no regulation and 1 means reflection/adaptation is present.</li> </ul> <p>This annotated dataset was used in the following paper to model challenge moments.</p> <p>For more details on how the dataset was generated, please refer to the paper.</p> <div> <div>Suraworachet, W., Seon, J., & Cukurova, M. (2024). Predicting challenge moments from students’ discourse: A comparison of GPT-4 to two traditional natural language processing approaches. <em>Proceedings of the 14th Learning Analytics and Knowledge Conference</em>, 473–485. <a href="https://doi.org/10.1145/3636555.3636905">https://doi.org/10.1145/3636555.3636905</a></div> <div> </div> </div>
Real-world grasp data of a dual-arm Yumi robot with a parallel gripper and suction cup end-effectors
<p>The attached txt file contains indexes to a cleaner subset of the data issued in the first version.</p> <p>Note: <br>Version 1 contains samples with failure cases due to environment constraints, which work well for the platform used in GraspAgent 1.0 (https://doi.org/10.1109/LRA.2024.3502066). However, this can degrade the performance if used on another platform with different constraints. To solve this, version 2 reports a subset of the raw data, excluding the failure modes due to environmental causes. </p>
Data related to publication "Coherent phase transfer for real-world twin-field quantum key distribution; Supplementary Information"
<p>These files contains datasets from which the Figures appearing in the Supplementary Information have been calculated. </p> <p>Description of datasets:</p> <p>Datasets related to SupplFig1 contain two columns: Frequency in Hz and phase noise in rad^2/Hz</p> <p>Data_SupplFig1_stabilised_fringes: psd of the phase noise calculated from the interference fringes in a stabilised condition</p> <p>Data_SupplFig1_unstabilised_fringes: psd of the phase noise calculated from the interference fringes in an unstabilised condition</p> <p>Data_SupplFig1_roundtrip_sensing_laser: psd of the sensing laser signal after a round-trip in the interferometer, calculated from self-heterodyne beatnote</p> <p>Data_SupplFig1_differential_roundtrip_sensing_vs_reference_laser: psd of the difference between the round-trip self-heterodyne beatnotes at the sensing and reference laser wavelengths</p> <p>Datasets related to SupplFig2 contain two columns: time in seconds and normalised intensity (calculated as detailed in the main publication).</p> <p>Data_SupplFig2_High_power_PD_free_evol: normalised intensity of the interference signal obtained with classical power level at the source. This trace was recorded with a photodiode when no artificial phase drift was applied</p> <p>Data_SupplFig2_High_power_PD_phase_drift: normalised intensity of the interference signal obtained with classical power level at the source. This trace was recorded with a photodiode when an artificial phase drift was applied (8pi/s)</p> <p>Data_SupplFig2_High_power_SPD_free_evol: normalised intensity of the interference signal obtained with classical power level at the source. This trace was recorded on an SPD (after suitable attenuation) when no artificial phase drift was applied </p> <p>Data_SupplFig2_High_power_SPD_phase_drift: normalised intensity of the interference signal obtained with classical power level at the source. This trace was recorded on an SPD (after suitable attenuation) when an artificial phase drift was applied (8pi/s)</p> <p>Data_SupplFig2_Attenuated_SPD_free_evol: normalised intensity of the interference signal obtained with attenuated beams at the source. This trace was recorded on an SPD when no artificial phase drift was applied </p> <p>Data_SupplFig2_Attenuated_SPD_phase_drift: : normalised intensity of the interference signal obtained with attenuated beams at the source. This trace was recorded on an SPD when an artificial phase drift was applied (8pi/s)</p> <p> </p>
Smart Analyser of Variability Requirements of Unknown Spaces (SAVRUS) Dataset of a study with 5 real-world large numerical variability models.
<p>The publications and research associated to cite is in:</p><p><a href="https://doi.org/10.1016/j.knosys.2023.110558">https://doi.org/10.1016/j.knosys.2023.110558</a></p><p>In that research we detail the Smart Analyser of Variability Requirements of Unknown Spaces (SAVRUS) approach, and provide a web-tool prototype in <a href="https://hadas.caosd.lcc.uma.es/savrus">https://hadas.caosd.lcc.uma.es/savrus</a></p><p>In the study, we model 5 different real-world software product lines to then analysed them with SAVRUS:</p><p>Detailed real-world variability models ordered by their search space size, of which GEC QA is incompletely measured NVM Description #Booleans #Numericals Space QA #Measurements </p><p>Dune1</p><p> </p><p>Multi-grid solver</p><p> </p><p>11</p><p> </p><p>3</p><p> </p><p>2,304</p><p> </p><p>Complex..</p><p> </p><p>2,304</p><p> </p><p>HSMGP1</p><p> </p><p>Stencil-grid solver</p><p> </p><p>14</p><p> </p><p>3</p><p> </p><p>3,456</p><p> </p><p>..equation..</p><p> </p><p>3,456</p><p> </p><p>HiPAcc1</p><p> </p><p>Image processing framework</p><p> </p><p>33</p><p> </p><p>2</p><p> </p><p>13,485</p><p> </p><p>..solving..</p><p> </p><p>13,485</p><p> </p><p>Trimesh2</p><p> </p><p>Triangle mesh library</p><p> </p><p>13</p><p> </p><p>4</p><p> </p><p>239,360</p><p> </p><p>..time</p><p> </p><p>239,360</p><p> </p><p>GEC</p><p> </p><p>Generic edge computing</p><p> </p><p>552</p><p> </p><p>2</p><p> </p><p>~5.3*108</p><p> </p><p>Energy Consumption</p><p> </p><p>132500</p><p> </p><p>The dataset zip file contains:</p><ul><li>5 numerical variability models in Clafer format (.txt) for each software product line.</li><li>5 CSV files with the respective quality attribute measurements</li><li>An .xlsx file containing SAVRUS scalability results divided in different tabs.</li></ul><p>References:</p><p>[1] N. Siegmund, A. Grebhahn, S. Apel, C. Kastner, Performance-influence models for highly configurable systems, in: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2015, Association for Computing Machinery, New York, NY, USA, 2015, p.284–294. doi:10.1145/2786805.2786845.</p><p>[2] M. Bauer, A comparison of six constraint solvers for variability analysis, Tech. rep., University of Passau (2019).</p>
Dataset of "PEMFC performance decay during real-world automotive operation: evincing degradation mechanisms and heterogeneity of ageing"
<p>This is the underlying dataset of "PEMFC performance decay during real-world automotive operation: evincing degradation mechanisms and heterogeneity of ageing"</p>
SIMBED - Offline Real-World Wireless Networking Experimentation using ns-3
<p>R&D in wireless networking typically depends on experimentation to make realistic evaluations, since simulation is inherently a simplification of the real-world. However, experimentation is limited in aspects where simulation excels, such as repeatability and reproducibility.</p> <p>Real wireless experiments are hardly repeatable. Given the same input they can produce very different output results, since wireless communications are influenced by external random phenomena such as noise, interference, and multipath. Real experiments are also difficult to reproduce: either the original community testbed is unavailable – offline or running other experiments – or the custom testbed used is inaccessible.</p> <p>Fed4FIRE+ wireless testbeds such as w-iLab.t and NITOS, although deployed in controlled environments, do not fully address the problem. The CONCRETE tool used in such testbeds assures the repeatability and reproducibility of experiments, but ignores executions whose results are also representative of the system operation and often reveal unpredicted behaviour that must be understood.</p> <p>What if we could make any wireless experiment repeatable and reproducible under the same exact conditions? What if we could share the same Fed4FIRE+ testbed execution conditions among an "infinite" number of users? What if we could run wireless experiments faster than in real time?</p> <p>INESC TEC has been developing the Offline Experimentation (OE) approach that combines the best of simulation and experimentation to achieve the above-mentioned goals. By relying on Network Simulator 3 (ns-3) and its good simulation capabilities from the MAC to the application layer, we have been exploring how ns-3 can be used to replicate real-world wireless experiments using real traces containing 1) position of nodes and 2) the quality of each radio link.</p> <p>The <strong>SIMBED </strong>project aimed at running a set of wireless experiments on top of the controlled environments of w-ilab.t and NITOS Fed4FIRE+ testbeds to further validate the OE approach. For that purpose, we configured different fixed and mobile experimental scenarios, representative of Wi-Fi range of operation, and measured the attained network performance using metrics such as throughput and Round-Trip Time (RTT). Then, we repeated each experiment using, both, Pure Simulation (PS) and OE approaches based on ns-3, also measuring the network performance for the same set of executions of experiments for all the different scenarios.</p> <p>By comparing the performance metrics of each real experiment with its PS and OE counterparts, we were able to measure the relative error of each simulation approach relatively to the real experiments, as well as the accuracy gains introduced by the OE approach when compared to the PS traditional alternative. The main results show that it is possible to repeat and reproduce real experiments in ns-3, using the OE approach, achieving closer to real performance than using the PS approach. For all the experiments performed in SIMBED, using the OE approach resulted in an average accuracy gain of 59% when comparing to the PS approach. </p> <p>These results were important for validating a PhD thesis contribution related to the OE approach, as well as for producing two conference papers and one journal paper. The SIMBED results increased our confidence on the accuracy of the OE approach and are envisioned to foster the adoption of the OE approach by the networking community, in complement to the use of real experimentation.</p> <p> </p> <p>The following dataset presents the results of the SIMBED project, organized in different folders, for each subset of experiments carried on:</p> <ul> <li><strong><em>SubExp#1: </em></strong><em>Static point-to-point Wi-Fi communications using auto-rate (Minstrel) </em> <ul> <li><strong><em>SubExp#1.1:</em></strong><em> Using w-iLab.2 (medium to high SNR scenarios)</em></li> <li><strong><em>SubExp#1.2:</em></strong><em> Using w-iLab.2 (low SNR scenarios)</em></li> <li><strong><em>SubExp#1.3:</em></strong><em> Using NITOS</em></li> <li><strong><em>SubExp#1.4:</em></strong><em> Using w-iLab.1 (datacenter room)</em></li> </ul> </li> <li><strong><em>SubExp#2: </em></strong><em>Static point-to-point Wi-Fi communications using fixed</em> rate</li> <li><strong><em>SubExp#3: </em></strong><em>Mobile point-to-point Wi-Fi communications using auto-rate (Minstrel)</em></li> <li><strong><em>SubExp#4: </em></strong><em>Static multiple access Wi-Fi communications using auto-rate (Minstrel) </em> <ul> <li><strong><em>SubExp#4.1:</em></strong><em> Using w-iLab.2 (bidirectional) (medium to high SNR scenarios)</em></li> <li><strong><em>SubExp#4.2:</em></strong><em> Using w-iLab.2 (bidirectional) (low SNR scenarios)</em></li> <li><strong><em>SubExp#4.3:</em></strong><em> Using NITOS (bidirectional)</em></li> <li><strong><em>SubExp#4.4:</em></strong><em> Using w-iLab.1 (bidirectional)</em></li> <li><strong><em>SubExp#4.5:</em></strong><em> Using NITOS (2 STAs)</em></li> <li><strong><em>SubExp#4.6:</em></strong><em> Using w-iLab.2 (2 STAs)</em></li> </ul> </li> <li><strong><em>SubExpExample</em></strong>: contains raw experimental logs, parsed data and simulation results, to show how data extracted from the nodes is processed to be compatible with the OE approach and comparable with OE and PS simulation results.</li> </ul> <p>Each experiment has an individual folder, named according to the date and time of the experiment and the nodes used. Inside, there’s a folder for the <strong>parsed</strong> experimental results, which contains</p> <p>This folder contains the details and parsed logs of the experiment, as follows:</p> <ul> <li><em>date_time</em><strong>.cfg </strong>– configuration details of the experiment</li> <li><em>date_time_NodeID<sup><a href="#_ftn1"><strong>[1]</strong></a></sup>_SenderID<sup><a href="#_ftn2"><strong>[2]</strong></a></sup>_ReceiverID<sup><a href="#_ftn3"><strong>[3]</strong></a></sup>_FlowType<sup><a href="#_ftn4"><strong>[4]</strong></a></sup>_Params<sup><a href="#_ftn5"><strong>[5]</strong></a></sup></em><strong>.snr </strong>– logs of the Signal/Noise ratio (1 file per node/flow) </li> <li><em>date_time_NodeID_SenderID_ReceiverID_FlowType_Params</em><strong>.stats</strong> – logs of the packets received (1 file per node/flow) </li> <li><em>NodeID</em><strong>.</strong><strong>waypoints</strong> – coordinates of the static nodes</li> <li><em>date_time_MobileNodeID</em><strong>.</strong><strong>waypoints</strong> – waypoints of the mobile nodes (when applicable)</li> </ul> <p>The experiment’s folder also contains a folder for the simulations <strong>output</strong> with the simulations statistics files, for the multiple simulations approaches considered, as follows:</p> <ul> <li><em>date_time_NodeID_SenderID_ReceiverID_FlowType_Params</em>.<strong>simstats </strong>– logs of the packets received (simulation)</li> </ul> <p> </p> <p><sub><a href="#_ftnref1">[1]</a> ID of the node Logging node</sub></p> <p><sub><a href="#_ftnref2">[2]</a> ID of the Sender node</sub></p> <p><sub><a href="#_ftnref3">[3]</a> ID of the Receiver node</sub></p> <p><sub><a href="#_ftnref4">[4]</a> Flow type: Unidirectional, Bidirectional or Unidirectional with Multiple Access</sub></p> <p><sub><a href="#_ftnref5">[5]</a> Configurable parameters: Sender/Receiver Transmission Power and Data Rate (when applicable)</sub></p>
Artifacts supplementing the EuroUSEC '22 paper "Assessing Real-World Applicability of Redesigned Developer Documentation for Certificate Validation Errors"
<p>This upload supplements the conference EuroUSEC 2022 submission by providing the full questionnaire, anonymized dataset and all performed analyses presented in the paper specified below.</p> <ul> <li>Title: <strong>Assessing Real-World Applicability of Redesigned Developer Documentation for Certificate Validation Errors</strong></li> <li>Authors: Martin Ukrop, Michaela Balážová, Pavol Žáčik, Eric Vincent Valčík, Vashek Matyas</li> <li>Paper details: https://crocs.fi.muni.cz/public/papers/eurousec2022</li> <li>Paper abstract: <em>We face certificate validation errors commonly, yet the related tools and documentation had been shown to have very poor usability. Previous research suggests that just improving the error messages and corresponding documentation can have significantly positive effects. Our work aims at increasing the usability of certificate validation by 1) redesigning the API error messages and the corresponding documentation, and 2) validating the real-world applicability of the redesign by investigating the opinions of 180 IT professionals. We focus on the perceived obstacles, desired ideal form and overall satisfaction. The redesigned documentation exhibits a reliable significant decrease in perceived incompleteness, with a small amount of perceived bloat and tangle. The redesigned documentation, now published on a dedicated website, is preferred by 89% of our study participants.</em></li> </ul> <p>The artifacts accompanying this paper contain three major parts:</p> <ul> <li>The questionnaire used in the main study (described in Sections 3.1 and 3.2 of the paper and mostly present in Appendices A and B of the paper).</li> <li>The anonymized dataset (multiple formats) of all valid questionnaire answers and qualitative coding performed. Analyses of this dataset are the core of the paper and are present in subsection 3.4 and all parts of Sections 4 and 5.</li> <li>The set of analyses files (IBM SPSS scripts and outputs) producing all statistical results presented in the paper are included.</li> </ul> <p>More details about the artifacts can be found in the README file in the artifacts archive.</p>
WikiChurches – A Fine-Grained Dataset of Architectural Styles with Real-World Challenges
<p>WikiChurches is a dataset for architectural style classification, consisting of 9,485 images of church buildings. Both images and style labels were sourced from Wikipedia. The dataset can serve as a benchmark for various research fields, as it combines numerous real-world challenges: fine-grained distinctions between classes based on subtle visual features, a comparatively small sample size, a highly imbalanced class distribution, a high variance of viewpoints, and a hierarchical organization of labels, where only some images are labeled at the most precise level. In addition, we provide 631 bounding box annotations of characteristic visual features for 139 churches from four major categories. These annotations can, for example, be useful for research on fine-grained classification, where additional expert knowledge about distinctive object parts is often available.</p> <p>Please refer to the README.md file for information about the different files contained in this dataset.</p>
Processed Synthetic Real-World Data for tristate modelling
<p>This model learning dataset is created out of the <a href="https://zenodo.org/record/7409763">Raw Synthetic RWD</a> raw dataset, including some of the original attributes. It is distributed in JOBLIB files, where .joblib files contain the vectors and _ids.joblib contain the ID of the person from which each vector is extracted.</p> <p>This is useful in case it is needed to map the vectors to metadata about the people that are found in the original raw dataset. Note that corresponds to , or , depending on the dataset.</p> <p>The split is roughly 60% of the people are in the training dataset, and 20% in each of the validation and the testing datasets. The input attributes are the age, the short-term averages and the trends of the current week’s BMI, steps walked, calories burned, sleep quality, mood and water consumption, as well as the previous week’s short-term average and trend of the answer to the health self-assessment question.</p> <p>The outcome to be predicted is a tristate quantized version of the health self-assessment answer to be given in the current week. The dataset is normalized based on the training set. The means and standard deviations used can be found in the train_statistics.joblib file. Finally, the output_descriptions.joblib file contains descriptions of the outcomes to be predicted (not actually needed, since included here).</p>
Processed Synthetic Real-World Data for binary modelling
<p>This model learning dataset is created out of the <a href="https://zenodo.org/record/7409763">Raw Synthetic RWD</a> raw dataset, including some of the original attributes. It is distributed in JOBLIB files, where .joblib files contain the vectors and _ids.joblib contain the ID of the person from which each vector is extracted.</p> <p>This is useful in case it is needed to map the vectors to metadata about the people that are found in the original raw dataset. Note that corresponds to , or , depending on the dataset. The split is roughly 60% of the people are in the training dataset, and 20% in each of the validation and the testing datasets. The input attributes are the age, the short-term averages and the trends of the current week’s BMI, steps walked, calories burned, sleep quality, mood and water consumption, as well as the previous week’s short-term average and trend of the answer to the health self-assessment question.</p> <p>The outcome to be predicted is the binary quantized health self-assessment answer to be given in the current week. The dataset is normalized based on the training set. The means and standard deviations used can be found in the train_statistics.joblib file. Finally, the output_descriptions.joblib file contains descriptions of the outcomes to be predicted (not actually needed, since included here).</p>
Raw Synthetic Real-World Data
<p>Real World Data from a simulator for 1,000 people belonging to any of four behavioural groups: athletic, normal, unfit and feeble, simulated over 2 years and 3 months.</p> <p>The dataset is organised into 4 CSV files:</p> <ul> <li>aggregations.csv contains the daily physiological data, including their short- & long-term averages and their trends. Every row corresponds to a day for a person.</li> <li>answers.csv contains the answers to questions. Each row corresponds to a question answered.</li> <li>activity_summaries.csv contains summaries of the detected activities. Each row corresponds to an activity someone performed.</li> <li>participants_features.csv contains the demographic info of each simulated person. This is static info, every row corresponding to one person. An ML engineer can combine information from any of these files to setup model learning datasets. This is done in the “Processed Synthetic RWD for binary modelling” and the “Processed Synthetic RWD for tristate modelling” datasets that are also shared here.</li> </ul>
RealVAD: A Real-world Dataset for Voice Activity Detection
<p><strong>RealVAD: A Real-world Dataset for Voice Activity Detection</strong></p> <p>The task of automatically detecting “Who is Speaking and When” is broadly named as Voice Activity Detection (VAD). Automatic VAD is a very important task and also the foundation of several domains, e.g., human-human, human-computer/ robot/ virtual-agent interaction analyses, and industrial applications.</p> <p>RealVAD dataset is constructed from a YouTube video composed of a panel discussion lasting approx. 83 minutes. The audio is available from a single channel. There is one static camera capturing all panelists, the moderator and audiences.</p> <p>Particular aspects of RealVAD dataset are:</p> <ul> <li> It is composed of panelists with different nationalities (British, Dutch, French, German, Italian, American, Mexican, Columbian, Thai). This aspect allows studying the effect of ethnic origin variety to the automatic VAD.</li> <li> There is a gender balance such that there are four female and five male panelists.</li> <li> The panelists are sitting in two rows and they can be gazing audience, other panelists, their laptop, the moderator or anywhere in the room while speaking or not-speaking. Therefore, they were captured not only from frontal-view but also from side-view varying based on their instant posture and head orientation.</li> <li> The panelists are moving freely and are doing various spontaneous actions (e.g., drinking water, checking their cell phone, using their laptop, etc.), resulting in different postures.</li> <li> The panelists’ body parts are sometimes partially occluded by their/other's body part or belongings (e.g., laptop).</li> <li> There are also natural changes of illumination and shadow rising on the wall behind the panelists in the back row.</li> <li> Especially, for the panelists sitting in the front row, there is sometimes background motion occurring when the person(s) behind them moves.</li> </ul> <p>The annotations includes:</p> <ul> <li> The upper body detection of nine panelists in bounding box form.</li> <li> Associated VAD ground-truth (speaking, not-speaking) for nine panelists.</li> <li> Acoustic features extracted from the video: MFCC and raw filterbank energies.</li> </ul> <p><em>All info regarding the annotations are given in the ReadMe.txt and Acoustic Features README.txt files.</em></p> <p><strong>When using this dataset for your research, please cite the following paper in your publication:</strong></p> <ol> <li>C. Beyan, M. Shahid and V. Murino, "RealVAD: A Real-world Dataset and A Method for Voice Activity Detection by Body Motion Analysis", in IEEE Transactions on Multimedia, 2020.</li> </ol>
OCDetect - A Real-World Dataset to Detect Handwashing in Daily-Life using Wrist Motion Data from Wearables
<p>Handwashing detection is a relevant research topic with applications in healthcare and professional environments. While usually related to hygiene improvement, handwashing detection could also be used to support individuals with obsessive-compulsive disorder (OCD). For these individuals, compulsive, long, and frequent handwashing has a negative impact. An automated system could spot compulsive handwashing in real-time and augment the therapy process. No activity recognition datasets containing in-the-wild-recorded compulsive handwashing are available. With this work, we present the OCDetect Dataset, the first dataset with unscripted, compulsive handwashing. It contains recordings from inertial measurement units (IMUs) of 22 participants over 28 days, with ~3000 recorded hand washes. For each hand wash, we supply its user-annotated kind (compulsive / routine). We provide an overview of related datasets and describe the recording, cleaning, labeling, and final features of our dataset. We reach a maximum F1 score of 0.77 (avg.: 0.33, chance level: 0.03) when spotting handwashing from all background activities on unseen participants. Our dataset and code for the reproduction of our results are publicly available.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.