Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
832
datasets available to search
ShareScore release 0.9.0
Dataset results
832 results for “automatic”
Data Access Made Easy: flexible, on the fly data standardization and processing (for research automatic weather stations)
<pre>Automatic Weather Stations (AWS) deployed in the context of research projects provide very valuable data thanks to the flexibility they offer in term of measured meteorological parameters, choice of sensors and quick deployment and redeployment. However this flexibility is a challenge in terms of metadata and data management. Traditional approaches based on networks of standard stations can not accommodate these needs and often no tools are available to manage these research AWS, leading to wasted data periods because of difficult data reuse, low reactivity in identifying potential measurement problems, and lack of metadata to document what happened. The Data Access Made Easy (DAME) effort is our answer to these challenges. At its core, it relies on the mature and flexible open source MeteoIO meteorological pre-processing library. It was originally developed as a flexible data processing engine for the needs of numerical models consuming meteorological data and further developed as a data standardization engine for the Global Cryosphere Watch (GCW) of the World Meteorological Organization (WMO). For each AWS, a single configuration file describes how to read and parse the data, defines a mapping between the available fields and a set of standardized names and provides relevant Attribute Conventions Dataset Discovery (ACDD) metadata fields, if necessary on a per input file basis. Low level data editing is also available, such as excluding a given sensor, swapping sensors or merging data from another AWS, for any given time period. Moreover an arbitrary number of filters can be applied on each meteorological parameter, restricted to specific time periods if required. This allows to describe the whole history of an AWS within a single configuration file and to deliver a single, consistent, standardized output file possibly spanning many years, many input data files and many changes both in format and available sensors. Finally, all configuration files are versionned in order to document their history. A web interface has been developed that allows data owners to manage the configuration files for their stations, refresh their data at regular intervals, inspect the data QA log files and allow on-demand data generation. The same interface allows other users to request data on-demand for any time period.<br><br>This presentation and software has received funding from the World Meteorological Organization under grant agreement No. 29539/2022-1.9 as well as the European Union’s Horizon 2020 research and innovation program under grant agreement No. 101003472 (Arctic Passion). It has also been supported by the WSL/SLF over many years and projects.</pre>
Repository for: "Using automatic calibration to improve the physics behind complex numerical models: An example from a 3D lake model"
<p>Set of numerical experiments supporting the paper entitled "Using automatic calibration to improve the physics behind complex numerical models: An example from a 3D lake model" by Marina Amadori, Abolfazl Irani Rahaghi, Damien Bouffard and Marco Toffolon. Submitted to GMD. </p> <p>The folder contains: </p> <p>simulations: DYNO-PODS + Delft3D experiments on Lake Morat. See https://github.com/louisXW/DYNO-pods for more insights on DYNO-PODS and instructions for installation.</p> <p>scripts: extraction and plotting scripts</p> <p>source_code: modified Delft3D src as available at: https://github.com/eawag-surface-waters-research/Delft3D/tree/d3d4/research/surface_heat_transfer</p>
Ice Anatomy: A Benchmark Dataset and Methodology for Automatic Ice Boundary Extraction from Radio-Echo Sounding Data
<p>The measurement of ice thickness is of great importance for the accurate estimation of glacier volume and the delineation of their bedrock topography. In particular, this is a crucial factor in forecasting the future evolution of glaciers in the context of a changing climate. In order to derive the ice thickness, the travel time of electromagnetic waves in radargrams acquired by radio-echo sounding (RES) systems is analyzed. This can only be achieved by identifying the ice surface and underlying ice bottom in corresponding radargrams. Manually identifying these two reflection horizons in RES data is a laborious and time-consuming process. Consequently, scientists are attempting to automate this task through the use of techniques such as deep learning. Such automation can significantly reduce the time between a field campaign and the calculation of the glacier's ice thickness distribution. In this paper, we present the first benchmark dataset for delineating the ice surface and bottom boundaries in RES data, to facilitate straightforward comparisons of deep learning models in the future. The ``IceAnatomy'' dataset comprises radargrams and the corresponding manual picks, amounting to a total of over 45,000km of observations. The RES data originates from three sources: FAU, CReSIS, and AWI. The dataset comprises different RES systems as well as different pre-processing methods. In addition, the data was acquired over a large range of geographical and glaciological settings, featuring different thermal regimes present in Antarctica and the Southern Patagonian Icefield. This diversity ensures that the models' behaviors can be analyzed in different scenarios. We define a standardized train-test split for each source in the dataset. This allows us to introduce not only a baseline model trained on the entire training set (the ``omni'' model), but also three source-specific baseline models. The source-specific models are trained exclusively on the subset of the training data acquired by the specified source. The baseline models provide an initial benchmark against which subsequent models can be compared. The source-specific models demonstrate more accurate results than the omni model. For the FAU, CReSIS, and AWI test sets, the source-specific models achieve Mean Meter Errors of 2.1m, 23.1m, and 4.9m for the ice surface and 9.1m, 78.2m, and 29.3m for the ice bottom. In relation to the mean measured ice thickness, these errors equate to 1.2%, 3.1%, and 0.3% for the ice surface and 4.9%, 10.4%, and 1.5% for the ice bottom.</p> <p> </p> <p> For more information, please read the following paper:</p> <p>[Coming soon. Currently under review.]</p> <p>Please also cite this paper if you plan on using the dataset.</p> <p> </p> <p>For the implementation of a baseline model please visit:</p> <p>[Coming soon]</p> <p> </p>
Data for Quantifying the Impact of Parametric Uncertainty on Automatic Mechanism Generation for CO2 Hydrogenation on Ni(111)
<p>Data, scripts, and all generated mechanisms for the preprint and article "Quantifying the Impact of Parametric Uncertainty on Automatic Mechanism Generation for CO<sub>2</sub> Hydrogenation on Ni(111)"</p>
Data Sets and Results for "Improved data sets and evaluation methods for the automatic prediction of DNA-binding proteins"
<p>Data sets and results for "Improved data sets and evaluation methods for the automatic prediction of DNA-binding proteins"<br> <br> The file "dna_binding_protein_sequences.zip" has the training and testing sets from the paper:<br> <br> RLL - "random_<train/test>_full_1000.csv"<br> RSL - "random_<train/test>_40.csv"<br> RS&LL - "random_<train/test>_40_1000.csv"<br> RLL where included positive examples have verified DNA binding activity - "random_<train/test>_hq_1000.csv"<br> The 10 RS&LL data sets - "random_<train/test>_40_1000.csv" + "random_<train/test>_40_1000_cv_<0-8>.csv"</p> <p>The results files are named similarly. See "see_results.ipynb" in the <a href="https://doi.org/10.5281/zenodo.5153683">codebase</a> that supplement these data sets<br> The species data sets are derived from "uniprot_data_bac.tab" and "uniprot_data_not_bac.tab." See <a href="https://doi.org/10.5281/zenodo.5153683">code</a>. <br> <br> The ESM embeddings used by the XGBoost model are in "dna_binding_protein_esm.zip"</p>
Developer-Centric Test Amplification: The Interplay Between Automatic Generation and Human Exploration --- Appendix
<p>This online appendix contains the accumulated code occurences during the interviews performed to evaluate our developer-centric test amplification approach and the TestCube plugin. In addition, it documents the inter-rater-reliability analysis we performed.</p>
Automatic extraction of opinions of users of social networks on reproductive behavior issues
<p>The database contains an upload of text comments in Russian from the social network Vkontakte in .tsv format (UTF-8 encoding). Comments are collected from communities, which discuss pregnancy, childhood, motherhood, paternity, etc. Comments are collected from communities, which discuss pregnancy, childhood, motherhood, paternity, etc. <br> The database contains train, test and valid sets for machine learning processing. For the analysis of opinions in the field of reproductive behavior, nine groups of the social network Vkontakte were selected, in the names or descriptions of which the words "childfree" and their variations were clearly present, and 341 groups, in the names or descriptions of which the keywords "mother", " mothers "," children ", etc. and the number of subscribers of which was more than 10,000 people. This distribution of groups depended on the different activity of groups - supporters of a childless lifestyle produced significantly more posts and comments on our topics. Using data from different groups of the social network Vkontakte avoids data homogeneity - one of the weak points of sentiment analysis. Thus, the presented database is suitable for the analysis of specific demographic groups, which we conditionally called “anti-natalists” and “pronatalists”. Note that in the group of pronatalists there are largely representatives of a small child model of reproductive behavior. The sample contains data about stance on 6 topics: "maternity capital / benefits", "abortion", "large families", "childlessness", "parental leave", "individualism".The topics are selected by a set of keywords such as abortion, childfree, rest, no child and so on.</p> <p>Sentences from the collected sample were randomly selected for annotator marking. Each sentence was marked with three annotators. Since each sentence could discuss several issues, the annotator marked each sentence on all seven topics. The proposals were marked up mainly by professional demographers and linguists.</p>
Towards a Data-Driven Requirements Engineering Approach: Automatic Analysis of User Reviews
<p>6000 French user reviews from three applications on Google Play (Garmin Connect, Huawei Health, Samsung Health) are labelled manually. We selected four labels: rating, bug report, feature request and user experience.</p> <ul> <li><strong>Ratings</strong> are simple text which express the overall evaluation to that app, including praise, criticism, or dissuasion.</li> <li><strong>Bug reports</strong> show the problems that users have met while using the app, like loss of data, crash of app, connection error, etc.</li> <li><strong>Feature requests</strong> reflect the demande of users on new function, new content, new interface, etc.</li> <li>In <strong>user experience</strong>, users describe their experience in relation to the functionality of the app, how does certain functions be helpful.</li> </ul> <p>As we can observe from the following table, that shows examples of labelled user reviews, each review belongs to one or more categories.</p> <table> <tbody> <tr> <th>App</th> <th>Total</th> <th>Rating</th> <th>Bug report</th> <th>Feature request</th> <th>User experience</th> </tr> </tbody> <tbody> <tr> <td>Garmin Connect</td> <td>2000</td> <td>1260</td> <td>757</td> <td>170</td> <td>493</td> </tr> <tr> <td>Huawei Health</td> <td>2000</td> <td>1068</td> <td>819</td> <td>384</td> <td>289</td> </tr> <tr> <td>Samsung Health</td> <td>2000</td> <td>1324</td> <td>491</td> <td>486</td> <td>349</td> </tr> </tbody> </table> <p> </p> <h2>New Dataset</h2> <p>Based on this dataset, we developed a labeled dataset containing 6,000 English and 6,000 French reviews for classification, as well as 1,200 bilingual reviews for clustering. The new dataset has been made publicly available on Zenodo at the following link: <a href="../records/11066414">https://zenodo.org/records/11066414</a></p>
Characterizing the automatic radon flux Transfer Standard system Autoflux: laboratory calibration and field experiments
<p>Data from the manuscript: Characterizing the automatic radon flux Transfer Standard system Autoflux: laboratory calibration and field experiments.</p> <p>The present folder contains:</p> <ol> <li>Exhalation Bed data (data and plots from the characterization of the exhalation bed of the Cantabria University)</li> <li>ANSTO Autoflux (data and plots of the calibration of the Autoflux radon flux system)</li> <li>INTE_UPC (data and plots of the calibration of the INTE_UPC radon flux system)</li> </ol>
Real-time Automatic Temperature Regulation During In Vivo MRI-guided Laser Induced Thermotherapy (MR-LITT)
<p>This dataset is associated with the article.</p> <p>For each example (in vitro/in vivo), two matrices are provided, each containing :</p> <p>- coefficients estimation curves ;</p> <p>- automatic regulation curves and recontructed data (thermometry, phase, module) of the MRI sequence.</p>
Automatic global climate classification, preserving Köppen thermal classes with added aridity information and a hypertropical class
<p>A streamlined system using growing degree days for climate classification that delivers both temperature and aridity information in all classes and an added hypertropical class. Primary thermal classes remain consistent with the primary Köppen climate classes and their established connection to biomes.</p> <p>The eight class to tif tool ends with a .tif file of climate classes. </p> <p>The steps may need to be modified if using different inputs than Envirem and WorldClim 1.4. Envirem is missing datum information, but ArcGIS makes the same calculations <br> without the project raster step. Note that WorldClim 1.4 applies a scaling factor of 10 for temperature data, which is propagated in Envirem.</p> <p>Import the symbology layer file climate_8_class. If ArcGIS is being cranky about displaying the (blue) hypertropical class, try reseting with the 'Add All Values' button,<br> followed by hitting 'Default Colors'.</p>
Overview of Automatic Preliminary Boundary Delineation Tool developed as part of UC2 (Prefilled Application) in NIVA4CAP
<p>UC2 Pre-filled application seeks to make it easier and more accurate for farmers to annually declare their lands by reducing the time needed to draw the new parcels boundaries or update the existing ones, thus, prevent the mistakes with input of the data on crops grown. The work focussed on (1) Creating an automatic preliminary boundary delineation tool, (2) Testing an early stage corp type/ land use classification tool and (3) Finding and testing a robotic process automation tool suitable for data harvesting from external registers. This document gives an overview of the parcel boundary delineation tool and is supplemented by a complementary report summarising results from tests of that tool in Lithuania and Spain and referenced below.</p>
Automatically Extracted SHACL Shapes for WikiData, DBpedia, YAGO-4, and LUBM & Associated Coverage Statistics
<p>The uploaded datasets contain <strong>automatically extracted </strong>SHACL shapes for the following datasets:</p> <ul> <li>WikiData (the truthy dump from September 2021 filtered by removing non-English strings) [1]</li> <li>DBpedia [2]</li> <li>YAGO-4 [3] </li> <li>LUBM (scale factor 500) [4]</li> </ul> <p>The validating shapes for these datasets are generated by a program that parses the corresponding RDF files (in `.nt` format). The extracted shapes encode various SHACL constraints, e.g., sh:minCount, sh:path, sh:class, sh:datatype etc. For each shape we encode coverage in terms of number of entities satisfying such shape, this information is encoded using the <a href="http://vocab.deri.ie/void#entities">void:entities</a> predicate. </p> <p>We have provided as executable Jar file the program we developed to extract these SHACL shapes.<br> More details about the datasets used to extract these shapes and <em>how to run the Jar</em> are available on our GitHub repository <a href="https://github.com/dkw-aau/qse">https://github.com/dkw-aau/qse</a>.</p> <p>Read more about our Quality Shapes Extraction (QSE) tool on our website <a href="https://relweb.cs.aau.dk/qse/">https://relweb.cs.aau.dk/qse/</a></p> <p>[1] Vrandečić, Denny, and Markus Krötzsch. "Wikidata: a free collaborative knowledgebase." Communications of the ACM 57.10 (2014): 78-85.</p> <p>[2] Auer, Sören, et al. "Dbpedia: A nucleus for a web of open data." The semantic web. Springer, Berlin, Heidelberg, 2007. 722-735.</p> <p>[3] Pellissier Tanon, Thomas, Gerhard Weikum, and Fabian Suchanek. "Yago 4: A reason-able knowledge base." European Semantic Web Conference. Springer, Cham, 2020.</p> <p>[4] Guo, Yuanbo, Zhengxiang Pan, and Jeff Heflin. "LUBM: A benchmark for OWL knowledge base systems." Journal of Web Semantics 3.2-3 (2005): 158-182.</p>
Automatic Choroid Vascularity Index Calculation in Optical Coherence Tomography Images with Low Contrast Sclerocho-roidal Junction Using Deep Learning
<p>This project aims to calculate Choroid Vascularity Index (CVI) in optical coherenece tomography (OCT) images, using loss modified U-Net. The method is detailed in "Automatic Choroid Vascularity Index Calculation in Optical Coherence Tomography Images low contrast sclerochoroidal junction Using Deep Learning". The dataset consists of Enhanced-depth imaging optical coherence tomography images from two patient groups.</p> <p>• First dataset is including Raster OCT B-scans from patients with diabetic retinopathy.</p> <p>• Second dataset is including EDI-HD OCT B-scans from patients with pachychoroid spectrum.</p>
Automatic Specialization of Third-Party Java Dependencies - Experimental Results
<p>This entry contains the experimental results of the research paper titled: <strong>Automatic Specialization of Third-Party Java Dependencies</strong><em>. </em>It includes the build results (e.g., executions logs, test results, JAR files, etc.) for the original, debloated, and specialized variants of the dependency trees for the 30 Java project used as study subjects in the paper.</p>
Supplementary data for: "Towards automatic generation of control structures for Process Flow Diagrams (PFDs) with Artificial Intelligence"
<p>Uploaded on 20. February 2023</p> <p>This is the supplementary data for the publication</p> <p>"Towards automatic generation of control structures for Process Flow Diagrams (PFDs) with Artificial Intelligence" (2023) by Edwin Hirtreiter, Lukas Schulze Balhorn, Artur M. Schweidtmann</p> <p>Corresponding author: A. M. Schweidtmann, E-mail: a.schweidtmann@tudelft.nl<br> Delft University of Technology, Department of Chemical Engineering, Process Intelligence Group, Van der Maasweg 9, 2629 HZ Delft, The Netherlands</p> <p>The folder contains json files with the training (train), test (test), and augmented training (train_augm) data files. The json files contain syntetically generated SFILES. </p> <p>The pre-print of the manuscript is accessible at https://doi.org/10.48550/arXiv.2211.05583</p>
TreeFlow: Probabilistic programming and automatic differentiation for phylogenetics
<p>Probabilistic programming frameworks are powerful tools for statistical modelling and inference. They are not immediately generalisable to phylogenetic problems due to the particular computational properties of the phylogenetic tree object. TreeFlow is a software library for probabilistic programming and automatic differentiation with phylogenetic trees. It implements inference algorithms for phylogenetic tree times and model parameters, given a tree topology. We demonstrate how TreeFlow can be used to quickly implement and assess new models. We also show that it provides reasonable performance for gradient-based inference algorithms compared to specialized computational libraries for phylogenetics.</p>
AutoMate: a dataset and learning approach for automatic mating of CAD assemblies
<p>Assembly modeling is a core task of computer aided design (CAD), comprising around one third of the work in a CAD workflow. Optimizing this process therefore represents a huge opportunity in the design of a CAD system, but current research of assembly based modeling is not directly applicable to modern CAD systems because it eschews the dominant data structure of modern CAD: parametric boundary representations (BREPs). CAD assembly modeling defines assemblies as a system of pairwise constraints, called mates, between parts, which are defined relative to BREP topology rather than in world coordinates common to existing work. We propose SB-GCN, a representation learning scheme on BREPs that retains the topological structure of parts, and use these learned representations to predict CAD type mates. To train our system, we compiled the first large-scale dataset of BREP CAD assemblies, which we are releasing along with benchmark mate prediction tasks. Finally, we demonstrate the compatibility of our model with an existing commercial CAD system by building a tool that assists users in mate creation by suggesting mate completions, with 72.2% accuracy.</p>
Image dataset for the creation of an automatic system for meteor fall detection
<p>Image dataset with sky photos showing the occurrence or non-occurrence of falling meteors. The database comprises 7,000 images in JPEG format -- 3,850 (55%) images show the event of falling meteors, and 3,150 (45%) images show no meteors. Different instruments captured the photos from 2014 to 2023. We used the images to train a deep-learning neural network for an automatic falling meteor detector.</p> <p>The primary image data sources were the Brazilian Meteor Observation Network (BRAMON -- <a href="http://www.bramonmeteor.org">http://www.bramonmeteor.org</a>), UK Meteor Network (UKMON -- <a href="https://ukmeteornetwork.co.uk">https://ukmeteornetwork.co.uk</a>), and <em>Base des Observateurs Amateurs de Météores</em> (BOAM -- <a href="http://boam.fr">http://boam.fr</a>) repositories.</p> <p><strong>Folders Structure</strong></p> <p>We divided the folder structure into two levels. In the first level, we have two folders: RawImages, which holds images with captions stored in the repositories; and CroppedImages, which contains images without the captions (we cropped a band of 24 pixels in the lower part of the image).</p> <p>In the second level, in each of the previous folders, we have another two folders: meteor, which has images with meteors; and non-meteors, with images without occurrences of meteors.</p> <p><strong>Naming pattern for the files</strong></p> <p>The naming pattern in the meteor folder follows the format <source>_<date>_<id>.jpg where:</p> <ul> <li><source> is one of the 3 data sources: bramon, ukmon, or boam.</li> <li><date> is the date-time the instrument captured the image in the format yyyymmdd_hhnnss (y:year, m:month, d:day, h:hours, n:minutes, s:seconds).</li> <li><id> is an identifier from a specific source to avoid date-time conflicts: <ul> <li>BRAMON: radiant identifier.</li> <li>UKMON: station identifier.</li> <li>BOAM: station identifier.</li> </ul> </li> </ul> <p>For the non-meteor folder, the naming pattern is <source>_<date>_nonmeteor.jpg to avoid homonyms (with the same date-time) and to identify that they are images of non-meteors.</p>
PolyMed: A Medical Dataset Addressing Disease Imbalance for Robust Automatic Diagnosis Systems
<p>We introduce the PolyMed dataset, designed to address the limitations of existing medical case data for Automatic Diagnosis Systems (ADS). ADS assists doctors by predicting diseases based on patients' basic information, such as age, gender, and symptoms. However, these systems face challenges due to imbalanced disease label data and difficulties in accessing or collecting medical data. To tackle these issues, the PolyMed dataset has been developed to improve the evaluation of ADS by incorporating medical knowledge graph data and diagnosis case data. The dataset aims to provide comprehensive evaluation, include diverse disease information, effectively utilize external knowledge, and perform tasks closer to real-world scenarios.</p> <p>We have also made the data collection tools publicly available to enable researchers and other interested parties to contribute additional data in a standardized format. These tools feature a range of customizable input fields that can be selectively utilized according to the user's specific requirements, ensuring consistency and professionalism in the data collection process.</p> <p>All train and test code of our data available in https://github.com/krchanyang/PolyMed</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.