Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
1,782
datasets available to search
ShareScore release 0.7.1
Dataset results
1,782 results for “algorithms”
A Building Imagery Database for the Calibration of Machine Learning Algorithms
<p>An open database of 5276 building images from a parish in Lisbon (Alvalade), whose buildings have been classified according to a uniform taxonomy. This open database can be used for the testing and calibration of machine learning algorithms, as well as for the direct assessment of earthquake risk in Alvalade.</p> <p><strong>Full Changelog</strong>: <a href="https://github.com/vsilva028/ML/commits/v1.0.0">https://github.com/vsilva028/ML/commits/v1.0.0</a></p>
Computing Star Discrepancies with Numerical Black-Box Optimization Algorithms - Code and Data
<p>This repository contains the code and data for reproducibility of the paper 'Computing Star Discrepancies with Numerical Black-Box<br> Optimization Algorithms'. </p> <p>The following files are included:</p> <p>- TA.zip and DEM.zip: The code used for the TA and DEM algorithms respectively.</p> <p>- experiment_runner: Python file which was used to run the black-box optimization algorithms on the discrepancy problems from IOHexperimenter (requires package 'ioh', version 0.3.6 or higher). This generates data in IOH-format, which is included in 'raw_data.zip'</p> <p>- process_stardicr.R: R script which uses IOHanalyzer to extract the performance from the raw data into csv files for visualization. The resulting csvs are included in 'csv_with_pos' for the final results including the corresponding coordinates and 'csv_perf.zip', which contains the convergence information.</p> <p>- Found_Values: The discrepancy values found by TA and DEM, separated by sampler.</p> <p>- A csv file of the relative performance of each of the optimizers compared to the values found by TA is included in 'final_precision_table.csv'</p> <p>- Plot_StarDiscr: the python notebook used to generate all figures, except figure 3 which was created using the IOHanalyzer GUI (iohanalyzer.liacs.nl). The full dataset is available on the website under the source 'star_discrepancy'</p> <p>- Figures: some additional figures which were not included in the paper because of space constraints + higher quality versions of some of the landscape plots.</p>
Dataset and Application of Algorithms of "Towards a More Set of Acceptance Criteria"
<p>This dataset contains the requirements of the Corona Warn App together with the results of the application of the algorithms described in "Towards a More Complete Set of Acceptance Criteria Using User Story Similarity" on this data set.</p>
yukimayuli-gmz/data: Two parts data used in our paper about Lpnet algorithm
<p>Two parts data used in our paper about Lpnet algorithm:</p> <ol> <li>the tree files, sequence alignments and distance matrices for the simulated data sets.</li> <li>the distance matrices for the random distances.</li> </ol>
Input data for Reversible Unwrapping Algorithm for Constant-Pressure Molecular Dynamics Simulations
<p>As described in the main text, here is the input data used for simulation, as well as analysis directories. The archive was generated in my project folder with "tar --exclude=*trr --exclude=pbctools --exclude=qtwrap --exclude=old* --exclude=*npz --exclude=*pdf --exclude=*png --exclude=*ppm --exclude=*dcd* --exclude=*xtc --exclude=*slurm* --exclude=core* --exclude=*sh --exclude=*xvg --exclude=*out --exclude=*git* --exclude=*edr --exclude=*log --dereference -zcvf kulke-$(date +"%F").tar.gz data figures scripts Simulations". Big data and trajectory files were excluded to keep the archive size small. The archive includes all necessary files to reproduce the simulations, analysis and figures for the publication.</p>
Simple Multimodal Algorithmic Reasoning Task Dataset (SMART-101)
<p><strong>Introduction</strong></p> <p>Recent times have witnessed an increasing number of applications of deep neural networks towards solving tasks that require superior cognitive abilities, e.g., playing Go, generating art, ChatGPT, etc. Such a dramatic progress raises the question: how generalizable are neural networks in solving problems that demand broad skills? To answer this question, we propose SMART: a Simple Multimodal Algorithmic Reasoning Task (and the associated SMART-101 dataset) for evaluating the abstraction, deduction, and generalization abilities of neural networks in solving visuo-linguistic puzzles designed specifically for children of younger age (6--8). Our dataset consists of 101 unique puzzles; each puzzle comprises a picture and a question, and their solution needs a mix of several elementary skills, including pattern recognition, algebra, and spatial reasoning, among others. To train deep neural networks, we programmatically augment each puzzle to 2,000 new instances; each instance varied in appearance, associated natural language question, and its solution. To foster research and make progress in the quest for artificial general intelligence, we are publicly releasing our SMART-101 dataset, consisting of the full set of programmatically-generated instances of 101 puzzles and their solutions.</p> <p>The dataset was introduced in our paper <a href="https://arxiv.org/pdf/2212.09993.pdf">Are Deep Neural Networks SMARTer than Second Graders?</a> by Anoop Cherian, Kuan-Chuan Peng, Suhas Lohit, Kevin A. Smith, and Joshua B. Tenenbaum, CVPR 2023</p> <p>Files in the unzipped folder:</p> <ol> <li>./README.md: This Markdown file</li> <li>./SMART101-Data: Folder containing all the puzzle data. See below for details.</li> <li>./puzzle_type_info.csv: Puzzle categorization (into 8 skill classes).</li> </ol> <p><strong>Dataset Organization</strong></p> <p>The dataset consists of `101` folders (numbered from 1-101); each folder corresponds to one distinct puzzle (root puzzle). There are 2000 puzzle instances programmatically created for each root puzzle, numbered from 1-2000. Every root puzzle index (in [1,101]) folder contains: (i) `img/` and (ii) `puzzle_<index>.csv`. The folder `img/` is the location where the puzzle instance images are stored, and `puzzle_<index>.csv` the non-image part of a puzzle. Specifically, a row of `puzzle_<index>.csv` is the following tuple: `<id, Question, image, A, B, C, D, E, Answer>`, where `id` is the puzzle instance id (in [1,2000]), `Question` is the puzzle question associated with the instance, `image` is the name of the image (in `img/` folder) corresponding to this instance `id`, `A, B, C, D, E` are the five answer candidates, and `Answer` is the answer to the question. </p> <p><strong>At a Glance</strong></p> <ul> <li>The size of the unzipped dataset is ~12GB. </li> <li>The dataset consists of `101` folders (numbered from 1-101); each folder corresponds to one distinct puzzle (root puzzle). </li> <li>There are 2000 puzzle instances programmatically created for each root puzzle, numbered from 1-2000. </li> <li>Every root puzzle index (in [1,101]) folder contains: (i) `img/` and (ii) `puzzle_<index>.csv`. </li> <li>The folder `img/` is the location where the puzzle instance images are stored, and `puzzle_<index>.csv` contains the non-image part of a puzzle. Specifically, a row of `puzzle_<index>.csv` is the following tuple: `<id, Question, image, A, B, C, D, E, Answer>`, where `id` is the puzzle instance id (in [1,2000]), `Question` is the puzzle question associated with the instance, `image` is the name of the image (in `img/` folder) corresponding to this instance `id`, `A, B, C, D, E` are the five answer candidates, and `Answer` is the correct answer to the question. </li> </ul> <p><strong>Other Details</strong><br> In our paper <a href="https://arxiv.org/pdf/2212.09993.pdf">Are Deep Neural Networks SMARTer than Second Graders?</a>, we provide four different dataset splits for evaluation: (i) Instance Split (IS), (ii) Answer Split (AS), (iii) Puzzle Split (PS), and (iv) Few-shot Split (FS). Below, we provide the details of each split to make fair comparisons to the results reported in our paper. </p> <p><em>Puzzle Split (PS)</em><br> We use the following root puzzle ids as the `Train` and `Test` sets. </p> <table> <thead> <tr> <th scope="col">Split</th> <th scope="col">Root Puzzle Id Sets</th> </tr> </thead> <tbody> <tr> <td>`Test`</td> <td>{ 94,95, 96, 97, 98, 99, 101, 61,62, 65, 66,67, 69, 70, 71,72,73,74,75,76,77}</td> </tr> <tr> <td>`Train`</td> <td>{1,2,...,101} \ Test</td> </tr> </tbody> </table> <p>Evaluation is done on all the `Test` puzzles and their accuracies averaged. For the 'Test' puzzles, we use the instance indices 1701-2000 in the evaluation.</p> <p><em>Few-shot Split (FS)</em></p> <p>We randomly select `k` number of instances from the `Test` sets (that are used in the PS split above) for training in FS split (e.g., `k=100`). These `k` few-shot samples are taken from instance indices 1-1600 of the respective puzzles and evaluation is conducted on all instance ids from 1701-2000.</p> <p><em>Instance Split (IS)</em></p> <p>We split the instances under every root puzzle as: Train = 1-1600, Val = 1601-1700, Test = 1701-2000. We train the neural network models using the `Train` split puzzle instances from all the root puzzles together and evaluate on the `Test` split of all puzzles.</p> <p><em>Answer Split (AS)</em></p> <p>We find the median answer value among all the 2000 instances for every root puzzle and only use this set of the respective instances (with the median answer value) as the `Test` set for evaluation (this set is excluded from the training of the neural networks).</p> <p><em>Puzzle Categorization</em></p> <p>Please see puzzle_type_info.csv for details on the categorization of the puzzles into eight classes, namely (i) counting, (ii) logic, (iii) measure, (iv) spatial, (v) arithmetic, (vi) algebra, (vii) pattern finding, and (viii) path tracing.</p> <p><strong>Other Resources</strong></p> <p>PyTorch code for using the dataset to train deep neural networks is available <a href="https://www.merl.com/publications/TR2023-014">here</a>.</p> <p><strong>Contact</strong><br> Anoop Cherian (cherian@merl.com), Kuan-Chuan Peng (kpeng@merl.com), or Suhas Lohit (slohit@merl.com)</p> <p><br> <strong>Citation</strong><br> If you use the SMART-101 dataset in your research, please cite our paper:</p> <pre><code>@article{cherian2022deep, title={Are Deep Neural Networks SMARTer than Second Graders?}, author={Cherian, Anoop and Peng, Kuan-Chuan and Lohit, Suhas and Smith, Kevin and Tenenbaum, Joshua B}, journal={arXiv preprint arXiv:2212.09993}, year={2022} }</code></pre> <p><strong>Copyright and Licenses</strong></p> <p>The SMART-101 dataset is released under `CC-BY-SA-4.0`.</p> <pre><code>Created by Mitsubishi Electric Research Laboratories (MERL), 2022-2023 SPDX-License-Identifier: CC-BY-SA-4.0 </code></pre> <p> </p>
Datasets for benchmarking RNA 2D structure prediction algorithms.
<p>Datasets for benchmarking ML approaches in RNA 2D structure prediction task.</p>
Lateral flow immunoassay-based laboratory algorithm for rapid diagnosis of diphtheria
<p><strong>Results of the ‘DIFTERIA’ Lateral Flow Immunoassay (LFIA) validation</strong></p>
Pento-DIARef: A Diagnostic Dataset for Learning the Incremental Algorithm for Referring Expression Generation from Examples
<p>We present a <strong>D</strong>iagnostic dataset of <strong>IA</strong> <strong>Ref</strong>erences in a <strong>Pento</strong>mino domain (Pento-DIARef) that ties extensional and intensional definitions more closely together, insofar as the latter is the generative process creating the former.</p> <p>We create a novel synthetic dataset of examples that pairs visual scenes with generated referring expressions; examine two variants of the dataset, representing two different ways to exemplify the underlying task; and evaluate an LSTM-based baseline, a transformer and a modified version with region embeddings on them.</p> <p>See https://github.com/clp-research/pento-diaref for more information.</p>
Supplementary data for "Fault-tolerant quantum algorithm for symmetry-adapted perturbation theory"
<p>Supplementary data belonging to "Fault-tolerant quantum algorithm for symmetry-adapted perturbation theory".</p> <p>The data includes geometries for the molecules in the paper as well as the Hamiltonian matrix elements, orbital coefficients and overlap matrices to reproduce the data in the paper.</p>
Datasets related to the Eggsplorer rapid plant–insect resistance determination tool, using an automated whitefly egg quantification algorithm
<p>These datasets support the data provided directly in the scientific publication titled "Eggsplorer: a rapid plant–insect resistance determination tool using an automated whitefly egg quantification algorithm", published in the <a href="https://plantmethods.biomedcentral.com/articles/10.1186/s13007-023-01027-9#Abs1">BMC Plant Methods journal</a>. </p> <p>The figures and tables included in the datasets highlight various materials and methods employed to build a proof-of-concept of the Eggsplorer tool. These included whitefly assays, leaf image acquisitions, an automatic egg quantification algorithm, image pre-processing, object detection and image stitching, detection post-processing, algorithm evaluation and software programming.</p> <p>The provided figures and tables in the datasets are further discussed and interpreted in detail, as well as their subsequent results, in the scientific publication.</p> <p>This research was conducted within the VIRTIGATION project, which is part of the EU Open Research Data pilot. This project has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No. 101000570.</p>
Out-of-distribution detection algorithms for robust insect classification dataset and models
<p>This folder contains trained models and datasets for reproducing the results in the paper on out-of-distribution detection algorithms for robust insect classification. Specifically, it contains the following folders: </p> <p> </p> <ul> <li>OODInsect (out-of-distribution data)</li> <li>MSP, MAH, and EBM trained models, each wrapped around the three classifiers of ResNet50, RegNet32, and VGG11, and different combinations of ID and OOD test data for reproducing RQ1, RQ2, and RQ3.</li> <li>ID3 (in-distribution test data)</li> </ul>
Simulated and real datacubes for developing and testing changepoint algorithms for spatially correlated, short and noisy time-series
<p>A set of datacubes where z-dimension is time, thus each (x,y,.) is a timeseries. The idea is to detect sudden changes in each series, assuming 1) the series can be quite short and noisy 2) the change occurs in spatial patches. The set has synthetic examples with known change-events, and a real-world dataset with unknown change-events. The data files are related to the pape</p> <p>T Rajala, P Packalen, M Myllymäki, A Kangas (2023): Improving detection of changepoints in short and noisy time-series with local correlations: Connecting the events in pixel neighbourhoods, "Journal of Agricultural, Biological and Environmental Statistics", https://doi.org/10.1007/s13253-023-00546-1</p> <p>More of the NFI data is available from Natural Resources Institute Finland, https://kartta.luke.fi/index-en.html</p> <p>See `data/00data_readme.txt` for further details.</p> <p> </p>
Problem Solving and Algorithmic Development with Flowcharts
<p>This study reports on an in-depth research into student-learning using a "thinking-first" framework combined with stepwise heuristics, to provide students structure throughout the entire programming process.</p> <p>The study targetted secondary education students in an elective computer science course. There was one class with<br> 11 Dutch high school students, of which 2 females and 9 males. The group was heterogeneous, with students from different academic<br> levels and age-groups. Each student’s level and previous experience with CS was determined a priori using a pretest.</p> <p>For this study we developed sets of quizes, tasks and tests comprised of code comprehension, code composition questions (including reading and creating flowchart designs). The student responses to each were anaylzed.</p> <p>This repository contains the following data:<br> - taxonomyPerQ.pdf: indicates taxonomy level of each (quiz, task, test) question answered by students <br> - assessments_unanswered: all questions (quizes, tasks, tests) administered to students<br> - pretask (responses): anonymized student responses to pretask questions<br> - midtask (responses): anonymized student responses to midtask questions<br> - finaltask (responses): anonymized student responses to finaltask questions<br> - quiz 1 (responses): anonymized student responses to quiz 1 questions<br> - quiz 2 (responses): anonymized student responses to quiz 2 questions<br> - final test (responses): anonymized student responses to final test questions</p> <p><br> The student's handwritten work was scanned, saved as pdf and coded in atlas.ti. These coded pdf's cannot be anonymized anymore, and thus not openly distributed or published.</p>
Data for: Defining usual oral temperature ranges in outpatients using an unsupervised learning algorithm
<p><strong>Importance</strong>: Although oral temperature is commonly assessed in medical examinations, the range of usual or "normal" temperature is poorly defined.</p> <p><strong>Objective</strong>: To determine normal oral temperature ranges by age, sex, height, weight and time of day.</p> <p><strong>Design</strong>: We applied a filtering algorithm (LIMIT) to 10 years of outpatient temperature measurements. LIMIT iteratively removed encounters with primary diagnoses overrepresented in the tails of the temperature distribution, leaving only those diagnoses unrelated to temperature. Mixed effects modeling was applied to the remaining temperature measurements to identify independent predictors of normal oral temperature and to generate individualized normal temperature ranges.</p> <p><strong>Setting</strong>: Single large medical care system, divisions of Internal Medicine and Family Medicine.</p> <p><strong>Participants</strong>: All adult outpatient encounters that included temperature measurements, April 2008 - June 2017.</p> <p><strong>Exposures</strong>: Primary diagnoses and medications, age, sex, height, weight, time of day and month, abstracted from each outpatient encounter.</p> <p><strong>Main outcomes and measures</strong>: Normal temperature ranges by age, sex, height, weight, and time of day.</p> <p><strong>Results</strong>: From 618,306 encounters, 36% were removed by LIMIT because they included diagnoses or medications that fell disproportionately in the tails of the temperature distribution. The encounters removed due to overrepresentation in the upper tail were primarily linked to infectious diseases (76.81% of all removed encounters); type 2 diabetes mellitus was the only diagnosis removed for over-representation in the lower tail (15.71% of all removed encounters). Prior to running LIMIT, the mean overall oral temperature was 36.71°C (±0.43); following LIMIT, the mean temperature was 36.64°C (±0.35). Using mixed effects modeling, age, sex, height, weight and time of day accounted for 6.86% (overall) and up to 25.52% (per subject) of the observed variability in temperature. Mean normal oral temperature did not reach 37°C for any subgroup; the upper 99th percentile ranged from 36.8°C (tall underweight old men in the morning) to 37.9°C (short obese young women in the afternoon).</p> <p><strong>Conclusion and relevance</strong>: Normal oral temperature varies in a predictable manner based on sex, age, height, weight and time of day, allowing individualized normal temperature ranges to be established. </p>
Replication package for: "Corrupted by Algorithms? How AI-generated and Human-written Advice Shape (Dis)honesty"
<p>Package to the following paper:</p> <p>Leib, M; Köbis, N; Rilke, R M; Hagens, M; Irlenbusch, B (2023) Corrupted by Algorithms? How AI-generated and Human-written Advice Shape (Dis)honesty</p>
Ensemble of optimised machine learning algorithms for predicting surface soil moisture content at global scale (v1.0)
<p>This study investigates the estimation of daily SSM using eight optimised ML algorithms and ten ensemble models (constructed via model bootstrap aggregating techniques and five-fold cross-validation). The algorithmic implementations were trained and tested using the international soil moisture network (ISMN) data collected from 1722 stations distributed across the World. </p>
A Multi-algorithm Approach for Modeling Coastal Wetland Eco-geomorphology
<p>This dataset includes the input data for MACES to run simulations on three representative coastal wetland sites: two saltmarsh wetland (Venice Lagoon and Plum Island Estuary) and a mangrove wetland (Hunter Estuary).</p>
A Novel Algorithm for Estimating Web Page Ranking in Search Engine Results Pages
<p><em><strong>Abstract:</strong> </em>Search engine optimization (SEO) can make a big improvement in the traffic to a web page. Because search engines keep their main rules of ranking undeclared, it’s important to develop models that can estimate the ranking of a web page in the search engine to be able to optimize web pages to rank higher in the search engine. The available research methodologies used machine learning algorithms to provide solutions for this target with the help of generated datasets by scraping the search engine results pages (SERP) and crawling web pages. Their proposed models suffered from the inability to be updated dynamically if the search engine updated its ranking algorithm, and their input data did not include the diversity of web pages and languages. This research will propose a novel original rank estimation algorithm that’s able to overcome other research challenges, with a set of comparative experiments and complexity analysis. Results will show that the proposed algorithm could achieve higher values of accuracy, precision, and recall.</p> <p><strong><em>Dataset: </em></strong></p> <p>For research purpose, the dataset will play two roles, first, it will act the role of search engine result pages (SERP), and second, it will be used to test algorithms and calculate performance measurements. Dataset is consisting of 9930 web pages, aimed to identify search results pages, focusing on the top 3 pages of SERP, with 31 extracted attributes that's related to search engine optimization (SEO). The distribution of examples between class labels was balanced, with changes due to scraping operation issues, but not significantly different, with fractions of 39.9%, 34.6%, and 25.5% for the class labels page1, page2, and page 3. Feature names are: 'Title 1 Length', 'Title 2 Length', 'Meta Description 1 Length', 'Meta Description 2 Length', 'Meta Keywords 1 Length', 'H1-1 Length', 'H1-2 Length', 'H2-1 Length', 'H2-2 Length', 'Size (bytes)', 'Word Count', 'Text Ratio', 'Inlinks', 'Unique Inlinks', 'Unique JS Inlinks', '% of Total', 'Outlinks', 'Unique Outlinks', 'Unique JS Outlinks', 'External Outlinks', 'Unique External Outlinks', 'Unique External JS Outlinks', 'Response Time', 'Status Code', 'Keyword in MetaDescription1', 'Keyword in Title1', 'Keyword in MetaKeywords1', 'Keyword in URL', 'Has LastModified', 'Keyword in Headers', and 'Keyword in Emphasized Text'.</p> <p>The process of dataset generation involved scraping the search engine, extracting URLs for selected keywords, focusing on feature extraction, cleaning and preprocessing, and generating new attributes related to keywords in web pages. It involved also removing missing values, duplicates, and data type conversions to obtain a comprehensive dataset.<br> Keyword selection involves selecting keywords from various categories and considering diversity, including high and low traffic, long-term and short-term keywords, and generic and branded keywords. Apify online tool was used for search engine scraping with default language and US country, resulting in 388 selected keywords with 30 results per keyword. Dataset included extracted SEO features from 9991 web pages using screamingFrog desktop software and Rapidminer desktop software, determining page SEO-friendliness and comparing it to SERP rankings. Dataset cleaning involved removing redundant attributes, removing paid SERP results, replacing missing values, and converting data types. Rapidminer was used for data cleaning and preprocessing, generating new attributes related to keyword usage in web pages.<br> </p>
Data set for the article: Modifying twist algorithms for determining the key length of a Vigenere cipher
<p>Data supporting the work in the article: Modifying twist algorithms for determining the key length of a Vigen\`{e}re cipher.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.