Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
832
datasets available to search
ShareScore release 0.9.0
Dataset results
832 results for “automatic”
InsectSet47 & InsectSet66: Expanded datasets for automatic acoustic identification of insects (Orthoptera and Cicadidae)
<p><strong>Updated full version with training, validation and test sets.</strong></p> <p>Two newly compiled datasets for training neural networks to automatically identify insect species while comparing adaptive, waveform-based frontends to conventional mel-spectrogram frontends for audio feature extraction. This work was <a href="https://doi.org/10.1371/journal.pcbi.1011541">published in PLOS</a> Computational Biology and the machine learning implementations were published on <a href="https://github.com/mariusfaiss/InsectSet47-InsectSet66-Adaptive-Representations-of-Sound-for-Automatic-Insect-Recognition">Github</a>.</p> <p>These datasets expand on the previously published <a href="https://doi.org/10.5281/zenodo.7072196">InsectSet32</a> by including recently published collections of insect recordings by citizen scientists from around the world. Recordings from <a href="https://bio.acousti.ca/">BioAcoustica</a>, <a href="http://xeno-canto.org/">xeno-canto</a> and <a href="http://inaturalist.org/">iNaturalist</a>, as well as private collections by <a href="https://orcid.org/0000-0002-8929-2737">Baudewijn Odé</a> were downloaded and manually inspected. Files with strong noise interference or intense filtering, as well as files containing sounds of multiple species were removed to compile these datasets. The files were standardised to 44.1 kHz mono WAV files ranging in length from less than one second to several minutes. Files containing long periods without insect sounds were edited into multiple smaller files with silent periods no longer than 5 seconds. These files are marked as edits in the annotation file and should be assigned together into train/validation/test sets to prevent data leakage. The annotation files contain information for each recording, including the file name, species name and identifier, as well as the data subset they were included in for training the neural network (training, test, validation).</p> <p>InsectSet47 expands on <a href="https://doi.org/10.5281/zenodo.7072196">InsectSet32</a> with recordings from <a href="http://xeno-canto.org/">xeno-canto</a> and contains 1006 original recordings from 47 species, with at least ten files per species. The total length of InsectSet47 is 22 hours. InsectSet66 further expands on InsectSet47 by adding research-grade audio observations from <a href="http://inaturalist.org/">iNaturalist</a>, with a total of 1554 recordings from 66 species, a total length of over 24 hours and a minimum of ten files per species.</p> <p>The datasets were split into the training, validation and test sets while ensuring a roughly equal distribution of audio files and audio material for every species in all three subsets. This resulted in a 60/20/20 split (train/validation/test) by file number and a 64/19.5/16.5 split by file length.</p>
Data deposit accompanying Accurate Energy Barriers for Catalytic Reaction Pathways: An Automatic Training Protocol for Machine Learning Force Fields
<p>Dataset accompanying the paper: <em>"Accurate Energy Barriers for Catalytic Reaction Pathways: An Automatic Training Protocol for Machine Learning Force Fields"</em>. Contains the training sets curated during active learning as well as .xyz files used for creating the Figures. <br> <br> The paper highlights that the computational efficiency of ML force fields not only results in decreased computational costs for routine catalytic investigations but also facilitates more comprehensive exploration of catalytic pathways.</p> <p><strong>Published in NPJ Computational Materials</strong>: <a href="https://www.nature.com/articles/s41524-023-01124-2">https://www.nature.com/articles/s41524-023-01124-2</a><br> Formerly on Arxiv: <a href="https://arxiv.org/abs/2301.09931">https://arxiv.org/abs/2301.09931</a></p>
BgMA-ESy: Expert system for automatic classification of vegetation plots of subalpine tall-herb vegetation (class Mulgedio-Aconitetea) from Bulgaria
<p>*****</p> <p>BgMA-ESy is an expert system that classifies vegetation plots of the class <em>Mulgedio-Aconitetea</em> (<a href="https://doi.org/10.1111/avsc.12257">Mucina et al. 2016</a>) occurring in Bulgaria. The expert system can be run using the JUICE program (<a href="https://doi.org/10.1111/j.1654-1103.2002.tb02069.x">Tichý 2002</a>; <a href="https://www.sci.muni.cz/botany/juice/">https://www.sci.muni.cz/botany/juice/</a>).</p> <p>The aggregation of vascular plants included within the BgMA-ESy is adopted from EUNIS-ESy (<a href="https://doi.org/10.1111/avsc.12519">Chytrý et al. 2020</a>; <a href="https://doi.org/10.5281/zenodo.4812736">https://doi.org/10.5281/zenodo.4812736</a>), and in a few cases, it is adjusted.</p> <p>*****</p> <p><strong>Specifications</strong></p> <p>The analyzed data (vegetation plots) cannot: </p> <ul> <li>include scrub vegetation (cover of tall shrub species > 8%; e.g., <em>Pinus mugo</em>, <em>Salix </em>spp.).</li> <li>contain tree species with cover > 1% (e.g., <em>Fagus sylvatica</em>, <em>Picea abies</em>).</li> <li>contain <em>Pteridium aquilinum </em>as a dominant species.</li> </ul> <p>The expert system was trained on vegetation plots with 5–100 m<sup>2</sup> area that occur above 1000 m a. s. l.</p> <p>* Exceptions from EUNIS-ESy aggregation:</p> <p>Heracleum sphondylium agg. does not include H. sphondylium subsp. verticillatum.</p> <p> </p> <p>*****</p> <p>When using this work, please cite:</p> <p>Szokala D., Kočí M. & Vassilev K. (2024): Subalpine tall-herb vegetation in Bulgaria: diversity and ecology. – Plant Biosystems 158: 490–510. <a href="https://doi.org/10.1080/11263504.2024.2327865">https://doi.org/10.1080/11263504.2024.2327865</a>.</p> <p>*****</p>
El Verde Field Station Air temperature from automatic sensor
Hourly air temperature for El Verde Field Station (2007 – present). Temperature is measured in the same box where we measure manual min-max temperature. The box is mostly shaded by surrounding trees, but it is more exposed to sunlight after hurricanes defoliate the forest canopy (e.g., hurricane Maria in 2017). Data is measured hourly using a HOBO Pendant data logger placed inside a wooden box. Notes: • Daily maximum and minimum air temperatures are available in separate data sets. This is a long-term monitoring of air temperature in the same box where we measure manual min-max temperature at El Verde Field Station. The data set is meant as a back up to the long-term manual data set. Data is measured hourly using a HOBO Pendant data logger placed inside a wooden box: under the shade of trees. The min-max thermometer that is measured manually during work days is located next to the HOBO in the same box. Support for this work was provided by grants BSR-8811902, DEB-9411973, DEB-9705814 , DEB-0080538, DEB-0218039 , DEB-0620910 , DEB-1239764, DEB-1546686, and DEB-1831952 from the National Science Foundation to the University of Puerto Rico as part of the Luquillo Long-Term Ecological Research Program. Additional support provided by the University of Puerto Rico and the International Institute of Tropical Forestry, USDA Forest Service.
CC20 Artifact - Automatic Fusion
<p>1. Getting started</p> <p>The title of our paper submitted to CC20 is</p> <p><strong>Improving Database Query Performance with Automatic Fusion</strong></p> <p>This repository is created for showing the reproducibility of our experiments in this paper. We provide the details of scripts and original data used in the experiments. There are mainly two systems: <em>HorsePower</em> and <em>RDBMS MonetDB</em>. We supply step-by-step instructions to configure and deploy both systems in the experiments.</p> <p>On this page, you will see:</p> <ul> <li>how to run experiments (Section 2); and</li> <li>the results used in the paper (Section 3);</li> </ul> <p>2. Experiments</p> <p>All experiments were run on a server called <code>sable-intel</code> equipped with</p> <ul> <li>Ubuntu 16.04.6 LTS (64-bit)</li> <li>4 Intel Xeon E7-4850 2.00 GHz</li> <li>total 40 cores with 80 threads</li> <li>128GB RAM</li> </ul> <p>Docker setup</p> <p>Download the docker image: cc20-docker.tar (About 13GB)</p> <pre><code>docker load < cc20-docker.tar </code></pre> <p>Generate a named container (then exit)</p> <pre><code>docker run --hostname sableintel -it --name=container-cc20 wukefe/cc20-docker exit </code></pre> <p>Then, you can run the container</p> <pre><code>docker start -ai container-cc20 </code></pre> <p>Open a new terminal to access the container (optional)</p> <pre><code>docker exec -it container-cc20 /bin/bash </code></pre> <p>Introduction to MonetDB</p> <p>Work directory for MonetDB</p> <pre><code>/home/hanfeng/cc20/monetdb </code></pre> <p>Start MonetDB (use all available threads)</p> <pre><code>./run.sh start </code></pre> <p>Login MonetDB using its client tool, <code>mclient</code></p> <pre><code>mclient -d tpch1 ## ... MonetDB version v11.33.3 (Apr2019) sql> SELECT 'Hello world'; +-------------+ | L2 | +=============+ | Hello world | +-------------+ 1 tuple </code></pre> <p>Show the list of tables in the current database</p> <pre><code>sql> \d TABLE sys.customer TABLE sys.lineitem TABLE sys.nation TABLE sys.orders TABLE sys.part TABLE sys.partsupp TABLE sys.region TABLE sys.supplier </code></pre> <p>Leave the session</p> <pre><code>sql> \q </code></pre> <p>Stop MonetDB before we can continue our experiments</p> <pre><code>./run.sh stop </code></pre> <p>Reference: <a href="https://www.monetdb.org/Documentation/Guide/Installation">How to install MonetDB and the introduction of server and client programs.</a></p> <p>Run MonetDB with TPC-H queries</p> <p>MonetDB: server mode</p> <p>Invoke MonetDB with a specific number of threads (e.g. 1)</p> <pre><code>mserver5 --set embedded_py=true --dbpath=/home/hanfeng/datafarm/2019/tpch1 --set monet_vault_key=/home/hanfeng/datafarm/2019/tpch1/.vaultkey --set gdk_nr_threads=1 </code></pre> <p>Open a new terminal</p> <pre><code>docker exec -it container-cc20 /bin/bash cd cc20/monetdb </code></pre> <p><strong><em>Note</em></strong>: Type <code>\q</code> to exit the server mode.</p> <p>Run with a specific number of threads (Two terminals required)</p> <p>1 thread</p> <pre><code>## terminal 1 mserver5 --set embedded_py=true --dbpath=/home/hanfeng/datafarm/2019/tpch1 --set monet_vault_key=/home/hanfeng/datafarm/2019/tpch1/.vaultkey --set gdk_nr_threads=1 ## terminal 2 (time ./runtest | mclient -d tpch1) &> "log/log_thread_1.log" </code></pre> <p>2 threads</p> <pre><code>## terminal 1 mserver5 --set embedded_py=true --dbpath=/home/hanfeng/datafarm/2019/tpch1 --set monet_vault_key=/home/hanfeng/datafarm/2019/tpch1/.vaultkey --set gdk_nr_threads=2 ## terminal 2 (time ./runtest | mclient -d tpch1) &> "log/log_thread_2.log" </code></pre> <p>4 threads</p> <pre><code>## terminal 1 mserver5 --set embedded_py=true --dbpath=/home/hanfeng/datafarm/2019/tpch1 --set monet_vault_key=/home/hanfeng/datafarm/2019/tpch1/.vaultkey --set gdk_nr_threads=4 ## terminal 2 (time ./runtest | mclient -d tpch1) &> "log/log_thread_4.log" </code></pre> <p>8 threads</p> <pre><code>## terminal 1 mserver5 --set embedded_py=true --dbpath=/home/hanfeng/datafarm/2019/tpch1 --set monet_vault_key=/home/hanfeng/datafarm/2019/tpch1/.vaultkey --set gdk_nr_threads=8 ## terminal 2 (time ./runtest | mclient -d tpch1) &> "log/log_thread_8.log" </code></pre> <p>16 threads</p> <pre><code>## terminal 1 mserver5 --set embedded_py=true --dbpath=/home/hanfeng/datafarm/2019/tpch1 --set monet_vault_key=/home/hanfeng/datafarm/2019/tpch1/.vaultkey --set gdk_nr_threads=16 ## terminal 2 (time ./runtest | mclient -d tpch1) &> "log/log_thread_16.log" </code></pre> <p>32 threads</p> <pre><code>## terminal 1 mserver5 --set embedded_py=true --dbpath=/home/hanfeng/datafarm/2019/tpch1 --set monet_vault_key=/home/hanfeng/datafarm/2019/tpch1/.vaultkey --set gdk_nr_threads=32 ## terminal 2 (time ./runtest | mclient -d tpch1) &> "log/log_thread_32.log" </code></pre> <p>64 threads</p> <pre><code>## terminal 1 mserver5 --set embedded_py=true --dbpath=/home/hanfeng/datafarm/2019/tpch1 --set monet_vault_key=/home/hanfeng/datafarm/2019/tpch1/.vaultkey --set gdk_nr_threads=64 ## terminal 2 (time ./runtest | mclient -d tpch1) &> "log/log_thread_64.log" </code></pre> <p>Post data processing - MonetDB</p> <p>Fetch average execution time (ms)</p> <pre><code>grep -A 3 avg_query log/log_thread_1.log | python cut.py 699.834133333 // q1 85.9178666667 // q4 65.0172 // q6 101.730666667 // q12 58.212 // q14 60.1138666667 // q16 248.926466667 // q19 77.6482 // q22 grep -A 3 avg_query log/log_thread_2.log | python cut.py grep -A 3 avg_query log/log_thread_4.log | python cut.py grep -A 3 avg_query log/log_thread_8.log | python cut.py grep -A 3 avg_query log/log_thread_16.log | python cut.py grep -A 3 avg_query log/log_thread_32.log | python cut.py grep -A 3 avg_query log/log_thread_64.log | python cut.py </code></pre> <p><strong><em>Note</em></strong>: The above numbers can be copied to an Excel file for further analysis before plotting figures. Details can be found in Section 3.</p> <p>Run with HorseIR</p> <p>The HorsePower project can be found on GitHub. In the docker image, it has been placed in <code>/home/hanfeng/cc20/horse</code>.</p> <pre><code>https://github.com/Sable/HorsePower </code></pre> <p>Execution time</p> <p>We then run each query 15 times to get the average execution time (ms).</p> <pre><code>(cd /home/hanfeng/cc20/horse/ && time ./run_all.sh) </code></pre> <p>The script <code>run_all.sh</code> runs over three versions of generated C code based on different levels of optimizations.</p> <pre><code>- naive : no optimization - opt1 : with optimizations - opt2 : with automatic fusion </code></pre> <p>In each version, it first compiles its C code and runs the generated binary with a different number of threads (i.e. 1/2/4/8/16/32/64). Each run computes a query 15 times and returns the average.</p> <p>As a result, all output is saved into a log file, for example, <code>log/naive/log_q6.log</code> contains the result of query 6 in the naive version with all different number of threads.</p> <p>Log file structures</p> <pre><code>log/naive/*.txt log/opt1/*.txt log/opt2/*.txt </code></pre> <p>Fetch a brief summary of execution time from a log file</p> <pre><code>cat log/naive/log_q6.txt | grep -E 'Run with 15 times' q06>> Run with 15 times, last 15 average (ms): 266.638 | 278.999 266.134 266.417 <12 more> # 1 thread q06>> Run with 15 times, last 15 average (ms): 138.556 | 144.474 137.837 137.579 <12 more> # 2 threads q06>> Run with 15 times, last 15 average (ms): 71.8851 | 75.339 72.102 72.341 <12 more> # 4 threads q06>> Run with 15 times, last 15 average (ms): 73.111 | 75.867 72.53 72.936 <12 more> # 8 threads q06>> Run with 15 times, last 15 average (ms): 56.1003 | 59.263 56.057 56.039 <12 more> # 16 threads q06>> Run with 15 times, last 15 average (ms): 56.8858 | 59.466 56.651 57.109 <12 more> # 32 threads q06>> Run with 15 times, last 15 average (ms): 53.4254 | 55.884 54.457 52.878 <12 more> # 64 threads </code></pre> <p>It may become verbose when you have to extract information for all queries over three different kinds of versions. We provide a simple solution for it.</p> <pre><code>./run.sh fetch log | python gen_for_copy.py </code></pre> <p>Output data in the following format</p> <pre><code>// query id | naive | opt1 | opt2 | ----------------------- | ... | ... | ... | # 1 thread | ... | ... | ... | # 2 threads ... ... ... | ... | ... | ... | # 64 threads </code></pre> <p>Note that we copy the generated numbers into an Excel described in Section 3. Within an Excel file, we compare the performance difference in MonetDB and different versions of the generated C code.</p> <p>Compilation time</p> <p>Work directory</p> <pre><code>/home/hanfeng/cc20/horse/codegen </code></pre> <p>Fetch compilation time for different kinds of C code</p> <pre><code>./run.sh compile naive &> log_cc20_compile_naive.txt ./run.sh compile opt1 &> log_cc20_compile_opt1.txt ./run.sh compile opt2 &> log_cc20_compile_opt2.txt </code></pre> <p>Let's look into the result of query 1 in the log file <code>log_cc20_compile_naive.txt</code>.</p> <pre><code>Time variable usr sys wall GGC phase setup : 0.00 ( 0%) 0.00 ( 0%) 0.01 ( 5%) 1266 kB ( 18%) phase parsing : 0.07 ( 54%) 0.07 ( 88%) 0.14 ( 64%) 3897 kB ( 55%) phase opt and generate : 0.06 ( 46%) 0.01 ( 12%) 0.07 ( 32%) 1899 kB ( 27%) dump files : 0.00 ( 0%) 0.00 ( 0%) 0.02 ( 9%) 0 kB ( 0%) df reg dead/unused notes : 0.01 ( 8%) 0.00 ( 0%) 0.00 ( 0%) 31 kB ( 0%) register information : 0.00 ( 0%) 0.00 ( 0%) 0.01 ( 5%) 0 kB ( 0%) preprocessing : 0.03 ( 23%) 0.02 ( 25%) 0.08 ( 36%) 1468 kB ( 21%) lexical analysis : 0.00 ( 0%) 0.03 ( 38%) 0.05 ( 23%) 0 kB ( 0%) parser (global) : 0.04 ( 31%) 0.02 ( 25%) 0.01 ( 5%) 2039 kB ( 29%) tree SSA other : 0.00 ( 0%) 0.01 ( 12%) 0.00 ( 0%) 3 kB ( 0%) integrated RA : 0.01 ( 8%) 0.00 ( 0%) 0.01 ( 5%) 726 kB ( 10%) thread pro- & epilogue : 0.02 ( 15%) 0.00 ( 0%) 0.00 ( 0%) 41 kB ( 1%) shorten branches : 0.00 ( 0%) 0.00 ( 0%) 0.01 ( 5%) 0 kB ( 0%) final : 0.00 ( 0%) 0.00 ( 0%) 0.01 ( 5%) 56 kB ( 1%) initialize rtl : 0.01 ( 8%) 0.00 ( 0%) 0.01 ( 5%) 12 kB ( 0%) rest of compilation : 0.01 ( 8%) 0.00 ( 0%) 0.00 ( 0%) 62 kB ( 1%) TOTAL : 0.13 0.08 0.22 7072 kB </code></pre> <p>The whole compilation time is split into many parts. We take the total wall time as the actual time spent on the code compilation. In this query, it needs 0.22 seconds to complete the whole compilation. (Note that manual work is required for retrieving the compilation time.)</p> <p>3. Results</p> <p>We have a lot of numbers generated by our experiments. We use R and Excel to process these performance numbers and R for plotting figures. There are two kinds of data: (1) the data can be used in R directly, such as the execution time of various versions of C code with different number of threads; and (2) the data needs to pre-processed in Excel before it is sent to R. In order to make it easy for further data analysis, we intentionally provide scripts to make an Excel-friendly format that allows us to copy and paste numbers easily.</p> <p>Work directory</p> <pre><code>/home/hanfeng/cc20/plot </code></pre> <p>For example, the R script for generating figure 10</p> <pre><code>fig10/plot-gmeans.R </code></pre> <p>The Excel file</p> <pre><code>result-book.xlsx </code></pre> <p>Note: We use RStudio Version 1.2.1335 to generate figures from our R scripts. Since RStudio is a GUI-based software, you can install it on another machine with GUI and run scripts from it.</p>
Dataset for paper "Automatically Identifying Archival-worthy, Software-related Slack Conversations"
<p>This dataset consists of 2000 conversations from 5 programming related Q&A channels, hosted on Slack, and accompanies the paper "Automatically Identifying Archival-worthy, Software-related Slack Conversations". In addition to the text of the conversations, each conversation has been annotated as either archival worthy or not. Our definition of archival-worthiness is:</p> <p><em>"If a conversation contains information that could be useful to other users, whether in the Slack channel or elsewhere, then it should be archived. These conversations have no determinate length and no need for objectivity. A conversation should be archived based on the availability and ease of identifying information that could help a person to gain useful software-related knowledge."</em></p> <p><strong>Data Origin: </strong>Numerous public Slack chat channels (<a href="https://slack.com/">https://slack.com/</a>) have recently become available that are focused on specific software engineering-related discussion topics, e.g., Python Development (<a href="https://pyslackers.com/web/slack">https://pyslackers.com/web/slack</a>). The data reflects a portion of the conversations on public channels related to Python, Clojure, Elm and Racket programming.</p> <p><strong>Data Pre-Processing:</strong> To protect privacy, we replace usernames with fake names, and replace absolute times with relative times (in seconds). The conversations are disentangled from the overall chat stream with each unique <em>thread </em>in the dataset specifying a conversation in the channel. Archival-worthy conversations are marked with 1, while non-archival-worthy with 0.</p>
Automatically computed correspondence patterns among six Burmish languages (based on 500 concepts in Huang 1992)
<p>This document is a printout of automatically computed correspondence patterns amount six Burmish languages (Old Burmese, Longchuan Achang, Xiandao, Atsi, Bola, and Maru). It uses as input 500 concepts taken from Huang 1992.</p>
Dataset for "Fmmgen: Automatic Code Generation of Operators for Cartesian Fast Multipole and Barnes-Hut Methods"
<p>This repository provides the dataset, build and run scripts for the paper "Fmmgen: Automatic Code Generation of Cartesian Fast Multipole and Barnes-Hut Operators", by Ryan Alexander Pepper and Hans Fangohr.</p> <p><br> <strong>Organisation</strong><br> The repository is organised as follows:</p> <p><strong><em>sim-scripts</em></strong></p> <p>sim-scripts/ contains the source code, build and run scripts for running the FMM calculations described in the paper. To reproduce the results from the paper, you need as prerequisites:<br> * An installation and license of the Intel Compiler (Parallel Studio 2019 Update 3 was used for the paper).<br> * An installation of the GNU compiler suite.<br> * GNU Make<br> * Python 3.6<br> * A copy of fmmgen v1.0 (available at https://github.com/rpep/fmmgen or https://zenodo.org/record/3842591)<br> * An installation of Fidimag v3.0 (available at http://github.com/computationalmodelling/fidimag or http://dx.doi.org/10.5281/zenodo.3841935)</p> <p>With these prerequisites, simply run from the sim-scripts directory:<br> ```<br> # To build the executables<br> make build<br> # To run the studies<br> make run<br> ```</p> <p>The four scripts are:<br> * run-harmonic-cse-comparison.sh - Runs the Fast Multipole Method for 50000 Coulomb particles, varying the order of expansion, and evaluating the performance benefits of various optimisation strategies introduced in the code generation stage.</p> <p>* run-scaling-comparison.sh - Runs comparisons between the Barnes-Hut and Fast Multipole Methods for different expansion orders and values of theta, the 'opening angle' parameter.</p> <p>* run-error-comparison.sh - Runs the FMM and Barnes-Hut calculations, saving the fields and performing the direct calculation, allowing evaluation of the errors for the two methods.</p> <p>* run-fidimag-tests.sh - Runs the Fidimag scaling tests for a series of magnetic dipoles placed on a lattice.</p> <p><strong><em>results</em></strong></p> <p>results contains the output data from the sim-scripts scripts. Note: running the scripts will overwrite this data!</p> <p><br> <strong><em>figure-scripts</em></strong></p> <p>This contains Python scripts needed to reproduce the figures from the paper. These generated figures are included in the repository for convenience. To run these scripts, you require:</p> <p>* Python >= 3.6<br> * Matplotlib >= 3.1.1<br> * Seaborn >= 0.9.1<br> * NumPy >= 1.17.1</p> <p><strong><em>figures</em></strong></p> <p>This contains the output figures included in the paper.<br> </p>
Replication package of "Revisiting Test Smells in Automatically Generated Tests: Limitations, Pitfalls, and Opportunities"
<p><strong>Abstract:</strong><br> Test smells attempt to capture design issues in test code that reduce their maintainability. Previous work found such smells to be highly common in automatically generated test-cases, but based this result on specific static detection rules; although these are based on the original definition of “test smells”, a recent empirical study showed that developers perceive these as overly strict and non-representative of the maintainability and quality of test suites. This leads us to investigate how effective such test smell detection tools are on automatically generated test suites. In this paper, we build a dataset of 2,340 test cases automatically generated by EVOSUITE for 100 Java classes. We performed a multi-stage, cross-validated manual analysis to identify six types of test smells and label their instances. We benchmark the performance of two test smell detection tools: one widely used in prior work, and one recently introduced with the express goal to match developer perceptions of test smells. Our results show that these test smell detection strategies poorly characterized the issues in automatically generated test suites; the older tool’s detection strategies, especially, misclassified over 70% of test smells, both missing real instances (false negatives) and marking many smell-free tests as smelly (false positives). We identify common patterns in these tests that can be used to improve the tools, refine and update the definition of certain test smells, and highlight as of yet uncharacterized issues. Our findings suggest the need for (i) more appropriate metrics to match development practice; and (ii) more accurate detection strategies, to be evaluated primarily in industrial contexts.</p>
Automatic plankton image classification - can capsules and filters help coping with data set shift?
<p>This data set is related to the article 'Automatic plankton image classification - can capsules and filters help coping with data set shift?' published in 'Limnology and Oceanography: Methods' by Plonus <em>et al.</em> (2021).</p> <p>The images belong to the trainings set used to train the models in the aforementioned paper (training_) and three different additional data sets which were used to evaluate the performance of the trained models in application mode (fs446_; fs466_; fs534_). The Python-Script 'separate_files.py' can be used to move all the images in different folders for each data set and class respectively.</p>
A Small Dataset of Jazz Guitar Licks for Automatic Transcription Experiments
<p>A dataset consisting of ten (mostly monophonic) jazz guitar licks that were performed by François Pachet and recorded at the Sony Computer Science Laboratory (CSL) in Paris.</p>
Validation data set for automatic blood vessel segmentation in colorectal cancer histology (IHC)
<p><strong>Content</strong></p> <p>This data set contains 100 histological image patches of 1000 * 1000 px size. The samples were immunostained for CD34 (3,3'-Diaminobenzidine, DAB [brown]) with hematoxylin (blue) counterstain.</p> <p>Furthermore, the data set contains a table of blood vessel counts in each image by three blinded observers as well as an automatic count with a method based on the following paper:</p> <p>Kather, Jakob Nikolas et al. "Continuous Representation Of Tumor Microvessel Density And Detection Of Angiogenic Hotspots In Histological Whole-Slide Images". <em>Oncotarget</em> 6.22 (2015): 19163-19176. http://dx.doi.org/10.18632/oncotarget.4383</p> <p><strong>Image format</strong></p> <p>All images are RGB, 0.50 µm per pixel, digitized with an Aperio ScanScope (Aperio/Leica biosystems), magnification 20x. Histological samples are fully anonymized images of formalin-fixed paraffin-embedded human colorectal adenocarcinomas (primary tumors and liver metastases) from our pathology archive (Institute of Pathology, University Medical Center Mannheim, Heidelberg University, Mannheim, Germany).</p> <p><strong>Ethics statement</strong></p> <p>All experiments were approved by the institutional ethics board (medical ethics board II, University Medical Center Mannheim, Heidelberg University, Germany; approval 2015-868R-MA). The institutional ethics board waived the need for informed consent for this retrospective analysis of anonymized samples. All experiments were carried out in accordance with the Declaration of Helsinki.</p> <p><strong>Contact</strong></p> <p>For questions, please contact:<br> Dr. Jakob Nikolas Kather<br> http://orcid.org/0000-0002-3730-5348<br> ResearcherID: D-4279-2015</p>
The FAIR-Device - a non-lethal and generalist semi-automatic Malaise trap for insect biodiversity monitoring: Proof of concept - Supplementary Material
<h3>Abstract</h3> <p>Field monitoring plays a crucial role in understanding insect dynamics within ecosystems. It facilitates pest distribution assessment, control measure evaluation, and prediction of pest outbreaks. Additionally, it provides important information on bioindicators with which the state of biodiversity and ecological integrity in specific habitats and ecosystems can be accurately assessed. However, traditional monitoring systems can present various difficulties, leading to a limited temporal and spatial resolution of the obtained information. Despite recent advancements in automatic insect monitoring traps, also called e-traps, most of these systems focus exclusively on studying agricultural pests, rendering them unsuitable for monitoring diverse insect populations. To address this issue, we introduce the Field Automatic Insect Recognition (FAIR)-Device, a novel non-lethal field tool that relies on semi-automatic image capture and species identification using artificial intelligence via the iNaturalist platform. Our objective was to develop an automatic, cost-effective, and non-specific monitoring solution capable of providing high-resolution data for assessing insect diversity. During a 26-day proof-of-concept evaluation, the FAIR-Device recorded 24.8 GB of video, identifying 431 individuals from 9 orders, 50 families, and 69 genera. While improvements are possible, our device demonstrated potential as a cost-effective, non-lethal tool for monitoring insect biodiversity. Looking ahead, we envision new monitoring systems such as e-traps as valuable tools for real-time insect monitoring, offering unprecedented insights for ecological research and agricultural practices.</p> <h3>Description of the data and file structure</h3> <p>This repository complements the publication <a href="https://www.biorxiv.org/content/10.1101/2024.03.22.586299v2" target="_blank" rel="noopener">"The FAIR-Device - a non-lethal and generalist semi-automatic Malaise trap for insect biodiversity monitoring: Proof of concept".</a> It contains the result data from the proof of concept field test of V1.0 of the FAIR-Device, conducted between July and August 2021 at the Thünen Institute of Agricultural Technology in Braunschweig. The repository comprises three compressed files (.zip):</p> <ul> <li><strong>FAIR-D_captures.zip</strong>: <ul> <li>Video captures from the field tests organized by recording day.</li> <li>Filenames indicating recording time (hh-mm-ss).</li> </ul> </li> </ul> <ul> <li><strong>FAIR-D_Tables&Code.zip</strong>: <ul> <li>Processed results from the obtained image captures, organized into: <ul> <li><strong>Monitoring2021_TotalPeriod.xlsx: </strong>Result table with taxonomic classifications, data analysis, and charts.</li> <li><strong>iNat_observations.xlsx: </strong>iNaturalist reviews analysis table.</li> <li><strong>R: </strong>code for generating article graphics.</li> </ul> </li> </ul> </li> </ul> <ul> <li><strong>FAIR-D_V1.0_3D_Models.zip</strong>: <ul> <li>Complete 3D design of the FAIR-D V1.0 in .stl format, organized into: <ul> <li><strong>3D_print_parts</strong>: 3D-printable parts with spatial coordinates for correct positioning in CAD software.</li> <li><strong>other_3Dparts</strong>: Non-printable parts, also in .stl format with spatial coordinates.</li> </ul> </li> </ul> </li> </ul> <p> </p> <p>NOTE: For visualizing the videos, we recommend <strong>VLC media player</strong> - <a href="https://www.videolan.org/vlc/">https://www.videolan.org/vlc/</a> </p>
Dataset related to article "Phantom‑based analysis of variations in automatic exposure control across three mammography systems: implications for radiation dose and image quality in mammography, DBT, and CEM"
<p>The dataset comprises information from several DICOM tags extracted from digital mammography (DM), digital breast tomosynthesis (DBT), and contrast-enhanced mammography (CEM) images acquired in a phantom study aimed at characterizing the automatic exposure control (AEC) behavior of diverse mammography equipment. The final ten columns of the datasets encompass signal (mena pixel values, MPV) and noise (standard deviation, SD) measurements derived from phantom images. These measurements are used to compute several image quality metrics, including contrast, signal-to-noise ratio (SNR), contrast-to-noise ratio (CNR), CNR relative difference in comparison to the 45 mm reference thickness, and a figure of merit (FOM) obtained by diving the squared CNR by the mean glandular dose (MGD).</p>
RafanoSet: Dataset of raw, manual and automatically annotated Raphanus Raphanistrum weed images for object detection and segmentation in Heterogenous Agriculture Environment
<p>This dataset is a collection of raw and annotated Multispectral (MS) images acquired in a heterogenous agricultural environment with MicaSense RedEdge-M camera. The spectra particularly Green, Blue, Red, Red Edge and Near Infrared (NIR) were acquired at sub-metre level.. <br><br>The MS images were labelled manually using VIA and automatically using Grounding DINO in combination with Segment Anything Model. The segmentation masks obtained using these two annotation techniqes over as well as the source code to perform necessary image processing operations are provided in the repository. The images are focussed over Horseradish (Raphanus Raphanistrum) infestations in Triticum Aestivum (wheat) crops.</p> <p>The nomenclature of sequecncing and naming images and annotations has been in this format: IMG_<scene number>_<spectral channel number><br><strong>_1</strong>: Blue<br><strong>_2</strong>: Green<br><strong>_3</strong>: Red<br><strong>_4</strong>: Near Infrared<br><strong>_5</strong>: RedEdge<br><br>Example: An image name <strong>IMG_0200_3 </strong>represents the scene number<strong> 200</strong> in <strong>Red channel</strong></p> <p>This dataset 'RafanoSet'is categorized in 6 directories namely 'Raw Images', 'Manual Annotations', 'Automated Annotations', 'Binary Masks - Manual', 'Binary Masks - Automated' and 'Codes'. The sub-directory 'Raw Images' consists of manually acquired 85 images in .PNG format. over 17 different scenes. The sub-directory 'Manual Annotations' consists of annotation file 'region_data' in COCO segmentation format. The sub-directory 'Automated Annotations' consists of 80 automatically annotated images in .JPG format and 80 .XML files in Pascal VOC annotation format.</p> <p>The scientific framework of image acquisition and annotations are explained in the Data in Brief paper which is the course of peer review. This is just a prerequisite to the data article. <br><br>Field experimentation roles:</p> <p>The image acquisition was performed by Mariano Crimaldi, a researcher, on behalf of Department of Agriculture and the hosting institution University of Naples Federico II, Italy.</p> <p>Shubham Rana has been the curator and analyst for the data under the supervision of his PhD supervisor Prof. Salvatore Gerbino. They are affiliated with Department of Engineering, University of Campania 'Luigi Vanvitelli'. </p> <p>Domenico Barretta, Department of Engineering has been associated in consulting and brainstorming role particularly with data validation, annotation management and litmus testing of the datasets.</p>
Automatic monitoring of neural activity with single-cell resolution in behaving Hydra
<p>The ability to record every spike from every neuron in a behaving animal is one of the holy grails of neuroscience. Here, we report coming one step closer towards this goal with the development of an end-to-end pipeline that automatically tracks and extracts calcium signals from individual neurons in the cnidarian <em>Hydra vulgaris</em>. We imaged dually labeled (nuclear tdTomato and cytoplasmic GCaMP7s) transgenic <em>Hydra </em>and developed an open-source Python platform (TraSE-IN) for the Tracking and Spike Estimation of Individual Neurons in the animal during behavior. The TraSE-IN platform comprises a series of modules that segments and tracks each nucleus over time and extracts the corresponding calcium activity in the GCaMP channel. Another series of signal processing modules allows robust prediction of individual spikes from each neuron's calcium signal. This complete pipeline will facilitate the automatic generation and analysis of large-scale datasets of single-cell resolution neural activity in <em>Hydra</em>, and potentially other model organisms, paving the way towards deciphering the neural code of an entire animal.</p>
Automatic detection of clefts and corridors within the sandstone plateau - Szczeliniec Wielki & Szczeliniec Mały mesas, Poland
<p>This dataset presents the method of automatic clefts and corridors detection within areas of highly dissected relief. This methodical approach was developed for a geomorphological study on Szczeliniec Wielki and Szczeliniec Mały sandstone mesas (Stołowe Mts., SW Poland) (Migoń et al. 2023)</p> <p>Contents:</p> <ol> <li>CleftHunter_manual_v1_0.pdf - short method description</li> <li>kernel_examples.zip - set of exemplary kernel files </li> <li>Szczeliniec_Wielki_clefts_threshold_depth_2m.zip - raster dataset of automatically detected clefts within the plateau of Szczeliniec Wielki (.tif)</li> <li>Szczeliniec_Maly_clefts_threshold_depth_2m.zip - raster dataset of automatically detected clefts within the plateau of Szczeliniec Mały (.tif)</li> <li>Szczeliniec_Wielki_mesa_caprock_base.zip - Szczeliniec Wielki caprock zone (.shp, polygon)</li> <li>Szczeliniec_Maly_mesa_caprock_base.zip - Szczeliniec Mały caprock zone (.shp, polygon)</li> <li>Szczeliniec_Wielki_mesa_caprock_hillshade.zip - shaded relief of the Szczeliniec Wielki plateau (.tif)</li> <li>Szczeliniec_Maly_mesa_caprock_hillshade.zip - shaded relief of the Szczeliniec Mały plateau (.tif)</li> </ol> <p>Preferred citation:</p> <p>Migoń P., Duszyński F., Jancewicz K., Kotowska M., Porębna W. (2023), Surface-subsurface connectivity in the morphological evolution of sandstone-capped tabular hills – how much analogy to karst?. Geomorphology vol. 440, Id 108884, 1–22<br>DOI: 10.1016/j.geomorph.2023.108884</p> <p> </p> <p>This research was funded by National Science Centre, Poland, research project no. 2020/39/D/ST10/00861.</p>
Automatic Classification of Final Assignments at the Nuclear Polytechnic Library
<p>This study aimed to look for a method to automatically classify the final projects of Indonesian Nuclear Technology Polytechnic students.</p>
Automatic message sequence chart creation from simulation run of the parametric colored Petri net model of the Chandy-Lamport algorithm with four processes
<p><span>The video shows the creation of the message sequence chart from a simulation run of the proposed parametric colored Petri net model of the Chandy-Lamport algorithm using the CPN tool with four constituting processes. The picture shows the resulting message sequence chart. </span></p> <p><strong><span>Message Sequence Chart of Parametric Model With 4 Processes via Automatic Simulation Run_SuppInfo.mp4</span></strong><span>: This video shows the automatic generation of a message sequence chart of the proposed parametric colored Petri net model of the Chandy-Lamport distributed global snapshot algorithm using the CPN tool version 4.0.0. The model's number of constituting processes is parametric and was set to four. The video was generated using the authors' updated CPN tool extension server. The automatic simulation run of the model has been used to create this video. The CPN tool randomly selects the enabled transition at each step in an automatic simulation run.</span></p> <p><strong><span>Picture of Message Sequence Chart of Parametric Model With 4 Processes_SuppInfo.png:</span></strong><span> This picture shows the automatically generated message sequence chart of the proposed parametric colored Petri net model of the Chandy-Lamport algorithm that is visible in the above video clip. The number of constituting processes was set to four. </span></p>
US Automatic Number Plate Recognition Market 2024 To 2033
<p><a href="https://www.custommarketinsights.com/report/us-automatic-number-plate-recognition-market/" target="_blank" rel="noopener">US Automatic Number Plate Recognition Market Size</a>, Trends and Insights By Application (Traffic Management, Toll Collection, Law Enforcement, Parking Management), By Product Type (Fixed ANPR Systems, Mobile ANPR Systems, Portable ANPR Systems), By End User (Government Agencies, Law Enforcement, Toll Operators, Parking Management Firms), and By Region - Industry Overview, Statistical Data, Competitive Analysis, Share, Outlook, and Forecast 2024–2033</p> <p><strong>Reports Description</strong></p> <p>As per the current market research conducted by the CMI Team, the <a href="https://www.custommarketinsights.com/report/us-automatic-number-plate-recognition-market/"><strong>US Automatic Number Plate Recognition Market</strong></a> is expected to record a CAGR of <strong>9.06%</strong> from 2024 to 2033. In 2024, the market size is projected to reach a valuation of USD <strong>3.19 Billion</strong>. By 2033, the valuation is anticipated to reach USD 6.93 billion<strong>.</strong></p> <p>This chart shows the count of images and instances trained and validated for car and license plate detection that form a part of ANPR systems in the US marketplace. Shares images were analyzed: 183 images of the training subset and 20 images of the validation subset. In the case of car detection, there were 819 instances 740 were designated for training and 79 for validation.</p> <p>Moreover, 246 license plate instance samples were used, with 227 in the training and 19 in the validation sets. These datasets illustrate the tremendous amount of work and precision involved in the ANPR technology because of the increasing use of such systems for managing traffic issues, enforcing laws and collecting tolls on highways across the United States. This extensive data training emphasizes the accuracy and sharpness of the ANPR systems that authorities provide to the market by definition of vehicle identification and control.</p> <p>DOWNLOAD FREE SAMPLE Now at <a href="https://www.custommarketinsights.com/request-for-free-sample/?reportid=58799" target="_blank" rel="noopener">https://www.custommarketinsights.com/request-for-free-sample/?reportid=58799</a></p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.