Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
9,302
datasets available to search
ShareScore release 0.9.0
Dataset results
9,302 results for “DETECT”
Satellite-based shoreline detection: macrotidal high-energy coasts dataset
<p>This dataset accompanies the article by Konstantinou <em>et al.</em> (2022) titled ‘Satellite-based shoreline detection: macrotidal high-energy coasts’. This study assesses the ability of existing satellite image analysis technology to capture shoreline position change at relevant magnitudes and timescales for two different coastal environments in the United Kingdom. It addresses the influence of tidal elevation and wave-induced water-level fluctuations at two sites representing end members of beach morphological type in a region of low satellite useability (high cloud cover combined with low image availability). The study uses 14 years of monthly topographic surveys at two macrotidal sites in the UK, combined with modelled wave data and harmonic tidal predictions to investigate the influence of tidal elevation and wave action on SDS accuracy.</p> <p><strong>Description</strong></p> <p>The dataset consists of two matlab files each containing the information listed below for the two test sites (Slapton Sands (SLSdata); Perranporth (PPTdata)):</p> <ul> <li>dnum: date of satellite image capture in matlab datenum format</li> <li>sat: name of satellite (L5=Landsat 5; L7=Landsat 7; L8=Landsat 8; S2=Sentinel-2)</li> <li>transects: a structure containing the following variables:</li> </ul> <ul> <li>profName – the name of the survey profile</li> <li>startEast; startNorth; endEast; endNorth: OSGB coordinates of the start (landward) and end (seaward) of the transect line</li> <li>startUTMlat; startUTMlon; endUTMlat; endUTMlon: WGS84-UTM30 coordinates of the start (landward) and end (seaward) of the transect line</li> <li>orientation: profile orientation</li> <li>shoreOrient: shoreline orientation</li> <li>transAngle: profile angle to shore-normal.</li> </ul> <ul> <li>transTimeSeries: time series of the intersection of the SDW at each transect in three columns that include chainage (m), longitude, latitude.</li> <li>profData: a nested structure containing the survey data at organised by profile including survey date, OSGB and WGS84-UTM30 coordinates of surveyed points.</li> </ul>
The Potential of UAV Imagery for the Detection of Rapid Permafrost Degradation: Assessing the Impacts on Critical Arctic Infrastructure
<p>Dataset and Python code complementing the publication </p> <p>Kaiser, S.; Boike, J.; Grosse, G.; Langer, M. The Potential of UAV Imagery for the Detection of Rapid Permafrost Degradation: Assessing the Impacts on Critical Arctic Infrastructure. <em>Remote Sens.</em> <strong>2022</strong>, <em>14</em>, 6107. https://doi.org/10.3390/rs14236107</p> <ul> <li><strong>AROSICS.zip</strong> contains the orthomosaic of 2018 shifted to 2019 with the AROSICS algorithm. The .txt file contains the x-/y-shift in map units [m].</li> <li><strong>CC_DistancePointClouds.zip</strong> contains the distance point clouds as calculated via Multiscale Model to Model Comparison (M3C2 after Lague et. al, 2013) at each post-processing level (I-IV) and the validation.</li> <li><strong>CC_PointCloudProcessing.zip</strong> contains the point clouds at post-processing levels II-IV.</li> <li><strong>ODM_Orthomosaics.zip</strong> contains the orthomosaics of 2018 and 2019 as processed in WebODM (based on OpenDroneMap).</li> <li><strong>ODM_PointClouds.zip</strong> contains the raw point clouds of 2018 and 2019 (post-processing level I) as processed in WebODM (based on OpenDroneMap).</li> <li><strong>PointCloudStatistics.zip</strong> contains the M3C2 distance statistics at each post-processing level (I-IV) and the validation for the whole point cloud and the two subsets.</li> <li><strong>Python_ChangeDetection.zip</strong> contains the Python (v 3.6) script for calculating the displacement vectors Dx, Dy, Dz for each distance point cloud, rasterizing the attribute "vertical displacement (Dz)" of the distance point cloud with the highest accuracy (post-processing level IV), applying a Sobel edge detection filter to highlight high image gradients and clustering the image into two categories: change (high image gradient) and no change (low image gradient). Needed data input is <strong>CC_DistancePointClouds.zip.</strong></li> <li><strong>Subsets.zip </strong>contains shapefiles of the two subsets.</li> </ul>
Benchmark datasets for detection and identification of insects from camera trap images with deep learning
<p><strong>Insect benchmark datasets for training, validation and test (train1201.zip, val1201.zip and test1201.zip) with time-lapse images as described in paper:</strong></p> <p><a href="https://www.biorxiv.org/content/10.1101/2022.10.25.513484v1">Bjerge K, Alison J, Dyrmann M, Frigaard C.E., Mann H. M. R., Høye T.T., Accurate detection and identification of insects from camera trap images with deep learning, bioRxiv:10.1101/2022.10.25.513484v1</a></p> <p>Labels in <strong>YOLO format: <a href="https://github.com/ultralytics/yolov5/issues/2293">ultralytics/yolov5: label format</a></strong></p> <p>The annotated training and validation datasets contains insects of nine different species as listed below:</p> <table> <tbody> <tr> <td>0 <em>Coccinellidae septempunctata</em></td> </tr> <tr> <td>1 <em>Apis mellifera</em></td> </tr> <tr> <td>2 <em>Bombus lapidarius</em></td> </tr> <tr> <td>3 <em>Bombus terrestris</em></td> </tr> <tr> <td>4 <em>Eupeodes corolla</em></td> </tr> <tr> <td>5 <em>Episyrphus balteatus</em></td> </tr> <tr> <td>6 <em>Aglais urticae</em></td> </tr> <tr> <td>7 <em>Vespula vulgaris</em></td> </tr> <tr> <td>8 <em>Eristalis tenax</em></td> </tr> </tbody> </table> <p>The test dataset contains additional classes of insects.</p> <table> <tbody> <tr> <td>9 Non-Bombus Anthophila</td> </tr> <tr> <td>10 Bombus spp.</td> </tr> <tr> <td>11 Syrphidae</td> </tr> <tr> <td>12 Fly spp.</td> </tr> <tr> <td>13 Unclear insect</td> </tr> <tr> <td>14 Mixed animals:<br> ——————————<br> Rhopalocera<br> Non-Anthophila Hymenoptera<br> Non-Syrphidae Diptera<br> Non-Conccinalidae Coleoptera<br> Concinellidae<br> Other animals</td> </tr> </tbody> </table> <p><strong>There are two naming conventions for image (.jpg) and label (.txt) files.</strong></p> <p><em>Background images without insects are named</em>:<br> “<strong>X_Seq-YYYYMMDDHHMMSS</strong>-snapshot”.<br> E.g.:<br> Background image: 12_13-20190704172200-snapshot.jpg<br> Empty label file: 12_13-20190704172200-snapshot.txt</p> <p><em>Images annotated with insects are named:</em><br> “<strong>SZ_IP-MonthDate_C_Seq-YYYYMMDDHHMMSS</strong>”.<br> E.g.:<br> Image file: S1_146-Aug23_1_156-20190822133230.jpg<br> Label file: S1_146-Aug23_1_156-20190822133230.txt</p> <p><strong>Abbreviations</strong>:</p> <p><strong>YYYYMMDDHHMMSS </strong>– Capture timestamp with year, month, date, hour, minutes, and second<br> <strong>Seq</strong> – Sequence number created by the motion program to separate images<br> <strong>C</strong> – Identification of two cameras with Id=0 or Id=1 in system identified by <strong>SZ_IP</strong><br> <strong>MonthDate </strong>– Folder name for where the original image were stored in the system<br> <strong>SZ_IP</strong> – Identification of five camera systems: S1_123, S2_146, S3_194, S4_199, S5_187 (Two cameras in each system)<br> <strong>X</strong> – An index number related to a specific camera and folder ensuring unique file names of background images from different camera systems.<br> <br> The important information in a filename is system (<strong>SZ_IP</strong>), camera Id (<strong>C</strong>) and timestamp (<strong>YYYYMMDDHHMMSS</strong>).</p> <p><strong>The three best YOLOv5 models (YOLOv5models.zip) from the paper are available in pytorch format.</strong></p> <p>All models are tested with YOLOv5 release v7.0 (22-11-2022): <a href="https://github.com/ultralytics/yolov5">ultralytics/yolov5: YOLOv5 in PyTorch</a></p> <p><strong>insect1201-bestF1-640v5m.pt</strong>: Model no. 6 in Table 2 (F1=0.912)<br> <strong>insect1201-bestF1-1280v5m6.pt</strong>: Model no. 8 in Table 2 (F1=0.925)<br> <strong>insect1201-bestF1-1280v5m6.pt</strong>: Model no. 10 in Table 2 (F1=0.932)</p> <p><strong>insects-1201val.yaml</strong>: YAML file with label names to train YOLOv5</p> <p><strong>trainInsects-1201m.sh</strong>: Linux bash shell script with parameters to train YOLOv5m6<br> <strong>valInsectsF1-1201.sh</strong>: Linux bash shell script with parameters to validated models</p> <p> </p>
CSEM-MISD - CSEM's Multi-Illumination Surface Defect Detection Dataset
<p>In automated surface visual inspection, it is often necessary to capture the inspected part under many different illumination conditions to capture all the defects. To address this issue, at <a href="http://www.csem.ch/">CSEM</a> we have acquired a real-world multi-illumination defect segmentation dataset, called CSEM-MISD and we release it for research purposes to benefit the community.</p> <p>The dataset consists of three different types of metallic parts -- washers, screws, and gears. Parts were captured in a half-spherical <a href="https://register.epo.org/application?number=EP13197867">light-dome system</a> that filtered out all the ambient light and successively illuminated it from 108 distinct illumination angles. Each 12 illumination angles share the same elevation level and the relative azimuthal difference between the adjacent illumination angles on the same level is 30 degrees. For more details, please read Sections 3 and 4 of our paper.</p> <p>The washers dataset features 70 defective parts. The gears and screws datasets feature 35 defective, 35 intact and several hundred unannotated parts. Some defects, such as notches and holes, are visible in most images (illuminations) with intensity and texture variations among them, while others, such as scratches, are only visible in a few.</p> <p>We split the datasets into train and test sets. The train sets contain 32 samples, and the test set 38 samples. Each sample comprises 108 images (each captured under a different illumination angle), an automatically extracted foreground segmentation mask, and a hand-labeled defect segmentation mask.</p> <p>This dataset is challenging mainly because:</p> <ul> <li>each raw sample consists of 108 gray-scale images of resolution 512×512 and therefore takes 27MB of space;</li> <li>the metallic surfaces produce many specular reflections that sometimes saturate the camera sensors;</li> <li>the annotations are not very precise because the exact extent of defect contours is always subjective;</li> <li>the defects are very sparse also in the spatial dimensions: they cover only about 0.2% of the total image area in gears, 0.8% in screws, and 1.4% in washers; this creates an unbalanced dataset with a highly skewed class representation. </li> </ul> <p>The dataset is organized as follows:</p> <ul> <li>each sample resides in the Test, Train, or Unannotated directory;</li> <li>each sample has its own directory which contains the individual images, the foreground, and defect segmentation masks;</li> <li>each image is stored in 8-bit greyscale png format and has a resolution of 512 x 512 pixels;</li> <li>Image file names are formatted using three string fields separated with the underscore character: prefix_sampleNr_illuminationNr.png, where the prefix is e.g. washer, the sampleNr might be a three-digit number 001, and the illuminationNr is formed of 3 digits, first corresponding to the elevation index (1 - highest angle, 9 - lowest angle), and the additional two corresponding to the azimuth index (01-12).</li> <li>Each dataset contains light_vectors.csv, which contains the illumination angles (in lexicographic order of the illuminationNr), and light_intensities.csv that contains the numbers corresponding to the light intensity on the scale from 0 to 127. Please, be aware, that the azimuth angles were not calibrated and might be a few degrees misaligned.</li> </ul> <p>We provide data loaders implemented in python at the project's <a href="https://github.com/DawyD/illumination-preserving-rotations">repository</a>.</p> <p>If you find our dataset useful, please cite our paper:</p> <blockquote> <p>Honzátko, D., Türetken, E., Bigdeli, S. A., Dunbar, L. A., & Fua, P. (2021). Defect segmentation for multi-illumination quality control systems. <em>Machine vision and Applications</em>.</p> </blockquote>
Additional files of scTensor paper "scTensor detects many-to-many cell-cell interactions from single cell RNA-sequencing data"
<p>Complex biological systems are described as a multitude of cell-cell interactions (CCIs). Recent single-cell RNA-sequencing studies focus on CCIs based on ligand-receptor (L-R) gene co-expression. However, the analytical methods are still not mature; such methods cannot detect CCIs and the related L-R pairs simultaneously or also are not appropriate to detect many-to-many CCIs.</p> <p>In this work, we propose scTensor, a novel method for extracting representative triadic relationships (or hypergraphs), which include ligand-expression, receptor-expression, and related L-R pairs. Through extensive studies with simulated and empirical datasets, we have shown that scTensor could detect some hypergraphs, which cannot be detected by conventional methods, especially when those CCIs are many-to-many relationships.</p>
ARTIMATION Conflict detection and resolution visualisation questionnaire results
<p>This dataset reports the results of the questionnaires (post-condition and post-scenario) that were applied for the human performance assessment of the conflict resolution visualisation tool human in the loop (HITL) simulation of ARTIMATION project.</p>
Deep Geo Stat WP3 Land Use Change Detection Dataset
<p>This dataset is created as part of work package WP3 of the Deep Geo Stat project, as part of the ESS topic B5674-2020-GEOS (project 101033951: 2020-NL-GEOS-DEEP-GEO-STAT). The objective was to research whether a Siamese Convolutional Neural Network (SCNN) could be used to automatically detect changes in land use.</p> <p>At Statistics Netherlands, every few years, the so-called "Bestand Bodemgebruik" (BBG) is created, which is a file containing information about the land use of the Netherlands for a given year. The country is split up into many polygons, whereby each polygon is labelled with the most common type of land use for that specific area. Creating the BBG is a time-consuming process and the assumption is that it can be speed up if we can automatically detect the changes in land use. During the research, SCCNs were created to decide the changes in land use for three classes. The SCCN models were then applied to the whole of The Netherlands. The results were then converted to GIS files.</p> <p>This dataset contains the GIS files for the classes 34 (building sites), 51 (other agricultural), and 60 (forest). These files can be loaded in QGIS, for example. Each polygon contains the prediction as it rolls out of the model to which we add the label (0: changed, 1: unchanged). Predictions are made on polygons that belonged to the specific class in 2017 and the new label is the prediction for 2018 and 2020.</p>
Hourly detections of echolocation clicks in Hawaiian Island HARP data from Hawai`i, Kaua`i, and Manawai with species labels
<p>This dataset consists of counts of detections of echolocation clicks at three sites in the Hawaiian Islands Archipelago. These sites are Hawaii, Kauai, and Manawai (also known as Pearl and Hermes Reef). Echolocation clicks have been labeled using a neural network classifier that was trained and tested on data from the Hawaiian Islands and can successfully identify many species of regionally present odontocetes. During the labeling process, clicks were grouped into one-minute bins and each bin was given a species' label. The data provided here is further binned at an hourly level, where counts of a given species represent the number of one-minute bins within a given hour that were labeled as that species (up to a maximum of 60). One file is provided per site, and files are in .csv format that can be read using any desired coding language.<span> </span></p>
The PS-Battles Dataset - an Image Collection for Image Manipulation Detection
<p>The boost of available digital media has led to a significant increase in derivative work. With tools for manipulating objects becoming more and more mature, it can be very difficult to determine whether one piece of media was derived from another one or tampered with. As derivations can be done with malicious intent, there is an urgent need for reliable and easily usable tampering detection methods. However, even media considered semantically untampered by humans might have already undergone compression steps or light post-processing, making automated detection of tampering susceptible to false positives. In this paper, we present the PS-Battles dataset which is gathered from a large community of image manipulation enthusiasts and provides a basis for media derivation and manipulation detection in the visual domain. The dataset consists of 102'028 images grouped into 11'142 subsets, each containing the original image as well as a varying number of manipulated derivatives.</p> <p>Mirror of the git repository: <a href="https://github.com/dbisUnibas/PS-Battles">https://github.com/dbisUnibas/PS-Battles</a></p> <p>Paper on arxiv: <a href="https://arxiv.org/abs/1804.04866">https://arxiv.org/abs/1804.04866</a></p>
Image Dataset for Object Detection of Small Size Construction Tools
<p> This is an image dataset established as input data for object detection model of small-sized construction tools. In the dataset, there are 12 classes of target tools (bucket, cutter, drill, grinder, hammer, knife, saw, shovel, spanner, tacker, trowel, and wrench) which are typically used at indoor construction sites. 25,084 sets of image and the corresponding label data have been established and shared. </p> <p> The diversity of objects in the images of the 12 small tools was considered by photographing tools of various shapes, sizes, and colors. In addition, to improve the model performance, images were also captured with various changes (e.g., image resolution, occlusion, lighting, and background). Among the 25,084 images in the dataset, 6,258 (25%) were obtained from the actual construction site. </p> <p> Object annotations in each image were done by bounding boxes and were saved into a text file. The coordinates of the bounding box have the form of (Class, Center X, Center Y, Width, Height). Class refers to one of 12 construction tool types. Center X and Center Y are the center coordinates of the bounding box for an object from an image when the resolution of the image has min-max normalized. Width and Height are the width and height of the bounding box for an object, respectively, also from the image with the min-max normalized resolution.</p> <p> </p> <p>The peer-reviewed publication for this dataset has now been published in " KSCE Journal of Civil Engineering" a Springer journal as follows:</p> <p><strong>* Lee, K., Jeon, C., and Shin, D. (2023, In press) "Small Tool Image Database and Object Detection Approach for Indoor Construction Site Safety" <em>KSCE Journal of Civil Engineering</em>. DOI: https://doi.org/10.1007/s12205-022-1011-7</strong></p> <p>Please cite this reference when using the dataset.</p>
Datasets of the manuscript "Rational design of profile HMMs for sensitive and specific sequence detection with case studies applied to viruses, bacteriophages, and casposons"
<p><strong>DATASETS</strong></p> <p>Rational design of profile HMMs for sensitive and specific sequence detection with case studies applied to viruses, bacteriophages, and casposons</p> <p>Liliane S. Oliveira, Alejandro Reyes, Bas E. Dutilh and Arthur Gruber<sup>*</sup></p> <p>* Correspondence: <a href="mailto:argruber@usp.br">argruber@usp.br</a> (AG); Tel. +55 11 3091 7274</p> <p> </p> <p>Here we provide different data of <em>Microviridae</em>, <em>Flavivirus</em> and casposons used throughout the work:</p> <ul> <li>Microviridae folder <ul> <li>conserved_HMMs – profile HMMs constructed with TABAJARA in Conservation mode for <em>Microviridae</em></li> <li>discriminative_HMMs – profile HMMs constructed with TABAJARA in Discrimination mode for <em>Microviridae</em></li> <li>sequences – different sequence datasets and respective multiple sequence alignments <ul> <li>Microviridae_113-seq_training_set.fasta - 113 VP1 sequences covering diversity of the <em>Microviridae</em> family</li> <li>Microviridae_113-seq.aln – multiple sequence alignment of the 113-protein dataset</li> <li>Microviridae_1836-seq_testset.fasta - 1,836 sequence dataset covering 1,836 sequences of the major capsid protein (VP1) comprising 501 <em>Alpavirinae</em> sequences, 1,040 <em>Gokushovirinae</em> sequences and 295 <em>Pichovirinae</em> sequences</li> <li>Microviridae_1866-seq.aln - multiple sequence alignment of the 1,866-protein <em>Microviridae</em> dataset used in the experiment of Figure 4</li> </ul> </li> </ul> </li> <li>Flavivirus folder <ul> <li>conserved_HMMs – profile HMMs constructed with TABAJARA in Conservation mode for <em>Flavivirus</em></li> <li>discriminative_HMMs – profile HMMs constructed with TABAJARA in Discrimination mode for <em>Flavivirus</em> <ul> <li>full-length – models constructed from full-length protein sequences</li> <li>short - models constructed from selected short alignment blocks of the protein sequences</li> </ul> </li> <li>sequences – different sequence datasets and respective multiple sequence alignments <ul> <li>Flavivirus_127-seq_training_set.fasta - 127 polyprotein sequences covering species diversity of the genus <em>Flavivirus</em></li> <li>Flavivirus_127-seq.aln – multiple sequence alignment of the 127-protein dataset</li> <li>Flavivirus_6364-seq_testset.fasta - 6,364 sequence dataset covering species diversity of <em>Flavivirus</em>, including 3,919 of dengue virus (DENV), 327 of Zika virus (ZIKV), 63 of yellow fever virus (YFV), and the remaining 2,055 sequences covering other available flaviviruses</li> <li>Flavivirus_6364-seq.aln - multiple sequence alignment of the 6,364-protein <em>Flavivirus</em> dataset</li> </ul> </li> </ul> </li> <li>Casposons folder <ul> <li>casposon_generic_HMMs – profile HMMs constructed with TABAJARA in Discrimination mode for the generic detection of all casposons and discrimination from CRISPRs.</li> <li>casposon_family_discriminative_HMMs – profile HMMs constructed with TABAJARA in Discrimination mode for the specific discrimination among casposon families and from CRISPRs.</li> <li>sequences – different sequence datasets and respective multiple sequence alignments <ul> <li>casposons_crisprs.fasta – 106 Cas1 <em>bona fide</em> sequences derived from 52 CRISPRs and 54 casposons</li> <li>casposon_family_discrimination.aln - multiple sequence alignment of 52 <em>bona fide</em> CRISPR and 54 casposon sequences, with appropriate nomenclature to run TABAJARA for the discrimination of each casposon family.</li> <li>casposons_crisprs_discrimination.aln - multiple sequence alignment of 52 <em>bona fide</em> CRISPR and 54 casposon sequences, with appropriate nomenclature to run TABAJARA for discrimination of CRISPRs and casposons.</li> </ul> </li> </ul> </li> </ul>
Mediterranean Marine Heatwaves (MHWs) as detected from ESA CCI SST 0.05°x0.05° covering 1982-2021
<p>Daily records of Mediterranean Marine Heatwaves (MHW) resulting from detection applied to the European Space Agency (ESA) Climate Change Initiative (CCI) Sea Surface Temperature (SST) satellite product on a regular 0.05°x0.05° grid, covering the period 01/01/1982-31/12/2021.</p> <p>The MHW detection has been carried out via Hobday's method (Hobday et al. 2016).</p> <p>The <strong>mhw_original</strong> field provides the intensity of anomaly [°C] of MHW events detected on the original SST data, while the <strong>mhw_detrended</strong> field describes the intensity of anomaly [°C] of MHW events detected on detrended SST data, via X-11 seasonal adjustment procedure. </p> <p>The production of this dataset has been sustained with the support of the European Space Agency (ESA) "deteCtion and threAts of maRinE Heat waves" project (CAREHeat; grant number: 4000137121/21/I-DT).</p> <p>References:<br> Hobday, A. J., Alexander, L. V., Perkins, S. E., Smale, D. A., Straub, S. C., Oliver, E. C., ... & Wernberg, T. (2016). <br> A hierarchical approach to defining marine heatwaves. Progress in Oceanography 141:227–238, https://doi.org/10.1016/j.pocean.2015.12.014.</p>
Visual Genome - Visual Relationship Detection - Scene Graph Generation using Message Passing Neural Networks and Graph Convolutional Networks
<p>This repository contains a processed version of <strong>Visual Genome</strong> for <em>Visual Relationship Detection</em>, from the Diploma (MSc) thesis <strong>Scene Graph Generation using Message Passing Neural Networks and Graph Convolutional Networks</strong> by Miltiadis Kofinas, supervised by Christos Diou and Anastasios Delopoulos.</p> <p>The original thesis is written in Greek</p> <blockquote> <p><strong>Νευρωνικά Δίκτυα Ανταλλαγής Μηνυμάτων και Συνελικτικά Δίκτυα Γράφων για Εξαγωγή Γράφου Σκηνής Εικόνων</strong><br> Μιλτιάδης Κοφινάς<br> <a href="https://ikee.lib.auth.gr/record/300900">https://ikee.lib.auth.gr/record/300900</a></p> </blockquote> <p>A summarized English version of the thesis can be accessed <a href="https://www.dropbox.com/s/m87ixw8c8ecrswm/mkofinas_thesis_english_scene_graph_generation.pdf?dl=0">here</a>.</p> <p>It contains region proposals for VGG-16 for all images, and metadata about the bounding box distribution and the predicate classes.</p>
Data from: RockNet: Rockfall and earthquake detection and association via multitask learning and transfer learning
<p>Seismological data can provide timely information for slope failure hazard assessments, among which rockfall waveform identification is challenging for its high waveform variations across different events and stations. A rockfall waveform does not have typical body waves as earthquakes do, so researchers have made enormous efforts to explore characteristic function parameters for automatic rockfall waveform detection. With recent advances in deep learning, algorithms can learn to automatically map the input data to target functions. We develop RockNet via multitask and transfer learning; the network consists of a single-station detection model and an association model. The former discriminates rockfall and earthquake waveforms. The latter determines the local occurrences of rockfall and earthquake events by assembling the single-station detection model representations with multiple station recordings. RockNet achieves macro F1 scores of 0.990 and 0.981 in terms of discriminating earthquakes and rockfalls from other events with the single-station detection and association models, respectively.</p>
Datasets for Evaluating Metagenomic Phage Detection Tools
<p>These datasets include sequences that can be used for evaluating computational tools that detect bacteriophage in metagenomes.</p> <p>Datasheets are supplied for each dataset, and a README describes how to extract each dataset. Once the directories are extracted, there is a README in each directory describing the individual dataset.</p>
Data for: Distinguishing intrinsic photon correlations from external noise with frequency-resolved homodyne detection
<p>Data for the publication C. Lüders, M. Aßmann, Distinguishing intrinsic photon correlations from external noise with frequency-resolved homodyne detection, Scientific Reports 10 (1), 1-11 (2020).</p> <p>In this work, we apply homodyne detection to investigate the frequency-resolved photon statistics of a cw light field emitted by a driven-dissipative semiconductor system in real time. We demonstrate that studying the frequency dependence of the photon number noise allows us to distinguish intrinsic noise properties of the emitter from external noise sources such as mechanical noise while maintaining a sub-picosecond temporal resolution. We further show that performing postselection on the recorded data opens up the possibility to study rare events in the dynamics of the emitter. By doing so, we demonstrate that in rare instances, additional external noise may actually result in reduced photon number noise in the emission.</p> <p>Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – SFB-Geschäftszeichen TRR142/3-2022 – Projektnummer 231447078, Project A04.</p>
Data from: Testing background matching and disruptive colouration in a sexually dichromatic grasshopper: a computer detection experiment.
<p>Cryptic colouration is an adaptative mechanism against predators. Colour patterns can become cryptic through background matching and disruptive colouration, which breaks up the outlines of an animal because the pattern does not coincide with the shape and outline of the animal’s body. Background matching could be advantageous in chromatically homogeneous microhabitats, whereas disruptive colouration can be favoured in visually heterogeneous microhabitats. Grasshoppers of the genus <em>Sphenarium</em> (Orthoptera: Pyrogomorphidae) inhabit very heterogeneous environments and exhibit both strategies. Adults show substantial continuous variation in colouration and longitudinal and transverse bands on the thorax and abdomen. However, males often exhibit considerably more variation in the number of longitudinal and transverse bands than females, which tend to have more uniform colouring (flatter patterns). In this study, we analysed the cryptic properties of the colour patterns of males and females of <em>Sphenarium </em><em>zapotecum</em><em> </em>Sanabria-Urbán, H. Song & Cueva del Castillo and tested the effectiveness of background matching and disruptive colouration using humans as ‘predators’ in a computer detection experiment. We found that the females and males are dichromatic and seem to follow different cryptic strategies in their colouration: males are more disruptive to the background than females, whereas females have a higher level of background matching. In addition, in visually heterogeneous areas, predators spent most time searching for striped male morphs with lower background matching and higher disruptive properties, as well as for female morphs with high background matching, potentially increasing prey survival. As background matching is associated with females and disruptive colouration with males, our results could help explain the evolution of sexual dichromatism in this and other species of grasshoppers of the genus <em>Sphenarium.</em></p> <p> </p>
M100 dataset: time-aggregated data for anomaly detection
<p>This entry is a part of a larger data set collected from the most recent Tier-0 supercomputer hosted at CINECA (Marconi100, <a href="https://www.hpc.cineca.it/hardware/marconi100">https://www.hpc.cineca.it/hardware/marconi100</a>). The data covers the entirety of the system, ranging from the computing nodes (980+ computing nodes) internal information such as core loads, temperatures, frequencies, memory write/read operations, CPU power consumption, fan speed, GPU usage details, etc., to the system-wide information, including the liquid cooling infrastructure, the air conditioning system, the power supply units, workload manager statistics, and job-related information, system status alerts, and weather forecast. <br> It comprises hundreds of metrics measured on each computing node, in addition to hundreds of other metrics gathered from sensors monitored along all system components.</p> <p>This particular dataset is made for anomaly detection purposes, it contains the same data as the main dataset but aggregated over time, with one Parquet file for each node. The data is distributed in tarballs, each one including all the files relative to the nodes contained in a given rack. For each file, the rows represent periods of 15 minutes, with the columns being aggregated values (average, standard deviation, min, max) over all the IPMI metrics that are available for the node; an additional column contains anomaly labels from Nagios.</p> <p>More details can be found in the companion repository: <a href="https://gitlab.com/ecs-lab/exadata">https://gitlab.com/ecs-lab/exadata</a>, including the spatial distribution of the nodes in the room.</p>
Data and scripts from "Unsupervised learning for structure detection in plastically deformed crystals"
<p>This documents contains the scripts and dataset used for the paper "Unsupervised learning for structure detection in plastically deformed crystals".</p> <p> </p> <p>More precisely it contains 4 folders :</p> <p><br> DumpForFigures : subfolder containing the atomic positions in .dump format (see lammps documentation) used for the article figures.</p> <p>DumpForTraining : subfolder containing the atomic position in .dump format (see lammps documentation) used for training the autoencoder.</p> <p>ScriptsToDetectStructuresFromDump : subfolder containing the script sused to detect the substructures of the system by combining autoencoder and clustering methods. This folder contains a readme with the details of the contents.</p> <p>ScriptToGenerateDump : subfolder containing the scripts used to generate the atomic data with molecular dynamics. These data are then used to train the autoencoder. This folder contains a readme with the details of the contents.</p> <p>REQUIREMENTS :</p> <p> </p> <p>Lammps</p> <p>Python3 with packages :</p> <p>-numpy</p> <p>-matplotlib</p> <p>-pyscal</p> <p>-sci-kit learn</p> <p>-pytorch</p> <p>-glob</p> <p> </p> <p> </p> <p> </p>
DisTEMIST corpus: detection and normalization of disease mentions in spanish clinical cases
<p><strong>DisTEMIST corpus: training set + MULTILINGUAL RESOURCES + CROSSMAPPINGS + test set + background set</strong></p><p> </p><p><strong>Please cite if you use this dataset:</strong></p><p>Miranda-Escalada, A., Gascó, L., Lima-López, S., Farré-Maduell, E., Estrada, D., Nentidis, A., Krithara, A., Katsimpras, G., Paliouras, G., & Krallinger, M. (2022). Overview of DisTEMIST at BioASQ: Automatic detection and normalization of diseases from clinical texts: results, methods, evaluation and multilingual resources. <i>Working Notes of Conference and Labs of the Evaluation (CLEF) Forum. CEUR Workshop Proceedings</i></p><p>@article{miranda2022overview, title={Overview of DisTEMIST at BioASQ: Automatic detection and normalization of diseases from clinical texts: results, methods, evaluation and multilingual resources}, author={Miranda-Escalada, Antonio and Gascó, Luis and Lima-López, Salvador and Farré-Maduell, Eulàlia and Estrada, Darryl and Nentidis, Anastasios and Krithara, Anastasia and Katsimpras, Georgios and Paliouras, Georgios and Krallinger, Martin}, booktitle={Working Notes of Conference and Labs of the Evaluation (CLEF) Forum. CEUR Workshop Proceedings}, year={2022} }</p><p> </p><p><strong>Introduction</strong></p><p>The DisTEMIST corpus is a collection of 1000 clinical cases with <strong>disease mention annotations manually mapped to</strong> <strong>Snomed-CT concepts</strong>. All documents are released in the context of the BioASQ DisTEMIST track for CLEF 2022. For more information about the track and its schedule, please <a href="https://temu.bsc.es/distemist/">visit the website</a>.</p><p> </p><p><strong>File structure:</strong></p><p>The DisTEMIST corpus has been randomly divided into a training set, containing 750 clinical cases, and a test set (584 in the case of subtrack2), consisting of 250 additional cases. Participants must train their systems using the train set and submit predictions for the test set, on which they will be evaluated. The file structure of the corpus is as follows:</p><ul><li><strong>train_set:</strong><ul><li><strong>text_files</strong>: Folder with plain text files of the clinical cases</li><li><strong>subtrack1_entities</strong>: It contains annotations in a tab-separated file (TSV) with the following columns:<ul><li><i>filename</i>: document name</li><li><i>mark</i>: identifier mention id</li><li><i>label</i>: mentions type (ENFERMEDAD)</li><li><i>off0</i>: starting position of the mention in the document</li><li><i>off1</i>: ending position of the mention in the document</li><li><i>span</i>: text span</li></ul></li><li><strong>subtrack2_linking: </strong>It contains annotations in a tab-separated file (TSV) with the following columns:<ul><li><i>filename</i>: document name</li><li><i>mark</i>: identifier mention id</li><li><i>label</i>: mentions type (ENFERMEDAD)</li><li><i>off0</i>: starting position of the mention in the document</li><li><i>off1</i>: ending position of the mention in the document</li><li><i>span</i>: text span</li><li><i>codes</i>: List of Snomed-CT concept codes linked to the mention. If there is more than one code associated with a mention, they will be concatenated by the symbol "+".</li><li><i>semantic relation</i>: the relationship between the assigned code and the mention. It can be EXACT, when the code corresponds exactly with the mention, or NARROW, when the mention corresponds to a narrower concept than the Snomed-CT code. For instance, the concept "Chorioretinal lacunae" does not exist in Snomed-CT. Then, it is normalized to the Snomed-CT ID 302893000 ("Chorioretinal disorder").</li><li>(Note: the training for entity linking were released in two parts, you must join them)</li></ul></li></ul></li></ul><p> </p><ul><li><strong>test_annotated:</strong><ul><li><strong>text_files</strong>: Folder with plain text files of the clinical cases</li><li><strong>brat</strong>: Folder with text files and their annotations in brat's .ann format</li><li><strong>subtrack1_entities: </strong>It contains annotations in a tab-separated file (TSV) with the same columns as the train set</li><li><strong>subtrack2_linking: </strong>It contains annotations in a tab-separated file (TSV) with the same columns as the train set</li></ul></li></ul><p> </p><ul><li><strong>test_background_unannotated/text_files: </strong>3000 clinical cases (test + background). In the the original task, participants had to make predictions for these 3000 files and they were evaluated on a subset of them.</li></ul><p> </p><ul><li><strong>multilingual-resources</strong>: we have generated the annotated training and validation sets in <strong>6 languages</strong>: <i><strong>English</strong></i>, <i><strong>Portuguese</strong></i>, <i><strong>Catalan</strong></i>, <i><strong>Italian</strong></i>, <i><strong>French</strong></i> and <i><strong>Romanian</strong></i>. The process was:<ol><li>The text files were translated with a neural machine translation system.</li><li>The annotations were translated with the same neural machine translation system.</li><li>The translated annotations were transferred to the translated text files using an annotation transfer technology.</li><li>The text files are stored in the multilingual_resources/training-text-files subfolder.</li><li>The annotated TSV files are stored in the multilingual_resources/lang subfolders.</li><li>If you want to visualize the multilingual resources, check out this Brat server: <a href="https://temu.bsc.es/mDistemist/#/translations/">https://temu.bsc.es/mDistemist/#/translations/</a><br>For instance, you can see the parallel annotations in <a href="http://temu.bsc.es/mDistemist/diff.xhtml#/translations/fr/train/es-S0004-06142008000100011-1?diff=/translations/en/train/">English vs in French</a>, or in <a href="https://temu.bsc.es/mDistemist/diff.xhtml#/translations/cat/train/S0004-06142005000500011-1?diff=/gold-standard/train/">Spanish (the gold standard) vs in Catalan</a>.</li></ol></li></ul><p> </p><ul><li><strong>cross-mappings. </strong>We include the same entities as in DISTEMIST-linking but mapped to Snomed-CT, <a href="https://www.ncbi.nlm.nih.gov/mesh/"><strong>MeSH</strong></a>, <a href="https://icd.who.int/browse10/2019/en#/"><strong>ICD-10</strong></a>, <a href="https://hpo.jax.org/"><strong>HPO</strong></a>, and <a href="https://www.omim.org/"><strong>OMIM</strong></a>. The original mappings are manual and to Snomed-CT. The mapping to the other terminologies was done through the UMLS Metathesaurus.</li></ul><p> </p><p><strong>Resources</strong> </p><ul><li><a href="https://temu.bsc.es/distemist/">Task Web</a></li><li><a href="https://github.com/TeMU-BSC/distemist_evaluation_library">Evaluation Library</a></li><li><a href="https://doi.org/10.5281/zenodo.6458114"><strong>DISTEMIST gazetteer</strong></a></li><li><a href="https://doi.org/10.5281/zenodo.6458078"><strong>DISTEMIST guidelines</strong></a></li><li><strong>Citation: </strong>Miranda-Escalada, A., Gascó, L., Lima-López, S., Farré-Maduell, E., Estrada, D., Nentidis, A., Krithara, A., Katsimpras, G., Paliouras, G., & Krallinger, M. (2022). Overview of DisTEMIST at BioASQ: Automatic detection and normalization of diseases from clinical texts: results, methods, evaluation and multilingual resources. <i>Working Notes of Conference and Labs of the Evaluation (CLEF) Forum. CEUR Workshop Proceedings</i></li><li><a href="https://ceur-ws.org/Vol-3180/paper-11.pdf"><strong>Overview paper</strong></a></li><li><a href="https://ceur-ws.org/Vol-3180/"><strong>CLEF/BioASQ Workshop proceedings</strong></a></li><li><a href="https://www.youtube.com/watch?v=KvvhogUMjl8&list=PL5uSCzf1azhC_Sg9leXavmcA4nJ4HQa_n"><strong>Youtube videos (overview & teams)</strong></a></li><li><a href="https://www.slideshare.net/MartinKrallinger/overview-of-distemist-at-bioasq-automatic-detection-and-normalization-of-diseases-from-clinical-texts-results-methods-evaluation-and-multilingual-resources-at-bioasq-workshop-of-clef-conference"><strong>DISTEMIST overview talk slides</strong></a></li></ul><p> </p><p><strong>License</strong></p><p>This work is licensed under a <a href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.</p><p> </p><p><strong>Additional resources and corpora</strong></p><p>If you are interested in DisTEMIST, you might want to check out these corpora and resources:</p><ul><li><a href="https://zenodo.org/records/8413866">SymTEMIST</a> (Corpus of symptoms, signs and findings mentions and normalization to SNOMED CT, same document collection)</li><li><a href="https://zenodo.org/records/8224056">MedProcNER </a>(Corpus of clinical procedure mentions and normalization to SNOMED CT, same document collection)</li><li><a href="https://zenodo.org/records/4270158">PharmaCoNER</a> (Corpus of medications, drugs, chemical substances, genes, proteins and vaccine mentions and normalization, same document collection)</li><li><a href="https://zenodo.org/records/7116201">MEDDOPROF</a> (Corpus of mentions of professions, occupations and working status and normalization, different document collection with some overlapping documents)</li><li><a href="https://zenodo.org/records/8403498">MEDDOPLACE</a> (Corpus of mentions of place-related entity mentions, including departments, nationalities or patient movements etc.. and normalization, different document collection with some overlapping documents)</li><li><a href="https://zenodo.org/records/4279323">MEDDOCAN</a> (Corpus of mentions of Personal Health Identifiers (PHI), modified synthetic verions of the document collection)</li><li><a href="https://zenodo.org/records/3978041">CANTEMIST</a> (Corpus of cancer tumor morphology mentions and normalization, different document collection)</li><li><a href="https://zenodo.org/records/3837305">CodiESp</a> (Corpus of clinical case reportes with assigned clinical codes from ICD10, Spanish version, same document collection)</li><li><a href="https://zenodo.org/records/7684093">LivingNER</a> (Corpus of mentions of species, including human/family members, pathogens, food, etc.. and normalization to NCBI Taxonomy, different document collection with some overlapping documents)</li><li><a href="https://zenodo.org/records/2560344">SPACCC-POS</a> (Corpus of clinical case reports in Spanish annotated with POS-tags, same document collection)</li><li><a href="https://zenodo.org/records/2560338">SPACCC-TOKEN</a> (Corpus of clinical case reports in Spanish annotated with token-tags (word mention boundaries), same document collection)</li><li><a href="https://zenodo.org/records/2560338">SPACCC-SPLIT</a> (Corpus of clinical case reports in Spanish annotated with sentence boundary-tags, same document collection)</li><li><a href="https://zenodo.org/records/5602914">MESINESP-2</a> (Corpus of manually indexed records with DeCS /MeSH terms comprising scientific literature abstracts, clinical trials, and patent abstracts, different document collection)</li></ul><p><strong>Contact</strong></p><p>If you have any questions or suggestions, please contact us at:</p><p><br>- Martin Krallinger (<krallinger [dot] martin [at] gmail [dot] com>)</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.