Find research datasets worth reusing

opencc-by-4.0Jan 2023View details →

Ultrasensitive and multiplexed miRNA detection system with DNA-PAINT

This dataset contains the raw data that were used for the publication entitled, "Ultrasensitive and multiplexed miRNA detection system with DNA-PAINT" published in Biosensors and Bioelectronics on 30 December 2023.

opencc-by-4.0Jan 2023View details →

Detecting coarse beach sediment using remotely sensed imagery at the FRF, Duck, NC, USA: Labeled images, deep learning model, testing data, and predictions.

This data record contains 5 zip files all used to build and use a semantic segmentation model to operate on beach imagery taken at the Field Research Facility (FRF) in Duck, North Carolina, USA.  All data is from 2015-2021 The `training_data.zip` contains all data used to train the ML model. All images come from the north facing (c1) camera. This zip file includes: a list of classes used to label the imagery, and folders of 107 images, 107 sparse annotations (doodles), 107 labels, and 107 overlays. All labeling was done with the open-source labeling tool ‘Doodler (Buscombe et al., 2021). The `model.zip` file contains the ML model, and associated metadata. This includes: a JSON model configuration file, a figure showing model training statistics, an `.npz` file of model training output, a list of training and validation files, the model as an h5 file and in the Tensorflow ‘saved model’ format.  All modeling was done with Segmentation Gym (Buscombe & Goldstein 2022). The `test_data_c6.zip` file contains all data from the south facing (c6) camera to test the ML model. This includes: a list of classes used to label the imagery, and folders of 10 images, 10 sparse annotations (doodles), 10 labels, and 10 overlays.  All labeling was done with the open-source labeling tool ‘Doodler (Buscombe et al., 2021). Testing the model with this data was done with codes in: https://github.com/ebgoldstein/FRF_GrainSize The `test_data_c1.zip` file contains all data from the north facing (c1) camera to test the ML model. This includes: a list of classes used to label the imagery, and folders of 10 images, 10 sparse annotations (doodles), 10 labels, and 10 overlays.  All labeling was done with an open-source labeling tool ‘Doodler (Buscombe et al., 2021). Testing the model with this data was done with codes in: https://github.com/ebgoldstein/FRF_GrainSize The `predictions.zip` file contains 4418 images from the north facing (c1) camera that were run through the trained segmentation model as well as the resulting output (presented as side-by-side image and overlays). These images were created using codes in Segmentation Gym (Buscombe & Goldstein 2022).

opencc-by-4.0Feb 2023View details →

DeepAstroUDA: Semi-Supervised Universal Domain Adaptation for Cross-Survey Galaxy Morphology Classification and Anomaly Detection

We present the data used in "DeepAstroUDA: Semi-Supervised Universal Domain Adaptation for Cross-Survey Galaxy Morphology Classification and Anomaly Detection". It was also used in the conference paper presented in Machine Learning and the Physical Sciences workshop at NeurIPS 2022: "Semi-Supervised Domain Adaptation for Cross-Survey Galaxy Morphology Classification and Anomaly Detection". A plethora of AI methods, has already shown huge promise in increasing quality and speed of work with astronomical datasets, but high complexity of AI methods leads to extraction of dataset-specific non-robust features, which leads to models that cannot work on multiple datasets at the same time. We develop a Universal Domain Adaptation method DeepAstroUDA, capable of performing semi-supervised domain adaptation, that can be applied to datasets with different data distributions and class overlap. Extra classes can be present in any of the two datasets, and the method can even be used in the presence of unknown classes. We apply our model to three examples of galaxy morphology classification tasks of different complexities (3-class and 10-class problems), with anomaly detection i.e. in all our experiments we have one extra class in the unlabeled target dataset, which represents our anomaly class.   DATA: 1) DA across two different data releases of the same survey (LSST 1 and 10 years of observation): We use data from Ciprijanovic et al. 2022. which can also be found on Zenodoo: <a href="https://zenodo.org/record/5514180#.Y6SM7y-B2_w">https://zenodo.org/record/5514180</a> . Data contains three classes: spiral (0), elliptical (1) and merging galaxies (3, anomaly class). 2) DA across two surveys (SDSS and DeCALS): We create datasets using data and labels from the Galaxy Zoo project. Datasets contain 10 classes (9 known classes present in both SDSS and DeCALS data, and one unknown anomaly class present only in DeCALS data): disturbed (0), merging (1), round smooth (2), cigar shaped smooth (3), barred spiral (4), unbarred tight spiral (5), unbarred loose spiral (6), edge-on without bulge (7), edge-on with bulge (8), lenses (9, unknown anomaly class). SDSS (wide filed): datasets is split into two files  - sdss_1.h5, sdss_2.h5 DeCALS:  decals.zip 3) DA between wide and deep observing fields of the same survey (SDSS): We create datasets using data and labels from the Galaxy Zoo project. Datasets contain same 10 classes as in 2), with the final lens anomaly class being only present in the SDSS deep field. SDSS (wide filed): the same data as in 2) SDSS (Strip 82 deep field): sdss_stripe82.zip All SDSS and DECaLS files contain full datasets (train, validation and test). Exact split that we performed (0.6 : 0.2 : 0.2) can be done using the code that accompanies this publication: <a href="https://github.com/deepskies/DeepAstroUDA">https://github.com/deepskies/DeepAstroUDA</a> .

opencc-by-4.0Feb 2023View details →

Dataset for Detecting Motorcyclists in Pedestrian Areas

The dataset contains a total of 11361 images in .jpg format for a total of 0.99 GB, the resolution of the images is 1280x720 (HD), and the weight per image is 56. 7 KB, the images were obtained from a closed circuit television CCTV free public access located in the city of Medellín Colombia consisting of 80 cameras located on the road, these images were captured during both night and day hours, The time range in which they were acquired is from 04/10/2022 to 28/10/2022 a total of 24 days, the format of the name of each image is "image (camera number)_(date format"AAAAMMDD")_( time format "hhmmss"), where (AAAA) is the year, (MM) is the month, (DD) is the day, (HH) is the 24-hour hour, (MM) is the minute and (SS) is the second. The file (annotations.pickle) corresponds to a file in .pickle format that contains the labels made to each image and corresponds to a single main variable of dictionary type where its main key is the name of the image (image2_20221004_054300.jpg), and the sub keys are (box), which contains a list of lists of bounding boxes, (clippings), presents a list of arrays containing clippings made to the image, (class), which contains a list of integer values representing the class to which corresponds the nth bounding box or clipping.

opencc-by-4.0May 2022View details →

CRACK-CH: A Crack detection and classification dataset on complex stone masonry surfaces

Description This dataset includes various images with cracks from the test sites of HYPERION H2020 project (Grant Agreement No. 821054). Specifically, square image patches of 224x224 pixels from the test sites of Naillac and St. Nikolaos Fort are included. The Saint Nikolaos Fort is an important part of the great fortifications of the Medieval City of Rhodes located at the entrance of the Mantraki port. At this location, there was just a chapel dedicated to Saint Nikolaos until 1464, when it was turned to a fortification. Since then it has undergone reinforcements and expansions in order to defend the city. The outer walls were built in 1480 AD and in 1863 AD it was finally transformed to a lighthouse. The second study area is the Naillac at Saint Paul’s rampart where a monumental tower was located as part of the fortification of the Commercial Harbour of Rhodes. It was constructed around 1400 AD on the Hellenistic Pier, but it was destroyed in 1863 after a severe earthquake. In 2017, the Naillac Tower was graphically reconstructed and presented as it stood until 1863, during the Ottoman rule. The Rodini Roman Bridge is one of the few ancient bridges surviving in Greece and part of the Hellenistic fortification of the city, making it a monument of great importance. It was built across the stream of Rhodini, situated outside the Medieval City and has two arched openings. The Roman Bridge is in continuous use until today and its static efficiency has deteriorated, while the scaffoldings which now support the arches are gradually rusting and losing their efficiency. Those images were split into two separate categories, facilitating the later training and evaluation of the model: “Cracks” and “No cracks”. The dataset is used to train and evaluate the CNN models for crack detection on complex stone masonry surfaces. Publication The paper is availbale here: https://arxiv.org/abs/2303.17989 If you use this dataset please cite it as CRACK-CH [reference]. [Reference] Agrafiotis, Panagiotis, Doulamis, Anastasios, and Georgopoulos, Andreas. (2023) "Unsupervised crack detection on complex stone masonry surfaces", arXiv preprint arXiv:2303.17989 Bibtex entry: @misc{agrafiotis2023unsupervised,       title={Unsupervised crack detection on complex stone masonry surfaces},        author={Panagiotis Agrafiotis and Anastastios Doulamis and Andreas Georgopoulos},       year={2023},       eprint={2303.17989},       archivePrefix={arXiv},       primaryClass={cs.CV} }

opencc-by-4.0Sep 2022View details →

ESAA: an EEG-Speech auditory attention detection database

We build a database for AAD research, which consists of competing speech stimuli and associated human neural responses, i.e, electroencephalography (EEG) recordings, namely EEG-Speech AAD (ESAA) database.  This is the first AAD database with speech stimuli in a tonal language (Mandarin). Moreover, we develop an AAD baseline as a reference model for decoding which speech stream a listening subject is attending to (speaker attention detection), and a baseline for decoding which spatial locus a listening subject is attending to (speaker locus attention detection) on the ESAA database. We release the source code and the database for use in research purpose. This database consists of response data for 17 normal-hearing subjects (S1-S17). It includes: - 64-channel EEG data: responses to two-speaker speech stimuli - Auditory stimuli data (clean): Chinese short stories narrated by a female and a male professional story teller.  - Auditory stimuli data (hrtf): Auditory stimuli after head-related transfer function (HRTF) filtering (simulating sound coming from ± 90 deg). - Preprocessing code - AAD baseline (CNN model)

Multi-label Tweet Dataset for Textual Propaganda Detection related to anti-CAA protest in India (2019-2021)

This  is a collection of English Tweets  about the Citizenship (Amendment) Bill protests that occurred in India in 2019-2021. The dataset contains  tweet instances multiple labels for identified propaganda techniques. The data set consists of tweet ids, hashtags used, and corresponding propaganda techniques. Labels have been automatically generated using Weak Supervision.  As of 2023, there are very limited textual propaganda detection dataset for Tweets. This dataset is released to facilitate future research as propaganda has become omnipresent in modern social media.

FlexiGroBots Ground-level Blueberry Orchard Dataset v1 - RGB Bush Detection Dataset

Ground-level Blueberry Orchard Dataset v1 consists of 2000 RGB images of blueberry orchard scenes captured in the village of Babe, Serbia on three occasions in March, May, and August of 2022. Images are captured using the RGB module of Luxonis OAK-D device, with the resolution of 1920×1080 pixels and stored in the lossless PNG format.  The dataset is created for the purpose of training deep learning models for blueberry bush detection, for the task of autonomous UGV guidance. It contains sequences of images captured from the UGV moving and rotating in blueberry orchard rows. Images are captured from a height of approximately 0.5 meters, with the camera angled towards the base of a blueberry plant and the surrounding bank on which it grows. Dataset is captured in real-life outdoor conditions and contains multiple sources of variability (bush shape and size, lighting conditions, shadows, saturation etc.) and artifacts (occlusions by weeds, branches, presence of irregular objects etc.). There are two classes of annotated objects of interest: <ul> <li> Bush, corresponding to the base of the blueberry bush. </li> <li> Pole, corresponding to hail netting poles and similar obstructing objects such as lamp posts or wooden legs of bumblebee hives (distinguishing poles is important to prevent equipment damage in operations such as soil sampling and pruning). </li> </ul> Objects of interest are annotated with bounding boxes. Labels are saved in two formats: <ul> <li> LabelMe JSON format (x1, y1, x2, y2; in pixels) </li> <li> Yolo TXT format (x_center, y_center, width, height; as a ratio of total image size, with numerical labels 0 and 1 corresponding to Bush and Pole) </li> </ul> There are 61 images with no annotated objects, and there are no corresponding label files for these images. The dataset is split into train, validation and test sets with 75%, 10%, and 15% split (1490, 200, and 310 images, respectively). As the data contains sequences of images, the split is made based on sequences rather than individual images to prevent data leakage. Detailed description and statistics are available in: V. Filipović, D. Stefanović, N. Pajević, Ž. Grbović, N. Đurić and M. Panić, "Bush Detection for Vision-based UGV Guidance in Blueberry Orchards: Data Set and Methods," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Vancouver, Canada, 2023. (Accepted)

opencc-by-4.0Mar 2023View details →

Dataset: Correlative Light, Electron Microscopy and Raman Spectroscopy Workflow to Detect and Observe Microplastic Interactions with Whole Jellyfish

ABSTRACT Many researchers have turned their attention to understanding microplastic interaction with marine fauna. Efforts are being made to monitor exposure pathways and concentrations, and to assess the impact such interactions may have. To answer these questions, it is important to select appropriate experimental parameters and analytical protocols. This study focuses on medusae of Cassiopea andromeda jellyfish: a unique benthic jellyfish known to favor (sub-)tropical coastal regions which are potentially exposed to plastic waste from land-based sources. Juvenile medusae were exposed to fluorescent poly(ethylene terephthalate) and polypropylene microplastics (< 300 µm), resin embedded, and sectioned before analysis with confocal laser scanning microscopy as well as transmission electron microscopy and Raman Spectroscopy. Results show the fluorescent microplastics were stable enough to be detected with the optimized analytical protocol presented, and that their observed interaction with medusae occurs in a manner which is likely driven by the microplastic properties (e.g. density, hydrophobicity).

openother-openAug 2021View details →

Multicenter Validated Detection of Focal Cortical Dysplasia using Deep Learning

Lesional and non-lesional patches derived from 148 FCD patients is available as a HDF5 dataset (v1.0.0; doi: 10.5061/dryad.h70rxwdgm or 10.5281/zenodo.3239446). To create this dataset, for each of the 148 FCD patients, we sampled at most 1,000 (*_N1000.h5) or 1,500 (*_N1500.h5) cortical patches (or # voxels in the lesion, whichever is lower) of size 16×16×16 within the lesion.The same number of cortical patches were sampled randomly outside the lesion. The resulting lesional and non-lesional patches were concatenated, shuffled (to add another layer of randomization), and saved along with their binary labels (compressed HDF5 dataset). For axis=1, index 0 is T1 and 1 is FLAIR.

replicAnt - Plum2023 - Detection & Tracking Datasets and Trained Networks

This dataset contains all recorded and hand-annotated as well as all synthetically generated data as well as representative trained networks used for detection and tracking experiments in the replicAnt - generating annotated images of animals in complex environments using Unreal Engine manuscript. Unless stated otherwise, all 3D animal models used in the synthetically generated data have been generated with the open-source photgrammetry platform scAnt <a href="http://peerj.com/articles/11155/">peerj.com/articles/11155/</a>.  All synthetic data has been generated with the associated replicAnt project available from <a href="https://github.com/evo-biomech/replicAnt">https://github.com/evo-biomech/replicAnt</a>. Abstract: Deep learning-based computer vision methods are transforming animal behavioural research. Transfer learning has enabled work in non-model species, but still requires hand-annotation of example footage, and is only performant in well-defined conditions. To overcome these limitations, we created replicAnt, a configurable pipeline implemented in Unreal Engine 5 and Python, designed to generate large and variable training datasets on consumer-grade hardware instead. replicAnt places 3D animal models into complex, procedurally generated environments, from which automatically annotated images can be exported. We demonstrate that synthetic data generated with replicAnt can significantly reduce the hand-annotation required to achieve benchmark performance in common applications such as animal detection, tracking, pose-estimation, and semantic segmentation; and that it increases the subject-specificity and domain-invariance of the trained networks, so conferring robustness. In some applications, replicAnt may even remove the need for hand-annotation altogether. It thus represents a significant step towards porting deep learning-based computer vision tools to the field. Benchmark data Two video datasets were curated to quantify detection performance; one in laboratory and one in field conditions. The laboratory dataset consists of top-down recordings of foraging trails of Atta vollenweideri (Forel 1893) leaf-cutter ants. The colony was collected in Uruguay in 2014, and housed in a climate chamber at 25°C and 60% humidity. A recording box was built from clear acrylic, and placed between the colony nest and a box external to the climate chamber, which functioned as feeding site. Bramble leaves were placed in the feeding area prior to each recording session, and ants had access to the recording area at will. The recorded area was 104 mm wide and 200 mm long. An OAK-D camera (OpenCV AI Kit: OAK-D, Luxonis Holding Corporation) was positioned centrally 195 mm above the ground. While keeping the camera position constant, lighting, exposure, and background conditions were varied to create recordings with variable appearance: The “base” case is an evenly lit and well exposed scene with scattered leaf fragments on an otherwise plain white backdrop. A “bright” and “dark” case are characterised by systematic over- or underexposure, respectively, which introduces motion blur, colour-clipped appendages, and extensive flickering and compression artefacts. In a separate well exposed recording, the clear acrylic backdrop was substituted with a printout of a highly textured forest ground to create a “noisy” case. Last, we decreased the camera distance to 100 mm at constant focal distance, effectively doubling the magnification, and yielding a “close” case, distinguished by out-of-focus workers. All recordings were captured at 25 frames per second (fps). The field datasets consists of video recordings of Gnathamitermes sp. desert termites, filmed close to the nest entrance in the desert of Maricopa County, Arizona, using a Nikon D850 and a Nikkor 18-105 mm lens on a tripod at camera distances between 20 cm to 40 cm. All video recordings were well exposed, and captured at 23.976 fps. Each video was trimmed to the first 1000 frames, and contains between 36 and 103 individuals. In total, 5000 and 1000 frames were hand-annotated for the laboratory- and field-dataset, respectively: each visible individual was assigned a constant size bounding box, with a centre coinciding approximately with the geometric centre of the thorax in top-down view. The size of the bounding boxes was chosen such that they were large enough to completely enclose the largest individuals, and was automatically adjusted near the image borders. A custom-written Blender Add-on aided hand-annotation: the Add-on is a semi-automated multi animal tracker, which leverages blender’s internal contrast-based motion tracker, but also include track refinement options, and CSV export functionality. Comprehensive documentation of this tool and Jupyter notebooks for track visualisation and benchmarking is provided on the <a href="https://github.com/evo-biomech/replicAnt">replicAnt</a> and <a href="https://github.com/FabianPlum/blenderMotionExport">BlenderMotionExport</a> GitHub repositories. Synthetic data generation Two synthetic datasets, each with a population size of 100, were generated from 3D models of \textit{Atta vollenweideri} leaf-cutter ants. All 3D models were created with the scAnt photogrammetry workflow. A “group” population was based on three distinct 3D models of an ant minor (1.1 mg), a media (9.8 mg), and a major (50.1 mg) (see <a href="https://zenodo.org/record/7849059">10.5281/zenodo.7849059</a>)). To approximately simulate the size distribution of A. vollenweideri colonies, these models make up 20%, 60%, and 20% of the simulated population, respectively. A 33% within-class scale variation, with default hue, contrast, and brightness subject material variation, was used. A “single” population was generated using the major model only, with 90% scale variation, but equal material variation settings. A Gnathamitermes sp. synthetic dataset was generated from two hand-sculpted models; a worker and a soldier made up 80% and 20% of the simulated population of 100 individuals, respectively with default hue, contrast, and brightness subject material variation. Both 3D models were created in Blender v3.1, using reference photographs. Each of the three synthetic datasets contains 10,000 images, rendered at a resolution of 1024 by 1024 px, using the default generator settings as documented in the Generator_example level file (see documentation on <a href="https://github.com/evo-biomech/replicAnt">GitHub</a>). To assess how the training dataset size affects performance, we trained networks on 100 (“small”), 1,000 (“medium”), and 10,000 (“large”) subsets of the “group” dataset. Generating 10,000 samples at the specified resolution took approximately 10 hours per dataset on a consumer-grade laptop (6 Core 4 GHz CPU, 16 GB RAM, RTX 2070 Super). Additionally, five datasets which contain both real and synthetic images were curated. These “mixed” datasets combine image samples from the synthetic “group” dataset with image samples from the real “base” case. The ratio between real and synthetic images across the five datasets varied between 10/1 to 1/100. Funding This study received funding from Imperial College’s President’s PhD Scholarship (to Fabian Plum), and is part of a project that has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (Grant agreement No. 851705, to David Labonte). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

opencc-by-4.0Jul 2017View details →

Data for: Detecting Long-Term Balancing Selection Using Allele Frequency Correlation

Genome-wide and top 1% scores for 1KG project data output from BetaScan reported in: <a href="https://pubmed.ncbi.nlm.nih.gov/28981714/">Detecting Long-Term Balancing Selection Using Allele Frequency Correlation.</a> Siewert KM, Voight BF. Mol Biol Evol. 2017 Nov 1;34(11):2996-3005. doi: 10.1093/molbev/msx209. PMID: 28981714 Code available at: https://github.com/ksiewert/BetaScan

opencc-by-4.0Feb 2020View details →

Data for: BetaScan2: Standardized Statistics to Detect Balancing Selection Utilizing Substitution Data

Genome-wide scan using BetaScan2 in 1KG populations report in: <a href="https://pubmed.ncbi.nlm.nih.gov/32011695/">BetaScan2: Standardized Statistics to Detect Balancing Selection Utilizing Substitution Data.</a> Siewert KM, Voight BF.Genome Biol Evol. 2020 Feb 1;12(2):3873-3877. doi: 10.1093/gbe/evaa013. PMID: 32011695  Code available at: https://github.com/ksiewert/BetaScan

opencc-by-4.0Mar 2022View details →

Hazards&Robots: A Dataset for Visual Anomaly Detection in Robotics

This is the final version of our dataset; we further expand the Corridor scenario. This new version of Corridor includes 20 anomalies and the total frames are 324,408. In this version, we release feature embeddings extracted using a CLIP ViT-B/32 model. This dataset is part of a Data in Brief paper submission. For more information check https://github.com/idsia-robotics/hazard-detection

Products and Models for "Detection of carbon monoxide's 4.6 micron fundamental band structure in WASP-39b's atmosphere with JWST NIRSpec G395H"

Overview: Carbon monoxide (CO) is predicted to be the dominant carbon-bearing molecule in giant planet atmospheres, and, along with water, is important for discerning the oxygen and therefore carbon-to-oxygen ratio of these planets. The fundamental absorption mode of CO has a broad double-branched structure composed of many individual absorption lines from 4.3 to 5.1  µm, which can now be spectroscopically measured with JWST. Here we present a technique for detecting the rotational sub-band structure of CO at medium resolution with the NIRSpec G395H instrument. We use a single transit observation of the hot Jupiter WASP-39b from the JWST Transiting Exoplanet Community Early Release Science (JTEC ERS) program at the native resolution of the instrument (R ~ 2700) to resolve the CO absorption structure. We robustly detect absorption by CO, with an increase in transit depth of 264 \(\pm\) 68 ppm, in agreement with the predicted CO contribution from the best-fit model at low resolution. This detection confirms our theoretical expectations that CO is the dominant carbon-bearing molecule in WASP-39b's atmosphere, and further supports the conclusions of low C/O and super-solar metallicities presented in the JTEC ERS papers for WASP-39b.

Result dataset for our experimental analysis on multi-cepstral projection representation strategies for dysphonia detection

Database containing the results of the analyzed versions of the framework proposed in our paper submitted to the journal Sensors (Basel) under the title "An experimental analysis on multi-cepstral projection representation strategies for dysphonia detection". In this database, we have the following information: "gender": Gender of individuals referring to the selected voice database. For this field, we have the following possible values: “male” for a selection of male individuals, “female” for a selection of female individuals, and “both” for a selection considering both genders. “Techniques”: Concerns about the techniques for extracting cepstral coefficients that we are analyzing. The identifier “nonceps” refers to the use of non-cepstral features. “vowel”: Vowel considered in the database selection. The following values are possible: “a”, “i” and “u”. “intonation”: Tone used by individuals when pronouncing the analyzed vowel. Possible values are: “h” for high; “l” for low; “n” is normal; and “lhl” for low-high-low. “coordinates”: Number of coordinates that make up the feature vector that represents the voice signal after the dimensionality reduction routines. “scale”: Normalization function used on the feature vector. The possible values of this field are the following: “MinMax” for the min-max scale; “Robust” for the robust scale; “Standard” for the standard scale; and “Unscaled” for the unscaled vector. “ACC”: Accuracy obtained by the analyzed version on the considered voice database clipping. “AUC”: Area under the ROC curve obtained by the analyzed version on the considered voice database clipping. “EER”: Equal Error Rate obtained by the analyzed version on the considered voice database clipping. “F1”: F1-score obtained by the analyzed version on the considered voice database clipping. “EH”: Rate of healthy voice signals classified as pathological on the considered voice database clipping. “EP”: Rate of pathological voice signals classified as healthy on the considered voice database clipping. “KFCV”: Average accuracy score of a 5-fold Cross Validation over the training dataset on the considered voice database clipping. “Balancing”: Indication of the use of balancing technique (SMOTE) by the considered framework version. “Classifier”: Classifier used, being possible the use of Random Forest (RF), Logistic Regression (LR), and Support Vector Machine (SVM). “Multi-Projection”: Multi-projection strategies employed by the evaluated technique. “Features”: Type of feature that defines the feature vector. In this case, the following values are possible in this field: “NonCeps” for non-cepstral features; “Ceps” for cepstral features only; and “Ceps and NonCeps” for features of cepstral and non-cepstral types. . It is worth noting that the symbol “-”, present in some fields, represents the “non-use” of any technique of the type indicated by the field. For example, in the case of the “Balancing” field, the value “-” means that no data balancing technique was used in the evaluated version of the framework.

Data for "Detection of large-scale cloud microphysical changes within a major shipping corridor after implementation of the IMO 2020 fuel sulfur regulations"

Processed data used for the manuscript "Detection of large-scale cloud microphysical changes within a major shipping corridor after implementation of the IMO 2020 fuel sulfur regulations". Includes input data for kriging algorithm as "SSF1deg_shipkrige_Terra.nc" and output data files as "Data_Terra_[VAR]_[YEAR]_C_M[MONTH].nc" for [VAR] Acld (overcast albedo) or cer (cloud droplet effective radius), [YEAR] the starting year of a three-year period starting with 2002 and ending at 2020 or "clim" for the 2002-2019 climatology, and [MONTH] 1to12 (annual mean) or 9to11 (austral spring). For the output data, "Obs" is the original data, "Est" is the mean counterfactual field obtained via kriging, "lowEst" and "highEst" are the 2.5th and 97.5th percentiles of the kriged fields for each grid box, "krSims" stores the results of the 5,000 simulated kriged fields, "Semivariance" is the binned empirical variogram values, "pVal" is the raw field significance (not adjusted for multiple testing), "nOut" is the number of individually significant grid boxes, "tran" is the transform applied (none for cer, logit for Acld), "iniPhi" and "iniSigma2" are the initial values for the fitted variogram, "Phi" and "Sigma2" are the fitted values using weighted least squares, and "parSel" is the list of selected regressors for the mean function that minimize the Bayesian information criterion.

Federated Learning for Distributed Intrusion Detection Systems in Public Networks - Validation Dataset

This dataset has been meticulously prepared and utilized as a validation set during the evaluation phase of "Meta IDS" to asses the performance of various machine learning models. It is  now made available for interested users and researchers who seek a reliable and diverse dataset for training and testing their own custom models. The validation dataset comprises a comprehensive collection of labeled entries, that determines whether the packet type is "malicious" or "benign." It covers complex design patterns that are commonly encountered in real-world applications. The dataset is designed to be representative, encompassing edge and fog layers that are in contact with cloud layer, thereby enabling thorough testing and evaluation of different models. Each sample in the dataset is labeled with the corresponding ground truth, providing a reliable reference for model performance evaluation.   To ensure convenient distribution and storage, the dataset has been broken down into three separate batches, each containing a portion of the dataset. This allows for convenient downloading and management of the dataset. The three batches are provided as individual compressed files.   In order to extract the data, follow the following instructions: <ul> <li>Download and install bzip2 (if not already installed) from the official website or your package manager.</li> <li>Place the compressed dataset file in a directory of your choice.</li> <li>Open a terminal or command prompt and navigate to the directory where the compressed dataset file is located.</li> <li>Execute the following command to uncompress the dataset: <ul> <li>bzip2 -d filename.bz2</li> </ul> </li> <li>Replace "filename.bz2" with the actual name of the compressed dataset file.</li> </ul> Once uncompressed, you will have access to the dataset in its original format for further exploration, analysis, and model training etc. The total storage required for extraction is approximately 800 GB in total, with the first batch requiring approximately 302 GB, the second batch requiring approximately 203 GB, and the third batch requiring approximately 297 GB of data storage.   The first batch contains 1,049,527,992 entries, where as the second batch contains 711,043,331 entries, and for the third and last batch we have 1,029,303,062 entries. The following table provides the feature names along with their explanation and example value once the dataset is extracted.   <table align="left"> <thead> <tr> <th scope="col">Feature</th> <th scope="col">Description</th> <th scope="col">Example Value</th> </tr> </thead> <tbody> <tr> <td>ip.src</td> <td>Source IP address in the packet</td> <td>a05d4ecc38da01406c9635ec694917e969622160e728495e3169f62822444e17</td> </tr> <tr> <td>ip.dst</td> <td>Destination IP address in the packet</td> <td>a52db0d87623d8a25d0db324d74f0900deb5ca4ec8ad9f346114db134e040ec5</td> </tr> <tr> <td>frame.time_epoch</td> <td>Epoch time of the frame</td> <td>1676165569.930869</td> </tr> <tr> <td>arp.hw.type</td> <td>Hardware type</td> <td>1</td> </tr> <tr> <td>arp.hw.size</td> <td>Hardware size</td> <td>6</td> </tr> <tr> <td>arp.proto.size</td> <td>Protocol size</td> <td>4</td> </tr> <tr> <td>arp.opcode</td> <td>Opcode</td> <td>2</td> </tr> <tr> <td>data.len</td> <td>Length</td> <td>2713</td> </tr> <tr> <td>eth.dst.lg</td> <td>Destination LG bit</td> <td>1</td> </tr> <tr> <td>eth.dst.ig</td> <td>Destination IG bit</td> <td>1</td> </tr> <tr> <td>eth.src.lg</td> <td>Source LG bit</td> <td>1</td> </tr> <tr> <td>eth.src.ig</td> <td>Source IG bit</td> <td>1</td> </tr> <tr> <td>frame.offset_shift</td> <td>Time shift for this packet</td> <td>0</td> </tr> <tr> <td>frame.len</td> <td>frame length on the wire</td> <td>1208</td> </tr> <tr> <td>frame.cap_len</td> <td>Frame length stored into the capture file</td> <td>215</td> </tr> <tr> <td>frame.marked</td> <td>Frame is marked</td> <td>0</td> </tr> <tr> <td>frame.ignored</td> <td>Frame is ignored</td> <td>0</td> </tr> <tr> <td>frame.encap_type</td> <td>Encapsulation type</td> <td>1</td> </tr> <tr> <td>gre</td> <td>Generic Routing Encapsulation</td> <td>'Generic Routing Encapsulation (IP)’</td> </tr> <tr> <td>ip.version</td> <td>Version</td> <td>6</td> </tr> <tr> <td>ip.hdr_len</td> <td>Header length</td> <td>24</td> </tr> <tr> <td>ip.dsfield.dscp</td> <td>Differentiated Services Codepoint</td> <td>56</td> </tr> <tr> <td>ip.dsfield.ecn</td> <td>Explicit Congestion Notification</td> <td>2</td> </tr> <tr> <td>ip.len</td> <td>Total length</td> <td>614</td> </tr> <tr> <td>ip.flags.rb</td> <td>Reserved bit</td> <td>0</td> </tr> <tr> <td>ip.flags.df</td> <td>Don't fragment</td> <td>1</td> </tr> <tr> <td>ip.flags.mf</td> <td>More fragments</td> <td>0</td> </tr> <tr> <td>ip.frag_offset</td> <td>Fragment offset</td> <td>0</td> </tr> <tr> <td>ip.ttl</td> <td>Time to live</td> <td>31</td> </tr> <tr> <td>ip.proto</td> <td>Protocol</td> <td>47</td> </tr> <tr> <td>ip.checksum.status</td> <td>Header checksum status</td> <td>2</td> </tr> <tr> <td>tcp.srcport</td> <td>TCP source port</td> <td>53425</td> </tr> <tr> <td>tcp.flags</td> <td>Flags</td> <td>0x00000098</td> </tr> <tr> <td>tcp.flags.ns</td> <td>Nonce</td> <td>0</td> </tr> <tr> <td>tcp.flags.cwr</td> <td>Congestion Window Reduced (CWR)</td> <td>1</td> </tr> <tr> <td>udp.srcport</td> <td>UDP source port</td> <td>64413</td> </tr> <tr> <td>udp.dstport</td> <td>UDP destination port</td> <td>54087</td> </tr> <tr> <td>udp.stream</td> <td>Stream index</td> <td>1345</td> </tr> <tr> <td>udp.length</td> <td>Length</td> <td>225</td> </tr> <tr> <td>udp.checksum.status</td> <td>Checksum status</td> <td>3</td> </tr> <tr> <td>packet_type</td> <td>Type of the packet which is either "benign" or "malicious"</td> <td>0</td> </tr> </tbody> </table> Furthermore, in compliance with the GDPR and to ensure the privacy of individuals, all IP addresses present in the dataset have been anonymized through hashing. This anonymization process helps protect the identity of individuals while preserving the integrity and utility of the dataset for research and model development purposes.   Please note that while the dataset provides valuable insights and a solid foundation for machine learning tasks, it is not a substitute for extensive real-world data collection. However, it serves as a valuable resource for researchers, practitioners, and enthusiasts in the machine learning community, offering a compliant and anonymized dataset for developing and validating custom models in a specific problem domain.   By leveraging the validation dataset for machine learning model evaluation and custom model training, users can accelerate their research and development efforts, building upon the knowledge gained from my thesis while contributing to the advancement of the field.

Antarctic ice sheet daily surface melt detection from ASCAT (2007-2022)

Antarctic ice sheet-wide surface melt detection using enhanced resolution ASCAT C-band radar scatterometer data. Data are daily temporal resolution spanning 2007-2022 and gridded at 4.45 km. Melt detection approach follows Trusel et al., (2012) with updates to threshold and masking procedures. These data were used as a binary melt presence/absence estimate and a predictor in a machine learning-based estimation of Antarctic Peninsula surface meltwater production (https://zenodo.org/record/7995543).  Please reach out to Luke Trusel with any questions!