Find research datasets worth reusing

opencc-by-4.0Dec 2023View details →

Dataset for Machine Learning Assisted Citation Screening for Systematic Reviews

The work "Machine Learning Assisted Citation Screening for Systematic Reviews" explored the problem of citation screening automation using machine-learning (ML) with an aim to accelerate the process of generating <a href="https://en.wikipedia.org/wiki/Systematic_review#:~:text=Systematic%20reviews%20are%20a%20type,synthesize%20findings%20qualitatively%20or%20quantitatively." rel="nofollow">systematic reviews</a>. Manual process of citation screening involve two reviewers manually screening the searched studies using a predefined inclusion criteria. If the study passes the "inclusion" criteria, it is included for further analysis or is excluded. As apparant through manual screening process, the work considered citation screening as a binary classification problem whereby any ML classifier could be trained to separate the searched studies into these two classes (include and exclude).   A physiotherapy citation screening dataset was used to test automation approaches and the dataset includes the studies identified for citation screening in an update to the systematic review by Hilfiker et al. The dataset included titles and abstracts (citations) from 31,279 (deduplicated: 25,540) studies identified during the search phase of this SR. These studies were already manually assessed for relevance and labelled by two reviewers into two mutually exclusive labels. The uploaded file consists of 25,540 data samples, with each data sample separated by a new line. It is a tab separated file and the data in it is structured as shown below. This dataset was manually labelled into include and exclude by Hilfiker et al.   <table> <tbody> <tr> <td>Title</td> <td>PMID</td> <td>Abstract </td> <td>Class</td> <td>MeSH terms (separated by a pipe)</td> </tr> <tr> <td>Structured exercise improves physical functioning in women with stages I and II breast cancer: results of a randomized controlled trial.  </td> <td>11157015</td> <td>Abstract PURPOSE: Self-directed and supervised exercise were compared with usual care in a clinical trial designed to evaluate the effect of structured exercise on physical functioning and other dimensions of health-related quality of life in women with stages I and II breast cancer. PATIENTS AND METHODS: One hundred twenty-three women with stages I and II breast cancer completed baseline evaluations of generic and disease- and site-specific health-related quality of life, aerobic capacity, and body weight. Participants were randomly allocated to one of three intervention groups: usual care (control group), self-directed exercise, or supervised exercise. Quality of life, aerobic capacity, and body weight measures were repeated at 26 weeks...</td> <td>include or exclude</td> <td>Clinical Trial | Comparative Study | Randomized Controlled Trial | Research Support, Non-U.S. Gov't | Antineoplastic Combined Chemotherapy Protocols | Breast Neoplasms | Breast Neoplasms | Breast Neoplasms | Chemotherapy, Adjuvant | Exercise | Female | Humans | Middle Aged | Neoplasm Staging | Quality of Life | Radiotherapy, Adjuvant</td> </tr> </tbody> </table>   If you use this dataset in your research, please cite our papers.

opencc-by-4.0Oct 2023View details →

Stiffness Moduli Modelling and Prediction in Four-Point Bending of Asphalt Mixtures: A Machine Learning-Based Framework within Weave-UNISONO 2021 project, NCN project No 2021/03/Y/ST8/00079, and GACR project GA22-04047K

<div>Summary:</div> <div>Two selected mixtures were thoroughly investigated in an experimental trial carried out by means of a four-point bending test (4PBT) apparatus. The mixtures were prepared using spilite aggregate, a conventional 50/70 penetration grade bitumen, and limestone filler. Their stiffness moduli (SM) were determined while samples were exposed to 11 loading frequencies (from 0.1 to 50 Hz) and 4 testing temperatures (from 0 to 30 °C). Observations were recorded and used to develop a machine learning (ML) model. The main scope was the prediction of the stiffness moduli based on the volumetric properties and testing conditions of the corresponding mixtures, which would provide the advantage of reducing the laboratory efforts required to determine them.</div> <div> </div> <div>The dataset includes:</div> <div>Characteristics of bituminous binder, CSV raw data</div> <div> <ul> <li>bituminous binder.csv</li> </ul> </div> <div>Grading curves of tested asphalt mixtures</div> <ul> <li>AML16 Grading curves.csv</li> <li>AMP22 Grading curves.csv</li> </ul> <div>Volumetric characterizations of AML16 and AMP22 mixtures</div> <ul> <li>AML16 Volumetric characterizations.csv</li> <li>AMP22 Volumetric characterizations.csv</li> </ul> <div>Outcomes of the 4PBT experimental trial carried out on AML16 and AMP22 mixtures</div> <ul> <li>AML16 Stiffness Modulus 4PB.csv</li> <li>AMP22 Stiffness Modulus 4PB.csv</li> </ul>

opencc-by-4.0Nov 2022View details →

Database for machine learning of hydrogen storage materials properties

Database for machine learning of hydrogen storage materials properties Matthew Witmana, Mark Allendorfa, Vitalie Stavilaa aSandia National Laboratories, Livermore, CA   Description This ML-HydPARK dataset provides a csv file of metal hydride compositions, capacities, and thermodynamic values that can be used as target properties for building, training, and testing machine learning models. It has been parsed and cleaned from the DOE’s original publicly available HydPARK database according to the procedure in [1] to make it more suitable for immediate use with data-driven models. Generally, this removed duplicate entries, removed entries missing critical data, and attempted to fix various entries with obvious errors in the data. It is continuously updated under version control as new metal alloy hydrides are published in the open literature. Most entries contain data on the enthalpy and entropy of the hydriding reaction, as well the maximum hydrogen capacity, for which compositional machine learning models can be trained [1,2].   Acknowledgements The authors gratefully acknowledge research support from the U.S. Department of Energy, Office of Energy Efficiency and Renewable Energy, Fuel Cell Technologies Office through the Hydrogen Storage Materials Advanced Research Consortium (HyMARC). This work was supported by the Laboratory Directed Research and Development (LDRD) program at Sandia National Laboratories. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. This paper describes objective technical results and analysis. Any subjective views or opinions that might be expressed in the paper do not necessarily represent the views of the U.S. Department of Energy or the United States Government.   References <ol> <li>Witman, M.; Ling, S.; Grant, D. M.; Walker, G. S.; Agarwal, S.; Stavila, V.; Allendorf, M. D. Extracting an Empirical Intermetallic Hydride Design Principle from Limited Data via Interpretable Machine Learning. J. Phys. Chem. Lett. 2020, 11, 40–47.</li> <li>Witman, M.; Ek, G.; Ling, S.; Chames, J.; Agarwal, S.; Wong, J.; Allendorf, M. D.; Sahlberg, M.; Stavila, V. Data-Driven Discovery and Synthesis of High Entropy Alloy Hydrides with Targeted Thermodynamic Stability. Chem. Mater. 2021, 33, 4067–4076.</li> </ol>   Contact Please email <a href="mailto:mwitman@sandia.gov">mwitman@sandia.gov</a> , <a href="mailto:mdallen@sandia.gov">mdallen@sandia.gov</a>, or <a href="mailto:vnstavi@sandia.gov">vnstavi@sandia.gov</a> for questions or to request addition of recent data from the literature to this dataset.

opencc-by-4.0Feb 2024View details →

Machine learning predicts earthquakes in the continuum model of a rate-and-state fault with frictional heterogeneities

Numerical data used to make Figures in the manuscript entitled "Machine learning predicts earthquakes in the continuum model of a rate-and-state fault with frictional heterogeneities". We provide the data to create Figures 1 to 4 from the main text and Figures S1 to S9 from the supplementary information. We also provide Python scripts to plot them.

opencc-by-4.0Sep 2024View details →

3d Transition Metal K-edge XANES Dataset for Machine Learning Models

Data This dataset contains machine learning data for K-edge X-ray Absorption Near-Edge Structure (XANES) prediction models for eight 3d transition metals (Ti -Cu). <ul> <li>features_and_spectra: Material features (X) and corresponding XAS spectra (y) for each dataset split: training (train), validation (val), and test.</li> <li> material_id_and_site: Material identifiers and site indices (according to <a href="https://github.com/AI-multimodal/Lightshow">Lightshow</a>) for each dataset split. </li> </ul> Funding This research is based upon work supported by the U.S. Department of Energy, Office of Science, Office Basic Energy Sciences, under Award Number FWP PS-030. This research also used theory and computational resources of the Center for Functional Nanomaterials, which is a U.S. Department of Energy Office of Science User Facility, and the Scientific Data and Computing Center, at Brookhaven National Laboratory under Contract No. DE-SC0012704 and by Brookhaven National Laboratory (BNL), Laboratory Directed Research and Development (LDRD) grant no. 24-004.

EPL Points Prediction Using Machine Learning

This project leverages machine learning techniques to predict match outcomes and the final standings of the English Premier League (EPL). By utilizing historical match data collected through web scraping, we built a robust prediction model that forecasts the results of upcoming fixtures and projects the season's points table.The data pipeline includes comprehensive data preprocessing, feature engineering, and the application of five machine learning algorithms: K-Nearest Neighbors (KNN), Random Forest, AdaBoost, XGBoost, and CatBoost. These models were evaluated using accuracy metrics, achieving up to 96% accuracy in predicting match outcomes. Key features such as team form, home/away advantage, and head-to-head records were engineered to enhance the model's predictive power.The best-performing models, XGBoost and CatBoost, were used to simulate the remaining fixtures, allowing us to generate a predicted points table for the EPL. This project demonstrates the potential of data-driven approaches in sports analytics, offering insights for football clubs, analysts, and enthusiasts looking to understand team performance dynamics.

Machine learning-based quality assessment of Antarctic margins salinity - code, data and figures

The submission contains the data, functions and code needed to reproduce the figures in Sohail et al., 2025

Reference Mean and Low Streamflow for all Brazilian Catchments Generated Using Machine Learning Models

This dataset provides comprehensive hydrological information for river networks in Brazil, focusing on reference streamflows, specifically long-term mean flows (qm) and low flows exceeded 95% of the time (q95). Covering over 400,000 ungauged river points, the dataset was developed using advanced machine learning models trained on environmental descriptors and validated against data from 1,069 gauging stations spread across the country. The machine learning pipeline evaluated six regression models to achieve high predictive accuracy (R² > 0.8 for qm and > 0.7 for q95). The 62 environmental descriptors - encompassing climate, topography, land cover, lithology, and water storage characteristics - that were used as features for the models are also included. Key features: <ul> <li>Spatial Coverage: Brazilian territory and the Amazon River basin, based on the <a href="https://metadados.snirh.gov.br/geonetwork/srv/api/records/f7b1fc91-f5bc-4d0d-9f4f-f4e5061e5d8f" target="_blank" rel="noopener">BHO 5k</a> dataset of officially adopted river networks.</li> <li>Outputs: Predicted qm and q95 values for each river stretch, with 90% and 75% confidence intervals to account for prediction uncertainty.</li> <li>Environmental Descriptors: Aggregated from upstream catchment area.</li> </ul>

Data supplement: Spatial autocorrelation in machine learning for modelling soil organic carbon

Spatial autocorrelation in machine learning for modelling soil organic carbon: Data supplement Alexander Kmoch, Clay Taylor Harrison, Jeonghwan Choi, Evelyn Uuemaa Spatial autocorrelation, the relationship between nearby samples of a spatial random variable, is often overlooked in machine learning models, leading to biased results. This study investigates various methods to account for spa- tial autocorrelation when predicting soil organic carbon (SOC) using random forest models. Five models incorporating spatial structure were compared against baseline models that did not have any added spatial components. Cross-validation showed slight improvements in accuracy for models consid- ering spatial autocorrelation, while Shapley Additive Explanations confirmed the importance of spatial variables. However, no decrease in spatial autocor- relation of residuals was observed. Raster-based models exhibited enhanced prediction detail, but high-resolution validation data availability limited thor- ough validation. The findings emphasize the value of incorporating spatial autocorrelation for improved SOC prediction in machine learning models. Considerations such as the distribution of predictions and computational complexity should help guide the selection of suitable approaches for specific spatial modelling tasks.

opencc-by-4.0Dec 2024View details →

Glossed Hittite Texts with German Translation for Machine Learning

This dataset contains 7,099 processed Hittite texts from 143 CTH numbers, sourced with permission from the <a href="https://www.hethport.uni-wuerzburg.de/HPM/index.php" target="_blank" rel="noopener">Hethitologie Portal Mainz (HPM)</a>, which is the comprehensive resource of Hittite texts and culture (modern Turkey, c. 1,650 - 1,200 BCE). These texts have been converted from XML format to a tabular structure for computational and linguistic analysis, as well as machine learning applications, while attempting to preserve the full complexity of the source material. Key Features: <ul> <li>txtid: Identifier for the specific text (e.g., "IBoT 1.30+"), referencing entries in the HPM Konkordanz (S. Ko&scaron;ak, hethiter.net/: hetkonk (2.plus)).</li> <li>lnr: Surface, column (if extant), and line number within the text.</li> <li>cth_number: Catalogue des textes hittites (CTH) classification, organizing texts by genre and content (S. Ko&scaron;ak – G.G.W. Müller – S. Görke – Ch.W. Steitler, hethiter.net/: CTH (2022-10-26)).</li> <li>word: Orthographic form of the text as presented in the HPM, not in cuneiform but in a standard Latin script representation.</li> <li>translit: Detailed transliteration, preserving all nuances such as diacritics, broken parts, and editorial markings to reflect the original condition on the text.</li> <li>gloss: Linguistic glosses, including grammatical, morphological, and semantic information.</li> <li>trans_de: German translation of the word or phrase.</li> </ul>

opencc-by-4.0Apr 2023View details →

Antisemitism on Twitter: A Dataset for Machine Learning and Text Analytics

<h1>Dataset from the Institute for the Study of Contemporary Antisemitism (ISCA) at Indiana University:  </h1>   <div> <div> The Social Media & Hate research lab at the Institute for the Study of Contemporary Antisemitism compiled this dataset using an annotation portal (Jikeli, Soemer, and Karali 2024), which was used to label tweets as either antisemitic or non-antisemitic, among other labels. Note that annotation was done on live data, including images and context, such as threads. All data was annotated by two experts, and all discrepancies were discussed (Jikeli et al. 2023).  </div> </div> Content: This dataset contains 11311 tweets covering a wide range of topics common in conversations about Jews, Israel, and antisemitism between January 2019 and April 2023. The dataset consists of random samples of relevant keywords during this time period. 1,953 tweets (17%) are antisemitic according to the IHRA definition of antisemitism.    <div> The distribution of tweets by year is as follows: 1499 (13%) from 2019, 3712 (33%) from 2020, 2591 (23%) from 2021, 2644 from 2022 (23%) and 865 (8%) from 2023. 6365 (56%) contain the keyword "Jews," 4134 (37%) include "Israel," 529 (5%) feature the derogatory term "ZioNazi*," and 283 (3%) use the slur "K---s." Some tweets may contain multiple keywords.   </div> <div> 725 out of the 6365 tweets with the keyword "Jews" (11%) and 664 out of the 4134 tweets with the keyword "Israel" (16%) were classified as antisemitic. 97 out of the 283 tweets using the antisemitic slur "K---s" (34%) are antisemitic. Interestingly, many tweets featuring the slur "K---s" actually call out its use. In contrast, the majority of tweets using the derogatory term "ZioNazi*" are antisemitic, with 467 out of 529 (88%) being classified as such.   </div>   File Description:  <div> <div> The dataset is provided in a csv file format, with each row representing a single message, including replies, quotes, and retweets. The file contains the following columns:   </div> <div>    ‘ID’: Represents the tweet ID.   </div> <div> ‘Username’: Represents the username that posted the tweet.    </div> <div> ‘Text’: Represents the full text of the tweet (not pre-processed).  </div> <div> ‘CreateDate’: Represents the date on which the tweet was created.    </div> <div> ‘Biased’: Represents the label given by our annotations as to whether the tweet is antisemitic or not.  </div> <div> ‘Keyword’: Represents the keyword that was used in the query. The keyword can be in the text, including hashtags, mentioned users, or the username itself.    </div> </div>   Licences  Data is published under the terms of the "Creative Commons Attribution 4.0 International" licence (https://creativecommons.org/licenses/by/4.0)    Acknowledgements  We are grateful for the support of Indiana University’s Observatory on Social Media (OSoMe) (Davis et al. 2016) and the contributions and annotations of all team members in our Social Media & Hate Research Lab at Indiana University’s Institute for the Study of Contemporary Antisemitism, especially Grace Bland, Elisha S. Breton, Kathryn Cooper, Robin Forstenhäusler, Sophie von Máriássy, Mabel Poindexter, Jenna Solomon, Clara Schilling, and Victor Tschiskale.  This work used Jetstream2 at Indiana University through allocation HUM200003 from the Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS) program, which is supported by National Science Foundation grants #2138259, #2138286, #2138307, #2137603, and #2138296.

Parsimonious machine learning for the global mapping of aboveground biomass density

This repository hosts data and code presented in the article "Parsimonious machine learning for the global mapping of aboveground biomass potential". The repository contains a compressed file containing all the code needed to reproduce the methodology that we developed and to analyse its results. We did not upload all the temporary and intermediate data files that are created during the execution of the method. We rather uploaded "milestone" data, i.e. final results or important intermediate ones. This includes the final training dataset, model calibration data,  the final trained model, the global data for prediction, the final global map of potential aboveground biomass density (AGBD) at present times (raster files at 1km2 and 10km2 resolution), maps depicting regions where climatic conditions are outside of the training range of positive AGBD instances and maps depicting world regions without trees.  Files: <a href="../api/records/11580414/draft/files/code.zip/content" target="_blank" rel="noopener noreferrer">code.zip</a> : Compressed directory with all the code needed to reproduce the methodology presented in the manuscript. Contains a README file. Also contains temporary data generated in the process, the training dataset, the trained model, and model calibration data. <a href="../api/records/11580414/draft/files/potential_AGBD_Mgha_1km2_contemporary_climate.tif/content" target="_blank" rel="noopener noreferrer">potential_AGBD_Mgha_1km_present_climate_1980_2010.tif</a> : the predicted global potential AGBD under contemporary climate conditions and at a resolution of 1 squared kilometer. <a href="../api/records/11580414/draft/files/potential_AGBD_Mgha_1km2_contemporary_climate.tif/content" target="_blank" rel="noopener noreferrer">potential_AGBD_Mgha_10km_</a><a href="../api/records/11580414/draft/files/potential_AGBD_Mgha_1km2_contemporary_climate.tif/content" target="_blank" rel="noopener noreferrer">present_climate_1980_2010.tif</a> : the predicted global potential AGBD under contemporary climate conditions downsampled at a resolution of 10 squared kilometers. <a href="../api/records/11580414/draft/files/potential_AGBD_Mgha_1km2_contemporary_climate.tif/content" target="_blank" rel="noopener noreferrer">potential_AGBD_Mgha_10km_model_difference.tif</a> : the difference between our prediction of potential AGBD and the prediction from a complex state-of-the-art model from Walker et al. (2022).  <a href="../api/records/11580414/draft/files/potential_AGBD_Mgha_1km2_contemporary_climate.tif/content" target="_blank" rel="noopener noreferrer">potential_AGB_Mg_1km_</a><a href="../api/records/11580414/draft/files/potential_AGBD_Mgha_1km2_contemporary_climate.tif/content" target="_blank" rel="noopener noreferrer">present_climate_1980_2010.tif</a> : the predicted global potential pixel-level AGB under contemporary climate conditions downsampled at a resolution of 1 squared kilometers. <a href="../api/records/11580414/draft/files/number_predictors_out_of_range.zip/content" target="_blank" rel="noopener noreferrer">number_predictors_out_of_range.zip</a> : tiled maps representing the number of climatic predictors outside of the training range before including 0 AGBD instances in the training dataset.  <a href="../api/records/11580414/draft/files/tree_absence_map.zip/content" target="_blank" rel="noopener noreferrer">tree_absence_map.zip</a> : tiled maps representing world regions without trees. Based on Crowther et al. (2015) (https://elischolar.library.yale.edu/yale_fes_data/1/). <a href="../api/records/11580414/draft/files/potential_agbd_Mgha_climate_envelope.pkl/content" target="_blank" rel="noopener noreferrer">inference_pipeline_potential_agbd_Mgha_climate.pkl</a> : Calibrated model for the prediction of potential AGBD given bioclimatic conditions.  <a href="../api/records/11580414/draft/files/predictors_data_global.zip/content" target="_blank" rel="noopener noreferrer">predictors_data_global.zip</a> : Global predictors data to apply the model on.

opencc-by-4.0Sep 2024View details →

Machine Learning Predicts Meter-Scale Laboratory Earthquakes

Numerical data and Python scripts used to make Figures in the manuscript entitled "Machine Learning Predicts Meter-Scale Laboratory Earthquakes". We provide the data and scripts to create Figures 1 to 6 from the main text and Figures S1 to S15 from the supplementary information. Note that you should download the experimental catalog of Yamashita et al. (Nature Comm, 2021) and address the request for shear force data (experimental number LB12-011) to Futoshi Yamashita, the organizer of the target experiment. This code is developed using Python 3.11.7.

opencc-by-4.0Oct 2023View details →

Simulated datasets for detector and particle flow reconstruction: CLIC detector, machine learning format

Synopsis Machine-learning friendly format of tracks, clusters and target particles in electron-positron events, simulated with the CLIC detector. Ready to be used with <a href="https://zenodo.org/records/14930299">jpata/particleflow:v2.3.0</a>. Derived from the EDM4HEP ROOT files in <a href="https://zenodo.org/record/8260741">https://zenodo.org/record/8260741</a>. <ul> <li>clic_edm_ttbar_pf.zip: e+e- -> ttbar, center of mass energy at 380 GeV</li> <li>clic_edm_qq_pf.zip: e+e- -> Z* -> qqbar, center of mass energy at 380 GeV</li> <li>clic_edm_ww_fullhad_pf.zip: e+e- -> WW -> W decaying hadronically, center of mass energy at 380 GeV</li> <li>clic-tfds.ipynb: an example notebook on how to load the files</li> </ul> Contents Each .zip file contains the dataset in the <a href="https://github.com/tensorflow/datasets">tensorflow-datasets</a>, <a href="https://github.com/google/array_record">array_record</a> format. We have split the full datasets into 10 subsets, due to space considerations on zenodo, two subsets from each dataset are uploaded. Each dataset contains a train and test split of events. Dataset semantics (to be updated) Each dataset consists of events that can be iterated over using the tensorflow-datasets library and used in either tensorflow or pytorch. Each event has the following information available: <ul> <li>X: the reconstruction input features, i.e. tracks and clusters</li> <li>ytarget: the ground truth particles with the features ["PDG", "charge", "pt", "eta", "sin_phi", "cos_phi", "energy", "jet_idx"], with "jet_idx" corresponding to the gen-jet assignment of this particle</li> <li>ycand: the baseline Pandora PF particles with the features ["PDG", "charge", "pt", "eta", "sin_phi", "cos_phi", "energy", "jet_idx"], with "jet_idx" corresponding to the gen-jet assignment of this particle</li> </ul> The full semantics, including the list of features for X, are available at https://github.com/jpata/particleflow/blob/v2.3.0/mlpf/heptfds/clic_pf_edm4hep/utils_edm.py and https://github.com/jpata/particleflow/blob/v2.3.0/mlpf/data/key4hep/postprocessing.py.

opencc-by-4.0Dec 2021View details →

Machine-Learning of Aerosol-Cloud-Climate Interactions Reveals an increased Cloud Fraction

Data presented in the manuscript "Machine-Learning of Aerosol-Cloud-Climate Interactions Reveals an increased Cloud Fraction" by Chen et al. (2022).

opencc-by-4.0Oct 2021View details →

Machine learning code and dataset for "Nowcasting thunderstorm hazards using machine learning: the impact of data sources on performance"

This repository contains the code and dataset for the paper: Nowcasting thunderstorm hazards using machine learning: the impact of data sources on performance, Natural Hazards and Earth System Sciences, 2022, <a href="https://doi.org/10.5194/nhess-2021-171">https://doi.org/10.5194/nhess-2021-171</a> The GitHub code repository at <a href="https://github.com/meteoswiss-mdr/ts-nowcast-datasources">https://github.com/meteoswiss-mdr/ts-nowcast-datasources</a> may contain a more up-to-date version of the code if bug fixes etc. have been necessary. The file <a href="https://zenodo.org/api/files/41faa1b7-17f6-4a75-be09-7743426ef13c/ts-nowcast-datasources-publication.zip">ts-nowcast-datasources-publication.zip</a> in this Zenodo release contains the status of the GitHub repository at the time of the publication of the paper. For instructions for using the data, please see the <a href="https://github.com/meteoswiss-mdr/ts-nowcast-datasources">code repository</a>.

SMART - Self-adaptive Machine Learning Approach for Real-time Tuning of IEEE 802.11 PHY and MAC layers

Introduction Worldwide the demand for wireless access networks providing very high throughputs has been increasing exponentially, namely due to bandwidth-hungry applications such as high definition video streaming and augmented reality. In order to fulfil these requirements, the Wi-Fi standard was enriched with new amendments, such as IEEE 802.11n, IEEE 802.11ac, and recently IEEE 802.11ax (Wi-Fi 6). New parameters have been proposed for both physical (PHY) and media access control (MAC) layers, including channel bonding, short guard interval (SGI), and advanced modulation and coding schemes (MCS). However, the high variability of the signal strength in the wireless radio channel, allied to the channel asymmetry, makes the selection of optimal configurations for these parameters a challenge. Typically, these parameters are configured with a default value. For runtime optimization, some algorithms have already been proposed. Still, they were designed considering legacy IEEE 802.11 releases and static scenarios. Besides, these parameters have their trade-offs that need to be properly managed. To help dealing with this, machine learning has been recently introduced in wireless networks, providing the intelligence that networks need in order to be smart and self-adaptive. SWOP (Smart Wireless Optimization) is a cross-layer optimization approach for Wi-Fi networks extending the current Rate Adaptation (RA) approach, for instance, followed by the well-known Minstrel algorithm widely used in practice. Our approach takes advantage of Deep Reinforcement Learning (DRL) in order to learn the optimal Wi-Fi link configuration. By considering the wireless channel as the environment, the transmitter node (the agent) chooses the best link parameters (the action) in order to maximize the throughput (the reward) based on the channel metrics captured from the environment (the state). In this work we propose a simple DRL-based Wi-Fi Rate Adaptation (RA) algorithm, named Data-driven Algorithm for Rate Adaptation (DARA), which is one of the modules of Smart Wireless Optimization (SWOP) SMART aimed to run a set of wireless experiments on top of w-iLab.t testbeds provided by the Fed4FIRE+ project to directly validate our DRL model and learn a policy from the wireless experiments executed in a controlled environment. However, after facing difficulties with the scenarios we could achieve on the real testbed, we decided to train and test DARA using a trace-based simulation approach. In simulation, we could train our model in scenarios that are more complex and diverse whilst easy to configure, when compared to real testbeds. The w-iLab.t testbeds were still used to capture data traces (e.g. Signal-to-Noise Ratio, position of nodes, transmission power and link distance) that were then injected in ns-3 for validating DARA. With this work, we concluded that DARA performance is impacted when operating in scenarios with asymmetric links, which is common in the highly dynamic and unpredictable wireless environments. Furthermore, the asymmetry offset varies between scenarios and it may also change for the same scenario, as time progresses. This randomness is not addressed when solely considering the SNR as the link metric, posing a challenge in the learning phase of DARA. Despite these limitations, the results obtained show that DARA still achieves up to 14.9% higher throughput higher than Minstrel [1]  and slightly lower than Ideal  [2] for most of the scenarios. The results obtained will serve as a basis to support our ongoing and future research.   Folder Organization The following dataset presents the results of the SMART project, organized in different folders for each Rate Adaptation Algorithm, as well as the traces that were used to obtain such results: <ul> <li>DARA: Results obtained using our solution (Naming Convention #1, Folder Content #1)</li> <li>MIN: Results obtained using Minstrel-HT (Naming Convention #1, Folder Content #2)</li> <li>ID: Results obtained using Ideal (Naming Convention #1, Folder Content #2)</li> <li>TRACES: Trace files used to obtain the results present in this dataset (Naming Convention #2, Folder Content #3)</li> </ul> Naming Convention #1 – RAA TID TP TO: <ul> <li>Rate Adaptation Algorithm (RAA)  <ul> <li>drl – Data Driven Algorithm for Rate Adaptation</li> <li>min – MinstrelHTWifiManager</li> <li>id – IdealWifiManager</li> </ul> </li> <li>Trace ID (TID) <ul> <li>3 up to 8</li> </ul> </li> <li>Transport Protocol (TP) <ul> <li>udp – User Datagram Protocol</li> </ul> </li> <li>Traffic Orientation (TO) <ul> <li>normal – A->B</li> <li>reversed – B->A</li> </ul> </li> </ul> Naming Convention #2 – TID_TXP: <ul> <li>Trace ID (TID)  <ul> <li>3 up to 8</li> </ul> </li> <li>Transmitting Power in dBm (TXP)  <ul> <li>3, 5, 7, 9, 12 dBm </li> </ul> </li> </ul> Folder Content #1: <ul> <li>checkpoint_ RAA TID TP TO (Folder) <ul> <li>Policy Checkpoint with which the results were obtained</li> </ul> </li> <li> <ul> <li>Flowmonitor output for the configured scenario</li> </ul> </li> <li> <ul> <li>Column 1 – Step Counter</li> <li>Column 2 – Reward Value</li> <li>Column 3 – Observation Value</li> <li>Column 4 – Action Value</li> </ul> </li> <li> <ul> <li>Column 1 – Simulation Time (seconds)</li> <li>Column 2 – Throughput (Mbit/100ms)</li> </ul> </li> </ul> Folder Content #2: <ul> <li> <ul> <li>Flowmonitor output for the configured scenario</li> </ul> </li> <li> <ul> <li>Column 1 – Simulation Time (seconds)</li> <li>Column 2 – Throughput (Mbit/100ms)</li> </ul> </li> </ul> Folder Content #3 - Source: <a href="https://zenodo.org/record/3713271#.YjjBVDXLdhE">https://zenodo.org/record/3713271#.YjjBVDXLdhE</a>: ·  date_time.cfg configuration details of the experiment ·  date_time_NodeID<a href="https://zenodo.org/record/3713271#_ftn1">[1]</a>_SenderID<a href="https://zenodo.org/record/3713271#_ftn2">[2]</a>_ReceiverID<a href="https://zenodo.org/record/3713271#_ftn3">[3]</a>_FlowType<a href="https://zenodo.org/record/3713271#_ftn4">[4]</a>_Params<a href="https://zenodo.org/record/3713271#_ftn5">[5]</a>.snr – logs of the Signal/Noise ratio (1 file per node/flow)   ·  date_time_NodeID_SenderID_ReceiverID_FlowType_Params.stats – logs of the packets received (1 file per node/flow)   <a href="https://zenodo.org/record/3713271#_ftnref1">[1]</a> ID of the node Logging node <a href="https://zenodo.org/record/3713271#_ftnref2">[2]</a> ID of the Sender node <a href="https://zenodo.org/record/3713271#_ftnref3">[3]</a> ID of the Receiver node <a href="https://zenodo.org/record/3713271#_ftnref4">[4]</a> Flow type: Unidirectional, Bidirectional or Unidirectional with Multiple Access <a href="https://zenodo.org/record/3713271#_ftnref5">[5]</a> Configurable parameters: Sender/Receiver Transmission Power and Data Rate (when applicable)   References 1.          F. FietKau, “Minstrel_HT: New rate control module for 802.11n [LWN.net]”. Mrt-2010. 2.          “ns-3: ns3::IdealWifiManager Class Reference,” Jan 2021, [Online; accessed 23. Jun. 2021]. Available: <a href="https://www.nsnam.org/docs/release/3.33/doxygen/classns3_1_1_ideal_wifi%20manager.html">https://www.nsnam.org/docs/release/3.33/doxygen/classns3_1_1_ideal_wifi manager.html</a>

First Three-dimensional Quantification of Planktic Food Chain lower levels (Copepods) for the Ross Sea region Marine Protected Area (RSRMPA), Antarctica: Using FAIR-inspired legacy data with Machine Learning, and Open Source GIS

This dataset is relative to the paper entitled: "First Three-dimensional Quantification of Planktic Food Chain lower levels (Copepods) for the Ross Sea region Marine Protected Area (RSRMPA), Antarctica: Using FAIR-inspired legacy data with Machine Learning, and Open Source GIS" publishing in journal Diversity (MPDI). Abstract: Zooplankton is a fundamental group in all aquatic ecosystems located the base of the food chain. It forms a link between the lower trophic levels with secondary consumers and shows marked fluctuations of populations with environmental change, especially reacting to heating and water acidification. At sea copepod crustaceans account for app. 70% in abundance of zooplankton and are a target of monitoring activities in key areas such as the Southern Ocean. In this study we have used FAIR-inspired legacy data (dating back to the ‘80s) collected in the Ross Sea by the Italian National Antarctic Program in GBIF.org. Together with other open-access GIS data sources and tools it allows generating, for the first time, three-dimensional predictive distribution maps for twenty-six copepod species. These predictive maps were obtained by applying machine learning techniques to grey literature data, which were visualized in open-source GIS platforms. In a Species Distribution Modeling (SDM) framework we used machine learning with three types of algorithms (TreeNet, RandomForest and Ensemble) to analyze the presence and absence of copepods at different areas and depth classes in function of environmental descriptors obtained from the Polar Macroscope Layers present in Quantartica. The models allow for the first time to map-predict the food chain in quantitative terms showing the relative index of occurrence (RIO) and identified the presence for each copepod species analyzed in the Ross Sea. Our results show marked geographical preferences that vary with species and trophic strategy. This study demonstrates that machine learning is a successful method in accurately predicting Antarctic copepod presence, also providing useful data to orient future sampling and management of wildlife and conservation.

Image-based & machine learning-guided multiplexed serology test for SARS-CoV-2

Single-cell extracted imaging features created in project "Image-based & machine learning-guided multiplexed serology test for SARS-CoV-2". The dataset includes train (with annotations) and test features used in the manuscript. Four SARS-CoV-2 antigens (S, N, R, M) were imaged separately with serum samples presenting IgG, IgA and IgM antibodies.

Improving The Diagnosis of Thyroid Cancer by Machine Learning and Clinical Data

This repository contains the dataset used in the paper "Improving The Diagnosis of Thyroid Cancer by Machine Learning and Clinical Data" published in Scientific Reports. Please check our <a href="https://www.nature.com/articles/s41598-022-15342-z">formal publication</a> for the full details. The dataset contains 1232 nodules from 724 patients. Each row represents one nodule and each column represents one variable that describes the characteristics of the patient or nodule. The meaning of each variable is summarized below. <ul> <li>id: the unique identity of the patient who carries the nodule</li> <li>age: the age of the patient</li> <li>FT3: triiodothyronine test result</li> <li>FT4: thyroxine test result</li> <li>TSH: thyroid-stimulating hormone test result</li> <li>TPO: thyroid peroxidase antibody test result</li> <li>TGAb: thyroglobulin antibodies test result</li> <li>site: the nodule location, 0: right, 1: left, 2: isthmus</li> <li>echo_pattern: thyroid echogenicity, 0: even, 1: uneven</li> <li>multifocality: if multiple nodules exist in one location, 0: no, 1: yes</li> <li>size: the nodule size in cm</li> <li>shape: the nodule shape, 0: regular, 1: irregular</li> <li>margin: the clarity of nodule margin, 0: clear; 1: unclear</li> <li>calcification: the nodule calcification, 0: absent, 1: present</li> <li>echo_strength: the nodule echogenicity, 0: none, 1: isoechoic, 2: medium-echogenic, 3: hyperechogenic, 4: hypoechogenic</li> <li>blood_flow: the nodule blood flow, 0: normal, 1: enriched</li> <li>composition: the nodule composition, 0: cystic, 1: mixed, 2: solid</li> <li>multilateral: if nodules occur in more than one location, 0: no, 1: yes</li> <li>mal: the nodule malignancy, 0: benign, 1: malignant</li> </ul>