Find research datasets worth reusing

opencc-by-4.0Jun 2023View details →

ROADMAP - Mini Webinar Series

The ROADMAP project worked on rethinking the use of antimicrobials in livestock production systems. In a mini webinar series, we have asked task leaders to summarise their key learnings from the project. This has resulted in the following six episodes: <ol> <li>Massimo Canali (UNIBO): "Stakeholders' behaviour and strategies towards AMU"</li> <li>Lee-Ann Sutherland (HUT): "Identifying actors' motivations in the use and reduction of antimicrobials"</li> <li>Sophie Molia (CIRAD): "Creating impact from the assessed strategies"</li> <li>Mette Vaarst (AU): "Co-building levers and incentives"</li> <li>João Sucena Afonso (ULIV): "Key learnings on the impact of alternatives in livestock and aquaculture production"</li> <li>Bernadette Oehen (FiBL): "Implementing innovative strategies"</li> </ol> All mini webinars can be watched here: https://www.youtube.com/playlist?list=PLW5PYxzlSCfzzdqn2n6-eCxfkGhRAT6Qy

opencc-by-4.0Jun 2023View details →

Network traffic datasets created by Single Flow Time Series Analysis

Network traffic datasets created by Single Flow Time Series Analysis Datasets were created for the paper: Network Traffic Classification based on Single Flow Time Series Analysis -- Josef Koumar, Karel Hynek, Tomá&scaron; Čejka -- which was published at The 19th International Conference on Network and Service Management (CNSM) 2023. Please cite usage of our datasets as:   <blockquote> J. Koumar, K. Hynek and T. Čejka, "Network Traffic Classification Based on Single Flow Time Series Analysis," 2023 19th International Conference on Network and Service Management (CNSM), Niagara Falls, ON, Canada, 2023, pp. 1-7, doi: 10.23919/CNSM59352.2023.10327876. </blockquote> This Zenodo repository contains 23 datasets created from 15 well-known published datasets which are cited in the table below. Each dataset contains 69 features created by Time Series Analysis of Single Flow Time Series. The detailed description of features from datasets is in the file: feature_description.pdf   In the following table is a description of each dataset file: <table> <tbody> <tr> <td>File name</td> <td>Detection problem</td> <td>Citation of original raw dataset</td> </tr> <tr> <td>botnet_binary.csv </td> <td>Binary detection of botnet </td> <td>S. García et al. An Empirical Comparison of Botnet Detection Methods. Computers & Security, 45:100–123, 2014. </td> </tr> <tr> <td>botnet_multiclass.csv </td> <td>Multi-class classification of botnet </td> <td>S. García et al. An Empirical Comparison of Botnet Detection Methods. Computers & Security, 45:100–123, 2014. </td> </tr> <tr> <td>cryptomining_design.csv</td> <td>Binary detection of cryptomining; the design part </td> <td>Richard Plný et al. Datasets of Cryptomining Communication. Zenodo, October 2022 </td> </tr> <tr> <td>cryptomining_evaluation.csv </td> <td>Binary detection of cryptomining; the evaluation part </td> <td>Richard Plný et al. Datasets of Cryptomining Communication. Zenodo, October 2022 </td> </tr> <tr> <td>dns_malware.csv </td> <td>Binary detection of malware DNS </td> <td>Samaneh Mahdavifar et al. Classifying Malicious Domains using DNS Traffic Analysis. In DASC/PiCom/CBDCom/CyberSciTech 2021, pages 60–67. IEEE, 2021. </td> </tr> <tr> <td>doh_cic.csv </td> <td>Binary detection of DoH </td> <td> Mohammadreza MontazeriShatoori et al. Detection of doh tunnels using time-series classification of encrypted traffic. In DASC/PiCom/CBDCom/CyberSciTech 2020, pages 63–70. IEEE, 2020  </td> </tr> <tr> <td>doh_real_world.csv </td> <td>Binary detection of DoH </td> <td>Kamil Jeřábek et al. Collection of datasets with DNS over HTTPS traffic. Data in Brief, 42:108310, 2022 </td> </tr> <tr> <td>dos.csv </td> <td>Binary detection of DoS </td> <td>Nickolaos Koroniotis et al. Towards the development of realistic botnet dataset in the Internet of Things for network forensic analytics: Bot-IoT dataset. Future Gener. Comput. Syst., 100:779–796, 2019.</td> </tr> <tr> <td>edge_iiot_binary.csv </td> <td>Binary detection of IoT malware </td> <td>Mohamed Amine Ferrag et al. Edge-iiotset: A new comprehensive realistic cyber security dataset of iot and iiot applications: Centralized and federated learning, 2022.</td> </tr> <tr> <td>edge_iiot_multiclass.csv</td> <td>Multi-class classification of IoT malware</td> <td>Mohamed Amine Ferrag et al. Edge-iiotset: A new comprehensive realistic cyber security dataset of iot and iiot applications: Centralized and federated learning, 2022.</td> </tr> <tr> <td>https_brute_force.csv</td> <td>Binary detection of HTTPS Brute Force</td> <td>Jan Luxemburk et al. HTTPS Brute-force dataset with extended network flows, November 2020</td> </tr> <tr> <td>ids_cic_binary.csv</td> <td>Binary detection of intrusion in IDS</td> <td>Iman Sharafaldin et al. Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISSp, 1:108–116, 2018.</td> </tr> <tr> <td>ids_cic_multiclass.csv </td> <td>Multi-class classification of intrusion in IDS </td> <td>Iman Sharafaldin et al. Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISSp, 1:108–116, 2018. </td> </tr> <tr> <td>ids_unsw_nb_15_binary.csv </td> <td>Binary detection of intrusion in IDS </td> <td>Nour Moustafa and Jill Slay. Unsw-nb15: a comprehensive data set for network intrusion detection systems (unsw-nb15 network data set). In 2015 military communications and information systems conference (MilCIS), pages 1–6. IEEE, 2015.</td> </tr> <tr> <td>ids_unsw_nb_15_multiclass.csv </td> <td>Multi-class classification of intrusion in IDS </td> <td>Nour Moustafa and Jill Slay. Unsw-nb15: a comprehensive data set for network intrusion detection systems (unsw-nb15 network data set). In 2015 military communications and information systems conference (MilCIS), pages 1–6. IEEE, 2015.</td> </tr> <tr> <td>iot_23.csv </td> <td>Binary detection of IoT malware </td> <td>Sebastian Garcia et al. IoT-23: A labeled dataset with malicious and benign IoT network traffic, January 2020. More details here https://www.stratosphereips.org /datasets-iot23</td> </tr> <tr> <td>ton_iot_binary.csv </td> <td>Binary detection of IoT malware </td> <td>Nour Moustafa. A new distributed architecture for evaluating ai-based security systems at the edge: Network ton iot datasets. Sustainable Cities and Society, 72:102994, 2021</td> </tr> <tr> <td>ton_iot_multiclass.csv </td> <td>Multi-class classification of IoT malware </td> <td>Nour Moustafa. A new distributed architecture for evaluating ai-based security systems at the edge: Network ton iot datasets. Sustainable Cities and Society, 72:102994, 2021</td> </tr> <tr> <td>tor_binary.csv </td> <td>Binary detection of TOR </td> <td>Arash Habibi Lashkari et al. Characterization of Tor Traffic using Time based Features. In ICISSP 2017, pages 253–262. SciTePress, 2017. </td> </tr> <tr> <td>tor_multiclass.csv </td> <td>Multi-class classification of TOR </td> <td>Arash Habibi Lashkari et al. Characterization of Tor Traffic using Time based Features. In ICISSP 2017, pages 253–262. SciTePress, 2017. </td> </tr> <tr> <td>vpn_iscx_binary.csv </td> <td>Binary detection of VPN </td> <td>Gerard Draper-Gil et al. Characterization of Encrypted and VPN Traffic Using Time-related. In ICISSP, pages 407–414, 2016. </td> </tr> <tr> <td>vpn_iscx_multiclass.csv </td> <td>Multi-class classification of VPN </td> <td>Gerard Draper-Gil et al. Characterization of Encrypted and VPN Traffic Using Time-related. In ICISSP, pages 407–414, 2016. </td> </tr> <tr> <td>vpn_vnat_binary.csv </td> <td>Binary detection of VPN </td> <td>Steven Jorgensen et al. Extensible Machine Learning for Encrypted Network Traffic Application Labeling via Uncertainty Quantification. CoRR, abs/2205.05628, 2022</td> </tr> <tr> <td>vpn_vnat_multiclass.csv</td> <td>Multi-class classification of VPN </td> <td>Steven Jorgensen et al. Extensible Machine Learning for Encrypted Network Traffic Application Labeling via Uncertainty Quantification. CoRR, abs/2205.05628, 2022</td> </tr> </tbody> </table>

opencc-by-4.0Jun 2023View details →

Pre-processed and modeled GNSS time-series after the 2011 Tohoku Earthquake

The raw, pre-processed, and modeled GNSS time-series of the 213 GEONET sites in the Tohoku region, Japan, from Mar. 12, 2011 to Nov. 20, 2021, relative to the Okhotsk plate (Argus et al., 2011, Geochemistry, Geophysics, Geosystems). The original GNSS time-series are F5 solutions, which are distributed by Geospatial Information Authority of Japan (GSI, https://www.gsi.go.jp/). The details and availability of F5 solutions are written in Takamatsu et al. (2023, Earth, Planets, and Space) https://doi.org/10.1186/s40623-023-01787-7. The GNSS time-series processing was performed by Tomita (submitted), and the following signals were excluded from the raw time-series: seasonal variation, coseismic step, antenna maintenance offset, and common mode errors. Then, the pre-processed time-series were modeled by a trajectory modeling method considering postseismic deformation of the 2011 Tohoku earthquake, the Boso SSEs, and postseismic deformations due to aftershocks and L-ASE (long-term aseismic slip event) since late 2019. "sitelist.txt" - Site information file column 1: Full site ID column 2: 4digits site ID column 3: Longitude [deg] column 4: Latitude [deg] column 5: Height [m]  "pre-process/xxxx.txt" - Time-series at xxxx (4digits site ID) site column 1: days from Mar. 12, 2011 (1 corresponds to Mar. 12, 2011) column 2: raw East-West displacement [m] column 3: raw North-South displacement [m] column 4: raw Up-down displacement [m] column 5: pre-processed East-West displacement [m] column 6: pre-processed North-South displacement [m] column 7: pre-processed Up-down displacement [m] "model/xxxx/prediction_yy.txt" - Time-series for yy component (yy=EW, NS, UD) at xxxx (4digits site ID) site column 1: days from Mar. 12, 2011 (1 corresponds to Mar. 12, 2011) column 2: modeled displacement excluding the Boso SSEs [m] column 3: modeled displacement excluding the Boso SSEs and postseismic deformation due to aftershocks caused one year after the 2011 Tohoku Eq. [m] column 4: modeled displacement excluding the Boso SSEs, postseismic deformation due to aftershocks caused one year after the 2011 Tohoku Eq. and the 2019 L-ASE [m] The displacement on Mar. 12, 2011 was initially set to be zero before the pre-processing, but the removal of the above factors provided some deviation from zero. The raw time-series excluded outliers from the original F5 solutions, and the raw time-series were transformed into the Okhotsk plate reference. Following the above trajectory modeling, the fully-relaxed postseismic displacement fields due to 2015 Feb. 17 Sanriku-oki earthquake ("Table_displacement1.xlsx"), the 2015 May 13 Miyagi-oki earthquake ("Table_displacement2.xlsx"), and summation of the 2021 Feb. 13 Fukushima-oki, the 2021 Mar. 20 Miyagi-oki, and the 2021 May 1 earthquakes ("Table_displacement2.xlsx") were calculated. Moreover, the cumulative displacement field due to the 2019 L-ASE since Nov. 25, 2019 was also calculated. In those files, the estimation errors are also shown as 1σ standard deviation obtained from diagonal components of the model covariance matrices.    The details of these data are introduced in the corresponding paper (Tomita, submitted).

opencc-by-4.0May 2023View details →

High resolution and high cadence time series of land surface categories, land use land cover, and land use land cover changes

A prototype of monthly, 10 m resolution land surface categories, land use land cover (LULC) cover, and LULC change maps derived from Sentinel-2 data over three areas within Belgium, Portugal, and Sicily for the period 2018-2020. The LULC and LULC change maps were independently validated by IIASA. All products were generated within the framework of the RapidAI4EO project, funded from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 101004356. The data description can be found below. The validation report of the LULC and LULC change maps can be found in validation_LULC.pdf and validation_change.pdf, respectively, and the validation dataset can be found in Lesiv et al. (2023). Data description Increasing the cadence of the land cover updates from the typical (multi-)annual to monthly cadence poses several challenges. First, several land cover types are difficult to discriminate without any knowledge of temporal dynamics. For instance, croplands are characterized by a dynamic of vegetation growth and a harvest period (i.e. cycles of bare soil, sparsely vegetated and vegetated periods). This contrasts with grasslands that often lack the harvest period resulting in a bare soil cover. Without this temporal information, it is difficult to distinguish a vegetated cropland field from grassland. Second, phenological changes may introduce a large intra-class variability and thus also confusion between classes. For example, the shedding of leaves during autumn or wilting of herbaceous vegetation in dry summer periods introduces spectral variability within land cover classes. To overcome these challenges, we developed a workflow with two main phases. The first phase aims to map land surface categories (LSC) at a monthly resolution. The next phase uses the resulting monthly LSC probability time series to classify land cover. Land surface category (LSC) These LSC represent basic, observable bio-geophysical properties (categories) of the Earth surface that can be predicted directly from individual monthly composites. LSC classes contain a set of vegetated and non-vegetated surface categories. Discrete LSC classification legend: <table> <tbody> <tr> <td> Map code </td> <td> Land cover class </td> </tr> <tr> <td> 11 </td> <td> Tree (leaf-on) </td> </tr> <tr> <td> 12 </td> <td> Shrubland (leaf-on) </td> </tr> <tr> <td> 13 </td> <td> Grassland </td> </tr> <tr> <td> 14 </td> <td> Woody vegetation (leaf-off) </td> </tr> <tr> <td> 15 </td> <td> Wilted herbaceous vegetation </td> </tr> <tr> <td> 21 </td> <td> Bare/sparse vegetation </td> </tr> <tr> <td> 22 </td> <td> Water </td> </tr> <tr> <td> 24 </td> <td> Built-up </td> </tr> </tbody> </table> In order to predict the LSC, we trained a CatBoost model (Dorogush et al., 2018) using a DEM, spectral bands and vegetation indices, country, the timing (month) of the spectral data, and the pseudo-probability of a U-Net model trained to segment built-up surfaces as input. Labels were derived by post-processing the land cover labels of the ESA WorldCover product (Zanaga et al., 2021). Please note that the collection of these labels was suboptimal, likely having an impact on the LULC and change maps generated in the prototype. Land use land cover After predicting LSC over the three AOI’s, we trained a CatBoost model using the LSC probabilities over a window of one year, country, and the timing (month) as independent variable. The use of LSC probabilities over multiple months allows to incorporate information about dynamics, which is necessary to discriminate some classes (e.g. cropland and grassland or cropland and bare). Similar to the LSC labels, the LULC labels were derived from the ESA WorldCover product v100 (year 2020), resulting in a similar legend system. The use of a moving window approach to predict LULC allows to (i) incorporate temporal information that is necessary to discriminate land cover classes and (ii) is expected to lead to more consistent land cover maps. It however has the disadvantage that (i) no land cover predictions are available at the beginning and the end of the time series and (ii) the timing of the predicted land cover change is not always accurate. To resolve these issues, we applied a post-processing step that compares and integrates the LULC predictions and cleaned LSC predictions. Discrete LC classification legend: <table> <tbody> <tr> <td> Map code </td> <td> Land cover class </td> </tr> <tr> <td> 10 </td> <td> Tree cover </td> </tr> <tr> <td> 20 </td> <td> Shrubland </td> </tr> <tr> <td> 30 </td> <td> Grassland </td> </tr> <tr> <td> 40 </td> <td> Cropland </td> </tr> <tr> <td> 50 </td> <td> Built-up </td> </tr> <tr> <td> 60 </td> <td> Bare/sparse vegetation </td> </tr> <tr> <td> 80 </td> <td> Permanent water bodies </td> </tr> <tr> <td> 90 </td> <td> Herbaceous wetland </td> </tr> </tbody> </table> Land use land cover change Monthly change maps were finally derived from the land cover maps. The pixel values within the change maps represent the percentage of pixels that changed with respect to the previous month over an area of 90x90m. The maps contain values between 0-100, with larger values assigned to larger change patches. A value of 100 indicates that all pixels within an area of 90x90m around the pixel were flagged as change. Files The zip files contain the following data: <ul> <li>lsc.zip: land surface category maps over the three AOI’s</li> <li>lc.zip: LULC maps over the three AOI’s</li> <li>change.zip: change maps over the three AOI’s</li> </ul> These maps are generated for each month over the period 2018-2020 for each of the tiles (see tiles.gpkg for an overview of all tiles). The files names use the following naming convention: “tile-year-month.tif”. References Myroslava Lesiv, Halyna Bun, & Martina Duerauer. (2023). Validation data set on land cover changes for RapidAI4EO project [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7825963                                                                   Dorogush, A. V., Ershov, V., & Gulin, A. (2018). CatBoost: gradient boosting with categorical features support. arXiv preprint arXiv:1810.11363. Zanaga, D., et al.., 2021. ESA WorldCover 10 m 2020 v100. <a href="https://doi.org/10.5281/zenodo.5571936 ">https://doi.org/10.5281/zenodo.5571936 </a>

EnergyPROSPECTS Energy Citizenship Factsheet Series, Part 5: Aspects of ENCI I.: Hybridity, private/public, passive/active forms

This document is Part 5 of the EnergyPROSPECTS Factsheet Series. We have created the Series to publish the results of a mapping of energy citizenship in Europe, along with the first stage of our analysis of the respective data. The EnergyPROSPECTS consortium mapped 596 cases of energy citizenship (ENCI) between November 2020 and May 2021 using desk research, collecting data on many aspects of the cases. Although the analysis is a work in progress, we believe it is important to share our data and, through doing this, contribute to the understanding of energy citizenship in Europe. EnergyPROSPECTS (PROactive Strategies and Policies for Energy Citizenship Transformation), a H2020 project between 2021-2024, works with a critical understanding of energy citizenship that is grounded in state-of-the-art social sciences and humanities (SSH) insights.

EnergyPROSPECTS Energy Citizenship Factsheet Series, Part 4: Funding

This document is Part 4 of the EnergyPROSPECTS Factsheet Series. We have created the Series to publish the results of a mapping of energy citizenship in Europe, along with the first stage of our analysis of the respective data. The EnergyPROSPECTS consortium mapped 596 cases of energy citizenship between November 2020 and May 2021 using desk research, collecting data on many aspects of the cases. Although the analysis is a work in progress, we believe it is important to share our data and, through doing this, contribute to the understanding of energy citizenship in Europe. EnergyPROSPECTS (PROactive Strategies and Policies for Energy Citizenship Transformation), a H2020 project between 2021-2024, works with a critical understanding of energy citizenship that is grounded in state-of-the-art social sciences and humanities (SSH) insights.

EnergyPROSPECTS Energy Citizenship Factsheet Series, Part 2: Motivations and objectives

This document is Part 2 of the EnergyPROSPECTS Factsheet Series. We have created the Series to publish the results of a mapping of energy citizenship in Europe, along with the first stage of our analysis of the respective data. The EnergyPROSPECTS consortium mapped 596 cases of energy citizenship between November 2020 and May 2021 using desk research, collecting data on many aspects of the cases. Although the analysis is a work in progress, we believe it is important to share our data and, through doing this, contribute to the understanding of energy citizenship in Europe. EnergyPROSPECTS (PROactive Strategies and Policies for Energy Citizenship Transformation), a H2020 project between 2021-2024, works with a critical understanding of energy citizenship that is grounded in state-of-the-art social sciences and humanities (SSH) insights.

EnergyPROSPECTS Energy Citizenship Factsheet Series, Part 7: Aspects of ENCI III.: Towards social sustainability

This document is Part 7 of the EnergyPROSPECTS Factsheet Series. We have created the Series to publish the results of a mapping of energy citizenship in Europe, along with the first stage of our analysis of the respective data. The EnergyPROSPECTS consortium mapped 596 cases of energy citizenship (ENCI) between November 2020 and May 2021 using desk research, collecting data on many aspects of the cases. Although the analysis is a work in progress, we believe it is important to share our data and, through doing this, contribute to the understanding of energy citizenship in Europe. EnergyPROSPECTS (PROactive Strategies and Policies for Energy Citizenship Transformation), a H2020 project between 2021-2024, works with a critical understanding of energy citizenship that is grounded in state-of-the-art social sciences and humanities (SSH) insights.

EnergyPROSPECTS Energy Citizenship Factsheet Series, Part 9: Aspects of ENCI V.: Contesting the current system

This document is Part 9 of the EnergyPROSPECTS Factsheet Series. We have created the Series to publish the results of a mapping of energy citizenship in Europe, along with the first stage of our analysis of the respective data. The EnergyPROSPECTS consortium mapped 596 cases of energy citizenship (ENCI) between November 2020 and May 2021 using desk research, collecting data on many aspects of the cases. Although the analysis is a work in progress, we believe it is important to share our data and, through doing this, contribute to the understanding of energy citizenship in Europe. EnergyPROSPECTS (PROactive Strategies and Policies for Energy Citizenship Transformation), a H2020 project between 2021-2024, works with a critical understanding of energy citizenship that is grounded in state-of-the-art social sciences and humanities (SSH) insights.

Duck94 time series

Time series from pressure gages and current meters deployed along a cross-shore transect from near the shoreline to about 5 m water depth at the USACE Field Research Facility, Duck, NC in 1994. The README describes the 2Hz time series, the file with X-Y-Z locations, and provides links to bathymetry surveys and incident wave frequency-directional data. The Duck94 data set includes both offshore and onshore sandbar migration, incident waves from the north and the south of shore normal, and incident significant wave heights from 0.4 to 4.0 m.

opencc-by-4.0Dec 2022View details →

The long term series of characteristics of floods that happened to 12 rivers in Finland and northern Russia.

The dataset of flood’s characteristics (annual and spring): the volume of spring flood (in mm of the depth of runoff), the dates of spring flood begin and end, the length of spring flooding period, the yearly maximum daily discharge and its date were estimated for each year from the daily series of water discharges observed at the hydrometric sites. To define the dates of spring flood begin and end we applied the semi-empirical method given in Shevnina (2013). The yearly maximum water discharges have been obtained in Gudmundsson et al. (2018) for the period until 2017; this dataset gives a good agreement in the estimations for the overlapping periods. The series of volume of spring flood (in mm of the depth of runoff), the dates of spring flood begin and end, the length of spring flooding period, the yearly maximum daily discharge and its date are given in the dataset supplementing the study submitted to the Water resource research journal (<a href="https://agupubs.onlinelibrary.wiley.com/journal/19447973">https://agupubs.onlinelibrary.wiley.com/journal/19447973</a> ).  The daily series of water river discharges at the sites located in Finland were extracted  from (a) the Global runoff database <a href="https://portal.grdc.bafg.de/">https://portal.grdc.bafg.de/</a> (for the period from beginning of the observations to 2017); (b) the archive of the Finnish Environmental Institute <a href="https://www.syke.fi/">https://www.syke.fi</a> (for the period  2018–2020) and these series can be obtained after its representatives’ permission from the author. The daily series of water discharges at the sites located in the Russian Federation were extracted from (a) the yearly hydrological books published by the State Hydrological Institute <a href="http://www.hydrology.ru/en">http://www.hydrology.ru/en</a>  (for the period from the beginning of observation to 2007); (b) the automated information system for state monitoring of water bodies <a href="https://gmvo.skniivh.ru/">https://gmvo.skniivh.ru/</a> (for the period 2008–2020) and these series are available from its web-site after a registration.   The dataset consists of the CSV/TXT files, each file contains the long term series of the characteristics listed in the header: "year", "DFB" (date when a spring flooding period begins, day of year, DOY),"DFE" (date when the spring flooding period ends, DOY),"Length" (length of the spring flooding period, days), "DFMax" (date when the yearly maximum water discharge is recorded, DOY), "Qmax" (the yearly maximum water discharge, cubic m per second), "FRD" (the volume of spring flood expressed in mm per flooding period), "YRD" (volume of annual flow, expressed in mm per year),"Ftype" (the source of annual flood equaling to 1 of the yearly maximum water discharge is recorded in the spring flooding period or 0 if it is not).  The dataset was obtained in the study funded by the Academy of Finland under the contract number 317999. It will become freely available once the manuscript is published.  References Gudmundsson, L., Do, H. X., Leonard, M., & Westra, S. (2018), The Global Streamflow Indices and Metadata Archive (GSIM) – Part 2: Quality control, time-series indices and homogeneity assessment, Earth Syst. Sci. Data, 10, 787–804, https://doi.org/10.5194/essd-10-787-2018. Shevnina E. (2013), Method to calculate characteristics of spring flood from daily water discharges, Problems of the Arctic and Antarctic, 1(95), pp. 12-21. In Russian

opencc-by-4.0Feb 2023View details →

Controlled Anomalies Time Series (CATS) Dataset

The Controlled Anomalies Time Series (CATS) Dataset consists of commands, external stimuli, and telemetry readings of a simulated complex dynamical system with 200 injected anomalies. The CATS Dataset exhibits a set of desirable properties that make it very suitable for benchmarking Anomaly Detection Algorithms in Multivariate Time Series [1]: <ul> <li>Multivariate (17 variables) including sensors reading and control signals. It simulates the operational behaviour of an arbitrary complex system including: <ul> <li>4 Deliberate Actuations / Control Commands sent by a simulated operator / controller, for instance, commands of an operator to turn ON/OFF some equipment.</li> <li>3 Environmental Stimuli / External Forces acting on the system and affecting its behaviour, for instance, the wind affecting the orientation of a large ground antenna.</li> <li>10 Telemetry Readings representing the observable states of the complex system by means of sensors, for instance, a position, a temperature, a pressure, a voltage, current, humidity, velocity, acceleration, etc.</li> </ul> </li> <li>5 million timestamps. Sensors readings are at 1Hz sampling frequency. <ul> <li>1 million nominal observations (the first 1 million datapoints). This is suitable to start learning the "normal" behaviour.</li> <li>4 million observations that include both nominal and anomalous segments. This is suitable to evaluate both semi-supervised approaches (novelty detection) as well as unsupervised approaches (outlier detection).</li> </ul> </li> <li>200 anomalous segments. One anomalous segment may contain several successive anomalous observations / timestamps. Only the last 4 million observations contain anomalous segments.</li> <li>Different types of anomalies to understand what anomaly types can be detected by different approaches. The categories are available in the dataset and in the metadata.</li> <li>Fine control over ground truth. As this is a simulated system with deliberate anomaly injection, the start and end time of the anomalous behaviour is known very precisely. In contrast to real world datasets, there is no risk that the ground truth contains mislabelled segments which is often the case for real data.</li> <li>Suitable for root cause analysis. In addition to the anomaly category, the time series channel in which the anomaly first developed itself is recorded and made available as part of the metadata. This can be useful to evaluate the performance of algorithm to trace back anomalies to the right root cause channel.</li> <li>Affected channels. In addition to the knowledge of the root cause channel in which the anomaly first developed itself, we provide information of channels possibly affected by the anomaly. This can also be useful to evaluate the explainability of anomaly detection systems which may point out to the anomalous channels (root cause and affected).</li> <li>Obvious anomalies. The simulated anomalies have been designed to be "easy" to be detected for human eyes (i.e., there are very large spikes or oscillations), hence also detectable for most algorithms. It makes this synthetic dataset useful for screening tasks (i.e., to eliminate algorithms that are not capable to detect those obvious anomalies). However, during our initial experiments, the dataset turned out to be challenging enough even for state-of-the-art anomaly detection approaches, making it suitable also for regular benchmark studies.</li> <li>Context provided. Some variables can only be considered anomalous in relation to other behaviours. A typical example consists of a light and switch pair. The light being either on or off is nominal, the same goes for the switch, but having the switch on and the light off shall be considered anomalous. In the CATS dataset, users can choose (or not) to use the available context, and external stimuli, to test the usefulness of the context for detecting anomalies in this simulation.</li> <li>Pure signal ideal for robustness-to-noise analysis. The simulated signals are provided without noise: while this may seem unrealistic at first, it is an advantage since users of the dataset can decide to add on top of the provided series any type of noise and choose an amplitude. This makes it well suited to test how sensitive and robust detection algorithms are against various levels of noise.</li> <li>No missing data. You can drop whatever data you want to assess the impact of missing values on your detector with respect to a clean baseline.</li> </ul> Change Log Version 2 <ul> <li>Metadata: we include a metadata.csv with information about: <ul> <li>Anomaly categories</li> <li>Root cause channel (signal in which the anomaly is first visible)</li> <li>Affected channel (signal in which the anomaly might propagate) through coupled system dynamics</li> </ul> </li> <li>Removal of anomaly overlaps: version 1 contained anomalies which overlapped with each other resulting in only 190 distinct anomalous segments. Now, there are no more anomaly overlaps.</li> <li>Two data files: CSV and parquet for convenience.</li> </ul> [1] Example Benchmark of Anomaly Detection in Time Series: “Sebastian Schmidl, Phillip Wenig, and Thorsten Papenbrock. Anomaly Detection in Time Series: A Comprehensive Evaluation. PVLDB, 15(9): 1779 - 1797, 2022. doi:10.14778/3538598.3538602” About Solenix Solenix is an international company providing software engineering, consulting services and software products for the space market. Solenix is a dynamic company that brings innovative technologies and concepts to the aerospace market, keeping up to date with technical advancements and actively promoting spin-in and spin-out technology activities. We combine modern solutions which complement conventional practices. We aspire to achieve maximum customer satisfaction by fostering collaboration, constructivism, and flexibility.

opencc-by-4.0Oct 2023View details →

Ensemble Machine Learning Prediction of Potential FAPAR: Monthly time-series 2021 and Long-Term Comparison with Actual FAPAR

General Description The dataset contains composites at 250 m spatial resolution of (1)  monthly potential FAPAR for the year 2021 from ensemble ML model predictions, (2) the model deviance for each prediction, (3) the yearly average of potential FAPAR, (4) the yearly average of actual FAPAR and (5) the yearly average of the difference between actual and potential (actual minus potential) FAPAR. The dataset is based on the <a href="https://zenodo.org/record/8392976">95th percentile of the monthly aggregated FAPAR</a> derived from <a href="http://glass.umd.edu/Overview.html">250 m 8 d GLASS V6 FAPAR</a>. Potential FAPAR was predicted by fitting an ensemble ML model using globally distributed training points (cca 3 Mio) and a set of 52 biophysical covariates including several layers related to human pressure. The code for modeling potential FAPAR is openly available at <a href="http://github.com/Open-Earth-Monitor/Global_FAPAR_250m">https://github.com/Open-Earth-Monitor/Global_FAPAR_250m</a>. The dataset can be used in many applications like land degradation modeling, land productivity mapping, and land potential mapping.  Data Details <ul> <li>Time period: January 2021 - December 2021</li> <li>Type of data: Fraction of Absorbed Photosynthetically Active Radiation (FAPAR)</li> <li>How the data was collected or derived: Derived from 250m 8 d GLASS V6 FAPAR</li> <li>Statistical methods used: Ensemble machine learning</li> <li>Limitations or exclusions in the data: The dataset does not include data for Antarctica.</li> <li>Coordinate reference system: EPSG:4326</li> <li>Bounding box (Xmin, Ymin, Xmax, Ymax): (-180.00000, -62.0008094, 179.9999424, 87.37000)</li> <li>Spatial resolution: 1/480 d.d. = 0.00208333 (250m)</li> <li>Image size: 172,800 x 71,698</li> <li>File format: Cloud Optimized Geotiff (COG) format.</li> </ul> Support If you discover a bug, artifact, or inconsistency, or if you have a question please raise a GitHub issue: <a href="https://github.com/Open-Earth-Monitor/Global_FAPAR_250m/issues">https://github.com/Open-Earth-Monitor/Global_FAPAR_250m/issues</a> Reference Hackländer, J., Parente, L., Ho, Y.-F., Hengl, T., Simoes, R., Consoli, D., Şahin, M., Tian, X., Herold, M., Jung, M., Duveiller, G., Weynants, M., Wheeler, I., (2023?) "Land potential assessment and trend-analysis using 2000–2021 FAPAR monthly time-series at 250 m spatial resolution", submitted to PeerJ, preprint available at: <a href="https://doi.org/10.21203/rs.3.rs-3415685/v1">https://doi.org/10.21203/rs.3.rs-3415685/v1</a>   Name convention To ensure consistency and ease of use across and within the projects, we follow the standard Open-Earth-Monitor file-naming convention. The convention works with 10 fields that describes important properties of the data. In this way users can search files, prepare data analysis etc, without needing to open files. The fields are: <ol> <li>generic variable name: pot.fapar = Potential Fraction of Absorbed Photosynthetically Active Radiation</li> <li>variable procedure combination: eml = ensemble machine learning</li> <li>Position in the probability distribution / variable type: m = mean</li> <li>Spatial support: 250m</li> <li>Depth reference: s = surface</li> <li>Time reference begin time: 20210101 = 2021-01-01</li> <li>Time reference end time: 20211231 = 2021-12-31</li> <li>Bounding box: go = global (without Antarctica)</li> <li>EPSG code: epsg.4326 = EPSG:4326</li> <li>Version code: v20230924 = 2023-09-24 (creation date)</li> </ol>

opencc-by-4.0May 2019View details →

Host network traffic time series 2019/01

General info Dataset was collected over one month period in January 2019. The observation points for the collection of IP flows were located at the borders of the university campus network. The campus university network has /16 CIDR IPv4 network range at disposal and contains various network segments from segments connecting dormitories, over server segments, to a segment containing working stations of university administrative workers. The size of the raw IP flows used to create the dataset was over 860GB. A host in our dataset is identified by its source IPv4 address.     Variables The dataset contains the following variables: <ul> <li>Aggregations - created from five-minute total volumes aggregated over one-hour disjoint windows using mean/max/min aggregation functions <ul> <li># of flows (FL) - number of flows for a given source IP </li> <li># of packets (PKT) - number of packets for a given source IP</li> <li># of bytes (BYT) - number of packets for a given source IP</li> <li>flow duration (DUR) - average flow duration in seconds</li> </ul> </li> <li>Distinct Counts - count of distinct values for each variable in five-minute window aggregated over one-hour disjoint windows using mean/max/min aggregation functions <ul> <li># of peers (PEER) - number of distinct communication peers for a given source IP</li> <li># of ports (PORTS) - number of distinct destination ports for a given source IP</li> <li># of protocols (PROTO) - number of distinct communication protocols for a given source IP</li> <li># of AS numbers (AS) - number of distinct destination AS numbers for a given source IP</li> <li># of countries (CTRY) - number of distinct destination countries for a given source IP</li> </ul> </li> <li>Labels <ul> <li>Range (RNG) - a network range a host belongs to (anonymized)</li> <li>Unit (UNT) - an administrative unit owning the network range</li> <li>Sub-unit (SUB-UNT) - a sub-unit of the unit</li> </ul> </li> </ul>   Dataset format <ul> <li>The dataset is in comma-separated values (CSV) format. </li> <li>Header - multilevel, first 3 lines <ul> <li>1 level - aggregation type {mean|min|max}</li> <li>2 level - variable {see above}</li> <li>3 level - hour of a day {00,01,02,03,...,22,23}</li> </ul> </li> <li>Lablels - last 4 columns</li> <li>Dataset size  <ul> <li>rows: 65536 host records + 3 headers</li> <li>columns: 648 variables + 4 labels</li> </ul> </li> </ul>

openCC (other)May 2023View details →

Nutrient and stoichiometric time series measurements of decomposing coarse detritus in freshwaters worldwide from literature published between 1976-2020.

This data publication is a database of published estimates of nitrogen, phosphorus, and carbon content of decomposing coarse detritus though time in freshwater ecosystems worldwide. Nutrient content measurements are paired with estimates of detrital mass loss within decomposition time series (i.e., defined cohorts of decomposing material through time) with the goal of understanding patterns and drivers of temporal elemental dynamics of freshwater detritus. A systematic literature search for aquatic decomposition experiments conducted on 29 April 2020 generated 580 records (after trimming for duplicates and obvious relevance). From this literature pool, we extracted 810 decomposition time series and associated environmental data (e.g., temperature, water quality, detrital characteristics). Time series included in this synthesis include a range of detritus types (including terrestrial and aquatic plant material, carcasses, dung, and veneers), ecosystems (including streams, lakes, rivers, and wetlands), and settings (including natural ecosystems, field mesocosms, and laboratory microcosms).

openCC (other)Oct 2017View details →

LAGOS-NE v.1.054.1 Lake water clarity time series (1987-2011), climate, and geophysical data for 601 lakes across a 17-state region of the United States

Time series of median summer water clarity (secchi) values from 601 unique lakes in the Midwest and Northeast United States. Water clarity observations were derived from the Lake Multi-Scaled Geospatial and Temporal Database LAGOS-NELIMNO version 1.054.1. These data were used to assess long-term changes in water clarity from 1987-2011, and the potential drivers of those trends (Lottig et al. in press). Summer open water period was used to approximate the stratified period in the study lakes, which was defined as June 15 to September 15. Over the 25-year time period, each lake had to have at least a single summer water clarity observation for 22 of 25 years. The median number of secchi measurements that were used to derive a single annual median value for each lake was approximately 9. Of the over 14,000 annual estimates of water clarity that we generated, only two percent of those annual values were generated from a single observation and median number of observations for each lake over the 25-year study period was 223. Each unique lake with water clarity data also has supporting geophysical data, including climate, land use, hydrology, and topography derived at multiple spatial scales. Lake-specific characteristics, such as depth and area, are also reported. The geospatial data came from LAGOS-NEGEO version 1.03 except for the annual climate data which was aggregated at the HUC8 spatial scale from monthly PRISM data. For more specific information on how LAGOS-NE was created, see Soranno et al. 2015. Citations: Lottig, N.R., P-N. Tan, T. Wager, K.S. Cheruvelil, P.A. Soranno, E.H. Stanley, C.E Scott, C.A. Stow, and S. Yuan. in press. Macroscale patterns of synchrony identify complex relationships among spatial and temporal ecosystem drivers. Ecosphere Soranno P.A., Bissell E.G., Cheruvelil K.S., Christel S.T., Collins S.M., Fergus C.E., Filstrup C.T., Lapierre J.-F., Lottig N.R., Oliver S.K., Scott C.E., Smith N.J., Stopyak S., Yuan S., Bremigan M.T., Downing J.A., G

openCustomNov 2016View details →

Decomposition of Fine Woody Roots: a Time Series Approach, 1995 to 2006

We examined the effects of species, initial substrate quality, and site differences on woody root decomposition and its nitrogen dynamics in Sitka spruce (Picea sitchensis), Douglas-fir (Pseudotsuga menziesii), and ponderosa pine (Pinus ponderosa) dominated forests in Oregon, U.S.A. using a time series approach. Roots of fourteen species and five size classes were placed in the field to incubate and then collected at pre-planned intervals to determine mass loss and nitrogen content.

openCC (other)Dec 2024View details →

I-Series Bacterial Production 2000-2021, Toolik Lake catchment, North Slope Alaska

The dataset reports values for bacterial production in Toolik Lake catchment series of lakes (I-series), Alaska at depths of 1m for the time period 2000-2021. The method used is the 14-C-Leucine incorporation into bacterial cells.

openCC0May 2025View details →

Final chlorophyll and temperature measurements at 10m depth, offshore of Dana Point, California as part of an Ocean Institute time series, 2006 - 2025.

Time series of chlorophyll and temperature at 10m offshore of Dana Point, California. Measurements made as part of education/outreach programs involving students and teachers in oceanographic sampling and analytic sensitivity to time series. Cruises are conducted twice, monthly (or other) where sampling is performed by students assisted by technicians and supervisors.