Find research datasets worth reusing

opencc-by-4.0Apr 2023View details →

Dataset from VR Streaming Server (Emulated) and Radio Access Network for Streaming Traffic

The dataset contains an experiment in  a site where UEs attach to a gNodeB that provides access to a streaming server that is stressed with high demanding transcoding workloads to emulate VR/AR processes. The UEs are realized through the Remote UE mode enabled by Amarisoft Simbox emulator, and the gNodeB is realized through the Amarisoft Callbox, which also provides the user plane function. The emulated VR streaming server is deployed as a Nginx pod in a Kubernetes cluster. We rely on MonB5G sampling functions that feed monitoring data (CPU and RAN parameters) to the monitoring system. A streaming video server has been deployed with the help of a NGINX server. It provides video-on-demand and video streaming, which can be accessed by any user (or UE) for real-time reproduction. This VR video streaming emulation aids to assess the performance of the network and therefore the benefits that each solution has brought. The video “Big Buck Bunny” with h.264 encoding and a resolution of 1920x1080p has been used for the experiments.The description of dataset features are: 1-) Index Number, 2-) Time: Time of the experiment, 3-) N: number of VR streaming clients, 4-) C: Average CPU of VR streaming server [mc] 5-) O: Outbound traffic at the server average outbound traffic (O) flowing from the data interface of the video server.  6-) R: Instantaneous downlink bit rate [Mbps], The original video file information: <table> <tbody> <tr> <td> Video codec  </td> <td> Advanced Video Codec (AVC)  </td> </tr> <tr> <td> Width  </td> <td> 1920 pixels  </td> </tr> <tr> <td> Height  </td> <td> 1080 pixels  </td> </tr> <tr> <td> Display aspect radio  </td> <td> 16:9  </td> </tr> <tr> <td> Duration  </td> <td> 10 min 34 s  </td> </tr> <tr> <td> Max Bitrate  </td> <td> 16.7 Mb/s  </td> </tr> <tr> <td> Frame rate  </td> <td> 30 FPS  </td> </tr> </tbody> </table>

opencc-by-4.0May 2023View details →

Federated Learning for Distributed Intrusion Detection Systems in Public Networks - Validation Dataset

This dataset has been meticulously prepared and utilized as a validation set during the evaluation phase of "Meta IDS" to asses the performance of various machine learning models. It is  now made available for interested users and researchers who seek a reliable and diverse dataset for training and testing their own custom models. The validation dataset comprises a comprehensive collection of labeled entries, that determines whether the packet type is "malicious" or "benign." It covers complex design patterns that are commonly encountered in real-world applications. The dataset is designed to be representative, encompassing edge and fog layers that are in contact with cloud layer, thereby enabling thorough testing and evaluation of different models. Each sample in the dataset is labeled with the corresponding ground truth, providing a reliable reference for model performance evaluation.   To ensure convenient distribution and storage, the dataset has been broken down into three separate batches, each containing a portion of the dataset. This allows for convenient downloading and management of the dataset. The three batches are provided as individual compressed files.   In order to extract the data, follow the following instructions: <ul> <li>Download and install bzip2 (if not already installed) from the official website or your package manager.</li> <li>Place the compressed dataset file in a directory of your choice.</li> <li>Open a terminal or command prompt and navigate to the directory where the compressed dataset file is located.</li> <li>Execute the following command to uncompress the dataset: <ul> <li>bzip2 -d filename.bz2</li> </ul> </li> <li>Replace "filename.bz2" with the actual name of the compressed dataset file.</li> </ul> Once uncompressed, you will have access to the dataset in its original format for further exploration, analysis, and model training etc. The total storage required for extraction is approximately 800 GB in total, with the first batch requiring approximately 302 GB, the second batch requiring approximately 203 GB, and the third batch requiring approximately 297 GB of data storage.   The first batch contains 1,049,527,992 entries, where as the second batch contains 711,043,331 entries, and for the third and last batch we have 1,029,303,062 entries. The following table provides the feature names along with their explanation and example value once the dataset is extracted.   <table align="left"> <thead> <tr> <th scope="col">Feature</th> <th scope="col">Description</th> <th scope="col">Example Value</th> </tr> </thead> <tbody> <tr> <td>ip.src</td> <td>Source IP address in the packet</td> <td>a05d4ecc38da01406c9635ec694917e969622160e728495e3169f62822444e17</td> </tr> <tr> <td>ip.dst</td> <td>Destination IP address in the packet</td> <td>a52db0d87623d8a25d0db324d74f0900deb5ca4ec8ad9f346114db134e040ec5</td> </tr> <tr> <td>frame.time_epoch</td> <td>Epoch time of the frame</td> <td>1676165569.930869</td> </tr> <tr> <td>arp.hw.type</td> <td>Hardware type</td> <td>1</td> </tr> <tr> <td>arp.hw.size</td> <td>Hardware size</td> <td>6</td> </tr> <tr> <td>arp.proto.size</td> <td>Protocol size</td> <td>4</td> </tr> <tr> <td>arp.opcode</td> <td>Opcode</td> <td>2</td> </tr> <tr> <td>data.len</td> <td>Length</td> <td>2713</td> </tr> <tr> <td>eth.dst.lg</td> <td>Destination LG bit</td> <td>1</td> </tr> <tr> <td>eth.dst.ig</td> <td>Destination IG bit</td> <td>1</td> </tr> <tr> <td>eth.src.lg</td> <td>Source LG bit</td> <td>1</td> </tr> <tr> <td>eth.src.ig</td> <td>Source IG bit</td> <td>1</td> </tr> <tr> <td>frame.offset_shift</td> <td>Time shift for this packet</td> <td>0</td> </tr> <tr> <td>frame.len</td> <td>frame length on the wire</td> <td>1208</td> </tr> <tr> <td>frame.cap_len</td> <td>Frame length stored into the capture file</td> <td>215</td> </tr> <tr> <td>frame.marked</td> <td>Frame is marked</td> <td>0</td> </tr> <tr> <td>frame.ignored</td> <td>Frame is ignored</td> <td>0</td> </tr> <tr> <td>frame.encap_type</td> <td>Encapsulation type</td> <td>1</td> </tr> <tr> <td>gre</td> <td>Generic Routing Encapsulation</td> <td>'Generic Routing Encapsulation (IP)’</td> </tr> <tr> <td>ip.version</td> <td>Version</td> <td>6</td> </tr> <tr> <td>ip.hdr_len</td> <td>Header length</td> <td>24</td> </tr> <tr> <td>ip.dsfield.dscp</td> <td>Differentiated Services Codepoint</td> <td>56</td> </tr> <tr> <td>ip.dsfield.ecn</td> <td>Explicit Congestion Notification</td> <td>2</td> </tr> <tr> <td>ip.len</td> <td>Total length</td> <td>614</td> </tr> <tr> <td>ip.flags.rb</td> <td>Reserved bit</td> <td>0</td> </tr> <tr> <td>ip.flags.df</td> <td>Don't fragment</td> <td>1</td> </tr> <tr> <td>ip.flags.mf</td> <td>More fragments</td> <td>0</td> </tr> <tr> <td>ip.frag_offset</td> <td>Fragment offset</td> <td>0</td> </tr> <tr> <td>ip.ttl</td> <td>Time to live</td> <td>31</td> </tr> <tr> <td>ip.proto</td> <td>Protocol</td> <td>47</td> </tr> <tr> <td>ip.checksum.status</td> <td>Header checksum status</td> <td>2</td> </tr> <tr> <td>tcp.srcport</td> <td>TCP source port</td> <td>53425</td> </tr> <tr> <td>tcp.flags</td> <td>Flags</td> <td>0x00000098</td> </tr> <tr> <td>tcp.flags.ns</td> <td>Nonce</td> <td>0</td> </tr> <tr> <td>tcp.flags.cwr</td> <td>Congestion Window Reduced (CWR)</td> <td>1</td> </tr> <tr> <td>udp.srcport</td> <td>UDP source port</td> <td>64413</td> </tr> <tr> <td>udp.dstport</td> <td>UDP destination port</td> <td>54087</td> </tr> <tr> <td>udp.stream</td> <td>Stream index</td> <td>1345</td> </tr> <tr> <td>udp.length</td> <td>Length</td> <td>225</td> </tr> <tr> <td>udp.checksum.status</td> <td>Checksum status</td> <td>3</td> </tr> <tr> <td>packet_type</td> <td>Type of the packet which is either "benign" or "malicious"</td> <td>0</td> </tr> </tbody> </table> Furthermore, in compliance with the GDPR and to ensure the privacy of individuals, all IP addresses present in the dataset have been anonymized through hashing. This anonymization process helps protect the identity of individuals while preserving the integrity and utility of the dataset for research and model development purposes.   Please note that while the dataset provides valuable insights and a solid foundation for machine learning tasks, it is not a substitute for extensive real-world data collection. However, it serves as a valuable resource for researchers, practitioners, and enthusiasts in the machine learning community, offering a compliant and anonymized dataset for developing and validating custom models in a specific problem domain.   By leveraging the validation dataset for machine learning model evaluation and custom model training, users can accelerate their research and development efforts, building upon the knowledge gained from my thesis while contributing to the advancement of the field.

opencc-by-4.0May 2023View details →

CTU-SME-11: a labeled dataset with real benign and malicious network traffic mimicking a small medium-size enterprise environment

As technology advances, the number and complexity of cyber-attacks increase, forcing defense techniques to be updated and improved. To help develop effective tools for detecting security threats it is essential to have reliable and representative security datasets. Many existing security datasets have limitations that make them unsuitable for research, including lack of labels, unbalanced traffic, and outdated threats. CTU-SME-11 is a labeled network dataset designed to address the limitations of previous datasets. The dataset was captured in a real network that mimics a small-medium enterprise setting. Raw network traffic (packets) was captured from 11 devices using tcpdump for a duration of 7 days, from 20th to 26th of February, 2023 in Prague, Czech Republic. The devices were chosen based on the enterprise setting and consists of IoT, desktop and mobile devices, both bare metal and virtualized. The devices were infected with malware or exposed to Internet attacks, and factory reset to restore benign behavior.  The raw data was processed to generate network flows (Zeek logs) which were analyzed and labeled. The dataset contains two types of levels, a high level label and a descriptive label, which were put by experts. The former can take three values, benign, malicious or background. The latter contains detailed information about the specific behavior observed in the network flows. The dataset contains 99 million labeled network flows. The overall compressed size of the dataset is 80GB and the uncompressed size is 170GB.

opencc-by-4.0May 2023View details →

Network Data of the District Heating System for the city of Sønderborg from 2016-2019

The data set contains measurement data for heat load, as well as feed and return flow temperatures, from seven plants for the years 2016-2019 with a 15-minute time resolution. The heating plants belong to the district heating systems of Sønderborg, Denmark.

opencc-by-4.0Dec 2022View details →

Core-loss EELS dataset and neural networks for element identification

We present a large dataset containing simulated core-loss electron energy loss spectroscopy (EELS) spectra with the elemental content as ground-truth labels. Additionally we present some neural networks trained on this data for element identification.  The simulated dataset contains zero padded core-loss spectra from 0 to 3072 eV, which represents 107 core-loss edges through all 80 elements from Be up to Bi. The core-loss edges are calculated from the generalised oscillator strength (GOS) database presented by Zhang et al.[1] Generic fine structures using lifetime broadened peaks are used to imitate fine structure due to solid-state effects in experimental spectra. Generic low-loss regions are used to imitate the effect of multiple scattering. Each spectrum contains at least one edge of a given query element and possibly additional edges depending on samples drawn from The Materials Project [2]. The dataset contains for each of the 80 elements: 7000 training spectra, 1500 test spectra, 600 validation spectra and 100 spectra representing only the query element. This results in a total 736 000 labeled spectra. Code on how to  - read the simulated data - transform HDF5 format to TFRecord format - train and evaluate neural networks using the simulated data - use the trained networks for automated element identification is available on GitHub at arnoannys/EELS_ID A full report on the simulation of the dataset and the training and evaluation of the neural networks can be found at:                    Annys, A., Jannis, D. & Verbeeck, J. Deep learning for automated materials characterisation in core-loss electron energy loss spectroscopy. Sci Rep 13, 13724 (2023). https://doi.org/10.1038/s41598-023-40943-7 [1] Zezhong Zhang, Ivan Lobato, Daen Jannis, Johan Verbeeck, Sandra Van Aert, & Peter Nellist. (2023). Generalised oscillator strength for core-shell electron excitation by fast electrons based on Dirac solutions (1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7729585 [2] Anubhav Jain, Shyue Ping Ong, Geoffroy Hautier, Wei Chen, William Davidson Richards, Stephen Dacek, Shreyas Cholia, Dan Gunter, David Skinner, Gerbrand Ceder, Kristin A. Persson; Commentary: The Materials Project: A materials genome approach to accelerating materials innovation. __APL Mater__ 1 July 2013; 1 (1): 011002. [https://doi.org/10.1063/1.4812323](https://doi.org/10.1063/1.4812323)

Environmetal layers of the street network of Tallinn

These datasets contain the spatial layers of environmental variables of the street network of Tallinn. They are the result of a research assessment conducted to evaluate the environmental quality of the Tallinn street network. Environmental layers have been assigned to each street segment using the Zonal Statistics tool within a 25-metre buffer zone from street segments. The datasets are provided in the GeoPackage (gpkg) file format, ensuring compatibility with Geographic Information System (GIS) software and integration into existing spatial databases. The datasets include the following environmental layers: <ul> <li>ndvi_segment_tallinn: This layer represents greenery for street segments derived from the Normalised Difference Vegetation Index (NDVI), ranging between 0 and 1, from 18th June 2021.</li> <li>pm25_segment_tallinn: Annual PM2.5 (particulate matter with a diameter of 2.5 micrometres or smaller) mean street segment values for the year 2020.</li> <li>pm10_segment_tallinn: Annual PM10 (particulate matter with a diameter of 10 micrometres or smaller) mean street segment values for the year 2020.</li> <li>no2_segment_tallinn: Annual NO2 (nitrogen dioxide) mean street segment values for the year 2020.</li> <li>noise_segment_tallinn: Mean noise (Lden) street segment values from the year 2019.</li> </ul> These datasets serve as resources for analysing and understanding the environmental characteristics of the Tallinn street network. Researchers and urban planners can utilise these data.

opencc-by-4.0Feb 2023View details →

Dataset of Linkability Networks of Ethereum Accounts Involved in NFT trading of Top 15 NFT Collections

The data is organized in 32 files: <ul> <li> collection_metadata.txt stores basic information about each collection that was analyzed. The graph data of the collection is stored in a file named as the nickname of the collection (slug column in collection_metadata.txt). The most important columns/properties are: rank, slug – nickname, creation date, address, and volume data, ... </li> <li> 15 files reporting NFT ownership transfer, the names corresponding to the nicknames of the collections in collection_metadata.txt. </li> <li> 15 files with graph data, the names correspond to the nicknames of the collections in collection_metadata.txt. Each file gives the address linkage graph of one of the top 15 collections according to monetary volume on Opensea marketplace, computed as presented in methodology, exclusively on Ethereum blockchain. </li> </ul>

Network traffic datasets created by Single Flow Time Series Analysis

Network traffic datasets created by Single Flow Time Series Analysis Datasets were created for the paper: Network Traffic Classification based on Single Flow Time Series Analysis -- Josef Koumar, Karel Hynek, Tomá&scaron; Čejka -- which was published at The 19th International Conference on Network and Service Management (CNSM) 2023. Please cite usage of our datasets as:   <blockquote> J. Koumar, K. Hynek and T. Čejka, "Network Traffic Classification Based on Single Flow Time Series Analysis," 2023 19th International Conference on Network and Service Management (CNSM), Niagara Falls, ON, Canada, 2023, pp. 1-7, doi: 10.23919/CNSM59352.2023.10327876. </blockquote> This Zenodo repository contains 23 datasets created from 15 well-known published datasets which are cited in the table below. Each dataset contains 69 features created by Time Series Analysis of Single Flow Time Series. The detailed description of features from datasets is in the file: feature_description.pdf   In the following table is a description of each dataset file: <table> <tbody> <tr> <td>File name</td> <td>Detection problem</td> <td>Citation of original raw dataset</td> </tr> <tr> <td>botnet_binary.csv </td> <td>Binary detection of botnet </td> <td>S. García et al. An Empirical Comparison of Botnet Detection Methods. Computers & Security, 45:100–123, 2014. </td> </tr> <tr> <td>botnet_multiclass.csv </td> <td>Multi-class classification of botnet </td> <td>S. García et al. An Empirical Comparison of Botnet Detection Methods. Computers & Security, 45:100–123, 2014. </td> </tr> <tr> <td>cryptomining_design.csv</td> <td>Binary detection of cryptomining; the design part </td> <td>Richard Plný et al. Datasets of Cryptomining Communication. Zenodo, October 2022 </td> </tr> <tr> <td>cryptomining_evaluation.csv </td> <td>Binary detection of cryptomining; the evaluation part </td> <td>Richard Plný et al. Datasets of Cryptomining Communication. Zenodo, October 2022 </td> </tr> <tr> <td>dns_malware.csv </td> <td>Binary detection of malware DNS </td> <td>Samaneh Mahdavifar et al. Classifying Malicious Domains using DNS Traffic Analysis. In DASC/PiCom/CBDCom/CyberSciTech 2021, pages 60–67. IEEE, 2021. </td> </tr> <tr> <td>doh_cic.csv </td> <td>Binary detection of DoH </td> <td> Mohammadreza MontazeriShatoori et al. Detection of doh tunnels using time-series classification of encrypted traffic. In DASC/PiCom/CBDCom/CyberSciTech 2020, pages 63–70. IEEE, 2020  </td> </tr> <tr> <td>doh_real_world.csv </td> <td>Binary detection of DoH </td> <td>Kamil Jeřábek et al. Collection of datasets with DNS over HTTPS traffic. Data in Brief, 42:108310, 2022 </td> </tr> <tr> <td>dos.csv </td> <td>Binary detection of DoS </td> <td>Nickolaos Koroniotis et al. Towards the development of realistic botnet dataset in the Internet of Things for network forensic analytics: Bot-IoT dataset. Future Gener. Comput. Syst., 100:779–796, 2019.</td> </tr> <tr> <td>edge_iiot_binary.csv </td> <td>Binary detection of IoT malware </td> <td>Mohamed Amine Ferrag et al. Edge-iiotset: A new comprehensive realistic cyber security dataset of iot and iiot applications: Centralized and federated learning, 2022.</td> </tr> <tr> <td>edge_iiot_multiclass.csv</td> <td>Multi-class classification of IoT malware</td> <td>Mohamed Amine Ferrag et al. Edge-iiotset: A new comprehensive realistic cyber security dataset of iot and iiot applications: Centralized and federated learning, 2022.</td> </tr> <tr> <td>https_brute_force.csv</td> <td>Binary detection of HTTPS Brute Force</td> <td>Jan Luxemburk et al. HTTPS Brute-force dataset with extended network flows, November 2020</td> </tr> <tr> <td>ids_cic_binary.csv</td> <td>Binary detection of intrusion in IDS</td> <td>Iman Sharafaldin et al. Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISSp, 1:108–116, 2018.</td> </tr> <tr> <td>ids_cic_multiclass.csv </td> <td>Multi-class classification of intrusion in IDS </td> <td>Iman Sharafaldin et al. Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISSp, 1:108–116, 2018. </td> </tr> <tr> <td>ids_unsw_nb_15_binary.csv </td> <td>Binary detection of intrusion in IDS </td> <td>Nour Moustafa and Jill Slay. Unsw-nb15: a comprehensive data set for network intrusion detection systems (unsw-nb15 network data set). In 2015 military communications and information systems conference (MilCIS), pages 1–6. IEEE, 2015.</td> </tr> <tr> <td>ids_unsw_nb_15_multiclass.csv </td> <td>Multi-class classification of intrusion in IDS </td> <td>Nour Moustafa and Jill Slay. Unsw-nb15: a comprehensive data set for network intrusion detection systems (unsw-nb15 network data set). In 2015 military communications and information systems conference (MilCIS), pages 1–6. IEEE, 2015.</td> </tr> <tr> <td>iot_23.csv </td> <td>Binary detection of IoT malware </td> <td>Sebastian Garcia et al. IoT-23: A labeled dataset with malicious and benign IoT network traffic, January 2020. More details here https://www.stratosphereips.org /datasets-iot23</td> </tr> <tr> <td>ton_iot_binary.csv </td> <td>Binary detection of IoT malware </td> <td>Nour Moustafa. A new distributed architecture for evaluating ai-based security systems at the edge: Network ton iot datasets. Sustainable Cities and Society, 72:102994, 2021</td> </tr> <tr> <td>ton_iot_multiclass.csv </td> <td>Multi-class classification of IoT malware </td> <td>Nour Moustafa. A new distributed architecture for evaluating ai-based security systems at the edge: Network ton iot datasets. Sustainable Cities and Society, 72:102994, 2021</td> </tr> <tr> <td>tor_binary.csv </td> <td>Binary detection of TOR </td> <td>Arash Habibi Lashkari et al. Characterization of Tor Traffic using Time based Features. In ICISSP 2017, pages 253–262. SciTePress, 2017. </td> </tr> <tr> <td>tor_multiclass.csv </td> <td>Multi-class classification of TOR </td> <td>Arash Habibi Lashkari et al. Characterization of Tor Traffic using Time based Features. In ICISSP 2017, pages 253–262. SciTePress, 2017. </td> </tr> <tr> <td>vpn_iscx_binary.csv </td> <td>Binary detection of VPN </td> <td>Gerard Draper-Gil et al. Characterization of Encrypted and VPN Traffic Using Time-related. In ICISSP, pages 407–414, 2016. </td> </tr> <tr> <td>vpn_iscx_multiclass.csv </td> <td>Multi-class classification of VPN </td> <td>Gerard Draper-Gil et al. Characterization of Encrypted and VPN Traffic Using Time-related. In ICISSP, pages 407–414, 2016. </td> </tr> <tr> <td>vpn_vnat_binary.csv </td> <td>Binary detection of VPN </td> <td>Steven Jorgensen et al. Extensible Machine Learning for Encrypted Network Traffic Application Labeling via Uncertainty Quantification. CoRR, abs/2205.05628, 2022</td> </tr> <tr> <td>vpn_vnat_multiclass.csv</td> <td>Multi-class classification of VPN </td> <td>Steven Jorgensen et al. Extensible Machine Learning for Encrypted Network Traffic Application Labeling via Uncertainty Quantification. CoRR, abs/2205.05628, 2022</td> </tr> </tbody> </table>

Dependency networks of PyPI, npm, CRAN and Bioconductor repositories

A set of datasets with a list of dependencies from the software repositories PyPI, npm, CRAN and Bioconductor.

Geochemical data of bottom sediments from a network of drainage canals located in the low-lying coastal area of Ravenna, Italy.

This dataset contains all raw geochemical data of bottom sediments from a network of drainage canals located in the low-lying coastal area of Ravenna. The dataset is divided in three separated excel worksheets:  - Focus Area. Sediment composition of the 21 sediment samples collected in 2022 in the Focus Area. Refer to Figs. 1 and 2 in the manuscript Giambastiani et al., 2024 for the sample locations. Listed are also other information related to sampling, such as depositional facies (BR: beach ridge deposits; IF: Interfluvial floodplain deposits), distance from the sea, altimetry, amount of fertilizer applied based on the land use, and EC of drainage water.  The sediment samples were collected in March 2022 along the drainage system of the lowlying coastal aquifer of Ravenna (Italy) by the authors. - LRC, Land Reclamation Consortium. PTEs composition of the sediment samples of the Land Reclamation Consortium dataset. Refer to Fig. 1 and 2 in the manuscript Giambastiani et al., 2024 for the location. Listed are also other information related to sampling, such as distance from the sea, altimetry, and amount of fertilizer applied based on the land use.  The sediment samples were collected since 2010 along the drainage system of the lowlying coastal aquifer of Ravenna (Italy) by The Land Reclamation Consortium of Romagna (Italy). No other uses apart from scientific purpose is allowed without notice to the authors. - Wells. Physical and chemical groundwater parameters of 4 wells localted within the Focus Area. Refer to Fig.2  in the manuscript Giambastiani et al., 2024 for the location.  Data were collected during previous studies by Greggio et al. (2020) and reprocessed to obtain vertical profiles of EC, pH, Eh, and chemical concentrations along the coastal aquifer depth. More informations regarding the source, ownership, collection methodologies and analytical techniques are in Giambastiani et al., 2024.

opencc-by-4.0Jan 2023View details →

Geographic range size and species morphology determines the organization of sponge host-guest interaction networks across tropical coral reefs (Raw data)

Datasets for the analysis developed in the Article "Geographic range size and species morphology determines the organization of sponge host-guest interaction networks across tropical coral reefs". For more information, please refer to the original publication. Network_Structural_Index_&_SpogeTraits.csv <- Structural Index for the sponge-dwelling fauna network, sponge accumulated area and sponges’ morphology. NWTA_CoralReefs_Sponges_ interactions.csv <- Relationship between host sponges and guest fauna in the Northwester Atlantic coral reefs NWTA_CoralReefs_Sponge_reacords.csv <- Sponge species incidence records in the Northwester Atlantic coral reefs sponges_morphological_description.csv <- Sponge morphological standardization Network.html <- Interactive sponge-dwelling fauna network Enjoy!

Current Harmonics Minimization of PMSM Based on Iterative Learning Control and Neural Networks: Motor Data

The provided motor data corresponds to an electrical machine with 24 stator slots and 16 poles. As is common in electrical machines, this motor generates unwanted flux and current harmonics. However, the accompanying paper presents an effective solution to suppress these harmonics through the combined use of Iterative Learning Control (ILC) and Neural Networks (NNs). The ILC method demonstrates proficient compensation for harmonics during operations with constant speed and current reference values. Additionally, Neural Networks are trained with data derived from ILC, proving to be highly effective in suppressing harmonics even during transient operation. The simulation model used in the study is based on flux and torque maps, dependent on dq-currents and the electrical angle. These maps are obtained from Finite Element Method (FEM) simulations of an interior permanent magnet synchronous machine (IPM) and are openly published here, intended to facilitate other researchers in making direct comparisons with their own methodologies. Simulation results presented in the paper confirm that the integration of ILC and NNs leads to superior elimination of current harmonics during transient operations compared to using ILC alone. If you use the provided maps and motor data, kindly cite the associated paper for reference: https://doi.org/10.3390/machines11080784, https://www.mdpi.com/2075-1702/11/8/784

Convolutional neural network for automated surface crack detection using inductive thermography

Two phase images of the samples AIT_01 and AIT_08, analysed in the publication "Convolutional neural network for automated surface crack detection using inductive thermography", submitted to the Journal of Electronic Imaging.

Datasets for Paper "BenchTemp: A General Benchmark for Evaluating Temporal Graph Neural Networks"

Datasets for Paper "BenchTemp: A General Benchmark for Evaluating Temporal Graph Neural Networks" URL: https://github.com/qianghuangwhu/benchtemp Openreview: https://openreview.net/forum?id=rnZm2vQq31 There are 19 (15+4)  benchmark temporal graph datasets: reddit, wikipedia, mooc, lastfm, enron, SocialEvo, uci, CollegeMsg, TaobaoSmall, CanParl, Contacts, Flights, UNtrade, USLegis, UNvote, DGraphFin, TaobaoLarge, YoutubeReddit, YoutubeRedditLarge   Each dataset has three files: 1. ml_{data_name}.csv - the csv file of the Temporal Graph. This file have five columns with properties: 'u': the id of the user. 'i': the id of the item. 'ts': the timestamp of the interaction (edge) between the user and the item. 'label': the label of the interaction (edge). 'idx': the index of the interaction (edge). For example: ,u,i,ts,label,idx 0,1,2,0.0,0.0,1 1,1,3,0.0,0.0,2 2,1,4,0.0,0.0,3 2. ml_{data_name}.npy - the edge features corresponding to the interactions (edges) in the the Temporal Graph.. 3. ml_{data_name}_node.npy - the initialization node features of the Temporal Graph.

Network traffic datasets with novel extended IP flow called NetTiSA flow

Network traffic datasets with novel extended IP flow called NetTiSA flow Datasets were created for the paper: NetTiSA: Extended IP Flow with Time-series Features for Universal Bandwidth-constrained High-speed Network Traffic Classification -- Josef Koumar, Karel Hynek, Jaroslav Pe&scaron;ek, Tomá&scaron; Čejka -- which is published in The International Journal of Computer and Telecommunications Networking <a href="https://doi.org/10.1016/j.comnet.2023.110147" rel="nofollow">https://doi.org/10.1016/j.comnet.2023.110147</a> Please cite the usage of our datasets as: <blockquote> Josef Koumar, Karel Hynek, Jaroslav Pe&scaron;ek, Tomá&scaron; Čejka, "NetTiSA: Extended IP flow with time-series features for universal bandwidth-constrained high-speed network traffic classification", Computer Networks, Volume 240, 2024, 110147, ISSN 1389-1286 <pre><code>@article{KOUMAR2024110147, title = {NetTiSA: Extended IP flow with time-series features for universal bandwidth-constrained high-speed network traffic classification}, journal = {Computer Networks}, volume = {240}, pages = {110147}, year = {2024}, issn = {1389-1286}, doi = {https://doi.org/10.1016/j.comnet.2023.110147}, url = {https://www.sciencedirect.com/science/article/pii/S1389128623005923}, author = {Josef Koumar and Karel Hynek and Jaroslav Pe&scaron;ek and Tomá&scaron; Čejka} } </code></pre> </blockquote> This Zenodo repository contains 23 datasets created from 15 well-known published datasets, which are cited in the table below. Each dataset contains the NetTiSA flow feature vector.   NetTiSA flow feature vector The novel extended IP flow called NetTiSA (Network Time Series Analysed) flow contains a universal bandwidth-constrained feature vector consisting of 20 features. We divide the NetTiSA flow classification features into three groups by computation. The first group of features is based on classical bidirectional flow information---a number of transferred bytes, and packets.  The second group contains statistical and time-based features calculated using the time-series analysis of the packet sequences. The third type of features can be computed from the previous groups (i.e., on the flow collector) and improve the classification performance without any impact on the telemetry bandwidth.   Flow features The flow features are: <ul> <li>Packets is the number of packets in the direction from the source to the destination IP address.</li> <li>Packets in reverse order is the number of packets in the direction from the destination to the source IP address.</li> <li>Bytes is the size of the payload in bytes transferred in the direction from the source to the destination IP address.</li> <li>Bytes in reverse order is the size of the payload in bytes transferred in the direction from the destination to the source IP address.</li> </ul>   Statistical and Time-based features The features that are exported in the extended part of the flow. All of them can be computed (exactly or in approximative) by stream-wise computation, which is necessary for keeping memory requirements low. The second type of feature set contains the following features: <ul> <li>Mean represents mean of the payload lengths of packets</li> <li>Min is the minimal value from payload lengths of all packets in a flow</li> <li>Max is the maximum value from payload lengths of all packets in a flow</li> <li>Standard deviation is a measure of the variation of payload lengths from the mean payload length</li> <li>Root mean square is the measure of the magnitude of payload lengths of packets</li> <li>Average dispersion is the average absolute difference between each payload length of the packet and the mean value</li> <li>Kurtosis is the measure describing the extent to which the tails of a distribution differ from the tails of a normal distribution</li> <li>Mean of relative times is the mean of the relative times which is a sequence defined as $st = \{t_1 - t_1, t_2 - t_1, ..., t_n - t_1\} $</li> <li>Mean of time differences is the mean of the time differences which is a sequence defined as $dt = \{ t_j - t_i | j = i + 1, i \in \{1, 2, \dots, n - 1\} \}.$</li> <li>Min from time differences is the minimal value from all time differences, i.e., min space between packets.</li> <li>Max from time differences is the maximum value from all time differences, i.e., max space between packets.</li> <li>Time distribution describes the deviation of time differences between individual packets within the time series. The feature is computed by the following equation: $tdist = \frac{ \frac{1}{n-1} \sum_{i=1}^{n-1} \left| \mu_{\{dt_{n-1}\}} - dt_i \right| }{ \frac{1}{2} \left(max\left(\{dt_{n-1}\}\right) - min\left(\{dt_{n-1}\}\right) \right) }$</li> <li>Switching ratio represents a value change ratio (switching) between payload lengths. The switching ratio is computed by equation: $sr = \frac{s_n}{\frac{1}{2} (n - 1)}$</li> </ul>            where $s_n$ is number of switches.    Features computed at the collector The third set contains features that are computed from the previous two groups prior to classification. Therefore, they do not influence the network telemetry size and their computation does not put additional load to resource-constrained flow monitoring probes. The NetTiSA flow combined with this feature set is called the Enhanced NetTiSA flow and contains the following features: <ul> <li>Max minus min  is the difference between minimum and maximum payload lengths</li> <li>Percent deviation is the dispersion of the average absolute difference to the mean value</li> <li>Variance is the spread measure of the data from its mean</li> <li>Burstiness is the degree of peakedness in the central part of the distribution</li> <li>Coefficient of variation is a dimensionless quantity that compares the dispersion of a time series to its mean value and is often used to compare the variability of different time series that have different units of measurement</li> <li>Directions describe a percentage ratio of packet direction computed as $\frac{d_1}{ d_1 + d_0}$, where $d_1$ is a number of packets in a direction from source to destination IP address and $d_0$ the opposite direction. Both  $d_1$ and $d_0$ are inside the classical bidirectional flow.</li> <li>Duration is the duration of the flow</li> </ul>   The NetTiSA flow is implemented into IP flow exporter <a href="https://github.com/CESNET/ipfixprobe">ipfixprobe</a>.   Description of dataset files In the following table is a description of each dataset file: <table> <tbody> <tr> <td> File name </td> <td> Detection problem </td> <td> Citation of the original raw dataset </td> </tr> <tr> <td>botnet_binary.csv </td> <td>Binary detection of botnet </td> <td>S. García et al. An Empirical Comparison of Botnet Detection Methods. Computers & Security, 45:100–123, 2014. </td> </tr> <tr> <td>botnet_multiclass.csv </td> <td>Multi-class classification of botnet </td> <td>S. García et al. An Empirical Comparison of Botnet Detection Methods. Computers & Security, 45:100–123, 2014. </td> </tr> <tr> <td>cryptomining_design.csv </td> <td>Binary detection of cryptomining; the design part </td> <td>Richard Plný et al. Datasets of Cryptomining Communication. Zenodo, October 2022 </td> </tr> <tr> <td>cryptomining_evaluation.csv </td> <td>Binary detection of cryptomining; the evaluation part </td> <td>Richard Plný et al. Datasets of Cryptomining Communication. Zenodo, October 2022 </td> </tr> <tr> <td>dns_malware.csv </td> <td>Binary detection of malware DNS </td> <td>Samaneh Mahdavifar et al. Classifying Malicious Domains using DNS Traffic Analysis. In DASC/PiCom/CBDCom/CyberSciTech 2021, pages 60–67. IEEE, 2021. </td> </tr> <tr> <td>doh_cic.csv </td> <td>Binary detection of DoH </td> <td>Mohammadreza MontazeriShatoori et al. Detection of doh tunnels using time-series classification of encrypted traffic. In DASC/PiCom/CBDCom/CyberSciTech 2020, pages 63–70. IEEE, 2020 </td> </tr> <tr> <td>doh_real_world.csv </td> <td>Binary detection of DoH </td> <td>Kamil Jeřábek et al. Collection of datasets with DNS over HTTPS traffic. Data in Brief, 42:108310, 2022 </td> </tr> <tr> <td>dos.csv </td> <td>Binary detection of DoS </td> <td>Nickolaos Koroniotis et al. Towards the development of realistic botnet dataset in the Internet of Things for network forensic analytics: Bot-IoT dataset. Future Gener. Comput. Syst., 100:779–796, 2019. </td> </tr> <tr> <td>edge_iiot_binary.csv </td> <td>Binary detection of IoT malware </td> <td>Mohamed Amine Ferrag et al. Edge-iiotset: A new comprehensive realistic cyber security dataset of iot and iiot applications: Centralized and federated learning, 2022. </td> </tr> <tr> <td>edge_iiot_multiclass.csv </td> <td>Multi-class classification of IoT malware </td> <td>Mohamed Amine Ferrag et al. Edge-iiotset: A new comprehensive realistic cyber security dataset of iot and iiot applications: Centralized and federated learning, 2022. </td> </tr> <tr> <td>https_brute_force.csv </td> <td>Binary detection of HTTPS Brute Force </td> <td>Jan Luxemburk et al. HTTPS Brute-force dataset with extended network flows, November 2020 </td> </tr> <tr> <td>ids_cic_binary.csv </td> <td>Binary detection of intrusion in IDS </td> <td>Iman Sharafaldin et al. Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISSp, 1:108–116, 2018. </td> </tr> <tr> <td>ids_cic_multiclass.csv </td> <td>Multi-class classification of intrusion in IDS </td> <td>Iman Sharafaldin et al. Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISSp, 1:108–116, 2018. </td> </tr> <tr> <td>unsw_binary.csv </td> <td>Binary detection of intrusion in IDS </td> <td>Nour Moustafa and Jill Slay. Unsw-nb15: a comprehensive data set for network intrusion detection systems (unsw-nb15 network data set). In 2015 military communications and information systems conference (MilCIS), pages 1–6. IEEE, 2015. </td> </tr> <tr> <td>unsw_multiclass.csv </td> <td>Multi-class classification of intrusion in IDS </td> <td>Nour Moustafa and Jill Slay. Unsw-nb15: a comprehensive data set for network intrusion detection systems (unsw-nb15 network data set). In 2015 military communications and information systems conference (MilCIS), pages 1–6. IEEE, 2015. </td> </tr> <tr> <td>iot_23.csv </td> <td>Binary detection of IoT malware </td> <td>Sebastian Garcia et al. IoT-23: A labeled dataset with malicious and benign IoT network traffic, January 2020. More details here https://www.stratosphereips.org /datasets-iot23 </td> </tr> <tr> <td>ton_iot_binary.csv </td> <td>Binary detection of IoT malware </td> <td>Nour Moustafa. A new distributed architecture for evaluating ai-based security systems at the edge: Network ton iot datasets. Sustainable Cities and Society, 72:102994, 2021 </td> </tr> <tr> <td>ton_iot_multiclass.csv </td> <td>Multi-class classification of IoT malware </td> <td>Nour Moustafa. A new distributed architecture for evaluating ai-based security systems at the edge: Network ton iot datasets. Sustainable Cities and Society, 72:102994, 2021 </td> </tr> <tr> <td>tor_binary.csv </td> <td>Binary detection of TOR </td> <td>Arash Habibi Lashkari et al. Characterization of Tor Traffic using Time based Features. In ICISSP 2017, pages 253–262. SciTePress, 2017. </td> </tr> <tr> <td>tor_multiclass.csv </td> <td>Multi-class classification of TOR </td> <td>Arash Habibi Lashkari et al. Characterization of Tor Traffic using Time based Features. In ICISSP 2017, pages 253–262. SciTePress, 2017. </td> </tr> <tr> <td>vpn_iscx_binary.csv </td> <td>Binary detection of VPN </td> <td>Gerard Draper-Gil et al. Characterization of Encrypted and VPN Traffic Using Time-related. In ICISSP, pages 407–414, 2016. </td> </tr> <tr> <td>vpn_iscx_multiclass.csv </td> <td>Multi-class classification of VPN </td> <td>Gerard Draper-Gil et al. Characterization of Encrypted and VPN Traffic Using Time-related. In ICISSP, pages 407–414, 2016. </td> </tr> <tr> <td>vpn_vnat_binary.csv </td> <td>Binary detection of VPN </td> <td>Steven Jorgensen et al. Extensible Machine Learning for Encrypted Network Traffic Application Labeling via Uncertainty Quantification. CoRR, abs/2205.05628, 2022 </td> </tr> <tr> <td>vpn_vnat_multiclass.csv </td> <td>Multi-class classification of VPN </td> <td>Steven Jorgensen et al. Extensible Machine Learning for Encrypted Network Traffic Application Labeling via Uncertainty Quantification. CoRR, abs/2205.05628, 2022 </td> </tr> </tbody> </table>

opencc-by-4.0Dec 2022View details →

Real-Time Sensor Network of Detroit Green Infrastructure: Datasets and Code

<ol> <li>MonitoredRainGardens_PaperTable.xlsx: Excel spreadsheet where the "Garden" sheet contains the 14 monitored green infrastructure (GI) sites with their design and physiographic features and the "Field Log" sheet contains the installation and field maintenance trips.</li> <li>WaterWells.xlsx: an Excel file containing all of the water wells in the Detroit region uses for interpolating groundwater levels.</li> <li>GI_GIS_Analysis.aparx: ArcGIS Pro project file which includes the 14 monitored GI sites and the GIS data for Detroit (percent imperviousness, elevation, slope, land use type, wells, interpolated groundwater levels, hydrologic soil group).</li> <li>Code.zip: Zip folder containing another folder titled "Code" which holds: (1) a folder titled "SensorData" containing 16 csv files with the raw pressure transducer data for the 16 monitored GI sites during the measurement period (including the two excluded sites); (2) a csv file titled "MonitoredRainGardens.csv" containing the 14 monitored green infrastructure (GI) sites with their design and physiographic features used in the correlation analysis; (3) a csv file titled "storm_constants.csv" which contain the computed decay constants for every storm in every GI during the measurement period;  (4) a csv file titled "GLWA_RainGaugesforStudy.csv" that contains rainfall from 9 rain gauges during the measurement period; (5) a Jupyter notebook titled "storm_constants_analysis.ipynb" which provides the code for calculating the decay constants for the monitored GI; (6) a Jupyter notebook titled "storm_constants.ipynb" which provides the code for analyzing the decay constants including the correlation analysis and surface plots; and (7) a Jupiter notebook titled "modeled_response.ipynb" which provides the code for plotting the drawdown curves based on the decay constant.</li> </ol>

UBGG-3m: Fine-grained urban blue-green-gray landscape dataset for 36 Chinese cities based on deep learning network

The UBGG dataset provides easily access and leverage to researchers and analysts, which is stored in the following Zenodo repository (<a href="https://doi.org/10.5281/zenodo.8053333">https://doi.org/10.5281/zenodo.8352777</a>). The UBGG dataset consists of two main components: <ul> <li>UBGG-3m: the fine-grained UBGG map product of 36 metropolises in China. The UBGG-3m dataset captures the intricate urban landscape features with remarkable precision, providing a detailed representation at an impressive 3-meter resolution. Fig. 1 in User Guides shows the classification results for 36 Chinese metropolises. Researchers can delve into the nuances of the UBGG continuum, gaining invaluable insights into the interplay between the blue, green, and gray elements of urban environments in each metropolis.</li> </ul> <ul> <li>UBGGset: the large-volume sample dataset to support the UBGG deep learning research. Complementing the UBGG-3m dataset, UBGGset serves as a large-volume sample dataset specifically tailored to support and foster UBGG research endeavors (Fig. 2). The UBGGset consists of 14,627 sample images (without data augmentation), with dimensions of 256 pixels in length and width, covering an urban area of approximately 2,272 km2. The UBGGset was constructed with co-registered pairs of 3 m Planet images and fine-annotated urban landscapes labeled on 1 m Google Earth image. This dataset encompasses 15 typical cities, offering researchers a rich and diverse resource to drive exploration, analysis, and innovation in the field of urban landscape studies.</li> </ul>   Citation format for paper and dataset: [1] Zhiyu Xu, Shuqing Zhao. Fine-grained urban blue-green-gray landscape dataset for 36 Chinese cities based on deep learning network. Sci Data 11, 266 (2024). https://doi.org/10.1038/s41597-023-02844-2 [2] Zhiyu Xu, Shuqing Zhao, Fine-grained urban landscape mapping reveals broad-scale homogeneity in urban environments, Science Bulletin, (2024). https://doi.org/10.1016/j.scib.2024.03.060 [3] Zhiyu Xu, Shuqing Zhao. UBGG-3m: Fine-grained urban blue-green-gray landscape dataset for 36 Chinese cities based on deep learning network (v1.0) [Data set]. (2023). Zenodo. https://doi.org/10.5281/zenodo.8352777

opencc-by-4.0Sep 2023View details →

High Energy Lightning Emission Network (HELEN) Flight 7a 06/19/2023

Data gathered during the flight of the High Energy Lightning Emission Network (HELEN) on June 19th, 2023. The first file, 1-Flight7a-Raw Data.zip, is the raw data taken directly from the payloads' micro SD cards. In the second file, 2-Flight7a-Data to Process.zip, the data has been cleaned and is suitable for further processing. The third file, 3-Flight7a-Processed Data.zip, contains data that has been combined and temporally synced. This data can be easily read into MATLAB and used to reproduce results.

opencc-by-4.0Dec 2023View details →

Identification of a lineage-specific protein network at the trypanosome nuclear envelope

The nuclear envelope (NE) separates translation and transcription and is the location of multiple functions, including chromatin organization, nucleocytoplasmic transport, ribosomal maturation and mRNA processing/quality control.  The molecular basis for many of these functions have diverged between different eukaryotic lineages.  Trypanosoma brucei, a member of the early branching eukaryotic lineage Discoba, highlight many of these, including a distinct lamina and kinetochore composition.  Here we describe a cohort of proteins interacting with both the lamina and NPC, which we term lamina-associated proteins (LAPs).  LAPs represent a diverse group of proteins, including two candidate NPC-anchoring pore membrane proteins (POMs) with architecture conserved with S. cerevisiae Pom152 and H. sapiens Nup210, and additional peripheral components of the NPC.  While many of the LAPs are specific to Trypanosomatids, we also identified broadly conserved proteins, indicating an amalgam of divergence and conservation within the NE proteome of trypanosomes, highlighting the diversity of nuclear biology across the eukaryotes and increasing our understanding of eukaryotic and NPC evolution.