Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
374
datasets available to search
ShareScore release 0.9.0
Dataset results
374 results for “traffic”
Network traffic datasets with novel extended IP flow called NetTiSA flow
<p><strong>Network traffic datasets with novel extended IP flow called NetTiSA flow</strong></p> <p>Datasets were created for the paper: NetTiSA: Extended IP Flow with Time-series Features for Universal Bandwidth-constrained High-speed Network Traffic Classification -- Josef Koumar, Karel Hynek, Jaroslav Pešek, Tomáš Čejka -- which is published in The International Journal of Computer and Telecommunications Networking <a href="https://doi.org/10.1016/j.comnet.2023.110147" rel="nofollow">https://doi.org/10.1016/j.comnet.2023.110147</a><br><br>Please cite the usage of our datasets as:</p> <blockquote> <p>Josef Koumar, Karel Hynek, Jaroslav Pešek, Tomáš Čejka, "NetTiSA: Extended IP flow with time-series features for universal bandwidth-constrained high-speed network traffic classification", Computer Networks, Volume 240, 2024, 110147, ISSN 1389-1286<br><br></p> <pre><code>@article{KOUMAR2024110147, title = {NetTiSA: Extended IP flow with time-series features for universal bandwidth-constrained high-speed network traffic classification}, journal = {Computer Networks}, volume = {240}, pages = {110147}, year = {2024}, issn = {1389-1286}, doi = {https://doi.org/10.1016/j.comnet.2023.110147}, url = {https://www.sciencedirect.com/science/article/pii/S1389128623005923}, author = {Josef Koumar and Karel Hynek and Jaroslav Pešek and Tomáš Čejka} } </code></pre> </blockquote> <p>This Zenodo repository contains 23 datasets created from 15 well-known published datasets, which are cited in the table below. Each dataset contains the NetTiSA flow feature vector.<br><br> </p> <p><strong>NetTiSA flow feature vector</strong></p> <p><br>The novel extended IP flow called NetTiSA (Network Time Series Analysed) flow contains a universal bandwidth-constrained feature vector consisting of 20 features. We divide the NetTiSA flow classification features into three groups by computation. The first group of features is based on classical bidirectional flow information---a number of transferred bytes, and packets. The second group contains statistical and time-based features calculated using the time-series analysis of the packet sequences. The third type of features can be computed from the previous groups (i.e., on the flow collector) and improve the classification performance without any impact on the telemetry bandwidth.</p> <p> </p> <p><strong>Flow features</strong></p> <p>The flow features are:</p> <ul> <li><strong><em>Packets</em></strong> is the number of packets in the direction from the source to the destination IP address.</li> <li><em><strong>Packets in reverse order</strong></em> is the number of packets in the direction from the destination to the source IP address.</li> <li><strong><em>Bytes</em> </strong>is the size of the payload in bytes transferred in the direction from the source to the destination IP address.</li> <li><strong><em>Bytes in reverse order</em></strong> is the size of the payload in bytes transferred in the direction from the destination to the source IP address.</li> </ul> <p> </p> <p><strong>Statistical and Time-based features</strong></p> <p>The features that are exported in the extended part of the flow. All of them can be computed (exactly or in approximative) by stream-wise computation, which is necessary for keeping memory requirements low. The second type of feature set contains the following features:</p> <ul> <li><strong><em>Mean</em></strong> represents mean of the payload lengths of packets</li> <li><strong><em>Min</em></strong> is the minimal value from payload lengths of all packets in a flow</li> <li><strong><em>Max</em></strong> is the maximum value from payload lengths of all packets in a flow</li> <li><strong><em>Standard deviation</em></strong> is a measure of the variation of payload lengths from the mean payload length</li> <li><strong><em>Root mean square</em></strong> is the measure of the magnitude of payload lengths of packets</li> <li><strong><em>Average dispersion</em></strong> is the average absolute difference between each payload length of the packet and the mean value</li> <li><strong><em>Kurtosis</em></strong> is the measure describing the extent to which the tails of a distribution differ from the tails of a normal distribution</li> <li><em><strong>Mean of relative times</strong></em> is the mean of the relative times which is a sequence defined as <span>\(st = \{t_1 - t_1, t_2 - t_1, ..., t_n - t_1\} \)</span></li> <li><em><strong>Mean of time differences</strong></em> is the mean of the time differences which is a sequence defined as <span>\(dt = \{ t_j - t_i | j = i + 1, i \in \{1, 2, \dots, n - 1\} \}.\)</span></li> <li><em><strong>Min from time differences</strong></em> is the minimal value from all time differences, i.e., min space between packets.</li> <li><em><strong>Max from time differences</strong></em> is the maximum value from all time differences, i.e., max space between packets.</li> <li><em><strong>Time distribution</strong></em> describes the deviation of time differences between individual packets within the time series. The feature is computed by the following equation:<br><span>\(tdist = \frac{ \frac{1}{n-1} \sum_{i=1}^{n-1} \left| \mu_{\{dt_{n-1}\}} - dt_i \right| }{ \frac{1}{2} \left(max\left(\{dt_{n-1}\}\right) - min\left(\{dt_{n-1}\}\right) \right) }\)</span></li> <li><em><strong>Switching ratio</strong></em> represents a value change ratio (switching) between payload lengths. The switching ratio is computed by equation:<br><span>\(sr = \frac{s_n}{\frac{1}{2} (n - 1)}\)</span></li> </ul> <p> where <span>\(s_n\)</span> is number of switches.</p> <p> </p> <p><strong>Features computed at the collector</strong><br>The third set contains features that are computed from the previous two groups prior to classification. Therefore, they do not influence the network telemetry size and their computation does not put additional load to resource-constrained flow monitoring probes. The NetTiSA flow combined with this feature set is called the Enhanced NetTiSA flow and contains the following features:</p> <ul> <li><em><strong>Max minus min</strong></em> is the difference between minimum and maximum payload lengths</li> <li><em><strong>Percent deviation</strong></em> is the dispersion of the average absolute difference to the mean value</li> <li><em><strong>Variance</strong></em> is the spread measure of the data from its mean</li> <li><em><strong>Burstiness</strong></em> is the degree of peakedness in the central part of the distribution</li> <li><em><strong>Coefficient of variation</strong></em> is a dimensionless quantity that compares the dispersion of a time series to its mean value and is often used to compare the variability of different time series that have different units of measurement</li> <li><em><strong>Directions</strong></em> describe a percentage ratio of packet direction computed as <span>\(\frac{d_1}{ d_1 + d_0}\)</span>, where <span>\(d_1\)</span> is a number of packets in a direction from source to destination IP address and <span>\(d_0\)</span> the opposite direction. Both <span>\(d_1\)</span> and <span>\(d_0\)</span> are inside the classical bidirectional flow.</li> <li><em><strong>Duration</strong></em> is the duration of the flow</li> </ul> <p> </p> <p>The NetTiSA flow is implemented into IP flow exporter <a href="https://github.com/CESNET/ipfixprobe">ipfixprobe</a>.</p> <p> </p> <p><strong>Description of dataset files</strong></p> <p>In the following table is a description of each dataset file:</p> <table> <tbody> <tr> <td> <p><strong>File name</strong></p> </td> <td> <p><strong>Detection problem</strong></p> </td> <td> <p><strong>Citation of the original raw dataset</strong></p> </td> </tr> <tr> <td>botnet_binary.csv </td> <td>Binary detection of botnet </td> <td>S. García et al. An Empirical Comparison of Botnet Detection Methods. Computers & Security, 45:100–123, 2014. </td> </tr> <tr> <td>botnet_multiclass.csv </td> <td>Multi-class classification of botnet </td> <td>S. García et al. An Empirical Comparison of Botnet Detection Methods. Computers & Security, 45:100–123, 2014. </td> </tr> <tr> <td>cryptomining_design.csv </td> <td>Binary detection of cryptomining; the design part </td> <td>Richard Plný et al. Datasets of Cryptomining Communication. Zenodo, October 2022 </td> </tr> <tr> <td>cryptomining_evaluation.csv </td> <td>Binary detection of cryptomining; the evaluation part </td> <td>Richard Plný et al. Datasets of Cryptomining Communication. Zenodo, October 2022 </td> </tr> <tr> <td>dns_malware.csv </td> <td>Binary detection of malware DNS </td> <td>Samaneh Mahdavifar et al. Classifying Malicious Domains using DNS Traffic Analysis. In DASC/PiCom/CBDCom/CyberSciTech 2021, pages 60–67. IEEE, 2021. </td> </tr> <tr> <td>doh_cic.csv </td> <td>Binary detection of DoH </td> <td>Mohammadreza MontazeriShatoori et al. Detection of doh tunnels using time-series classification of encrypted traffic. In DASC/PiCom/CBDCom/CyberSciTech 2020, pages 63–70. IEEE, 2020 </td> </tr> <tr> <td>doh_real_world.csv </td> <td>Binary detection of DoH </td> <td>Kamil Jeřábek et al. Collection of datasets with DNS over HTTPS traffic. Data in Brief, 42:108310, 2022 </td> </tr> <tr> <td>dos.csv </td> <td>Binary detection of DoS </td> <td>Nickolaos Koroniotis et al. Towards the development of realistic botnet dataset in the Internet of Things for network forensic analytics: Bot-IoT dataset. Future Gener. Comput. Syst., 100:779–796, 2019. </td> </tr> <tr> <td>edge_iiot_binary.csv </td> <td>Binary detection of IoT malware </td> <td>Mohamed Amine Ferrag et al. Edge-iiotset: A new comprehensive realistic cyber security dataset of iot and iiot applications: Centralized and federated learning, 2022. </td> </tr> <tr> <td>edge_iiot_multiclass.csv </td> <td>Multi-class classification of IoT malware </td> <td>Mohamed Amine Ferrag et al. Edge-iiotset: A new comprehensive realistic cyber security dataset of iot and iiot applications: Centralized and federated learning, 2022. </td> </tr> <tr> <td>https_brute_force.csv </td> <td>Binary detection of HTTPS Brute Force </td> <td>Jan Luxemburk et al. HTTPS Brute-force dataset with extended network flows, November 2020 </td> </tr> <tr> <td>ids_cic_binary.csv </td> <td>Binary detection of intrusion in IDS </td> <td>Iman Sharafaldin et al. Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISSp, 1:108–116, 2018. </td> </tr> <tr> <td>ids_cic_multiclass.csv </td> <td>Multi-class classification of intrusion in IDS </td> <td>Iman Sharafaldin et al. Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISSp, 1:108–116, 2018. </td> </tr> <tr> <td>unsw_binary.csv </td> <td>Binary detection of intrusion in IDS </td> <td>Nour Moustafa and Jill Slay. Unsw-nb15: a comprehensive data set for network intrusion detection systems (unsw-nb15 network data set). In 2015 military communications and information systems conference (MilCIS), pages 1–6. IEEE, 2015. </td> </tr> <tr> <td>unsw_multiclass.csv </td> <td>Multi-class classification of intrusion in IDS </td> <td>Nour Moustafa and Jill Slay. Unsw-nb15: a comprehensive data set for network intrusion detection systems (unsw-nb15 network data set). In 2015 military communications and information systems conference (MilCIS), pages 1–6. IEEE, 2015. </td> </tr> <tr> <td>iot_23.csv </td> <td>Binary detection of IoT malware </td> <td>Sebastian Garcia et al. IoT-23: A labeled dataset with malicious and benign IoT network traffic, January 2020. More details here https://www.stratosphereips.org /datasets-iot23 </td> </tr> <tr> <td>ton_iot_binary.csv </td> <td>Binary detection of IoT malware </td> <td>Nour Moustafa. A new distributed architecture for evaluating ai-based security systems at the edge: Network ton iot datasets. Sustainable Cities and Society, 72:102994, 2021 </td> </tr> <tr> <td>ton_iot_multiclass.csv </td> <td>Multi-class classification of IoT malware </td> <td>Nour Moustafa. A new distributed architecture for evaluating ai-based security systems at the edge: Network ton iot datasets. Sustainable Cities and Society, 72:102994, 2021 </td> </tr> <tr> <td>tor_binary.csv </td> <td>Binary detection of TOR </td> <td>Arash Habibi Lashkari et al. Characterization of Tor Traffic using Time based Features. In ICISSP 2017, pages 253–262. SciTePress, 2017. </td> </tr> <tr> <td>tor_multiclass.csv </td> <td>Multi-class classification of TOR </td> <td>Arash Habibi Lashkari et al. Characterization of Tor Traffic using Time based Features. In ICISSP 2017, pages 253–262. SciTePress, 2017. </td> </tr> <tr> <td>vpn_iscx_binary.csv </td> <td>Binary detection of VPN </td> <td>Gerard Draper-Gil et al. Characterization of Encrypted and VPN Traffic Using Time-related. In ICISSP, pages 407–414, 2016. </td> </tr> <tr> <td>vpn_iscx_multiclass.csv </td> <td>Multi-class classification of VPN </td> <td>Gerard Draper-Gil et al. Characterization of Encrypted and VPN Traffic Using Time-related. In ICISSP, pages 407–414, 2016. </td> </tr> <tr> <td>vpn_vnat_binary.csv </td> <td>Binary detection of VPN </td> <td>Steven Jorgensen et al. Extensible Machine Learning for Encrypted Network Traffic Application Labeling via Uncertainty Quantification. CoRR, abs/2205.05628, 2022 </td> </tr> <tr> <td>vpn_vnat_multiclass.csv </td> <td>Multi-class classification of VPN </td> <td>Steven Jorgensen et al. Extensible Machine Learning for Encrypted Network Traffic Application Labeling via Uncertainty Quantification. CoRR, abs/2205.05628, 2022 </td> </tr> </tbody> </table> <p> </p> <p> </p> <p> </p> <p> </p>
Host network traffic time series 2019/01
<p><em><strong>General info</strong></em></p> <p>Dataset was collected over one <strong>month period in January 2019</strong>. The observation points for the collection of IP flows were located at the borders of the university campus network. The campus university network has /16 CIDR IPv4 network range at disposal and contains various network segments from segments connecting dormitories, over server segments, to a segment containing working stations of university administrative workers. The size of the raw IP flows used to create the dataset was over 860GB. <strong>A host in our dataset is identified by its source IPv4 address. </strong><br> </p> <p><em><strong>Variables</strong></em></p> <p>The dataset contains the following variables:</p> <ul> <li><strong>Aggregations</strong> - created from five-minute total volumes aggregated over one-hour disjoint windows using mean/max/min aggregation functions <ul> <li><strong># of flows (FL) </strong>- number of flows for a given source IP </li> <li><strong># of packets (PKT)</strong> - number of packets for a given source IP</li> <li><strong># of bytes (BYT)</strong> - number of packets for a given source IP</li> <li><strong>flow duration (DUR)</strong> - average flow duration in seconds</li> </ul> </li> <li><strong>Distinct Counts </strong>- count of distinct values for each variable in five-minute window aggregated over one-hour disjoint windows using mean/max/min aggregation functions <ul> <li><strong># of peers (PEER)</strong> - number of distinct communication peers for a given source IP</li> <li><strong># of ports (PORTS)</strong> - number of distinct destination ports for a given source IP</li> <li><strong># of protocols (PROTO)</strong> - number of distinct communication protocols for a given source IP</li> <li><strong># of AS numbers (AS)</strong> - number of distinct destination AS numbers for a given source IP</li> <li><strong># of countries (CTRY)</strong> - number of distinct destination countries for a given source IP</li> </ul> </li> <li><strong>Labels</strong> <ul> <li><strong>Range (RNG)</strong> - a network range a host belongs to (anonymized)</li> <li><strong>Unit (UNT) </strong>- an administrative unit owning the network range</li> <li><strong>Sub-unit (SUB-UNT)</strong> - a sub-unit of the unit</li> </ul> </li> </ul> <p> </p> <p><em><strong>Dataset format</strong></em></p> <ul> <li>The dataset is in <strong>comma-separated values (CSV)</strong> format. </li> <li><strong>Header</strong> - multilevel, first 3 lines <ul> <li>1 level - aggregation type {mean|min|max}</li> <li>2 level - variable {see above}</li> <li>3 level - hour of a day {00,01,02,03,...,22,23}</li> </ul> </li> <li><strong>Lablels</strong> - last 4 columns</li> <li><strong>Dataset size </strong> <ul> <li>rows: 65536 host records + 3 headers</li> <li>columns: 648 variables + 4 labels</li> </ul> </li> </ul> <p> </p>
Brainport, Platooning, platoon with live traffic light
<p><strong>Scenario description</strong>:</p> <p>Platoon formation and platooning, from Helmond to Eindhoven and back to the Automotive Campus.<br> - Starting in urban area with speed limits of 15 and 30 km/h.<br> - Driving East on the Europaweg with speed limits of 50 and 70 km/h. This includes 3 crossings with traffic lights.<br> - Driving on the the N270, along the Automotive Campus. One crossing with traffic lights, just before the A270.<br> - Driving on the A270 (speed limit 100 km/h). Interrupted by one traffic light.<br> - U-turn at the fly-over or at the end of the A270, to return the same way to the Automotive Campus.</p> <p><strong>Session description</strong>:</p> <p>Platoon formation and platooning, with live traffic light data included in planner.<br> - Live traffic light data available for planner<br> - Driver uses the Android app<br> - Starting at default locations<br> - Platooning (CACC and lane keeping) on the A270 when possible.</p> <p><strong>Datasets descriptions</strong>:</p> <p><strong>AUTOPILOT_BrainPort_Platooning_DriverVehicleInteraction</strong>: Data extracted from the CAN of the vehicle</p> <p>This dataset contains e.g. throttlestatus, clutchstatus, brakestatus, brakeforce, wipersstatus, steeringwheel for the vehicle</p> <p><strong>AUTOPILOT_BrainPort_Platooning_EnvironmentSensorsAbsolute</strong>: Data extracted from the vehicle environment sensors</p> <p>This dataset contains information about detected object, with absolute coordinates</p> <p><strong>AUTOPILOT_BrainPort_Platooning_EnvironmentSensorsRelative</strong>: Data extracted from the vehicle environment sensors</p> <p>This dataset contains information about detected object, with relative coordinates</p> <p><strong>AUTOPILOT_BrainPort_Platooning_IotVehicleMessage</strong>: Data sent between all devices, vehicles and services</p> <p>Each sensor data submission is a Message. A Message has an Envelope, a Path, and optionally (but likely) Path Events and optionally Path Media. The envelope bears fundamental information about the individual sender (the vehicle) but not to a level that owner of the vehicle can be identified or different messages can be identified that originate from a single vehicle.</p> <p><strong>AUTOPILOT_BrainPort_Platooning_PlatoonFormation</strong>: Data sent from PlatoonService to vehicle</p> <p>This dataset contains information about the route and speed for a specific vehicle for forming a platoon</p> <p><strong>AUTOPILOT_BrainPort_Platooning_PlatooningAction</strong>: Data logged by vehicle</p> <p>This dataset contains information about the current status of the platooning</p> <p><strong>AUTOPILOT_BrainPort_Platooning_PlatooningEvent</strong>: Data logged by vehicle</p> <p>This dataset contains information about the identifiers used for each specific platooning event</p> <p><strong>AUTOPILOT_BrainPort_Platooning_PlatoonStatus</strong>: Data sent by vehicle to PlatoonService</p> <p>This dataset contains information about the current status of the platooning</p> <p><strong>AUTOPILOT_BrainPort_Platooning_PositioningSystem</strong>: Data from GPS on the vehicle</p> <p>This dataset contains speed, longitude, latitude, heading from the GPS</p> <p><strong>AUTOPILOT_BrainPort_Platooning_PositioningSystemResample</strong>: Data from GPS on the vehicle</p> <p>This dataset contains speed,longitude,latitude,heading from the GPS, resampled to 100 milliseconds</p> <p><strong>AUTOPILOT_BrainPort_Platooning_PSInfo</strong>: Data sent by PlatoonService to the vehicle</p> <p>This dataset contains speed and route information for the vehicle to create a platoon</p> <p><strong>AUTOPILOT_BrainPort_Platooning_Target</strong>: Data from sensors on the vehicle</p> <p>Target detection in the vicinity of the host vehicle, by a vehicle sensor or virtual sensor</p> <p><strong>AUTOPILOT_BrainPort_Platooning_Vehicle</strong>: Data from the CAN and sensors about the state of the vehicle</p> <p>This dataset contains a.o temperature and battery state of the vehicles</p> <p><strong>AUTOPILOT_BrainPort_Platooning_VehicleDynamics</strong>: Data from the CAN and sensors about the state of the vehicle</p> <p>This dataset contains a.o accelerations and speedlimit of the vehicle, as observed from the CAN and the external sensors</p> <p><strong>AUTOPILOT_BrainPort_Platooning_VehicleDynamics</strong>: Data from the CAN and sensors about the state of the vehicle</p> <p>This dataset contains a.o accelerations and speedlimit of the vehicle, as observed from the CAN and the external sensors</p>
Brainport, Platooning, formation improvement by traffic light data
<p><strong>Scenario description</strong>:</p> <p>Platoon formation with live traffic light data included in planner.<br> - Enabled live traffic light data included in planner<br> - Not using the Android app<br> - Starting at default locations<br> - This test was filmed, including the GUI.</p> <p><strong>Session description</strong>:</p> <p>Platoon formation improvement by traffic light data.</p> <p><strong>Datasets descriptions</strong>:</p> <p><strong>AUTOPILOT_BrainPort_Platooning_DriverVehicleInteraction</strong>: Data extracted from the CAN of the vehicle</p> <p>This dataset contains e.g. throttlestatus, clutchstatus, brakestatus, brakeforce, wipersstatus, steeringwheel for the vehicle</p> <p><strong>AUTOPILOT_BrainPort_Platooning_EnvironmentSensorsAbsolute</strong>: Data extracted from the vehicle environment sensors</p> <p>This dataset contains information about detected object, with absolute coordinates</p> <p><strong>AUTOPILOT_BrainPort_Platooning_EnvironmentSensorsRelative</strong>: Data extracted from the vehicle environment sensors</p> <p>This dataset contains information about detected object, with relative coordinates</p> <p><strong>AUTOPILOT_BrainPort_Platooning_IotVehicleMessage</strong>: Data sent between all devices, vehicles and services</p> <p>Each sensor data submission is a Message. A Message has an Envelope, a Path, and optionally (but likely) Path Events and optionally Path Media. The envelope bears fundamental information about the individual sender (the vehicle) but not to a level that owner of the vehicle can be identified or different messages can be identified that originate from a single vehicle.</p> <p><strong>AUTOPILOT_BrainPort_Platooning_PlatoonFormation</strong>: Data sent from PlatoonService to vehicle</p> <p>This dataset contains information about the route and speed for a specific vehicle for forming a platoon</p> <p><strong>AUTOPILOT_BrainPort_Platooning_PlatooningAction</strong>: Data logged by vehicle</p> <p>This dataset contains information about the current status of the platooning</p> <p><strong>AUTOPILOT_BrainPort_Platooning_PlatooningEvent</strong>: Data logged by vehicle</p> <p>This dataset contains information about the identifiers used for each specific platooning event</p> <p><strong>AUTOPILOT_BrainPort_Platooning_PlatoonStatus</strong>: Data sent by vehicle to PlatoonService</p> <p>This dataset contains information about the current status of the platooning</p> <p><strong>AUTOPILOT_BrainPort_Platooning_PositioningSystem</strong>: Data from GPS on the vehicle</p> <p>This dataset contains speed, longitude, latitude, heading from the GPS</p> <p><strong>AUTOPILOT_BrainPort_Platooning_PositioningSystemResample</strong>: Data from GPS on the vehicle</p> <p>This dataset contains speed,longitude,latitude,heading from the GPS, resampled to 100 milliseconds</p> <p><strong>AUTOPILOT_BrainPort_Platooning_PSInfo</strong>: Data sent by PlatoonService to the vehicle</p> <p>This dataset contains speed and route information for the vehicle to create a platoon</p> <p><strong>AUTOPILOT_BrainPort_Platooning_Target</strong>: Data from sensors on the vehicle</p> <p>Target detection in the vicinity of the host vehicle, by a vehicle sensor or virtual sensor</p> <p><strong>AUTOPILOT_BrainPort_Platooning_Vehicle</strong>: Data from the CAN and sensors about the state of the vehicle</p> <p>This dataset contains a.o temperature and battery state of the vehicles</p> <p><strong>AUTOPILOT_BrainPort_Platooning_VehicleDynamics</strong>: Data from the CAN and sensors about the state of the vehicle</p> <p>This dataset contains a.o accelerations and speedlimit of the vehicle, as observed from the CAN and the external sensors</p> <p><strong>AUTOPILOT_BrainPort_Platooning_VehicleDynamics</strong>: Data from the CAN and sensors about the state of the vehicle</p> <p>This dataset contains a.o accelerations and speedlimit of the vehicle, as observed from the CAN and the external sensors</p>
Brainport, Platooning, formation and platooning without traffic
<p><strong>Scenario description</strong>:</p> <p>Platoon formation and platooning, from Helmond to Eindhoven and back to the Automotive Campus.<br> - Starting in urban area with speed limits of 15 and 30 km/h.<br> - Driving East on the Europaweg with speed limits of 50 and 70 km/h. This includes 3 crossings with traffic lights.<br> - Driving on the the N270, along the Automotive Campus. One crossing with traffic lights, just before the A270.<br> - Driving on the A270 (speed limit 100 km/h). Interrupted by one traffic light.<br> - U-turn at the fly-over or at the end of the A270, to return the same way to the Automotive Campus.</p> <p><strong>Session description</strong>:</p> <p>Platoon formation and platooning, without live traffic light data included in planner.<br> - No live traffic light data available for planner<br> - Driver uses the Android app<br> - Starting at default locations<br> - Platooning (CACC and lane keeping) on the A270 when possible.</p> <p><strong>Datasets descriptions</strong>:</p> <p><strong>AUTOPILOT_BrainPort_Platooning_DriverVehicleInteraction</strong>: Data extracted from the CAN of the vehicle</p> <p>This dataset contains e.g. throttlestatus, clutchstatus, brakestatus, brakeforce, wipersstatus, steeringwheel for the vehicle</p> <p><strong>AUTOPILOT_BrainPort_Platooning_EnvironmentSensorsAbsolute</strong>: Data extracted from the vehicle environment sensors</p> <p>This dataset contains information about detected object, with absolute coordinates</p> <p><strong>AUTOPILOT_BrainPort_Platooning_EnvironmentSensorsRelative</strong>: Data extracted from the vehicle environment sensors</p> <p>This dataset contains information about detected object, with relative coordinates</p> <p><strong>AUTOPILOT_BrainPort_Platooning_IotVehicleMessage</strong>: Data sent between all devices, vehicles and services</p> <p>Each sensor data submission is a Message. A Message has an Envelope, a Path, and optionally (but likely) Path Events and optionally Path Media. The envelope bears fundamental information about the individual sender (the vehicle) but not to a level that owner of the vehicle can be identified or different messages can be identified that originate from a single vehicle.</p> <p><strong>AUTOPILOT_BrainPort_Platooning_PlatoonFormation</strong>: Data sent from PlatoonService to vehicle</p> <p>This dataset contains information about the route and speed for a specific vehicle for forming a platoon</p> <p><strong>AUTOPILOT_BrainPort_Platooning_PlatooningAction</strong>: Data logged by vehicle</p> <p>This dataset contains information about the current status of the platooning</p> <p><strong>AUTOPILOT_BrainPort_Platooning_PlatooningEvent</strong>: Data logged by vehicle</p> <p>This dataset contains information about the identifiers used for each specific platooning event</p> <p><strong>AUTOPILOT_BrainPort_Platooning_PlatoonStatus</strong>: Data sent by vehicle to PlatoonService</p> <p>This dataset contains information about the current status of the platooning</p> <p><strong>AUTOPILOT_BrainPort_Platooning_PositioningSystem</strong>: Data from GPS on the vehicle</p> <p>This dataset contains speed, longitude, latitude, heading from the GPS</p> <p><strong>AUTOPILOT_BrainPort_Platooning_PositioningSystemResample</strong>: Data from GPS on the vehicle</p> <p>This dataset contains speed,longitude,latitude,heading from the GPS, resampled to 100 milliseconds</p> <p><strong>AUTOPILOT_BrainPort_Platooning_PSInfo</strong>: Data sent by PlatoonService to the vehicle</p> <p>This dataset contains speed and route information for the vehicle to create a platoon</p> <p><strong>AUTOPILOT_BrainPort_Platooning_Target</strong>: Data from sensors on the vehicle</p> <p>Target detection in the vicinity of the host vehicle, by a vehicle sensor or virtual sensor</p> <p><strong>AUTOPILOT_BrainPort_Platooning_Vehicle</strong>: Data from the CAN and sensors about the state of the vehicle</p> <p>This dataset contains a.o temperature and battery state of the vehicles</p> <p><strong>AUTOPILOT_BrainPort_Platooning_VehicleDynamics</strong>: Data from the CAN and sensors about the state of the vehicle</p> <p>This dataset contains a.o accelerations and speedlimit of the vehicle, as observed from the CAN and the external sensors</p> <p><strong>AUTOPILOT_BrainPort_Platooning_VehicleDynamics</strong>: Data from the CAN and sensors about the state of the vehicle</p> <p>This dataset contains a.o accelerations and speedlimit of the vehicle, as observed from the CAN and the external sensors</p>
Curated Research on Network Traffic Analysis
<p>With the NTA Database we aim to collect relevant information about the research in network traffic analysis conducted during the last years. To this end, we have curated related papers from journals and conferences and stored the extracted data in JSON files. </p>
Traffic and Log Data Captured During a Cyber Defense Exercise
<p>This dataset was acquired during Cyber Czech – a hands-on cyber defense exercise (Red Team/Blue Team) held in March 2019 at Masaryk University, Brno, Czech Republic. Network traffic flows and a high variety of event logs were captured in an <em>exercise network</em> deployed in the KYPO Cyber Range Platform.</p> <p><strong>Contents</strong></p> <p>The dataset covers two distinct time intervals, which correspond to the official schedule of the exercise. The timestamps provided below are in the ISO 8601 date format. </p> <ul> <li>Day 1, March 19, 2019 <ul> <li>Start: 2019-03-19T11:00:00.000000+01:00 </li> <li>End: 2019-03-19T18:00:00.000000+01:00 </li> </ul> </li> <li>Day 2, March 20, 2019 <ul> <li>Start: 2019-03-20T08:00:00.000000+01:00 </li> <li>End: 2019-03-20T15:30:00.000000+01:00 </li> </ul> </li> </ul> <p>The captured and collected data were normalized into three distinct event types and they are stored as structured JSON. The data are sorted by a timestamp, which represents the time they were observed. Each event type includes a raw payload ready for further processing and analysis. The description of the respective event types and the corresponding data files follows. </p> <ul> <li><em>cz.muni.csirt.IpfixEntry.tgz</em> – an archive of IPFIX traffic flows enriched with an additional payload of parsed application protocols in raw JSON. </li> <li><em>cz.muni.csirt.SyslogEntry.tgz</em> – an archive of Linux Syslog entries with the payload of corresponding text-based log messages. </li> <li><em>cz.muni.csirt.WinlogEntry.tgz</em> – an archive of Windows Event Log entries with the payload of original events in raw XML. </li> </ul> <p>Each archive listed above includes a directory of the same name with the following four files, ready to be processed. </p> <ul> <li><em>data.json.gz </em>– the actual data entries in a single gzipped JSON file. </li> <li><em>dictionary.yml</em> – data dictionary for the entries. </li> <li><em>schema.ddl</em> – data schema for Apache Spark analytics engine. </li> <li><em>schema.jsch</em> – JSON schema for the entries. </li> </ul> <p>Finally, the exercise network topology is described in a machine-readable NetJSON format and it is a part of a set of auxiliary files archive – <em>auxiliary-material.tgz</em> – which includes the following. </p> <ul> <li><em>global-gateway-config.json</em> – the network configuration of the global gateway in the NetJSON format. </li> <li><em>global-gateway-routing.json</em> – the routing configuration of the global gateway in the NetJSON format. </li> <li><em>redteam-attack-schedule.{csv,odt}</em> – the schedule of the Red Team attacks in CSV and ODT format. Source for Table 2. </li> <li><em>redteam-reserved-ip-ranges.{csv,odt}</em> – the list of IP segments reserved for the Red Team in CSV and ODT format. Source for Table 1. </li> <li><em>topology.{json,pdf,png}</em> – the topology of the complete Cyber Czech exercise network in the NetJSON, PDF and PNG format. </li> <li><em>topology-small.{pdf,png}</em> – simplified topology in the PDF and PNG format. Source for Figure 1. </li> </ul> <p> </p>
Recording of filesharing traffic
<p>The recording was created with the General Purpose Network Testing Framework:</p> <p><a href="https://github.com/vs-uulm/General-Purpose-Network-Testing-Framework">https://github.com/vs-uulm/General-Purpose-Network-Testing-Framework</a></p>
[FED4FIRE+] [MMT-IoT] 6LoWPAN - IoT Traffic Dataset
<p>The dataset contains the pcap files of IoT network traffic captured in the experiments with w-iLab.t and Log-a-Tec testbeds in the context of FED4FIRE+ open call project. </p>
Zebra finches increase social behavior in traffic noise: implications for urban songbirds
<p>Statistical code & datasets for "Zebra finches increase social behavior in traffic noise: implications for urban songbirds" manuscript submitted to <em>Acta ethologica. </em>Also includes audio file for traffic noise playback in described experiment. </p>
An urban traffic dataset composed of visible images and their semantic segmentation generated by the CARLA simulator
<p><strong>If you use this dataset please cite this paper: Rosende, S.B.; Gavilán, D.S.J.; Fernández-Andrés, J.; Sánchez-Soriano, J. An Urban Traffic Dataset Composed of Visible Images and Their Semantic Segmentation Generated by the CARLA Simulator. <em>Data</em> 2024, <em>9</em>, 4. <a href="https://doi.org/10.3390/data9010004">https://doi.org/10.3390/data9010004</a></strong></p> <p>A dataset of aerial urban traffic images and their semantic segmentation is presented to be used to train computer vision algorithms, among which those based on convolutional neural networks stand out. The images have been generated using the CARLA simulator (but would be like those that could be obtained with fixed aerial cameras or by using AUVs) in the field of intelligent transportation management. The presented dataset is available and accessible to improve the performance of vision and road traffic management systems, especially for the detection of incorrect or dangerous maneuvers.</p>
Consistent traffic noise impacts few fitness-related traits in a field cricket
<p>Anthropogenic habitat change is occurring rapidly, and organisms can respond through within-generation responses that improve the match between their phenotype and the novel conditions they encounter. But, plastic responses can be adaptive or maladaptive and are most likely to be adaptive only when contemporary conditions reasonably mimic something experienced historically to which a response has already evolved. Noise pollution is a ubiquitous anthropogenic stressor that accompanies expanding urbanization. We tested whether the amplitude of traffic noise influences a suite of fitness-related traits (e.g. survival, life history, reproductive investment, immunity) and whether that depends on the life stage at which the noise is experienced (juvenile or adult). Our treatments mimic the conditions experienced by animals living in urban roadside environments with variable vehicle types, but continuous movement of traffic. We used the Pacific field cricket, an acoustically communicating insect that was previously shown to experience some negative behavioral and life history responses to very loud, variable traffic noise, as a model system. </p>
GRN-AV-traffic-entity
<p>The GRN-AV-traffic-entity dataset is a collection of labelled multi-modal synched audio-video (AV) data, which is labelled with traffic sounds (e.g., engine noise, wheel rolling and vehicle horn). The video frames are not labelled, but are included.</p>
GRN-AV-traffic-state
<p>This GRN-AV-traffic-state dataset is a collection of labelled multi-modal data for the purpose of training audio classification models </p>
CESNET-QUIC22: A large one-month QUIC network traffic dataset from backbone lines
<p><strong>Please refer to the original data article for further data description: </strong>Jan Luxemburk et al. CESNET-QUIC22: A large one-month QUIC network traffic dataset from backbone lines, Data in Brief, 2023, 108888, ISSN 2352-3409, <a href="https://doi.org/10.1016/j.dib.2023.108888">https://doi.org/10.1016/j.dib.2023.108888</a>. </p><p><strong>We recommend using the</strong> <strong>CESNET DataZoo python library, which facilitates the work with large network traffic datasets. </strong>More information about the DataZoo project can be found in the GitHub repository <a href="https://github.com/CESNET/cesnet-datazoo">https://github.com/CESNET/cesnet-datazoo</a>.</p><p>The QUIC (Quick UDP Internet Connection) protocol has the potential to replace TLS over TCP, which is the standard choice for reliable and secure Internet communication. Due to its design that makes the inspection of QUIC handshakes challenging and its usage in HTTP/3, there is an increasing demand for research in QUIC traffic analysis. This dataset contains one month of QUIC traffic collected in an ISP backbone network, which connects 500 large institutions and serves around half a million people. The data are delivered as enriched flows that can be useful for various network monitoring tasks. The provided server names and packet-level information allow research in the encrypted traffic classification area. Moreover, included QUIC versions and user agents (smartphone, web browser, and operating system identifiers) provide information for large-scale QUIC deployment studies.</p><p><strong>Data capture</strong> The data was captured in the flow monitoring infrastructure of the <a href="https://www.cesnet.cz">CESNET2</a> network. The capturing was done for four weeks between 31.10.2022 and 27.11.2022. The following list provides per-week flow count, capture period, and uncompressed size:</p><ul><li><strong>W-2022-44</strong><ul><li>Uncompressed Size: 19 GB</li><li>Capture Period: 31.10.2022 - 6.11.2022</li><li>Number of flows: 32.6M</li></ul></li><li><strong>W-2022-45</strong><ul><li>Uncompressed Size: 25 GB</li><li>Capture Period: 7.11.2022 - 13.11.2022</li><li>Number of flows: 42.6M</li></ul></li><li><strong>W-2022-46</strong><ul><li>Uncompressed Size: 20 GB</li><li>Capture Period: 14.11.2022 - 20.11.2022</li><li>Number of flows: 33.7M</li></ul></li><li><strong>W-2022-47</strong><ul><li>Uncompressed Size: 25 GB</li><li>Capture Period: 21.11.2022 - 27.11.2022</li><li>Number of flows: 44.1M</li></ul></li><li><strong>CESNET-QUIC22 </strong><ul><li>Uncompressed Size: 89 GB</li><li>Capture Period: 31.10.2022 - 27.11.2022</li><li>Number of flows: 153M</li></ul></li></ul><p> </p><p><strong>Data description</strong> The dataset consists of network flows describing encrypted QUIC communications. Flows were created using <a href="https://github.com/CESNET/ipfixprobe">ipfixprobe</a> flow exporter and are extended with packet metadata sequences, packet histograms, and with fields extracted from the QUIC Initial Packet, which is the first packet of the QUIC connection handshake. The extracted handshake fields are the Server Name Indication (SNI) domain, the used version of the QUIC protocol, and the user agent string that is available in a subset of QUIC communications.</p><p><strong>Packet Sequences</strong> Flows in the dataset are extended with sequences of packet sizes, directions, and inter-packet times. For the packet sizes, we consider payload size after transport headers (UDP headers for the QUIC case). Packet directions are encoded as ±1, <i>+1</i> meaning a packet sent from client to server, and <i>-1</i> a packet from server to client. Inter-packet times depend on the location of communicating hosts, their distance, and on the network conditions on the path. However, it is still possible to extract relevant information that correlates with user interactions and, for example, with the time required for an API/server/database to process the received data and generate the response to be sent in the next packet. Packet metadata sequences have a length of 30, which is the default setting of the used flow exporter. We also derive three fields from each packet sequence: its length, time duration, and the number of roundtrips. The roundtrips are counted as the number of changes in the communication direction (from packet directions data); in other words, each client request and server response pair counts as one roundtrip.</p><p><strong>Flow statistics</strong> Flows also include standard flow statistics, which represent aggregated information about the entire bidirectional flow. The fields are: the number of transmitted bytes and packets in both directions, the duration of flow, and packet histograms. Packet histograms include binned counts of packet sizes and inter-packet times of the entire flow in both directions (more information in the <a href="https://github.com/CESNET/ipfixprobe/tree/master#phists">PHISTS plugin documentation</a> There are eight bins with a logarithmic scale; the intervals are 0-15, 16-31, 32-63, 64-127, 128-255, 256-511, 512-1024, >1024 [ms or B]. The units are milliseconds for inter-packet times and bytes for packet sizes. Moreover, each flow has its end reason - either it was idle, reached the active timeout, or ended due to other reasons. This corresponds with the official <a href="https://www.iana.org/assignments/ipfix/ipfix.xhtml#ipfix-flow-end-reason">IANA IPFIX-specified values</a>. The <i>FLOW_ENDREASON_OTHER</i> field represents the <i>forced end</i> and <i>lack of resources</i> reasons. The <i>end of flow detected</i> reason is not considered because it is not relevant for UDP connections.</p><p><strong>Dataset structure</strong> The dataset flows are delivered in compressed CSV files. CSV files contain one flow per row; data columns are summarized in the provided list below. For each flow data file, there is a JSON file with the number of saved and seen (before sampling) flows per service and total counts of all received (observed on the CESNET2 network), service (belonging to one of the dataset's services), and saved (provided in the dataset) flows. There is also the <i>stats-week.json</i> file aggregating flow counts of a whole week and the <i>stats-dataset.json</i> file aggregating flow counts for the entire dataset. Flow counts before sampling can be used to compute sampling ratios of individual services and to resample the dataset back to the original service distribution. Moreover, various dataset statistics, such as feature distributions and value counts of QUIC versions and user agents, are provided in the <i>dataset-statistics</i> folder. The mapping between services and service providers is provided in the <i>servicemap.csv</i> file, which also includes SNI domains used for ground truth labeling. The following list describes flow data fields in CSV files:</p><ul><li><strong>ID:</strong> Unique identifier</li><li><strong>SRC_IP:</strong> Source IP address</li><li><strong>DST_IP:</strong> Destination IP address</li><li><strong>DST_ASN:</strong> Destination Autonomous System number</li><li><strong>SRC_PORT:</strong> Source port</li><li><strong>DST_PORT:</strong> Destination port</li><li><strong>PROTOCOL:</strong> Transport protocol</li><li><strong>QUIC_VERSION QUIC:</strong> protocol version</li><li><strong>QUIC_SNI:</strong> Server Name Indication domain</li><li><strong>QUIC_USER_AGENT:</strong> User agent string, if available in the QUIC Initial Packet</li><li><strong>TIME_FIRST:</strong> Timestamp of the first packet in format YYYY-MM-DDTHH-MM-SS.ffffff</li><li><strong>TIME_LAST:</strong> Timestamp of the last packet in format YYYY-MM-DDTHH-MM-SS.ffffff</li><li><strong>DURATION:</strong> Duration of the flow in seconds</li><li><strong>BYTES:</strong> Number of transmitted bytes from client to server</li><li><strong>BYTES_REV:</strong> Number of transmitted bytes from server to client</li><li><strong>PACKETS:</strong> Number of packets transmitted from client to server</li><li><strong>PACKETS_REV:</strong> Number of packets transmitted from server to client</li><li><strong>PPI:</strong> Packet metadata sequence in the format: [[inter-packet times], [packet directions], [packet sizes]]</li><li><strong>PPI_LEN:</strong> Number of packets in the PPI sequence</li><li><strong>PPI_DURATION:</strong> Duration of the PPI sequence in seconds</li><li><strong>PPI_ROUNDTRIPS:</strong> Number of roundtrips in the PPI sequence</li><li><strong>PHIST_SRC_SIZES:</strong> Histogram of packet sizes from client to server</li><li><strong>PHIST_DST_SIZES: </strong>Histogram of packet sizes from server to client</li><li><strong>PHIST_SRC_IPT: </strong>Histogram of inter-packet times from client to server</li><li><strong>PHIST_DST_IPT:</strong> Histogram of inter-packet times from server to client</li><li><strong>APP:</strong> Web service label</li><li><strong>CATEGORY:</strong> Service category</li><li><strong>FLOW_ENDREASON_IDLE:</strong> Flow was terminated because it was idle</li><li><strong>FLOW_ENDREASON_ACTIVE:</strong> Flow was terminated because it reached the active timeout</li><li><strong>FLOW_ENDREASON_OTHER:</strong> Flow was terminated for other reasons</li></ul><p> </p><p><strong>Link to other CESNET datasets</strong></p><ul><li><a href="https://www.liberouter.org/technology-v2/tools-services-datasets/datasets/">https://www.liberouter.org/technology-v2/tools-services-datasets/datasets/</a></li><li><a href="https://github.com/CESNET/cesnet-datazoo">https://github.com/CESNET/cesnet-datazoo</a></li></ul><p><strong>Please cite the original data article:</strong></p><blockquote><p>@article{CESNETQUIC22, author = {Jan Luxemburk and Karel Hynek and Tomáš Čejka and Andrej Lukačovič and Pavel Šiška}, title = {CESNET-QUIC22: a large one-month QUIC network traffic dataset from backbone lines}, journal = {Data in Brief}, pages = {108888}, year = {2023}, issn = {2352-3409}, doi = {https://doi.org/10.1016/j.dib.2023.108888}, url = {https://www.sciencedirect.com/science/article/pii/S2352340923000069} }</p></blockquote>
GRN-TXT-traffic-data
<p>This dataset is generated from the GDPR compliant AI models developed in MARVEL (objective 2) and/or GRN’s proprietary CATFlow system. These models take as input the audio or video streams and output non-binary structured data, which is further processed in the implementation of some use-cases.</p>
Fig 1 in Effect of traffic noise on Scinax nasicus advertisement call (Amphibia, Anura)
Fig 1. Noise backgrounds and Scinax nasicus (Cope, 1862) call parameters at reference (Site A) and noisy environments (Site B). Amplitude (oscillograms A1, B1) and frequency (spectrograms A2, B2) of the environments. Amplitude (oscillograms A3, B3) and frequency (spectrograms A4, B4) of the advertisement frog's call.
Fig 2 in Effect of traffic noise on Scinax nasicus advertisement call (Amphibia, Anura)
Fig 2. Non-metric multidimensional scaling (NMDS) ordination of call variables of Scinax nasicus (Cope, 1862) adult males in natural (filled circles, Site A) and noisy environments (empty circles, Site B) Sites. The minimum frequency and duration as well as the displayed note pulse of the frogs call were different between the Sites in the first dimension.
Vehicular traffic count measurements at Tampere City, Finland, Updated version
<p>Inductive loop vehicular traffic count measurements from Tampere City, Finland</p> <p>15-minute resolution, 2 locations, 2011-2014</p> <p>Data utilized in "Origin-destination matrix estimation with a conditionally binomial model" by P.Kuusela, I. Norros, J. Kilpi and T. Räty</p> <p>See VehicularTrafficCountMeasurements_TampereFinland_2011-2014.rtf</p>
Administrative OpenStack Traffic
<p>This is a simple dataset which groups the administrative network traffic volume from OpenStack clouds by Virtual Machine (VM) operations. This dataset considers ten different images of OS for the VMs.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.