AIT Netflow Data Set

<p><strong>AIT Netflow Data Sets</strong></p> <p>This repository contains labeled synthetic netflows suitable for evaluation of intrusion detection systems, federated learning, and alert aggregation. The netflows are generated from the packet captures contained in the <a href="http://doi.org/10.5281/zenodo.5789064">AIT-LDS-v2.0</a>. A detailed description of that dataset is available in [1]. The packet captures were collected from eight testbeds that were built at the Austrian Institute of Technology (AIT) following the approach by [2]. Please cite these papers if the data is used for academic publications.</p> <p>In brief, each of the datasets corresponds to a testbed representing a small enterprise network including mail server, file share, WordPress server, VPN, firewall, etc. Normal user behavior is simulated to generate background noise over a time span of 4-6 days. At some point, a sequence of attack steps is launched against the network. The following attacks are launched in the network:</p> <ul> <li>Scans (nmap, WPScan, dirb)</li> <li>Webshell upload (CVE-2020-24186)</li> <li>Password cracking (John the Ripper)</li> <li>Privilege escalation</li> <li>Remote command execution</li> <li>Data exfiltration (DNSteal)</li> </ul> <p>This repository contains the following files:</p> <ul> <li><em>&lt;testbed&gt;_netflows.zip</em>: CSV files of labeled TCP and UDP netflows for each testbed.</li> <li><em>label_info.txt</em>: File describing which labels in TCP and UDP are benign and which ones are malicious.</li> <li><em>README.md</em>: Instructions on how to reproduce the generation and labeling of the netflows from the <a href="http://doi.org/10.5281/zenodo.5789064">AIT-LDS-v2.0</a>. Note that it is only necessary to run the python scripts if you want to extend or change the labeling procedure.</li> <li><em>1_format_dataset_info.ipynb</em>: Generates the tables necessary for labeling (see README.md).</li> <li><em>2_label_logs.ipynb</em>: Labels the netflows (see README.md).</li> </ul> <p>Acknowledgements: Partially funded by the FFG projects INDICAETING (868306) and DECEPT (873980), and the EU projects GUARD (833456) and PANDORA (SI2.835928).</p> <p><strong>If you use the dataset, please cite the following publications:</strong></p> <p>[1] M. Landauer, F. Skopik, M. Frank, W. Hotwagner, M. Wurzenberger, and A. Rauber. <a href="https://ieeexplore.ieee.org/abstract/document/9866880">"Maintainable Log Datasets for Evaluation of Intrusion Detection Systems"</a>. IEEE Transactions on Dependable and Secure Computing, vol. 20, no. 4, pp. 3466-3482. [<a href="https://arxiv.org/pdf/2203.08580.pdf">PDF</a>]</p> <p>[2]&nbsp;M. Landauer, F. Skopik, M. Wurzenberger, W. Hotwagner and A. Rauber, <a href="https://ieeexplore.ieee.org/document/9262078">"Have it Your Way: Generating Customized Log Datasets With a Model-Driven Simulation Testbed,"</a> in IEEE Transactions on Reliability, vol. 70, no. 1, pp. 402-415, March 2021, doi: 10.1109/TR.2020.3031317. [<a href="https://www.skopik.at/ait/2020_trel.pdf">PDF</a>]</p>