Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
111
datasets available to search
ShareScore release 0.9.0
Dataset results
111 results for “DNS”
DNS Threats Dataset
<p>The dataset contains Normal, DGA and Tunneling domain names: i. the total number of normal domains are conformed by the Alexa top one million domains, 3,161 normal domains provided by the Bambenek Consulting feed, and another 177,017 normal domains; ii. the DGA domains were obtained from the repositories of DGA domains of Andrey Abakumov and John Bambenek, corresponding to 51 different malware families; iii. the DNS Tunneling consist of 8000 tunnel domains generated using a set of well known DNS tunneling tools under laboratory conditions: iodine, dnscat2 and dnsExfiltrator.</p> <p>The dataset is described in the paper:<br> Palau, F., Catania, C., Guerra, J., García, S. J., & Rigaki, M. (2019). Detecting DNS threats: A deep learning model to rule them all. In XX Simposio Argentino de Inteligencia Artificial (ASAI 2019)-JAIIO 48 (Salta).</p>
Phishing and Benign Domain Dataset (DNS, IP, WHOIS/RDAP, TLS, GeoIP)
<p>The dataset contains DNS records, IP-related features, WHOIS/RDAP information, information from TLS certificate fields, and GeoIP information for 432,572 verified benign domains from Cisco Umbrella and 36,993 verified phishing domains from PhishTank and OpenPhish services. The dataset is useful for statistical analysis of domain data or feature extraction for training machine learning-based classifiers, e.g. for phishing detection. The data was collected between March and July 2023.The final assessment of the data was conducted in July 2023 (this is why the names are suffixed with _2307).</p> <p>The upload contains: a) <strong>data</strong> files, b) the description of the data <strong>structure</strong>, and c) the <strong>veature vector</strong> we used for ML-based phishing domain detection.</p> <h1>Data Files</h1> <p>The data is located in two individual files:</p> <ul> <li><strong>benign_2307.json</strong> - data about 432,572 benign domains, and</li> <li><strong>phishing_2307.json</strong> - data about 36,993 phishing domains.</li> </ul> <h1>Data Structure</h1> <p>Both files are in the JSON Array format. The structure is as follows:</p> <pre><code>[ { "_id" : "A unique ID of the data record", "domain_name" : "Name of the domain (e.g., zenodo.com)", "dns" : { "//": "Data obtained from DNS records" }, "evaluated_on" : "// ISO Timestamp of data collection ", "ip_data" : [ "// Data for each related IP adddress ", { "//": "IP-related data, including RTT from ICMP echo attempts (from Brno, Czechia)", "//": "WHOIS/RDAP data for the given IP address", "//": "GeoIP data for the given IP address", "//": "NERD system reputation score (if available)", "//": "ASN info", "//": "remarks: ISO timestamps of collection of the individual data pieces" }, ], "label" : "benign_2307 for benign OR misp_2307 for phishing", "rdap" : { "//": "WHOIS/RDAP information for the domain name" }, "remarks" : { "dns_evaluated_on" : "ISO Timestamp of DNS data collection", "rdap_evaluated_on" : "ISO Timestamp of WHOIS/RDAP data collection", "tls_evaluated_on" : "ISO Timestamp of TLS certificate information collection", "dns_had_no_ips" : "true if no IPs were found in DNS records" }, "sourced_on" : "ISO Timestamp of the moment the domain was found", "tls" : { "cipher" : "Identifier of the TLS cipher suite", "count" : "Number of certificates in chain", "protocol" : "Version of the TLS protocol", "certificates" : [ "//": "Information from TLS certificate fields: issuer, extensions, etc." ] }, "category" : "Category of the record (could be ignored)", "source" : "Name of the file that we used to save the domain list" } ]</code></pre> <h1>Feature Vector</h1> <p>This section describes the veature vector used in the "Unmasking the Phishermen: Phishing Domain Detection with Machine Learning and Multi-Source Intelligence" paper that was accepted to the IEEE NOMS 2024 conference.</p> <h2>Lexical Features</h2> <p>The following features were extracted from the sole domain name:</p> <ul> <li><strong>lex_name_len</strong> - length of the domain name,</li> <li><strong>lex_begins_with_digit</strong> - true if the domain name begins with a digit,</li> <li><strong>lex_www_flag</strong> - true if the domain name begins with "www.",</li> <li><strong>lex_phishing_keyword_count</strong> - occurence count of 47 phishing-related keywords,</li> <li><strong>lex_consecutive_chars</strong> - length of the longest consecutive character sequence,</li> <li><strong>lex_tld_len</strong> - length of the top-level domain (TLD),</li> <li><strong>lex_tld_hash</strong> - hash of the TLD,</li> <li><strong>lex_sld_len </strong>- length of the second-level domain (SLD),</li> <li><strong>lex_sld_norm_entropy </strong>- normalized entropy of the SLD,</li> <li><strong>lex_stld_unique_char_count </strong>- number of unique characters in the TLD and the SLD,</li> <li><strong>lex_sub_count </strong>- number of subdomains,</li> <li><strong>lex_sub_digit_ratio </strong>- ratio of digits in subdomains,</li> <li><strong>lex_sub_hex_ratio </strong>- ratio of hex symbols in subdomains,</li> <li><strong>lex_sub_non_alpanum_ratio </strong>- ratio of non-alphanumeric symbols in subdomains,</li> <li><strong>lex_sub_vowel_ratio </strong>- ratio of vowels in subdomains,</li> <li><strong>lex_sub_consonant_ratio </strong>- ratio of consonants in subdomains,</li> <li><strong>lex_sub_max_consonant_len </strong>- length of the longest consonant sequence in subdomains,</li> <li><strong>lex_sub_norm_entropy </strong>- normalized entropy of a string made from all subdomains,</li> <li><strong>lex_phishing_bigram_matches</strong> - occurrence count of the top 300 phishing domain bigrams,</li> <li><strong>lex_phishing_trigram_matches</strong> - occurrence count of the top 2000 phishing domain trigrams,</li> <li><strong>lex_phishing_tetragram_matches</strong> - occurrence count of the top 5000 phishing domain tetragrams,</li> <li><strong>lex_phishing_pentagram_matches</strong> - occurrence count of the top 10000 phishing domain pentagrams.</li> </ul> <h2>DNS-based Features</h2> <p>The following features were extracted from DNS responses when querying about the domain:</p> <ul> <li><strong>dns_A_count</strong> - number of A records for the domain,</li> <li><strong>dns_AAAA_count</strong> - number of AAAA records for the domain,</li> <li><strong>dns_CNAME_count</strong> - number of CNAME records for the domain,</li> <li><strong>dns_MX_count</strong> - number of MX records for the domain,</li> <li><strong>dns_NS_count</strong> - number of nameserver (NS) records for the domain,</li> <li><strong>dns_TXT_count</strong> - number of TXT records for the domain,</li> <li><strong>dns_soa_primary_ns_len</strong> - number of characters in the primary NS's domain name,</li> <li><strong>dns_soa_primary_ns_level</strong> - number of subdomain in the primary NS's domain name,</li> <li><strong>dns_soa_primary_ns_digit_count</strong> - number of digits in the primary NS's domain name,</li> <li><strong>dns_soa_primary_ns_entropy </strong>- normalized entropy of the primary NS's domain name,</li> <li><strong>dns_soa_email_len</strong> - number of characters in the admin's email domain name part,</li> <li><strong>dns_soa_email_level</strong> - number of subdomains in the admin's email domain name part,</li> <li><strong>dns_soa_email_digit_count</strong> - number of digits in the admin's email domain name part,</li> <li><strong>dns_soa_email_entropy</strong> - normalized entropy of the admin's email domain name part,</li> <li><strong>dns_soa_refresh</strong> - SOA refresh parameter,</li> <li><strong>dns_soa_retry</strong> - SOA retry parameter,</li> <li><strong>dns_soa_expire</strong> - SOA expire parameter,</li> <li><strong>dns_mx_avg_len</strong> - average number of characters of the domain names in MX records,</li> <li><strong>dns_mx_avg_entropy</strong> - average normalized entropy of the domain names in MX records,</li> <li><strong>dns_domain_name_in_mx </strong>- true if the domain name is contained in the MX record's domains,</li> <li><strong>dns_txt_spf_exists</strong> - true if an SPF record is in the TXT RRs,</li> <li><strong>dns_txt_avg_entropy</strong> - average normalized entropy of the TXT records</li> <li><strong>dns_ttl_low</strong> - number of RRsets with TTL in [0,100],</li> <li><strong>dns_ttl_mid </strong>- number of RRsets with TTL in [101,500],</li> <li><strong>dns_zone_entropy</strong> - normalized entropy of the zone's domain name.</li> </ul> <h2>IP-based Features</h2> <p>These features were derived from IP addresses and ICMP echo replies:</p> <ul> <li><strong>ip_mean_average_rtt</strong> - average RTT of all ICMP echo attempts,</li> <li><strong>ip_entropy</strong> - total entropy of all /16 (/64 for v6) IP prefixes,</li> <li><strong>ip_count</strong> - total number of IP addresses for the domain,</li> <li><strong>ip_v4_count</strong> - total number of IPv4 addresses for the domain,</li> <li><strong>ip_v6_count</strong> - total number of IPv6 addresses for the domain,</li> </ul> <h2>TLS-based Features</h2> <p>The following features were extracted from TLS certificate chains and TLS handshakes:</p> <ul> <li><strong>tls_chain_len</strong> - length of the TLS certificate chain,</li> <li><strong>tls_broken_chain</strong> - true if there is a certificate that has never been valid,</li> <li><strong>tls_expired_chain </strong>- true if there is an expired certificate in the chain,</li> <li><strong>tls_total_extension_count</strong> - total extensions in all certificates in the chain,</li> <li><strong>tls_critical_extensions</strong> - total extensions flagged as "critical" in all certificates,</li> <li><strong>tls_with_policies_crt_count</strong> - number of certificates that include the "policies" extension,</li> <li><strong>tls_percentage_crt_with_policies</strong> - percentage of certificates that include the "policies" extension,</li> <li><strong>tls_x509_anypolicy_crt_count</strong> - number of certificates not enforcing any security policy,</li> <li><strong>tls_iso_policy_crt_count</strong> - total discovered policies from the 1.* OID space,</li> <li><strong>tls_joint_isoitu_policy_crt_count</strong> - total discovered policies from from the 2.* OID space,</li> <li><strong>tls_subject_count</strong> - number of subject alternative names (SANs) in the leaf certificate,</li> <li><strong>tls_server_auth_crt_count</strong> - number of certificates with the "Web Server Authentication",</li> <li><strong>tls_client_auth_crt_count</strong> - number of certificates with the "Web Client Authentication",</li> <li><strong>tls_CA_certs_in_chain_ratio</strong> - ratio of CA certificates in the chain,</li> <li><strong>tls_unique_SLD_count</strong> -number of unique second-level domains (SLD) in domain name SANs, </li> <li><strong>tls_common_name_count</strong> - number of common names in the chains,</li> <li><strong>tls_root_cert_validity_len</strong> - length of the validity period of the root certificate,</li> <li><strong>tls_leaf_cert_validity_len</strong> - length of the validity period of the leaf certificate.</li> </ul> <h2>WHOIS/RDAP-based Features</h2> <p>These features are based on the information gathered from WHOIS/RDAP when asking about: a) the domain name and b) domain-related IP addresses:</p> <ul> <li><strong>rdap_registration_period</strong> - difference between domain expiration and registration date,</li> <li><strong>rdap_has_dnssec</strong> - true if DNSSEC is used for the domain,</li> <li><strong>rdap_domain_age</strong> - days elapsed from the domain registration,</li> <li><strong>rdap_time_from_last_change</strong> - days elapsed from the last change of records,</li> <li><strong>rdap_domain_active_time</strong> - min(today, expiration) - registration date,</li> <li><strong>rdap_registrar_name_hash</strong> - hash of the domain's registrar,</li> <li><strong>rdap_ip_avg_admin_name_len</strong> - average length of the admin's name for IP addresses.</li> </ul> <h2>Geolocation Features</h2> <p>This set of features is based on information gathered from the GeoIP service when asked about domain-related IP addresses:</p> <ul> <li><strong>geo_countries_count</strong> - number of distinct countries where servers of domain-related IPs are located,</li> <li><strong>geo_countries_hash</strong> - a unique hash for each combination of countries amongst domain-related IPs,</li> <li><strong>geo_continent_hash</strong> - a uniuque hash for each combinations of continents where the countries are situated.</li> </ul>
Dataset used for fingerprinting of DNS over HTTPS responses.
<p><strong> </strong>The dataset consists of multiple different data sources:</p> <ol> <li>DoH enabled Firefox on Linux OS</li> <li>DoH enabled Firefox on Windows 10 OS</li> <li>DoH enabled Chrome on Windows 10 OS</li> </ol> <p> </p> <p>We captured the traffic from the DoH enabled web-browsers using tcpdump. To automate the process of traffic generation, we installed Google Chrome and Mozilla Firefox into separate virtual machines and controlled them with the Selenium framework shows detailed information about used browsers and environments). Selenium simulates a user's browsing according to the predefined script and a list of domain names (i.e., URLs from Alexa's top websites list in our case). The selenium was configured to visit pages in random order multiple times. For capturing the traffic, we used the default settings of each browser. We did not disable the DNS cache of the browser, and the random order of visiting webpages secures that the dataset contains traces influenced by DNS caching mechanisms. Each virtual machine was configured to export TLS cryptographic keys, that was used for decrypting the traffic using WireShark application. </p> <p>The WireShark text output of the decrypted traffic is provided in the dataset files. The detailed information about each file is provided in dataset README.</p> <p> </p> <p> </p> <p> </p>
The turbulent flow over the BARC rectangular cylinder: a DNS study
<p>This set of results stems from a database computed with a Direct Numerical simulation of the incompressible flow around a rectangular cylinder with chord-to-thickness ratio 5:1 (also known as BARC benchmark). The Reynolds number based on the cylinder thickness and the incoming velocity is set to Re=3000. We provide the two-dimensional mean field as well as the complete set of Reynolds stresses and the terms involved in their single-point budget equations. Further details can be found on an accompanying paper by Chiarini & Quadrio, Flow Turbul. Combust. 107, 875–899 (2021), available at https://doi.org/10.1007/s10494-021-00254-1.</p> <p>Data are provided in VTK file format, so that they can be visualized with several applications, as for example the open-source package ParaView.</p> <p>The file mean.vtk contains the mean flow in terms of the velocity components U and V and pressure field P in the x − y plane.<br> The files uu-Budget.vtk, vv-Budget.vtk, ww-Budget.vtk, uv-Budget.vtk, uw-Budget.vtk, vw-Budget.vtk and k-Budget.vtk contain the complete set of terms appearing in the budget equations for the components of the Reynolds stress tensor and for the turbulent kinetic energy.</p>
A User DNS Fingerprint Dataset
<p><span>Using a user DNS fingerprint allows one to identify a specific network user regardless of the knowledge of his IP address. This method is proper, for example, when examining the behavior of a monitored network user in more depth. In contrast to other studies, this work introduces a dataset for possible user identification based only on the knowledge of its DNS fingerprint created from the previously sent DNS queries.</span></p> <p><span>We created a large dataset from the real network traffic of a metropolitan Internet service provider. The dataset was created from 2.3 billion DNS queries representing 6.2 million different domain names. The data collection took place over three months from 12/2023 to 02/2024.</span></p> <p><span>The dataset contains a detailed user activity description in the sense of overall daily activity statistics and detailed 24-hour activity statistics. Each dataset record contains a list of 1137 classification attributes. The absolutely unique feature of this data set is the classification of user activity based on categories of content accessed by a user.</span></p> <p><span>The new dataset can be used for the creation of machine learning models, allowing the identification of a specific user without direct knowledge of their IP addresses or additional network location information. The dataset can also serve as a reference dataset for the creation of DNS fingerprints of users.</span></p>
A Dataset of Information (DNS, IP, WHOIS/RDAP, TLS, GeoIP) for a Large Corpus of Benign, Phishing, and Malware Domain Names 2024
<p>The dataset contains DNS records, IP-related features, WHOIS/RDAP information, information from TLS handshakes and certificates, and GeoIP information for 368,956 benign domains from Cisco Umbrella, 461,338 benign domains from the actual CESNET network traffic, 164,425 phishing domains from PhishTank and OpenPhish services, and 100,809 malware domains from various sources like ThreatFox, The Firebog, MISP threat intelligence platform, and other sources. The ground truth for the phishing dataset was double-check with the VirusTotal (VT) service. Domain names not considered malicious by VT have been removed from phishing and malware datasets. Similarly, benign domain names that were considered risky by VT have been removed from the benign datasets. The data was collected between March 2023 and July 2024. The final assessment of the data was conducted in August 2024.</p> <p>The dataset is useful for cybersecurity research, e.g. statistical analysis of domain data or feature extraction for training machine learning-based classifiers, e.g. for phishing and malware website detection.</p> <p>The dataset was created using software available in the associated GitHub repository <a href="https://github.com/nesfit/domainradar-dib" target="_blank" rel="noopener">nesfit/domainradar-dib</a>.</p> <h1>Data Files</h1> <ul> <li> <p>The data is located in the following individual files:</p> <ul> <li><strong>benign_umbrella.json</strong> - data for 368,956 benign domains from Cisco Umbrella,</li> <li><strong>benign_cesnet.json </strong>- data for 461,338 benign domains from the CESNET network,</li> <li><strong>phishing.json</strong> - data for 164,425 phishing domains, and</li> <li><strong>malware.json</strong> - data for 100,809 malware domains.</li> </ul> </li> <li>The <strong>schema.json</strong> file contains a JSON Schema with detailed description of the data entries.</li> </ul> <h1>Data Structure</h1> <div> <p>Both files contain a JSON array of records generated using mongoexport (in the MongoDB Extended JSON (v2) format in Relaxed Mode). The following table documents the structure of a record. Please note that:</p> <ul> <li>some fields may be missing (they should be interpreted as nulls), </li> <li>extra fields may be present (they should be ignored).</li> </ul> </div> <table> <tbody> <tr> <td> <div> <div> <p><strong>Field name </strong></p> </div> </div> </td> <td> <div> <div> <p><strong>Field type </strong></p> </div> </div> </td> <td> <div> <div> <p><strong>Nullable </strong></p> </div> </div> </td> <td> <div> <div> <p><strong>Description </strong></p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>domain_name </p> </div> </div> </td> <td> <div> <div> <p>String </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>The evaluated domain name </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>url </p> </div> </div> </td> <td> <div> <div> <p>String </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>The source URL for the domain name </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>evaluated_on </p> </div> </div> </td> <td> <div> <div> <p>Date </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>Date of last collection attempt </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>source </p> </div> </div> </td> <td> <div> <div> <p>String </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>An identifier of the source </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>sourced_on </p> </div> </div> </td> <td> <div> <div> <p>Date </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>Date of ingestion of the domain name </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>dns </p> </div> </div> </td> <td> <div> <div> <p>Object </p> </div> </div> </td> <td> <div> <div> <p>Yes </p> </div> </div> </td> <td> <div> <div> <p>Data from DNS scan </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>rdap </p> </div> </div> </td> <td> <div> <div> <p>Object </p> </div> </div> </td> <td> <div> <div> <p>Yes </p> </div> </div> </td> <td> <div> <div> <p>Data from RDAP or WHOIS </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>tls </p> </div> </div> </td> <td> <div> <div> <p>Object </p> </div> </div> </td> <td> <div> <div> <p>Yes </p> </div> </div> </td> <td> <div> <div> <p>Data from TLS handshake </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>ip_data </p> </div> </div> </td> <td> <div> <div> <p>Array of Objects </p> </div> </div> </td> <td> <div> <div> <p>Yes </p> </div> </div> </td> <td> <div> <div> <p>Array of data objects capturing the IP addresses related to the domain name </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>malware_type</p> </div> </div> </td> <td> <div> <div> <p>String</p> </div> </div> </td> <td> <div> <div> <p>No</p> </div> </div> </td> <td> <div> <div> <p>The malware type/family or “unknown” (only present in malware.json) </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p><strong>DNS data (dns field) </strong></p> </div> </div> </td> <td> </td> <td> </td> <td> </td> </tr> <tr> <td> <div> <div> <p>A </p> </div> </div> </td> <td> <div> <div> <p>Array of Strings </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>Array of IPv4 addresses </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>AAAA </p> </div> </div> </td> <td> <div> <div> <p>Array of Strings </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>Array of IPv6 addresses </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>TXT </p> </div> </div> </td> <td> <div> <div> <p>Array of Strings </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>Array of raw TXT values </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>CNAME </p> </div> </div> </td> <td> <div> <div> <p>Object </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>The CNAME target and related IPs </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>MX </p> </div> </div> </td> <td> <div> <div> <p>Array of Objects </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>Array of objects with the MX target hostname, priority and related IPs </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>NS </p> </div> </div> </td> <td> <div> <div> <p>Array of Objects </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>Array of objects with the NS target hostname and related IPs </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>SOA </p> </div> </div> </td> <td> <div> <div> <p>Object </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>All the SOA fields, present if found at the target domain name </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>zone_SOA </p> </div> </div> </td> <td> <div> <div> <p>Object </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>The SOA fields of the target’s zone (closest point of delegation), present if found and not a record in the target domain directly </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>dnssec </p> </div> </div> </td> <td> <div> <div> <p>Object </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>Flags describing the DNSSEC validation result for each record type </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>ttls </p> </div> </div> </td> <td> <div> <div> <p>Object </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>The TTL values for each record type </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>remarks </p> </div> </div> </td> <td> <div> <div> <p>Object </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>The zone domain name and DNSSEC flags </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p><strong>RDAP data (rdap field) </strong></p> </div> </div> </td> <td> </td> <td> </td> <td> </td> </tr> <tr> <td> <div> <div> <p>copyright_notice </p> </div> </div> </td> <td> <div> <div> <p>String </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>RDAP/WHOIS data usage copyright notice </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>dnssec </p> </div> </div> </td> <td> <div> <div> <p>Bool </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>DNSSEC presence flag </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>entitites </p> </div> </div> </td> <td> <div> <div> <p>Object </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>An object with various arrays representing the found related entity types (e.g. abuse, admin, registrant). The arrays contain objects describing the individual entities. </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>expiration_date </p> </div> </div> </td> <td> <div> <div> <p>Date </p> </div> </div> </td> <td> <div> <div> <p>Yes </p> </div> </div> </td> <td> <div> <div> <p>The current date of expiration </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>handle </p> </div> </div> </td> <td> <div> <div> <p>String </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>RDAP handle </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>last_changed_date </p> </div> </div> </td> <td> <div> <div> <p>Date </p> </div> </div> </td> <td> <div> <div> <p>Yes </p> </div> </div> </td> <td> <div> <div> <p>The date when the domain was last changed </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>name </p> </div> </div> </td> <td> <div> <div> <p>String </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>The target domain name for which the data in this object are stored </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>nameservers </p> </div> </div> </td> <td> <div> <div> <p>Array of Strings </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>Nameserver hostnames provided by RDAP or WHOIS </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>registration_date </p> </div> </div> </td> <td> <div> <div> <p>Date </p> </div> </div> </td> <td> <div> <div> <p>Yes </p> </div> </div> </td> <td> <div> <div> <p>First registration date </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>status </p> </div> </div> </td> <td> <div> <div> <p>Array of Strings </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>The state of the registered object (see RFC 7483, section 10.2.2)</p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>terms_of_service_url </p> </div> </div> </td> <td> <div> <div> <p>String </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>URL of the RDAP usage ToS </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>url </p> </div> </div> </td> <td> <div> <div> <p>String </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>URL of the RDAP entity </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>whois_server </p> </div> </div> </td> <td> <div> <div> <p>String </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>WHOIS server address </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p><strong>TLS data (tls field) </strong></p> </div> </div> </td> <td> </td> <td> </td> <td> </td> </tr> <tr> <td> <div> <div> <p>cipher </p> </div> </div> </td> <td> <div> <div> <p>String </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>TLS cipher suite description according to <a href="https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-4" target="_blank" rel="noopener">IANA</a></p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>protocol </p> </div> </div> </td> <td> <div> <div> <p>String </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>One of “TLS”, ”TLSv1.2”, ”TLSv1.3” </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>certificates </p> </div> </div> </td> <td> <div> <div> <p>Array of Objects </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>Array of objects representing the certificate chain, the first element is the root certificate </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p><strong>IP data (elements in the ip_data array) </strong></p> </div> </div> </td> <td> </td> <td> </td> <td> </td> </tr> <tr> <td> <div> <div> <p>ip </p> </div> </div> </td> <td> <div> <div> <p>String </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>The IP address </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>from_record </p> </div> </div> </td> <td> <div> <div> <p>String </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>The type of the DNS record the address was captured from </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>remarks </p> </div> </div> </td> <td> <div> <div> <p>Object </p> </div> </div> </td> <td> <div> <div> <p>No </p> </div> </div> </td> <td> <div> <div> <p>Ping round-trip time, “is alive” flag and rdap/geo/asn evaluation dates </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>rdap </p> </div> </div> </td> <td> <div> <div> <p>Object </p> </div> </div> </td> <td> <div> <div> <p>Yes </p> </div> </div> </td> <td> <div> <div> <p>RDAP data, similar to DNS RDAP, see the JSON Schema for details </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>geo </p> </div> </div> </td> <td> <div> <div> <p>Object </p> </div> </div> </td> <td> <div> <div> <p>Yes </p> </div> </div> </td> <td> <div> <div> <p>Geolocation data from the GeoLite2 City database (e.g. latitude, longitude, city, country, etc.) </p> </div> </div> </td> </tr> <tr> <td> <div> <div> <p>asn </p> </div> </div> </td> <td> <div> <div> <p>Object </p> </div> </div> </td> <td> <div> <div> <p>Yes </p> </div> </div> </td> <td> <div> <div> <p>Autonomous system data from the GeoLite2 ASN database (ASN, organization, network) </p> </div> </div> </td> </tr> </tbody> </table> <h1>Acknowledgements</h1> <p>We would like to thank the OpenPhish Team for grating permission to use and publish their dataset. We also thank VirusTotal for providing us access to the API for research purposes.</p> <p>This dataset includes GeoLite2 data created by MaxMind, available from <a href="https://www.maxmind.com" target="_blank" rel="noopener">https://www.maxmind.com</a>.</p> <p>The research has been supported by the <strong>Flow-based Encrypted Traffic Analysis</strong> project, no. VJ02010024, granted by the Ministry of the Interior of the Czech Republic and the <strong>Smart Information Technology for a Resilient Society </strong>project, no. FIT-S-23-8209, granted by Brno University of Technology.</p>
List of DNS over HTTPS resolvers on the internet
<p>The DoH Internet Servers dataset comprises a verified list of Internet servers offering DNS over HTTPS (DoH) service. The list was created by active scanning of the IPv4 address space. The scanning was done two times in April 2021 and in January 2022. Together, the list contains 1987 different IP addresses with their reverse DNS record (if available), supported DoH method, and TLS 1.3 support. </p> <p>The scanning was done in three phases:</p> <ol> <li>We scanned the IPv4 address space for opened port 443 using masscan. </li> <li>IP addresses found in the previous step were scanned for DoH support using a custom Nmap-NSE script.</li> <li>The IP addresses found in previous steps were reached by a slower python script that also validated responses, obtained domain names.</li> </ol> <p> </p> <p><strong>Scanning limitation: </strong>The main limitation of our scanning was that it could not find DoH resolvers hosted on infrastructures hosting multiple services behind a single IP address. In such cases, an SNI, or HTTP Host header, or HTTP/2 <em>:authority</em> header is needed for a successful request. Since we did not have the SNI, it was impossible to provide it. </p> <p><strong>Used Scripts:</strong></p> <p>NMap: https://github.com/cejkato2/dns-doh.nse<br> Python: https://github.com/hynekkar/DoH-Checker</p> <p> </p> <p> </p> <p> </p>
Dataset of DNS over HTTPS (DoH) Internet Servers
<p><strong>Description</strong></p> <p>The DoH Internet Servers dataset comprises a verified list of Internet servers offering DNS over HTTPS (DoH). This is an updated <a href="http://doi.org/10.17632/ny4m53g6bw.1">10.17632/ny4m53g6bw.1</a> The list was created through the aggregation of a previously existing, but incomplete, list of DoH servers. The servers in this dataset went through a verification phase where it was confirmed they were active and working as advertised. The verification was done between May 1st, 2022, and May 4th, 2022. The dataset contains a total of 254 unique DoH servers, out of which 136 are over IPv4 and 118 over IPv6. The DoH servers belong to 59 unique Autonomous Systems and are associated with a total of 106 unique domain names.</p> <p>The following public lists of existing DoH servers were used to create this dataset:</p> <ul> <li>https://developers.google.com/speed/public-dns/docs/doh/json</li> <li>https://blog.nightly.mozilla.org/2018/06/01/improving-dns-privacy-in-firefox/</li> <li>https://github.com/curl/curl/wiki/DNS-over-HTTPS</li> <li>https://help.keenetic.com/hc/en-us/articles/360007687159-DNS-over-TLS-and-DNS-over-HTTPS-proxy-servers-for-DNS-requests-encryption</li> <li>https://dnsprivacy.org/wiki/display/DP/DNS+Privacy+Public+Resolvers</li> <li>https://kb.adguard.com/en/general/dns-providers</li> <li>https://applied-privacy.net/services/dns/</li> <li>https://www.pacnog.org/pacnog24/presentations/DoT-DoH-DNS-Privacy.pdf</li> <li>https://www.privacytools.io/providers/dns/</li> </ul> <p> </p> <p>The verification of the DoH servers was performed using a custom-made python script. The script is available at: https://github.com/stratosphereips/DoH-Research/tree/main/validation-script</p>
DNS resolved hostnames with organization and geographical mapping
<p>Hostnames mapped to their CNAMEs, IP addresses, ASes, Organization. IP addresses are obtained after resolving each hostname from 90,000+ open DNS servers across the Internet.</p> <p>The dataset is segregated as online ads and regular web content serving hostnames.</p> <p>The data was collected during January, 2013.</p>
Supplementary files for Collection of Datasets with DNS over HTTPS Traffic
<p>The DNS over HTTPS (DoH) is becoming a default option for domain resolution in modern privacy-aware software. Therefore, research has already focused on various aspects; however, a comprehensive dataset from an actual production network is still missing. In this paper, we present a novel dataset, which comprises multiple PCAP files of DoH traffic. The captured traffic is generated towards various DoH providers to cover differences of various DoH server implementations and configurations. In addition to generated traffic, we also provide real network traffic captured on high-speed backbone lines of a large Internet Service Provider with around half a million users. Network identifiers (excluding network identifiers of DoH resolvers) in the real network traffic (e.g., IP addresses and transmitted content) were anonymized, but still, the important characteristics of the traffic can still be obtained from the data that can be used, e.g., for network traffic classification research. The real network traffic dataset contains DoH and also non-DoH HTTPS traffic as observed at the collection points in the network.</p> <p> </p> <p>This repository provides supplementary files for the "Collection of Datasets with DNS over HTTPS Traffic" :</p> <pre><code>─── supplementary_files | - Directory with supplementary files (scripts, DoH resolver list) used for dataset creation ├── chrome | - Generation scripts for Chrome browser and visited websites during generation ├── doh_resolvers | - The list of DoH resolvers used for filter creation during ISP backbone capture ├── firefox | - Generation scripts for Firefox browser and visited websites during generation └── pcap-anonymizer | - Anonymization script of real backbone captures</code></pre> <p>Collection of datasets:</p> <ul> <li><strong>DoH-Gen-F-AABBC</strong> --- <a href="https://doi.org/10.5281/zenodo.5957277">https://doi.org/10.5281/zenodo.5957277</a></li> <li><strong>DoH-Gen-F-FGHOQS</strong> --- <a href="https://doi.org/10.5281/zenodo.5957121">https://doi.org/10.5281/zenodo.5957121</a></li> <li><strong>DoH-Gen-F-CCDDD</strong> --- <a href="https://doi.org/10.5281/zenodo.5957420">https://doi.org/10.5281/zenodo.5957420</a></li> <li><strong>DoH-Gen-C-AABBCC </strong>--- <a href="https://doi.org/10.5281/zenodo.5957465">https://doi.org/10.5281/zenodo.5957465</a></li> <li><strong>DoH-Gen-C-DDD</strong> --<a href="https://doi.org/10.5281/zenodo.5957676"> https://doi.org/10.5281/zenodo.5957676</a></li> <li><strong>DoH-Gen-C-CFGHOQS</strong> --- <a href="https://doi.org/10.5281/zenodo.5957659">https://doi.org/10.5281/zenodo.5957659</a></li> <li><strong>DoH-Real-world</strong> --- <a href="https://doi.org/10.5281/zenodo.5956043">https://doi.org/10.5281/zenodo.5956043</a></li> </ul> <p> </p> <p> </p>
Seen Time Dataset of Malicious DNS IPs (October 2021)
<p><span>This collection consists of two complementary datasets focused on the seen time records of IP addresses involved in DNS-related attacks targeting port 53, identified in October 2021. The datasets were constructed using open-source intelligence (OSINT) sources and enriched with historical data from AbuseIPDB.</span></p> <p><strong><span>Raw Data</span></strong></p> <p><span>Description: This dataset includes the full historical seen time records for all IP addresses linked to DNS-related attacks on port 53, as identified by OSINT in October 2021. The data, sourced from AbuseIPDB, encompasses 173 IP addresses before preprocessing, providing a comprehensive view of the observed malicious activities.</span></p> <p><strong><span>Preprocessed Data</span></strong></p> <p><span>Description: This dataset provides the preprocessed seen time records for IP addresses involved in DNS-related attacks on port 53, sourced from OSINT and enriched with data from AbuseIPDB. The preprocessed dataset narrows the focus to 111 IP addresses, each with at least 20 seen time records, offering a high-quality subset for detailed cybersecurity analysis and research into DNS attack trends.</span></p>
DNS dataset for modelling homogeneous ignition processes of clustering solid particle clouds in isotropic turbulence
<h2>Abstract</h2> <p>This dataset is being published to enable the development of models for igniting and combusting solid particles in isotropic turbulence using flamelet tabulated chemistry. This dataset is generated using the forced homogeneous isotropic turbulence in order to investigate the effect of active turbulent forces on the ignition of particles, and it is used as the supplementary material for the manuscript "<em>Modeling homogeneous ignition processes of clustering </em><em>solid particle clouds in isotropic turbulence</em>", which was accepted for publication in the special issue of Fuel Journal for the proceeding of the 4th international Oxyflame workshop. The particles are chosen to have near unity Stokes numbers, which promote particle clustering to study the ignition phenomenon for particle clusters, which is common during ignition and combustion of particle clouds in large industrial solid fuel-powered burners. Since the ignition process of clustering solid particle clouds is transient, different time instances during the ignition process are presented to facilitate the modelling effort for the transient ignition behaviour of the particles. The dataset consists of gas-phase data, particle data and the most important routines required to process the dataset. </p> <p>This database is a valuable resource for users developing solid fuel ignition and combustion in turbulent conditions. </p> <p>It should be noted that this dataset is a reduced version of the full dataset in order to size limitations in the sharing platforms. More information and full dataset can be provided upon request. For more information, contact: p.farmand@itv.rwth-aachen.de</p> <h2>Technical details</h2> <p>The data provided in this dataset contains gas phase data, particle data, and some post-processing scripts for visualization of the data. The data is generated in forced homogeneous isotropic turbulence (HIT) with an initial preferential concentration of the particles in a hot atmosphere to study the impact of particle clustering on ignition. Simulations were performed within a region with the physical size of 12.8mm * 12.8mm * 12.8mm with periodic boundary conditions in all directions. The domain size is discretized with a three-dimensional cartesian mesh with a resolution of Δx = 50 μm. A forced isotropic turbulent field with Re_λ=30 and the Kolmogorov length scale η =100 microns has been chosen. The dispersed phase consists of 10,000 particles of Colombian coal with D_p= 20 microns and T0=300K and with an apparent density of 700kg/m3. Non-reactive particles are first randomly distributed in the box filled with air with 20% oxygen and an initial gas temperature of T = 1500K, which is relevant to practical PCC applications. These conditions lead to an initial Stokes number of around 5, for which a clustering behaviour in particle cloud motion is expected. The employed forced isotropic turbulence ensures maintaining the same turbulence statistics during non-reactive and reactive simulations, as summarised in the following table:</p> <table> <tbody> <tr> <td> <p><em>time [ms]</em></p> </td> <td> <p><em>Re_</em><em>λ</em></p> </td> <td> <p><em>Re_</em><em>Turb</em></p> </td> <td> <p><em>η[m]</em></p> </td> <td> <p><em>l_</em><em>t</em><em>[m]</em></p> </td> <td> <p><em>t_</em><em>η</em><em>[ms]</em></p> </td> <td> <p><em>t_</em><em>l</em><em>[ms]</em></p> </td> <td> <p><em>St</em></p> </td> </tr> <tr> <td>0</td> <td>30.7</td> <td>141.8</td> <td>1.04e-4</td> <td>4.27e-3</td> <td>4.47e-2</td> <td>5.33e-1</td> <td>6.199</td> </tr> <tr> <td>0.46</td> <td>30.9</td> <td>143.9</td> <td>9.94e-5</td> <td>4.13e-3</td> <td>4.16e-2</td> <td>4.99e-1</td> <td>6.207</td> </tr> <tr> <td>0.5</td> <td>30.5</td> <td>140.1</td> <td>9.96e-5</td> <td>4.06e-3</td> <td>4.27e-2</td> <td>5.05e-1</td> <td>6.333</td> </tr> <tr> <td>0.55</td> <td>29.4</td> <td>129.8</td> <td>1.01e-4</td> <td>3.88e-3</td> <td>4.34e-2</td> <td>4.94e-1</td> <td>6.032</td> </tr> <tr> <td>0.6</td> <td>28.5</td> <td>122.1</td> <td>1.02e-4</td> <td>3.76e-3</td> <td>4.46e-2</td> <td>4.92e-1</td> <td>5.775</td> </tr> <tr> <td>0.65</td> <td>27.7</td> <td>115.2</td> <td>1.04e-4</td> <td>3.66e-3</td> <td>4.48e-2</td> <td>4.81e-1</td> <td>5.365</td> </tr> </tbody> </table> <p><em>η and t_η are the respective Kolmogorov length and time scales, and l_t and t_l correspond to the integral length and time scales.</em></p> <p> </p> <h3>Gas phase and particle data:</h3> <p>The gas phase data has HDF5 formation, which contains selected scalars relevant to model developments. Since the ignition process is a transient process, two different time instances, t=0.5ms with the maximum number of ignited regions during the ignition process and t=0.65ms at the end of the ignition process, are chosen. Before starting the reactive simulations, a non-reactive simulation for 20.5ms was performed to form the particle clusters, and then the reactive simulation was started. Therefore, the data.out_2.100E-02.h5 corresponds to t = 0.5ms and data.out_2.115E-02.h5 corresponds to t = 0.65ms. Each HDF5 dataset has the following structure:</p> <p>Group Flow:</p> <ul> <li>U, V, W</li> </ul> <p>Group scalar: </p> <ul> <li>CO, CO2, H2O, C2H2, O2, N2, OH, progress variable (PV), temperature(T), enthalpy(h), heat capacity (Cp), density(RHO), pressure(P), mixture fraction(Z), dissipation rate, heat conductivity</li> </ul> <p>Group C_dot:</p> <ul> <li>Molar production rate of CO, CO2, O2, OH</li> </ul> <p>Group ST:</p> <ul> <li>Mass fraction rate of OH</li> </ul> <p>Group var (data required for subfilter analysis and PDF modelling):</p> <ul> <li>RHO * h</li> <li>RHO * h * h</li> <li>RHO *<em> </em>PV</li> <li>RHO *<em> </em>PV <em>* </em>PV</li> <li>RHO *<em> </em>RHO</li> <li>RHO *<em> </em>RHO <em>* </em>RHO</li> <li>RHO * T</li> <li>RHO *<em> </em>T <em>* </em>T</li> <li>RHO * Z</li> <li>RHO *<em> </em>Z <em>* </em>Z</li> </ul> <p>These gas phase data can be used to study the Eulerian field, which is impacted by the particles through the source terms, which are obtained from the Lagrangian framework. For the particle data, two CSV files containing information about each particle's position and temperature are provided. </p> <p>In each CSV file, which corresponds to the same time instance as the gas phase data, this structure can be found:</p> <ul> <li>Points_0, Points_1, Points_2: particle position x,y,z</li> <li>coal_defaultT: particle temperature</li> </ul> <h3>Scripts:</h3> <p>Different scripts are provided for a reader to enable the first-time use of the data, such as visualising the gas phase quantities, calculating the conditional mean and statistical analysis, and clustering analysis for the particles. Here is a brief information regarding the scripts:</p> <ul> <li><strong>hdf5_2D_Con_Mean_Plots.m</strong>: This script can be used for loading the HDF5 file and visualising the field, joint PDF, and joint correlation of different quantities with respect to different parameters. This script requires another module to calculate the conditional mean of the data.</li> <li><strong>Compute_ConditionalMean_histograms_2D.m: </strong>This module calculates the conditional average of a 2D array.</li> <li><strong>clustering_voronoi_3D.m</strong>: This script can analyse the particle data and calculate the clustering limit based on the Voronoi algorithm. It can also filter the clustered particles and separate different clusters using the nearest neighbour and DB-SCAN methods.</li> </ul> <p> </p> <p> </p>
Padding Ain't Enough: Assessing the Privacy Guarantees of Encrypted DNS – Subpage-Agnostic Domain Classification Firefox
<p>This dataset contains one part for the "Subpage-Agnostic Domain Classification" section of our FOCI 2020 paper "Padding Ain’t Enough: Assessing the Privacy Guarantees of Encrypted DNS".</p> <p><a href="https://www.usenix.org/conference/foci20/presentation/bushart">https://www.usenix.org/conference/foci20/presentation/bushart</a></p> <p>You can find the source code for this project on GitHub: <a href="https://github.com/jonasbb/padding-aint-enough">https://github.com/jonasbb/padding-aint-enough</a></p> <p>When using this software or our dataset, please cite our FOCI 20 paper.</p> <pre>@inproceedings {PaddingAintEnough, author = {Jonas Bushart and Christian Rossow}, booktitle = {10th {USENIX} Workshop on Free and Open Communications on the Internet ({FOCI} 20)}, month = aug, publisher = {{USENIX} Association}, title = {Padding Ain{\textquoteright}t Enough: Assessing the Privacy Guarantees of Encrypted {DNS}}, year = {2020}, }</pre>
Padding Ain't Enough: Assessing the Privacy Guarantees of Encrypted DNS – Web Scans
<p>This dataset contains the main data set of our FOCI 2020 paper "Padding Ain’t Enough: Assessing the Privacy Guarantees of Encrypted DNS".</p> <p><a href="https://www.usenix.org/conference/foci20/presentation/bushart">https://www.usenix.org/conference/foci20/presentation/bushart</a></p> <p>You can find the source code for this project on GitHub: <a href="https://github.com/jonasbb/padding-aint-enough">https://github.com/jonasbb/padding-aint-enough</a></p> <p>When using this software or our dataset, please cite our FOCI 20 paper.</p> <pre>@inproceedings {PaddingAintEnough, author = {Jonas Bushart and Christian Rossow}, booktitle = {10th {USENIX} Workshop on Free and Open Communications on the Internet ({FOCI} 20)}, month = aug, publisher = {{USENIX} Association}, title = {Padding Ain{\textquoteright}t Enough: Assessing the Privacy Guarantees of Encrypted {DNS}}, year = {2020}, }</pre>
Padding Ain't Enough: Assessing the Privacy Guarantees of Encrypted DNS – Subpage-Agnostic Domain Classification Tor Browser
<p>This dataset contains the second part of the "Subpage-Agnostic Domain Classification" section of our FOCI 2020 paper "Padding Ain’t Enough: Assessing the Privacy Guarantees of Encrypted DNS".</p> <p><a href="https://www.usenix.org/conference/foci20/presentation/bushart">https://www.usenix.org/conference/foci20/presentation/bushart</a></p> <p>You can find the source code for this project on GitHub: <a href="https://github.com/jonasbb/padding-aint-enough">https://github.com/jonasbb/padding-aint-enough</a></p> <p>When using this software or our dataset, please cite our FOCI 20 paper.</p> <pre>@inproceedings {PaddingAintEnough, author = {Jonas Bushart and Christian Rossow}, booktitle = {10th {USENIX} Workshop on Free and Open Communications on the Internet ({FOCI} 20)}, month = aug, publisher = {{USENIX} Association}, title = {Padding Ain{\textquoteright}t Enough: Assessing the Privacy Guarantees of Encrypted {DNS}}, year = {2020}, }</pre>
Emulator of PR-DNS: Accelerating Dynamical Fields with Neural Operators in Particle-Resolved Direct Numerical Simulation
<p>The codes directory includes the various machine learning models, such as FNO, UNet and ResNet. R128_init1 and R128_init2 are the PR-DNS time step simulations at different initial conditions. R64_init2, R128_init2 and R256_init2 are the PR-DNS time step simulations at different resolutions. </p> <p> </p>
Dataset used for detecting DNS over HTTPS by Machine Learning.
<p><strong> </strong>The dataset consists of three different data sources:</p> <ol> <li> DoH enabled Firefox</li> <li>DoH enabled Google Chrome</li> <li>Cloudflared DoH proxy</li> </ol> <p>The capture of web browser data was made using the Selenium framework, which simulated classical user browsing. The browsers received command for visiting domains taken from Alexa's top 10K most visited websites. The capturing was performed on the host by listening to the network interface of the virtual machine. Overall the dataset contains almost 5,000 web-page visits by Mozilla and 1,000 pages visited by Chrome.</p> <p>The Cloudflared DoH proxy was installed in Raspberry PI, and the IP address of the Raspberry was set as the default DNS resolver in two separate offices in our university. It was continuously capturing the DNS/DoH traffic created up to 20 devices for around three months.</p> <p>The dataset contains 1,128,904 flows from which is around 33,000 labeled as DoH. We provide raw pcap data, CSV with flow data, and CSV file with extracted features.</p> <p>The CSV with extracted features has the following data fields:</p> <p>- Label (1 - Doh, 0 - regular HTTPS)<br> - Data source<br> - Duration<br> - Minimal Inter-Packet Delay<br> - Maximal Inter-Packet Delay<br> - Average Inter-Packet Delay<br> - A variance of Incoming Packet Sizes<br> - A variance of Outgoing Packet Sizes<br> - A ratio of the number of Incoming and outgoing bytes<br> - A ration of the number of Incoming and outgoing packets<br> - Average of Incoming Packet sizes<br> - Average of Outgoing Packet sizes<br> - The median value of Incoming Packet sizes<br> - The median value of outgoing Packet sizes<br> - The ratio of bursts and pauses<br> - Number of bursts<br> - Number of pauses<br> - Autocorrelation<br> - Transmission symmetry in the 1st third of connection<br> - Transmission symmetry in the 2nd third of connection<br> - Transmission symmetry in the last third of connection</p> <p>The observed network traffic does not contain privacy-sensitive information. </p> <p>The zip file structure is:</p> <pre><code>|-- data | |-- extracted-features...extracted features used in ML for DoH recognition | | |-- chrome | | |-- cloudflared | | `-- firefox | |-- flows...............................................exported flow data | | |-- chrome | | |-- cloudflared | | `-- firefox | `-- pcaps....................................................raw PCAP data | |-- chrome | |-- cloudflared | `-- firefox |-- LICENSE `-- README.md</code></pre> <p><br> When using this dataset, please cite the original work as follows:</p> <pre><code>@inproceedings{vekshin2020, author = {Vekshin, Dmitrii and Hynek, Karel and Cejka, Tomas}, title = {DoH Insight: Detecting DNS over HTTPS by Machine Learning}, year = {2020}, isbn = {9781450388337}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3407023.3409192}, doi = {10.1145/3407023.3409192}, booktitle = {Proceedings of the 15th International Conference on Availability, Reliability and Security}, articleno = {87}, numpages = {8}, keywords = {classification, DoH, DNS over HTTPS, machine learning, detection, datasets}, location = {Virtual Event, Ireland}, series = {ARES '20} } </code></pre> <p> </p>
Evaluating the impact of design decisions on passive DNS-based domain rankings: ranking files (type: fqdn, weighting: IDENTITY, days: 1, dates: 15 November 2023-05 December 2023)
See <a href="https://domain-ranking-design-decisions.distrinet-research.be/">https://domain-ranking-design-decisions.distrinet-research.be/</a> for a full project description.
Evaluating the impact of design decisions on passive DNS-based domain rankings: ranking files (type: fqdn, weighting: IDENTITY, days: 7, dates: 15 November 2023-05 December 2023)
See <a href="https://domain-ranking-design-decisions.distrinet-research.be/">https://domain-ranking-design-decisions.distrinet-research.be/</a> for a full project description.
Evaluating the impact of design decisions on passive DNS-based domain rankings: ranking files (type: fqdn, weighting: IDENTITY, days: 7, dates: 16 December 2023-30 December 2023)
See <a href="https://domain-ranking-design-decisions.distrinet-research.be/">https://domain-ranking-design-decisions.distrinet-research.be/</a> for a full project description.
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.