Find research datasets worth reusing

Data and code for "Representing storylines with causal networks to support decision making: framework and example"

Data and code for the paper "Representing storylines with causal networks to support decision making: framework and example", along with the Shiny webapp code accompanying the paper. The paper has been submitted to the journal of Climate Risk Management and is currently under review.

Passive Operating System Fingerprinting Revisited - Network Flows Dataset

For the evaluation of OS fingerprinting methods, we need a dataset with the following requirements: <ul> <li>First, the dataset needs to be big enough to capture the variability of the data. In this case, we need many connections from different operating systems.</li> <li>Second, the dataset needs to be annotated, which means that the corresponding operating system needs to be known for each network connection captured in the dataset. Therefore, we cannot just capture any network traffic for our dataset; we need to be able to determine the OS reliably.</li> </ul> To overcome these issues, we have decided to create the dataset from the traffic of several web servers at our university. This allows us to address the first issue by collecting traces from thousands of devices ranging from user computers and mobile phones to web crawlers and other servers. The ground truth values are obtained from the HTTP User-Agent, which resolves the second of the presented issues. Even though most traffic is encrypted, the User-Agent can be recovered from the web server logs that record every connection’s details. By correlating the IP address and timestamp of each log record to the captured traffic, we can add the ground truth to the dataset. For this dataset, we have selected a cluster of five web servers that host 475 unique university domains for public websites. The monitoring point recording the traffic was placed at the backbone network connecting the university to the Internet. The dataset used in this paper was collected from approximately 8 hours of university web traffic throughout a single workday. The logs were collected from Microsoft IIS web servers and converted from W3C extended logging format to JSON. The logs are referred to as web logs and are used to annotate the records generated from packet capture obtained by using a network probe tapped into the link to the Internet. The entire dataset creation process consists of seven steps: <ol> <li>The packet capture was processed by the Flowmon flow exporter (<a href="https://www.flowmon.com">https://www.flowmon.com</a>) to obtain primary flow data containing information from TLS and HTTP protocols.</li> <li>Additional statistical features were extracted using GoFlows flow exporter (<a href="https://github.com/CN-TU/go-flows">https://github.com/CN-TU/go-flows</a>).</li> <li>The primary flows were filtered to remove incomplete records and network scans.</li> <li>The flows from both exporters were merged together into records containing fields from both sources.</li> <li>Web logs were filtered to cover the same time frame as the flow records.</li> <li>Web logs were paired with the flow records based on shared properties (IP address, port, time).</li> <li>The last step was to convert the User-Agent values into the operating system using a Python version of the open-source tool ua-parser (<a href="https://github.com/ua-parser/uap-python">https://github.com/ua-parser/uap-python</a>). We replaced the unstructured User-Agent string in the records with the resulting OS.</li> </ol> The collected and enriched flows contain 111 data fields that can be used as features for OS fingerprinting or any other data analyses. The fields grouped by their area are listed below: <ul> <li>basic flow properties - flow_ID;start;end;L3 PROTO;L4 PROTO;BYTES A;PACKETS A;SRC IP;DST IP;TCP flags A;SRC port;DST port;packetTotalCountforward;packetTotalCountbackward;flowDirection;flowEndReason;</li> <li>IP parameters - IP ToS;maximumTTLforward;maximumTTLbackward;IPv4DontFragmentforward;IPv4DontFragmentbackward;</li> <li>TCP parameters - TCP SYN Size;TCP Win Size;TCP SYN TTL;tcpTimestampFirstPacketbackward;tcpOptionWindowScaleforward;tcpOptionWindowScalebackward;tcpOptionSelectiveAckPermittedforward;tcpOptionSelectiveAckPermittedbackward;tcpOptionMaximumSegmentSizeforward;tcpOptionMaximumSegmentSizebackward;tcpOptionNoOperationforward;tcpOptionNoOperationbackward;synAckFlag;tcpTimestampFirstPacketforward;</li> <li>HTTP - HTTP Request Host;URL;</li> <li>User-agent - UA OS family;UA OS major;UA OS minor;UA OS patch;UA OS patch minor;</li> <li>TLS - TLS_CONTENT_TYPE;TLS_HANDSHAKE_TYPE;TLS_SETUP_TIME;TLS_SERVER_VERSION;TLS_SERVER_RANDOM;TLS_SERVER_SESSION_ID;TLS_CIPHER_SUITE;TLS_ALPN;TLS_SNI;TLS_SNI_LENGTH;TLS_CLIENT_VERSION;TLS_CIPHER_SUITES;TLS_CLIENT_RANDOM;TLS_CLIENT_SESSION_ID;TLS_EXTENSION_TYPES;TLS_EXTENSION_LENGTHS;TLS_ELLIPTIC_CURVES;TLS_EC_POINT_FORMATS;TLS_CLIENT_KEY_LENGTH;TLS_ISSUER_CN;TLS_SUBJECT_CN;TLS_SUBJECT_ON;TLS_VALIDITY_NOT_BEFORE;TLS_VALIDITY_NOT_AFTER;TLS_SIGNATURE_ALG;TLS_PUBLIC_KEY_ALG;TLS_PUBLIC_KEY_LENGTH;TLS_JA3_FINGERPRINT;</li> <li>Packet timings - NPM_CLIENT_NETWORK_TIME;NPM_SERVER_NETWORK_TIME;NPM_SERVER_RESPONSE_TIME;NPM_ROUND_TRIP_TIME;NPM_RESPONSE_TIMEOUTS_A;NPM_RESPONSE_TIMEOUTS_B;NPM_TCP_RETRANSMISSION_A;NPM_TCP_RETRANSMISSION_B;NPM_TCP_OUT_OF_ORDER_A;NPM_TCP_OUT_OF_ORDER_B;NPM_JITTER_DEV_A;NPM_JITTER_AVG_A;NPM_JITTER_MIN_A;NPM_JITTER_MAX_A;NPM_DELAY_DEV_A;NPM_DELAY_AVG_A;NPM_DELAY_MIN_A;NPM_DELAY_MAX_A;NPM_DELAY_HISTOGRAM_1_A;NPM_DELAY_HISTOGRAM_2_A;NPM_DELAY_HISTOGRAM_3_A;NPM_DELAY_HISTOGRAM_4_A;NPM_DELAY_HISTOGRAM_5_A;NPM_DELAY_HISTOGRAM_6_A;NPM_DELAY_HISTOGRAM_7_A;NPM_JITTER_DEV_B;NPM_JITTER_AVG_B;NPM_JITTER_MIN_B;NPM_JITTER_MAX_B;NPM_DELAY_DEV_B;NPM_DELAY_AVG_B;NPM_DELAY_MIN_B;NPM_DELAY_MAX_B;NPM_DELAY_HISTOGRAM_1_B;NPM_DELAY_HISTOGRAM_2_B;NPM_DELAY_HISTOGRAM_3_B;NPM_DELAY_HISTOGRAM_4_B;NPM_DELAY_HISTOGRAM_5_B;NPM_DELAY_HISTOGRAM_6_B;NPM_DELAY_HISTOGRAM_7_B;</li> <li>ICMP - ICMP TYPE;</li> </ul> The details of OS distribution grouped by the OS family are summarized in the table below. The Other OS family contains records generated by web crawling bots that do not include OS information in the User-Agent. <table> <thead> <tr> <th scope="col">OS Family</th> <th scope="col">Number of flows</th> </tr> </thead> <tbody> <tr> <td>Other</td> <td>42474</td> </tr> <tr> <td>Windows</td> <td>40349</td> </tr> <tr> <td>Android</td> <td>10290</td> </tr> <tr> <td>iOS</td> <td>8840</td> </tr> <tr> <td>Mac OS X</td> <td>5324</td> </tr> <tr> <td>Linux</td> <td>1589</td> </tr> <tr> <td>Ubuntu</td> <td>653</td> </tr> <tr> <td>Fedora</td> <td>88</td> </tr> <tr> <td>Chrome OS</td> <td>53</td> </tr> <tr> <td>Symbian OS</td> <td>1</td> </tr> <tr> <td>Slackware</td> <td>1</td> </tr> <tr> <td>Linux Mint</td> <td>1</td> </tr> </tbody> </table>

Evaluation of Compatibility among Network Adjustment Software - Results from the JAG3D software package

Together with colleagues from Conservatoire National des Arts et Métiers (CNAM), European Organization for Nuclear Research (CERN), and School of Management and Engineering Vaud (HEIG-VD), we evaluated the compatibility among network adjustment software packages. A detailed description of the procedure and the results was presented at the 5th Joint International Symposium on Deformation Monitoring JISDM 2022. We compared the results of several geodetic networks using software packages developed by the authors' institutions, namely Compensation de Mesures Topographiques (CoMeT), Logiciel Général de Compensation (LGC), Trinet+ as well as JAG3D. Moreover, we included further commercial software packages such as Columbus, Geolab, Move3 and Star*Net. The networks differ mainly in their extent, i.e. the side length and the height. Whereas the smallest network is about 30 m, the largest network under consideration is about 40 km. The height component varies in a range from 30 m to 2.5 km. The raw data and the obtained results can be found on the official CNAM website <a href="https://comet.esgt.cnam.fr/comparisons">comet.esgt.cnam.fr/comparisons</a>. This data set contains the raw data, the JAG3D database as well as the adjustment results.

Dataset for Unrolled-DOT: An Interpretable Deep Network for Diffuse Optical Tomography

Official dataset repository for Unrolled-DOT: An Interpretable Deep Network for Diffuse Optical Tomography. The repository contains both the experimental dataset (allTrainingDat_30-Sep-2021.mat) as well as data that is a dependency for running our code (5_29_21_src-det_10x10_scene_4cm.zip).

opencc-zeroFeb 2023View details →

Data for: Prescribed fire increases plant-pollinator network robustness to losses of rare native forbs

Restoration efforts often focus on changing the composition and structure of invaded plant communities, with two implicit assumptions: 1) functional interactions with species of other trophic levels, such as pollinators, will reassemble automatically when native plant diversity is restored; and 2) restored communities will be more resilient to future stressors. However, the impact of restoration activities on pollinator richness, plant-pollinator interaction network structure, and network robustness is incompletely understood. Leveraging a restoration chronosequence in Pacific Northwest prairies, we examined the effects of restoration-focused prescribed fire and native forb replanting on floral resources, pollinator visitation, and plant-pollinator network structure. We then simulated the effects of plant species loss/removal scenarios on secondary extinction cascades in the networks. Specifically, we explored three management-relevant plant loss scenarios (removal of an abundant exotic forb, removal of an abundant forb designated a noxious weed, and loss of the rarest native forb) and compared them to control scenarios. Pyrodiversity, proportion of area recently burned, and cumulative replanting effort (plugging and seeding) over the prior 10 years increased the abundance and diversity of floral resources, with concomitant increases in pollinator visitation and diversity. Pyrodiversity also decreased network connectance and nestedness, increased modularity, and buffered networks against secondary extinction cascades. Rare forbs contributed disproportionately to network robustness in less restored prairies, while removal of typical "problem" plants like exotic and noxious species had relatively small impacts on network robustness, particularly in prairies with a long history of restoration activities. Restoration actions aimed mainly at improving the diversity and abundance of pollinator-provisioning plants may also produce plant-pollinator networks with increased resilience to plant species losses.

opencc-by-4.0May 2022View details →

Weighted Link Schedules in 100-node Fixed Topology Wireless Networks

This is a data repo for the data samples used for learning the link scheduling in a fixed-topology placed networks. This data set contains samples for 100-node networks, and the scheduling decisions are made from delayed column generation (DCG) algorithm.

opencc-zeroMar 2023View details →

Orchid-mycorrhizal fungi interactions reveal a duality in their network structure in two European regions differing in climate

Network analysis is an effective tool to describe and quantify the ecological interactions between plants and root-associated fungi. Mycoheterotrophic plants, such as orchids, critically rely on mycorrhizal fungi for nutrients to survive, therefore, investigating the structure of those intimate interactions brings new insights into the plant community assembly and coexistence. So far, there is little consensus on the structure of those interactions, described either as nested (generalist interactions), modular (highly specific interactions) or of both topologies. Biotic factors (e.g., mycorrhizal specificity) were shown to influence the network structure, while there is less evidence of abiotic factor effects. By using next-generation sequencing of the orchid mycorrhizal fungal (OMF) community associated with 238 plant individuals belonging to 17 orchid species, we assessed the structure of four orchid-OMF networks in two European regions under contrasting climatic conditions (Mediterranean vs Continental). Each network contained four to 12 co-occurring orchid species, including up to eight species shared among the sites. All four networks were both nested and modular, and fungal communities were different between co-occurring orchid species, despite multiple sharing of fungi across some orchids. Co-occurring orchid species growing in Mediterranean climates were associated with more dissimilar fungal communities, consistent with a greater modular structure compared to the Continental ones. The OMF diversity was comparable among orchid species since most orchids were associated with multiple rarer fungi and with only a few highly dominant ones in the roots. Our results provide useful highlights on potential factors involved in structuring plant-mycorrhizal fungi interactions in different climatic conditions.

opencc-by-4.0Dec 2030View details →

Agta hunter-gatherer oral microbiomes are shaped by contact network structure

Here we investigate the effects of extensive sociality and mobility on the oral microbiome of 138 Agta hunter-gatherers from the Philippines. Our comparisons of microbiome composition showed that the Agta are more similar to Central African Bayaka hunter-gatherers than to neighbouring farmers. We also defined the Agta social microbiome as a set of 137 oral bacteria (only 7% of 1980 amplicon sequence variants) significantly influenced by social contact (quantified through wireless sensors of short-range interactions). We show that large interaction networks including strong links between close kin, spouses, and even unrelated friends, can significantly predict bacterial transmission networks across Agta camps. Finally, we show that more central individuals to social networks are also bacterial supersharers. We conclude that hunter-gatherer social microbiomes are predominantly pathogenic and were shaped by evolutionary tradeoffs between extensive sociality and disease spread.

Moravian Knowledge Network. Aktuelle Forschungsarbeiten und Perspektiven

Im Datenset sind die Präsentationen zu folgenden und nach Programm gereihten Vorträgen im Rahmen des Workshops “<a href="https://dhh.hypotheses.org/1151">Moravian Knowledge Network. Aktuelle Forschungsarbeiten und Perspektiven</a>“, Dresden, 6. und 7. März 2023, zusammengefasst: Alexander Lasch. “Herrnhuterische Quellen als Basis für konstruktionsgrammatische Analysen”. Josefine Salomo. “Die naturkundliche Ausbildung am Pädagogium und der Knabenanstalt in Niesky und deren Gartenanlagen”. Kerstin Roth. “Versicherung in meiner Seele. Über Neuwieder Lebensbeschreibungen”. Lubina Mahling. “Fromme, Prediger und Propheten. Zur religiösen Devianz im Umfeld der Herrnhuter Diaspora in der Lausitz”. Dominik Hetjens & Robert C. Schuppe. “Virtuelle Modelle für die Lehre – ein Reisetagebuch”. Marleen Schindler & Kerstin Roth. “Sprache und Klang – die Herrnhuterische Liedüberlieferung”.

opencc-zeroMar 2023View details →

Data and code for: Generation and applications of simulated datasets to integrate social network and demographic analyses

Social networks are tied to population dynamics; interactions are driven by population density and demographic structure, while social relationships can be key determinants of survival and reproductive success. However, difficulties integrating models used in demography and network analysis have limited research at this interface. We introduce the R package genNetDem for simulating integrated network-demographic datasets. It can be used to create longitudinal social networks and/or capture-recapture datasets with known properties. It incorporates the ability to generate populations and their social networks, generate grouping events using these networks, simulate social network effects on individual survival, and flexibly sample these longitudinal datasets of social associations. By generating co-capture data with known statistical relationships it provides functionality for methodological research. We demonstrate its use with case studies testing how imputation and sampling design influence the success of adding network traits to conventional Cormack-Jolly-Seber (CJS) models. We show that incorporating social network effects in CJS models generates qualitatively accurate results, but with downward-biased parameter estimates when network position influences survival. Biases are greater when fewer interactions are sampled or fewer individuals are observed in each interaction. While our results indicate the potential of incorporating social effects within demographic models, they show that imputing missing network measures alone is insufficient to accurately estimate social effects on survival, pointing to the importance of incorporating network imputation approaches. genNetDem provides a flexible tool to aid these methodological advancements and help researchers test other sampling considerations in social network studies.

Supporting information for "Incoming Neutron Flux Corrections for Cosmic-ray Soil and Snow Sensors Using the Global Neutron Monitor Network"

This dataset includes 2 files that represent supporting information for McJannet, D and Desilets, D (Submitted 2023)"Incoming Neutron Flux Corrections for Cosmic-ray Soil and Snow Sensors Using the Global Neutron Monitor Network" Water Resources Research. File 1 - Supporting Information 1 - Example calculation: Excel sheet showing demonstration calaculations using the neutron intensity correction described in the paper File 2 - Supporting information 2 - List of neutron moniotr stations used in the paper and acknowledgment of their contribtuion

Data for "Structure and drivers of social networks and their links with health in older adults"

Data, supplementary material and scripts for the paper "Structure and drivers of social networks and their links with health in older adults" Social network is an important factor in promoting healthy aging. However, the mechanisms linking social capital to health are complex. Moreover, most of the social network analysis studies on older adults consider only participants’ relationships and not how these relationships are themselves connected. In this study, we went further than current ego-centered network studies by determining global social network metrics and the structure of relationships among older adult participants of the RECORD Cohort using the Veritas-Social questionnaire. The aim of this study is to identify key dimensions of social networks of older adults, and to evaluate how these dimensions relate to depressive symptoms, life satisfaction, and well-being. Using Principal Component Analyses (PCA), we identified four social network dimensions with psychological meanings. Dimension 1 (homophily) was positively linked with perceived accessibility to services in one’s residential neighborhood but negatively linked with the level of study. Dimension 2 (social integration) as Dimension 3 (social support) was only linked to the number of people living with ego. Dimension 4 was linked with perceived accessibility to local services. Finally, and rather surprisingly, we found that none of the four network dimensions, even the degree, was linked to the three health status metrics.

opencc-zeroMar 2023View details →

Linking trait network parameters with plant growth across light gradients and seasons

1. Reduced light availability induced by eutrophication has dramatically affected the growth of submerged macrophytes and caused their rapid decline globally in lakes. Functional traits have usually been used to predict ecological processes and explain plant adaptation. Trait networks, which are constructed from a series of nodes (traits) and edges (trait-trait correlations), can reveal complex relationships among traits. Plant traits belonging to different organs are considered relevant for overall plant performance. Therefore, variation in trait network topology at the whole-plant level can better reflect plant adaptation and response to environments than traditional methods, but the mechanisms underlying the decline of plants from a trait network perspective are not well understood. 2. In this study, based on a one-year manipulation experiment for Potamogeton maackianus cultured with four levels of light intensity, we constructed trait networks from 20 traits belonging to different organs. 3. Our results showed that trait network connectivity decreases in harsh environments, probably due to increased trait modules responding independently to stress. Network connectivity was positively related to the plant relative growth rate (RGR), as high trait connectivity and coordination should be beneficial for plants to acquire and transport resources efficiently across the whole plant. Additionally, we found that specific stem length, leaf:root mass ratios, and leaf total nonstructural carbohydrates were hub traits with high connectivity. These hub traits expressed high phenotypic plasticity, had close links with plant growth, and consistently held their higher importance within the network across light gradients or seasons. 4. We found that low phenotypic integration in stressful environments may constrain plant growth, which can provide important implications for understanding plant adaptation strategies to low-light stress and even predicting community dynamics in the context of global environmental change.

opencc-by-sa-4.0Mar 2023View details →

unarXive: All arXiv Publications Pre-Processed for NLP, Including Structured Full-Text and Citation Network (open subset)

<h2>Description</h2>unarXive is a scholarly data set containing publications' structured full-text, annotated in-text citations, linked non-text content (mathematical notation, figure/table captions) and a citation network.The data is generated from all LaTeX sources on <a href="https://arxiv.org/">arXiv</a> and therefore of higher quality than data generated from PDF files.Typical uses are<ul><li>Training of ML models (citation recommendation, summarization, LLMs)</li><li>Citation context analysis</li><li>Bibliographic analyses</li></ul><h2>Access</h2>┏━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃  <a href="https://github.com/IllDepence/unarXive/raw/master/doc/unarXive_data_sample.tar.gz">D O W N L O A D   S A M P L E</a>   ┃ ┗━━━━━━━━━━━━━━━━━━━━━━━━━━┛Regarding the full data set, please note the following:<blockquote>Note: this Zenodo record is the "open subset" of unarXive, which contains all permissively licensed papers from arXiv.org. You can find the <a href="https://doi.org/10.5281/zenodo.7752754">full version here</a>.</blockquote>The code used for generating the data set is <a href="https://github.com/IllDepence/unarXive">publicly available</a>.

opencc-by-4.0May 2022View details →

Weighted Link Schedules in sub-100 node Random Topology Wireless Networks

This is a data repo for the data samples used for learning the link scheduling in a randomly placed networks. This data set contains samples for sub 100-node networks, and the scheduling decisions are made from delayed column generation (DCG) algorithm.

opencc-by-4.0May 2022View details →

Weighted Link Schedules in sub-100 node Fixed Topology Wireless Networks

This data repo contains the link schedules in a multi-hop wireless networks that aims to maximize the weighted throughput. Each instance is represented by a numpy data file that contains the necessary data fields to reconstruct the original problem instance.   Note: 100_7_new.tar.xz should be in the repo https://zenodo.org/deposit/7671940.

Illustratory figures on brain networks, sex differences, and vulnerability to disorders

These figures were initially created for a PhD thesis but may serve the wider (neuroscientific) community.

Marmoset Functional Maps, "A Vocalization-Processing Network in Marmosets"

Marmoset data and functional maps used in the article "A vocalization-processing network in marmosets". (Jafari et al., 2023)

opencc-by-4.0Apr 2023View details →

Shifting Dynamics in International Trade Networks: A Longitudinal Analysis of Major Exporting Economies (1992-2020): A case of the US, China, India, Japan and South Korea

This article aims to analyze the changes in trading networks by bringing out aspects of centrality and reciprocity among participating countries. The research considers the trade relations of five export countries, namely, The U.S., China, India, Japan, and South Korea from 1992 to 2020. We examine whether trade networks are truly becoming more bidirectional over the long term and how the political and economic crises that have occurred in the past 30 years affect the structure of trade networks. The article presents key findings on the changes in the network structure of international trade over the past 30 years. The network structure has become more reciprocal for most product categories, and the center of the trade network has shifted from the United States to China. The analysis also demonstrated that the center of the network has shifted from U to C for all product categories, except for raw materials. The study also highlights the significant impact of global events and crises on the international trade network structure, emphasizing the importance of considering geopolitical events and economic shocks in understanding trade dynamics.