Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
13
datasets available to search
ShareScore release 0.9.0
Dataset results
13 results for “I/O”
Server-side I/O request arrival traces
<p>Dataset generated for the "<strong>On server-side file access pattern matching</strong>" paper (Boito et al., HPCS 2019).</p> <p>The traces were obtained following the methodology described in the paper. In addition to the two data sets discussed in the paper, we are also making available an extra data set of server traces.</p> <p><strong>Traces from I/O nodes</strong></p> <ul> <li>IOnode_traces/output/commands has the list of commands used to generate them. Each test is identified by a label, and the test_info.csv file contains the mapping of labels to access patterns. Some files include information about experiments with 8 I/O nodes, but these were removed from the data set because they had some errors.</li> <li>IOnode_traces/output contains .map files that detail the mapping of clients to I/O nodes for each experiment, and .out files, which contain the output of the benchmark.</li> <li>IOnode_traces/ contains one folder per experiment. Inside this folder, there is one folder per I/O node, and inside these folders there are tracefiles for the read and write portions of the experiments. Due to a mistake during the integration between IOFSL and AGIOS, read requests appear as "W", and writes as "R". Once accounted for when processing the traces, that has no impact on results.</li> <li>pattern_length.csv contains the average pattern length for each experiment and operation (average number of requests per second), obtained with the get_pattern_length.py script.</li> </ul> <p>Each line of a trace looks like this:</p> <p><code>277004729325 00000000eaffffffffffff1f729db77200000000000000000000000000000000 W 0 262144</code></p> <p>The first number is an internal timestamp in nanoseconds, the second value is the file handle, and the third is the type of the request (inverted, "W" for reads and "R" for writes). The last two numbers give the request offset and size in bytes, respectively.</p> <p><strong>Traces from parallel file sytem data servers</strong></p> <p>These traces are inside the server_traces/ folder. Each experiment has two concurrent applications, "app1" and "app2", and its traces are inside a folder named accordingly:</p> <p><code>NOOP\_app1\_(identification of app1)\_app2\_(identification of app2)\_(repetition)\_pvfstrace/</code></p> <p>Each application is identified by:</p> <p><code>(contig/noncontig)\_(number and size of requests per process)\_(number of processes)\_(number of client machines)\_(nto1/nton regarding the number of files)</code></p> <p>Inside each folder there are eight trace files, two per data server, one for the read portion and another for the write portion. Each line looks like this:</p> <p><code>[D 02:54:58.386900] REQ SCHED SCHEDULING, handle: 5764607523034231596, queue_element: 0x2a11360, type: 0, offset: 458752, len: 32768</code></p> <p>The part between [] is a timestamp, "handle" gives the file handle, "type" is 0 for reads and 1 for writes, "offset" and "len" (length) are in bytes.</p> <ul> <li>server_traces/pattern_length.csv contains the average pattern length for each experiment and operation, obtained with the server_traces/count_pattern_length.py script.</li> </ul> <p><strong>Extra traces from data servers</strong></p> <p>These traces were not used for the paper because we do not have performance measurements for them with different scheduling policies, so it would not be possible to estimate the results of using the pattern matching approach to select scheduling policies. Still, we share them in the extra_server_traces/ folder in the hope they will be useful. They were obtained in the same experimental campaign than the other data server traces, and have the same format. The difference is that these traces are for single-application scenarios.</p>
Dataset: Data I/O Corporation (DAIO) Stock Performance
This dataset provides historical stock market performance data for specific companies. It enables users to analyze and understand the past trends and fluctuations in stock prices over time. This information can be utilized for various purposes such as investment analysis, financial research, and market trend forecasting.
HADDOCK file I/O benchmark
<p>The benchmark is composed of 1,000,000 structures in PDB format extracted from the results of applying HADDOCK to the cases of the Protein-Protein Docking Benchmark v5 (BM5), which is a well-accepted benchmark in the protein docking community. Due to the nature of the BM5 benchmark, the PDB files are different in length as they represent different molecular structures. Files are placed in a single directory and named according to the BM5 they originated from, docking procedure used, docking stage and model number, example; <em>1BVN_ti5-it1_167.pdb </em>and<em> 1AK4_cm-it0_1601.pdb</em>. The HADDOCK file I/O benchmark is related to our BioExcel project on <em><a href="https://bioexcel.eu/research/projects/high-throughput-modelling-of-interactomes/">high throughput modelling of interactomes</a></em>, this task will generate a gargantuan number of files and will only be possible by taking advantage of exascale computing resources both for processing and I/O handling.</p>
IISWC 2021 Characterizing and Mitigating the I/O Scalability Challenges for Serverless Applications (Dataset and Scripts)
<p>As serverless computing paradigm becomes widespread, it is important to understand the I/O performance characteristics on serverless computing platforms. To the best of our knowledge, we provide the first study that analyzes the observed I/O performance characteristics -- some expected and some unexpected findings that reveal the hidden, complex interactions between the application I/O characteristics, the serverless computing platform, and the storage engines. The goal of this analysis is to provide data-driven guidelines to serverless programmers and system designers about the performance trade-offs and pitfalls of serverless I/O.</p>
Dataset for IOMax: Maximizing Out-of-Core I/O Analysis Performance on HPC Systems
<p>I/O analysis is an essential task for improving the performance of scientific applications on high-performance computing (HPC) systems. However, current analysis tools, which often use data drilling techniques (iterative exploration for deeper insights), treat every query independently and do not optimize column data for data-slicing (extracting specific data subsets), resulting in subpar querying performance. In this paper, we designed IOMax, a tool for efficient data drilling analysis on large-scale I/O traces. IOMax utilizes a novel query optimization technique to improve the query performance by 8.6x while reducing the memory footprint required for analysis by 11x. Additionally, it employs data transformation techniques to improve data-slicing performance by up to 11.4x. In conclusion, IOMax optimizes I/O analysis for scientific workflows on the Lassen supercomputer, resulting in up to 7x improvement.</p> <p>This dataset contains optimized data that was used in the associated publication.</p>
data artifact for "Keeping It Real: Why HPC Data Services Don't Achieve I/O Microbenchmark Performance"
<p>This is the data artifact associated with the PDSW 20 workshop paper submission entitled "Keeping It Real: Why HPC Data Services Don’t Achieve I/O Microbenchmark Performance" in accordance with the PDSW 20 artifact submission guidelines.</p>
Capturing Periodic I/O Using Frequency Techniques [Data Set]
<div>This file contains the data set from the paper: "Capturing Periodic I/O Using Frequency Techniques," which was accepted at the IPDPS 2024. <div>The Instructions are provided in the <a href="https://github.com/tuda-parallel/FTIO/tree/main/artifacts/ipdps24">FTIO GitHub</a>: https://github.com/tuda-parallel/FTIO/tree/main/artifacts/ipdps24</div> </div> <div> </div> <div>After extracting data.zip, the folder named <em>data</em> has the following structure:</div> <div> <pre>data ├── application_traces │ ├── HACC-IO │ ├── IOR │ ├── LAMMPS │ ├── NEK5000 │ └── README.md ├── exps_with_synthetic_traces ├── iosets_ftio_experiments └── README.md</pre> </div> <div><br>The folder iosets_ftio_experiments and exps_with_synthetic_traces in data.zip are snapshots from the repositories: <ul> <li><a href="https://gitlab.inria.fr/hpc_io/iosets-ftio-experiments">https://gitlab.inria.fr/hpc_io/iosets-ftio-experiments</a></li> <li><a href="https://gitlab.inria.fr/hpc_io/ftio_paper_exps_with_synthetic_traces">https://gitlab.inria.fr/hpc_io/ftio_paper_exps_with_synthetic_traces</a></li> </ul> </div>
Yggdrasil: Reducing Network I/O Tax with (CXL-Based) Distributed Shared Memory
<p>used for ICPP24 - Yggdrasil:Reducing Network I/O Tax with (CXL-Based) Distributed Shared Memory Evaluation code and image.</p>
I/O Behind the Scenes [Data Set]
<div>This file contains the data set from the paper: "I/O Behind the Scenes: Bandwidth Requirements of<br>HPC Applications With Asynchronous I/O," which was accepted at the Cluster 2024. <div>The Instructions are provided in the <a href="https://github.com/tuda-parallel/TMIO/tree/main/artifacts/cluster24">TMIO GitHub</a>: <a href="https://github.com/tuda-parallel/TMIO/tree/main/artifacts/cluster24">https://github.com/tuda-parallel/TMIO/tree/main/artifacts/cluster24</a></div> </div> <div> </div> <div>After extracting data.zip, the folder named <em>data</em> has the following structure:</div> <div> <pre>data<br>└─── application_traces<br> ├── HACC-IO<br> │ ├── 1536<br> │ ├── 9216<br> │ └── time_distribution<br> └── WACOM++<br> ├── 9216_nolimit<br> ├── 9614_limit<br> ├── 96_limit<br> ├── 96_nolimit<br> └── time_distribution</pre> <p> </p> </div>
Study of PDS0101 and Pembrolizumab Combination I/O in Subjects With HPV16 + Recurrent and/or Metastatic HNSCC
ClinicalTrials.gov study NCT04260126. IPD Sharing: NO. Countries: 4. Publications: 1.
Arbitration of I/O Forwarding Resources
<p>This repository contains the datasets, source-codes, and parsing scripts from our paper.</p>
Mango-IO: I/O Metrics Consistency Analysis
<p>Performance tools are inseparable from complex HPC applications' performance analysis and engineering life cycles. Due to the application's complexity, various performance analysis tools are created to serve different analysis purposes and provide a deeper look at certain aspects of the applications. Although these tools might operate differently, having coherent information and consistent metrics across all tools is mandatory for ensuring analysis continuity. It is common for performance analysts to switch their usual performance tools due to various reasons and limitations. In this work, we look specifically at the I/O performance analysis tools landscape and introduce Mango-IO to verify the result consistencies between tools and provide tool-agnostic metrics calculation methods. Our analysis and case study provides lesson learned and guideline for ensuring measurement continuity and comparability.</p> <p>This is the collection of traces used in the paper.</p>
Arbitration of I/O Forwarding Resources
<p>This repository contains datasets, source-codes, and parsing scripts from our paper.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.