Find research datasets worth reusing

Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.

datasets available to search

ShareScore release 0.9.0

Dataset results

2 results for “Darshan”

Learn how ShareScore rates datasets ↗

zenodo40/100

April 2019 Darshan counters from the Cori supercomputer

Copyright (c) 2022, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved.   GENERAL INFORMATION   1. Title of Dataset: April 2019 Darshan counters from the Cori supercomputer   2. Author Information:       A. Principal Investigator Contact Information         - Name: Suren Byna         - Institution: Lawrence Berkeley National Laboratory         - Address: 1 Cyclotron Rd, Berkeley, CA 94720         - Email: sbyna@lbl.gov       B. Associate or Co-investigator Contact         - Name: Jean Luca Bez         - Institution: Lawrence Berkeley National Laboratory         - Address: 1 Cyclotron Rd, Berkeley, CA 94720         - Email: jlbez@lbl.gov   3. Date of data collection: 2019-04-01   4. Geographic location of data collection: Berkeley, California, USA   5. Information about funding sources that supported the collection of the data:       This research used resources of the National Energy Research Scientific Computing Center (NERSC), a U.S. Department of Energy Office of Science User Facility located at Lawrence Berkeley National Laboratory, operated under Contract No. DE-AC02-05CH11231.   SHARING/ACCESS INFORMATION   1. Licenses/restrictions placed on the data:       - Creative Commons - Attribution-NonCommercial-NoDerivatives 4.0 International (<a href="https://creativecommons.org/licenses/by-nc-nd/4.0">CC BY-NC-ND 4.0</a>)   2. Links to publications that cite or use the data:  N/A   3. Links to other publicly accessible locations of the data: N/A   4. Links/relationships to ancillary data sets: N/A   5. Was data derived from another source? YES       This dataset is derived from raw Darshan logs collected in Cori during April 2020.   6. Recommended citation for this dataset:       Jean Luca Bez, Ahmad Maroof Karimi, Arnab K. Paul, Bing Xie, Suren Byna, Philip Carns, Sarp Oral, Feiyi Wang, and Jesse Hanley. 2022. Access Patterns and Performance Behaviors of Multi-layer Supercomputer I/O Subsystems under Production Load. In Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing (HPDC '22). Association for Computing Machinery, New York, NY, USA.   DATA & FILE OVERVIEW   1. File List:       1. `darshan`: Dataset derived from raw Darshan logs collected in Cori during April 2020. Months are organized as directories with a `Darshan.csv` file in each day containing all the counters collected and relevant to our investigation.       2. `domains`: Dataset maps job identification to their primary associated science domain as reported by NERSC upon project registration. A single CSV comprises all jobs in April 2020 covered or not by Darshan.   2. Additional related data collected that was not included in the current data package:       - The original raw Darshan logs are not included in this package.   4. Are there multiple versions of the dataset?       - No, this is the first version of this dataset.   DESCRIPTIONS FOR VARIABLES USED IN THE FILES   1. Darshan counter dataset   * `uid`: User identifier that ran this job * `jobid`: Job identifier from the SLURM scheduler * `rank`: Rank identifier * `recordid`: File record identifier * `fstype`: File system type * `POSIX_OPENS`: Number of open calls in POSIX * `POSIX_FILENOS`: Number of POSIX fileno operations * `POSIX_DUPS`: Number of POSIX dup operations * `POSIX_READS`: Number of POSIX read operations * `POSIX_WRITES`: Number of POSIX write operations * `POSIX_SEEKS`: Number of POSIX seek operations * `POSIX_STATS`: Number of POSIX stat operations * `POSIX_MMAPS`: Number of POSIX mmap operations * `POSIX_FSYNCS`: Number of POSIX fsync operations * `POSIX_FDSYNCS`: Number of POSIX fdatasync operations * `POSIX_BYTES_READ`: Total number of bytes that were read from the file * `POSIX_BYTES_WRITTEN`: Total number of bytes written to the file * `POSIX_CONSEC_READS`: Number of consecutive reads (that were immediately adjacent to the previous access) * `POSIX_CONSEC_WRITES`: Number of consecutive writes (that were immediately adjacent to the previous access) * `POSIX_SEQ_READS`: Number of sequential reads (at a higher offset than where the previous access left off) * `POSIX_SEQ_WRITES`: Number of sequential writes (at a higher offset than where the previous access left off) * `POSIX_SIZE_READ_0_100`: Number of read access between 0 and 100 bytes at POSIX level * `POSIX_SIZE_READ_100_1K`: Number of read access between 100 bytes and 1KB at POSIX level * `POSIX_SIZE_READ_1K_10K`: Number of read access between 1KB and 10KB at POSIX level * `POSIX_SIZE_READ_10K_100K`: Number of read access between 10KB and 100KB at POSIX level * `POSIX_SIZE_READ_100K_1M`: Number of read access between 100KB and 1MB at POSIX level * `POSIX_SIZE_READ_1M_4M`: Number of read access between 1MB and 4MB at POSIX level * `POSIX_SIZE_READ_4M_10M`: Number of read access between 4MB and 10MB at POSIX level * `POSIX_SIZE_READ_10M_100M`: Number of read access between 10MB and 100MB at POSIX level * `POSIX_SIZE_READ_100M_1G`: Number of read access between 100MB and 1GB at POSIX level * `POSIX_SIZE_READ_1G_PLUS`: Number of read access greater than 1GB at POSIX level * `POSIX_SIZE_WRITE_0_100`: Number of write access between 0 and 100 bytes at POSIX level * `POSIX_SIZE_WRITE_100_1K`: Number of write access between 100 bytes and 1KB at POSIX level * `POSIX_SIZE_WRITE_1K_10K`: Number of write access between 1KB and 10KB at POSIX level * `POSIX_SIZE_WRITE_10K_100K`: Number of write access between 10KB and 100KB at POSIX level * `POSIX_SIZE_WRITE_100K_1M`: Number of write access between 100KB and 1MB at POSIX level * `POSIX_SIZE_WRITE_1M_4M`: Number of write access between 1MB and 4MB at POSIX level * `POSIX_SIZE_WRITE_4M_10M`: Number of write access between 4MB and 10MB at POSIX level * `POSIX_SIZE_WRITE_10M_100M`: Number of write access between 10MB and 100MB at POSIX level * `POSIX_SIZE_WRITE_100M_1G`: Number of write access between 100MB and 1GB at POSIX level * `POSIX_SIZE_WRITE_1G_PLUS`: Number of write access greater than 1GB at POSIX level * `POSIX_F_OPEN_START_TIMESTAMP`: Timestamp that the first POSIX file open operation began * `POSIX_F_READ_START_TIMESTAMP`: Timestamp that the first POSIX file read operation began * `POSIX_F_WRITE_START_TIMESTAMP`: Timestamp that the first POSIX file write operation began * `POSIX_F_CLOSE_START_TIMESTAMP`: Timestamp that the first POSIX file close operation began * `POSIX_F_OPEN_END_TIMESTAMP`: Timestamp that the last POSIX file open operation ended * `POSIX_F_READ_END_TIMESTAMP`: Timestamp that the last POSIX file read operation ended * `POSIX_F_WRITE_END_TIMESTAMP`: Timestamp that the last POSIX file write operation ended * `POSIX_F_CLOSE_END_TIMESTAMP`: Timestamp that the last POSIX file close operation ended * `POSIX_F_READ_TIME`: Cumulative time spent reading at the POSIX level * `POSIX_F_WRITE_TIME`: Cumulative time spent in write, fsync, and fdatasync at the POSIX level * `POSIX_F_META_TIME`: Cumulative time spent in open, close, stat, and seek at the POSIX level * `MPIIO_INDEP_OPENS`: Number of non-collective MPI opens * `MPIIO_COLL_OPENS`: Number of collective MPI opens * `MPIIO_INDEP_READS`: Number of non-collective MPI reads * `MPIIO_INDEP_WRITES`: Number of non-collective MPI writes * `MPIIO_COLL_READS`: Number of collective MPI reads * `MPIIO_COLL_WRITES`: Number of collective MPI writes * `MPIIO_SPLIT_READS`: Number of MPI split collective reads * `MPIIO_SPLIT_WRITES`: Number of MPI split collective writes * `MPIIO_NB_READS`: Number of MPI non-blocking reads * `MPIIO_NB_WRITES`: Number of MPI non-blocking writes * `MPIIO_SYNCS`: Number of MPI file syncs * `MPIIO_MODE`: MPI mode that the file was last opened in * `MPIIO_BYTES_READ`: Total number of bytes that were read from the file at MPI level * `MPIIO_BYTES_WRITTEN`: Total number of bytes written to the file at MPI level * `MPIIO_RW_SWITCHES`: Number of times that access toggled between read and write in consecutive MPI operations * `MPIIO_SIZE_READ_0_100`: Number of read access between 0 and 100 bytes at MPIIO level * `MPIIO_SIZE_READ_100_1K`: Number of read access between 100 bytes and 1KB at MPIIO level * `MPIIO_SIZE_READ_1K_10K`: Number of read access between 1KB and 10KB at MPIIO level * `MPIIO_SIZE_READ_10K_100K`: Number of read access between 10KB and 100KB at MPIIO level * `MPIIO_SIZE_READ_100K_1M`: Number of read access between 100KB and 1MB at MPIIO level * `MPIIO_SIZE_READ_1M_4M`: Number of read access between 1MB and 4MB at MPIIO level * `MPIIO_SIZE_READ_4M_10M`: Number of read access between 4MB and 10MB at MPIIO level * `MPIIO_SIZE_READ_10M_100M`: Number of read access between 10MB and 100MB at MPIIO level * `MPIIO_SIZE_READ_100M_1G`: Number of read access between 100MB and 1GB at MPIIO level * `MPIIO_SIZE_READ_1G_PLUS`: Number of read access greater than 1GB at MPIIO level * `MPIIO_SIZE_WRITE_0_100`: Number of write access between 0 and 100 bytes at MPIIO level * `MPIIO_SIZE_WRITE_100_1K`: Number of write access between 100 bytes and 1KB at MPIIO level * `MPIIO_SIZE_WRITE_1K_10K`: Number of write access between 1KB and 10KB at MPIIO level * `MPIIO_SIZE_WRITE_10K_100K`: Number of write access between 10KB and 100KB at MPIIO level * `MPIIO_SIZE_WRITE_100K_1M`: Number of write access between 100KB and 1MB at MPIIO level * `MPIIO_SIZE_WRITE_1M_4M`: Number of write access between 1MB and 4MB at MPIIO level * `MPIIO_SIZE_WRITE_4M_10M`: Number of write access between 4MB and 10MB at MPIIO level * `MPIIO_SIZE_WRITE_10M_100M`: Number of write access between 10MB and 100MB at MPIIO level * `MPIIO_SIZE_WRITE_100M_1G`: Number of write access between 100MB and 1GB at MPIIO level * `MPIIO_SIZE_WRITE_1G_PLUS`: Number of write access greater than 1GB at MPIIO level * `MPIIO_F_OPEN_START_TIMESTAMP`: Timestamp that the first MPIIO file open operation began * `MPIIO_F_READ_START_TIMESTAMP`: Timestamp that the first MPIIO file read operation began * `MPIIO_F_WRITE_START_TIMESTAMP`: Timestamp that the first MPIIO file write operation began * `MPIIO_F_CLOSE_START_TIMESTAMP`: Timestamp that the first MPIIO file close operation began * `MPIIO_F_OPEN_END_TIMESTAMP`: Timestamp that the last MPIIO file open operation ended * `MPIIO_F_READ_END_TIMESTAMP`: Timestamp that the last MPIIO file read operation ended * `MPIIO_F_WRITE_END_TIMESTAMP`: Timestamp that the last MPIIO file write operation ended * `MPIIO_F_CLOSE_END_TIMESTAMP`: Timestamp that the last MPIIO file close operation ended * `MPIIO_F_READ_TIME`: Cumulative time spent reading at MPI level * `MPIIO_F_WRITE_TIME`: Cumulative time spent write and sync at MPI level * `MPIIO_F_META_TIME`: Cumulative time spent in open and close at MPI level * `MPIIO_F_VARIANCE_RANK_TIME`: The population variance for MPI I/O time of all the ranks * `MPIIO_F_VARIANCE_RANK_BYTES`: The population variance for bytes transferred of all the ranks at MPI level * `STDIO_OPENS`: Number of stdio file open operations * `STDIO_FDOPENS`: Number of stdio fdopen operations * `STDIO_READS`: Number of stdio read operations * `STDIO_WRITES`: Number of stdio write operations * `STDIO_SEEKS`: Number of stdio seek operations * `STDIO_STATS`: Number of stdio stats operations * `STDIO_FLUSHES`: Number of stdio flush operations * `STDIO_BYTES_READ`: Total number of bytes read from the file using stdio operations * `STDIO_BYTES_WRITTEN`: Total number of bytes written to the file using stdio operations * `STDIO_META_TIME`: Cumulative time spent in stdio open/close/seek operations * `STDIO_READ_TIME`: Cumulative time spent in stdio read operations * `STDIO_WRITE_TIM`: Cumulative time spent in stdio write operations   2. Science domain dataset   * `job_id`: Unique job identifier automatically generated by SLURM upon submission * `date`: Date the job started to run as provided by SLURM * `science_domain`: Science domain of the job based on the information provided upon project allocation   RESOURCES   Helpful resources in understanding and utilizing this dataset:   1. Data collection methodology, understanding of Darshan logs and counters       - <a href="https://www.mcs.anl.gov/research/projects/darshan/documentation">Darshan</a>: Darshan HPC I/O Characterization Tool Documentation   2. General information about the NERSC Cori Supercomputer       - <a href="https://docs.nersc.gov/systems/cori">CORI</a>: General information about the Cori supercomputer     - <a href="https://docs.nersc.gov/tools/performance/darshan">Darshan</a>: Darshan I/O profiler documentation in Cori

opencc-by-4.0Apr 2022View details →

zenodo36/100

Fives Input dataset (Cobalt & Darshan traces, combined and preprocessed)

Dataset made of aggregated and curated Cobalt and Darshan logs from the Theta HPC platform at ALCF. Cobalt and Darshan logs were obtained from ALCF Public Data repository (https://reports.alcf.anl.gov/data/index.html) and cover the year 2022. This data was generated from resources of the Argonne Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC02-06CH11357. In order to use the scripts contained within this archive, these datasets must be downloaded and placed in the directory '2022' at the root of the extracted archive. The Darshan logs used in this datasets are originillay available in an aggregated form. The levels of details are usually the following :  <ul> <li>job (reservation made to a resource manager for some platform resources)</li> <li>application run (application running inside the job, on the reserved resources ; there may be multiple ones, sequentially or in parallel, during a job's execution)</li> <li>I/O operation (read or write registered to a file from a process of an application)</li> </ul> Darshan CSV files for Theta contain job and application runs informations, but individual I/O of each application run is aggregated into a single entry. This resource is organised as a single archive containing: <ul> <li>YAML files with our datasets, at various granularity levels (in 'preprocessed_datastets' directory): <ul> <li>48 files containing each 1 month worth of job traces for one of 3 job classes (4 files per month, one per job class and one with all job classes) </li> <li>4 files containing each the entire year worth of job traces ; 1 file per job class, 1 file with all job classes.</li> </ul> </li> <li>A Jupyter Lab notebook, which contains the necessary routines to create aformentionned datasets from raw logs files from ALCF, for the Theta system</li> <li>A requirements.txt file, describing required Python packages and their versions.</li> <li>Various empty directories meant to receive outputs from the Jupyter notebook.</li> </ul>

opencc-by-4.0Apr 2024View details →

ScienceDex guides

Understand access before you commit

These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.

Compare curated datasets

Allen Brain Atlas

Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.

allen-brain-atlas

neuroscienceopenDocumentation, web resources, and API references are available online.

Last verified 2026-04-30Open record

Annotated Behaviour and Observability Dataset (ABODe)

ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.

abode-home-cage

behavioral-neuroscienceopenThe DataShare record exposes download links for annotations, documentation, license text, and the zipped per-snippet data directory.

Last verified 2026-04-30Open record

DANDI Archive for NWB datasets

DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.

dandi-nwb

electrophysiologyopenPublished Dandiset metadata and archive endpoints are available through the production DANDI API.

Last verified 2026-04-30Open record

International Brain Laboratory public data

The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.

ibl

behavioral-neuroscienceopenPublic sessions can be searched and loaded from the IBL public data server through ONE.

Last verified 2026-04-29Open record

OpenNeuro

OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.

openneuro

neuroscienceopenPublished datasets are available on demand over the internet.

Last verified 2026-04-29Open record