Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
415
datasets available to search
ShareScore release 0.9.0
Dataset results
415 results for “pipeline”
Introduction to Ancient Metagenomics Textbook (Edition 2025): Ancient Metagenomic Pipelines
<p>Data and conda software environment file for the chapter 'Ancient Metagenomic Pipelines' of the SPAAM Community's textbook: Introduction to Ancient Metagenomics (https://www.spaam-community.org/intro-to-ancient-metagenomics-book).</p>
bollito: a flexible pipeline for comprehensive single-cell RNA-seq analyses - Melanoma tutorial
<p>Downsampled version of the melanoma dataset originally published by <em><a href="https://genome.cshlp.org/content/28/9/1353">Ho et al </a>(1)</em>. The dataset is composed by cells from the 451Lu cell line. There are two samples available:</p> <table> <tbody> <tr> <td><strong>Name</strong></td> <td><strong>Description</strong></td> <td><strong>R1/R2</strong></td> </tr> <tr> <td>451LU</td> <td>Parental cell line</td> <td>2500K_451LU_L003_R*_001.fastq.gz</td> </tr> <tr> <td>451LUBR3</td> <td>Vemurafenib-resistant sample treated with targeted BRAF inhibitors</td> <td>500K_451LUBR3_L004_R*_001.fastq.gz</td> </tr> </tbody> </table> <p><br> (1) Ho YJ, Anaparthy N, Molik D, et al. Single-cell RNA-seq analysis identifies markers of resistance to targeted BRAF inhibitors in melanoma cell populations. <em>Genome Res</em>. 2018;28(9):1353-1363. doi:10.1101/gr.234062.117</p>
Minimal dataset to test multiplexed DNA imaging (Hi-M) software pipelines
<p>This is a dataset of nuclei (DAPI), and 3 multiplexed DNA imaging cycles to test and validate processing software packages, such as pyHiM (https://github.com/marcnol/pyHiM). This dataset was acquired in a nc14 Drosophila embryo.</p> <p>File contents:</p> <p>scan_001_RT27_001_ROI_converted_decon_ch00.tif barcode 27, fiducial <br> scan_001_RT27_001_ROI_converted_decon_ch01.tif barcode 27<br> scan_001_RT29_001_ROI_converted_decon_ch00.tif barcode 29, fiducial <br> scan_001_RT29_001_ROI_converted_decon_ch01.tif barcode 29 <br> scan_001_RT37_001_ROI_converted_decon_ch00.tif barcode 37, fiducial <br> scan_001_RT37_001_ROI_converted_decon_ch01.tif barcode 37 <br> scan_006_DAPI_001_ROI_converted_decon_ch00.tif DAPI <br> scan_006_DAPI_001_ROI_converted_decon_ch01.tif DAPI, fiducial <br> scan_006_DAPI_001_ROI_converted_decon_ch02.tif RNA</p> <p> </p> <p>To test this dataset please refer to <a href="https://github.com/marcnol/pyHiM">pyHiM documentation page</a>.</p>
SubPipe: A Submarine Pipeline Inspection Dataset for Segmentation and Visual-inertial Localization
<h1><strong>Abstract</strong></h1> <p>This paper presents SubPipe, an underwater dataset for SLAM, object detection, and image segmentation. <br><br>SubPipe has been recorded using a lightweight autonomous underwater vehicle (LAUV), operated by OceanScan MST, and carrying a sensor suite including two cameras, a side-scan sonar, and an inertial navigation system, among other sensors. The AUV has been deployed in a pipeline inspection environment with a submarine pipe partially covered by sand. The AUV's pose ground truth is estimated from the navigation sensors. The side-scan sonar and RGB images include object detection and segmentation annotations, respectively. State-of-the-art segmentation, object detection, and SLAM methods are benchmarked on SubPipe to demonstrate the dataset's challenges and opportunities for leveraging computer vision algorithms.<br>To the authors' knowledge, this is the first annotated underwater dataset providing a real pipeline inspection scenario. The dataset and experiments are publicly available <a href="https://github.com/remaro-network/SubPipe-dataset">online.</a></p> <p>On Zenodo we provide <em>three</em> versions for SubPipe. One is the full version (<strong>SubPipe.zip</strong>, ~80GB unzipped) and two subsamples: <strong>SubPipeMini.zip</strong>, ~12GB unzipped and <strong>SubPipeMini2.zip</strong>, ~16GB unzipped. Both subsamples are only parts of the entire dataset (SubPipe.zip). SubPipeMini is a subset, containing semantic segmentation data, and it has interesting camera data of the underwater pipeline. On the other hand, SubPipeMini2 is mainly focused on underwater side-scan sonar images of the seabed including ground truth object detection bounding boxes of the pipeline.</p> <p><strong>For (re-)using/publishing SubPipe, please include the following copyright text:</strong></p> <p><em><strong>SubPipe</strong> is a public dataset of a submarine outfall pipeline, property of Oceanscan-MST. This dataset was acquired with a Light Autonomous Underwater Vehicle by Oceanscan-MST, within the scope of Challenge Camp 1 of the</em> <em>H2020 </em><a href="https://remaro.eu/"><em>REMARO</em></a><em> project.</em></p> <p><em>More information about OceanScan-MST can be found at </em><a href="https://www.oceanscan-mst.com/"><em>this link</em></a><em>.</em></p> <h1><strong>Cam0 — GoPro Hero 10</strong></h1> <h4>Camera parameters:</h4> <ul> <li>Resolution: 1520×2704</li> <li>fx = 1612.36</li> <li>fy = 1622.56</li> <li>cx = 1365.43</li> <li>cy = 741.27</li> <li>k1,k2, p1, p2 = [−0.247, 0.0869, −0.006, 0.001]</li> </ul> <h1><strong>Side-scan Sonars</strong></h1> <p>Each sonar image was created after 20 “ping” (after every 20 new lines) which corresponds to approx. ~1 image / second.</p> <p>Regarding the object detection annotations, we provide both COCO and YOLO formats for each annotation. A single COCO annotation file is provided per each chunk and per each frequency (low frequency vs. high frequency), whereas the YOLO annotations are provided for each SSS image file.</p> <p>Metadata about the side-scan sonar images contained in this dataset:</p> <table> <tbody> <tr> <td><strong>Images for object detection</strong></td> <td> </td> </tr> <tr> <td># Low Frequency (LF):</td> <td> 5000</td> </tr> <tr> <td>LF image size:</td> <td>2500 × 500</td> </tr> <tr> <td># High Frequency (HF):</td> <td> 5030</td> </tr> <tr> <td>HF Image size</td> <td>5000 × 500</td> </tr> <tr> <td><strong>Total number of images:</strong></td> <td>10030</td> </tr> <tr> <td><strong>Annotations</strong><strong><br></strong></td> <td> </td> </tr> <tr> <td># Low Frequency:</td> <td> 3163</td> </tr> <tr> <td># High Frequency:</td> <td> 3172</td> </tr> <tr> <td><strong>Total number of annotations:</strong></td> <td><strong> </strong>6335</td> </tr> </tbody> </table>
MUSE HUDF survey I, Section 4: data and reproduction pipeline for photometry and astrometry
<p>Necessary data and <a href="http://akhlaghi.org/reproducible-science.html">Reproduction pipeline</a> for <a href="https://www.aanda.org/articles/aa/full_html/2017/12/aa30833-17/aa30833-17.html#S14">Section 4</a> of "<em>The MUSE Hubble Ultra Deep Field Survey: I. Survey description, data reduction and source detection</em>", Bacon et al. (2017), <a href="https://www.aanda.org/articles/aa/abs/2017/12/aa30833-17/aa30833-17.html">Astronomy & Astrophysics, 608, A1</a>. The purpose of this section in the paper is to show the photometric and astrometric precision of the processed <a href="http://muse-vlt.eu/science/">MUSE</a> 3D data cubes discussed in the paper (pseudo-broad-band images created from the cubes) in comparison with broad-band images of the Hubble Space Telescope (HST).</p> <p>This repository on Zenodo contains all the necessary input data, software and <a href="http://akhlaghi.org/reproducible-science.html">reproduction pipeline</a> (containing the scripts, configuration files and settings to exactly reproduce the results in Section 4 of the paper). Below is a description of the contents:</p> <ul> <li> <p><a href="https://zenodo.org/record/1163746/files/gnuastro-0.2.51-bc56.tar.gz"><code>gnuastro-0.2.51-bc56.tar.gz</code></a>: The version of <a href="https://www.gnu.org/software/gnuastro">GNU Astronomy Utilities</a> (Gnuastro) that is necessary for this pipeline. Gnuastro is a large collection of programs for astronomical data analysis on the command-line (and in scripts). Note that the reproduction pipeline <em>only</em> works with Gnuastro version 0.2.51, it will complain and abort if another version is installed.</p> <p>IMPORTANT NOTE: Since version 0.2.51 of Gnuastro was released, CFITSIO (one of Gnuastro's dependencies) has added a dependency for the cURL library (to read https URLs). Therefore, to install Gnuastro 0.2.51, please install <a href="https://heasarc.gsfc.nasa.gov/FTP/software/fitsio/c/cfitsio3410.tar.gz">CFITSIO version 3.41</a> or earlier.</p> </li> <li> <p><a href="https://zenodo.org/record/1163746/files/gnuastro-dependencies.tar.gz"><code>gnuastro-dependencies.tar.gz</code></a>: Software libraries necessary to build Gnuastro as it is used here. With these, a working C compiler is enough (currently only tested in a GNU/Linux environment) to exactly reproduce the results (tables).</p> </li> <li> <p><a href="https://zenodo.org/record/1163746/files/hst-acs-images.tar.gz"><code>hst-acs-images.tar.gz</code></a>: Necessary images from HST's <a href="https://archive.stsci.edu/prepds/xdf/">eXtreme Deep Field</a> survey <a href="https://archive.stsci.edu/pub/hlsp/xdf">archives</a>. These images are not necessary to run the reproduction pipeline (they will be downloaded from the HST archives if not present). They are stored here for the self-sufficiency of this repository and faster download: in this lossless compressed format, they are roughly 1/3rd the volume of the same files in HST archives.</p> </li> <li> <p><a href="https://zenodo.org/record/1163746/files/hst-acs-throughputs.tar.gz"><code>hst-acs-throughputs.tar.gz</code></a>: The throughputs of HST Advanced Camera for Surveys (ACS) filters necessary in this study. These are also available from the <a href="http://www.stsci.edu/hst/acs/analysis/throughputs/tables">HST archives</a> and are kept here with similar reasons to above.</p> </li> <li> <p><a href="https://zenodo.org/record/1163746/files/muse-pseudo-broadband-images.tar.gz"><code>muse-pseudo-broadband-images.tar.gz</code></a>: Pseudo-broad-band images generated from the MUSE 3D data cube. These images are only released in this repository. However, to run the reproduction pipeline, it isn't necessary to download them directly from here. The script will download them from Zenodo automatically.</p> </li> <li> <p><a href="https://zenodo.org/record/1163746/files/reproduce-v1-4-gaafdb04.tar.gz"><code>reproduce-v1-4-gaafdb04.tar.gz</code></a>: The <a href="http://akhlaghi.org/reproducible-science.html">reproduction pipeline</a> (version 1-4-gaafdb04) that produces the results (tables) plotted in the paper. The full Git version controlled history of this repository is available on <a href="https://git-cral.univ-lyon1.fr/mohammad.akhlaghi/muse-udf-photometry-astrometry">git-cral.univ-lyon1.fr</a> or <a href="https://gitlab.com/makhlaghi/muse-udf-photometry-astrometry">gitlab.com</a>. We recommend cloning from the Git repository if it is available. This tarball is kept here in case those servers don't work or Git is no longer in common use. Please see the <code>README</code> file in this repository for instructions on how to run the reproduction pipeline and exactly reproduce the results. This pipeline will download all the necessary data if they aren't already present on the system (it is probably just necessary to install the required version of Gnuastro).</p> </li> </ul> <p>The Creative Commons Attribution-NonCommercial 4.0 copyright mentioned in the Zenodo webpage is only applicable to files that don't have an explicit copyright within them. The copyright of other files (mainly scripts and software) is mentioned within them (all are <a href="https://www.gnu.org/licenses/licenses.en.html">free licenses</a>).</p> <p>For any issues with the pipeline/processing, please contact <a href="http://akhlaghi.org">Mohammad Akhlaghi</a>.</p>
DeepCytometer pipeline parameter files, Klf14 mouse white adipose tissue histology and hand-traced training contours
<p>Latest description of this data set: <a href="https://github.com/MRC-Harwell/cytometer/blob/main/DATA.md">Data.md at cytometer project</a></p> <pre># Publications related to the data The data associated to the DeepCytometer project (https://github.com/MRC-Harwell/cytometer) is available from Zenodo (doi: 10.5281/zenodo.5137433 and 10.5281/zenodo.5149005). The histology and mouse measures were generated as part of the Small et al. 2018 study: > Small et al. "Regulatory variants at KLF14 influence type 2 diabetes risk via a female-specific effect on adipocyte size and body composition". Nature Genetics, 50:572–580, 2018. The hand traced data set, colour maps, and automatic segmentations were generated for the Casero et al. 2021 paper: > Casero et al. "Phenotyping of Klf14 mouse white adipose tissue enabled by whole slide segmentation with deep neural networks". bioRxiv, 2021. doi: [10.1101/2021.06.03.444997](https://www.biorxiv.org/content/10.1101/2021.06.03.444997v1.full). # Data protocols ## Histology and laboratory measures To develop and evaluate our methods we used Klf14tm1(KOMP)Vlcg C57BL/6NTac (B6NTac) mice tissue samples and additional data generated as part of the Small et al. 2018 study(Small et al. 2018). It should be noted that the single exon Klf14 gene is imprinted and only expressed from the maternally inherited allele(Parker-Katiraee et al. 2007). This was taken into account by (Small et al. 2018) by crossing a Het parent with a WT parent, so that each offspring inherited a WT allele from the WT parent, and the Klf14 gene knockout or a WT allele from the other parent (from the father, PAT, or the mother, MAT). We also take Klf14 imprinting into account by using as controls the PAT mice and comparing them to the MAT WT and MAT Het (or functional KO, FKO) mice. We used a total of 76 Klf14-B6NTac mice (nfemale=nmale=38), of which 20 mice from the Control and FKO groups were used for training and testing the DeepCytometer pipeline, as well as the hand traced population experiment (summary in Table MICE). The histopathology screen involved fixing, processing and embedding in wax, sectioning and staining with Hematoxylin and Eosin (H&E) both inguinal subcutaneous and gonadal adipose depots. For paraffin-embedded sections, all samples were fixed in 10% neutral buffered formalin (Surgipath) for at least 48 hours at RT and processed using an Excelsior™ AS Tissue Processor (Thermo Scientific). Samples were embedded in molten paraffin wax and 8 μm sections were cut through the respective depots using a Finesse™ ME+ microtome (Thermo Scientific). Sampling was conducted at 2sxns per slide, 3 slides per depot block onto simultaneous charged slides, stained with haematoxylin Gill 3 and eosin (Thermo scientific) and scanned using an NDP NanoZoomer Digital pathology scanner (RS C10730 Series; Hamamatsu). Body weight (BW) and depot weight (DW) were measured with Satorius BAL7000 scales. ## White adipose tissue segmentation For cell area quantification, we applied DeepCytometer v8 to 75 inguinal subcutaneous and 72 gonadal whole histology slides with DeepCytometer (with the Corrected method), including the 20 slides sampled for the hand-traced data set, corresponding to 73 females and 74 males, to produce 2,560,067 subcutaneous and 2,467,686 gonadal cells (on average, 34,134 and 34,273 cells per slide, respectively). Full segmentation of all whole slides was performed with script [klf14_b6ntac_exp_0106_full_slide_pipeline_v8.py](https://github.com/MRC-Harwell/cytometer/blob/39358ed1d79df07d1d522b98728c7efd745513f7/scripts/klf14_b6ntac_exp_0106_full_slide_pipeline_v8.py). In this case, the segmentation contours were grouped by tiles in the output AIDA annotation `.json` file (one contour per cell, one file per slide). Non-white adipocyte contours were filtered out, and white adipocyte contours were aggregated into an AIDA annotation `.json` file with a single tile with script [klf14_b6ntac_exp_0106_annotations_postprocessing_v8.py](https://github.com/MRC-Harwell/cytometer/blob/39358ed1d79df07d1d522b98728c7efd745513f7/scripts/klf14_b6ntac_exp_0106_annotations_postprocessing_v8.py) (one contour per cell, one file per slide). # List of directories and files ## Casero et al. (2021) "DeepCytometer pipeline parameter files, Klf14 mouse white adipose tissue histology and hand-traced training contours" (doi: 10.5281/zenodo.5137433) ### `deepcytometer_pipeline_v8.zip` (60.6 MB) Weights, colourmaps, etc. necessary to run the pipeline (v8, with mode colour correction). This is the version of the pipeline described in the paper. There are 10 weight files per convolutional neural network (CNN), corresponding to 10-fold cross-validation * `klf14_b6ntac_exp_0086_cnn_dmap_model_fold_[0..9].h5`: Keras weights for the **EDT CNN** (Histology to Euclidean Distance Transform regression) * `klf14_b6ntac_exp_0089_cnn_segmentation_correction_overlapping_scaled_contours_model_fold_[0..9].h5`: Keras weights for the **Correction CNN** (Segmentation Correction regression) * `klf14_b6ntac_exp_0091_cnn_contour_after_dmap_model_fold_[0..9].h5`: Keras weights for the **Contour CNN** (EDT to Contour detection) * `klf14_b6ntac_exp_0095_cnn_tissue_classifier_fcn_model_fold_[0..9].h5`: Keras weights for the **Tissue CNN** (Pixel-wise tissue classifier) * `klf14_b6ntac_exp_0094_generate_extra_training_images.pickle`: training dataset description * **'file_list'**: list of SVG files with hand-traced contours for network training. Each SVG file has a corresponding TIFF file with the histology used for segmentation * **'idx_test'**: 10 lists with file indices for testing in 10-fold cross-validation * **'idx_train'**: 10 lists with file indices for training in 10-fold cross-validation * **'fold_seed'**: seed number used for the random number generator to assign file indices to folds * `klf14_b6ntac_exp_0098_filename_area2quantile.npz`: quantile colour maps calculated in `klf14_b6ntac_exp_0098_full_slide_size_analysis_v7.py` using the whole Klf14 data set with v7 of the pipeline, and used in earlier experiments, including some where v8 of the pipeline was used for segmentation. * `klf14_b6ntac_exp_0106_filename_area2quantile_v8.npz`: quantile colour maps calculated in `klf14_b6ntac_exp_0106_full_slide_pipeline_v8.py` using the whole Klf14 data set with v8 of the pipeline, and used in later experiments. * `klf14_training_colour_histogram.npz`: statistics from Klf14 histology images to be used in colour correction * **'xbins_edge'**, **'xbins'**: edges and centres of the bins used for histogram calculations * **'hist_r_q1'**, **'hist_r_q2'**, **'hist_r_q3'** * **'hist_g_q1'**, **'hist_g_q2'**, **'hist_g_q3'** * **'hist_b_q1'**, **'hist_b_q2'**, **'hist_b_q3'**: density quartiles (Q1, Q2, Q3) for RGB channels for each bin the histogram * **'mode_r'**, **'mode_g'**, **'mode_b'**: modes for RGB channels (this corresponds to the most typical background colour in the histology images) * **'mean_l'**, **'mean_a'**, **'mean_b'**: mean intensity for L*a*b channels of the image * **'std_l'**, **'std_a'**, **'std_b'**: intensity standard deviations for L*a*b channels of the image * `klf14_exp_0112_training_colour_histogram.npz`: other statistics from Klf14 histology images to be used in colour correction * **'p'**: vector of quantile values used in ECDF calculations * **'val_r_klf14'**, **'val_g_klf14'**, **'val_b_klf14'**: all intensity values for the RGB channels of Klf14 training images that contain at least a white adipocyte * **'f_ecdf_to_val_r_klf14'**, **'f_ecdf_to_val_g_klf14'**, **'f_ecdf_to_val_b_klf14'**: linear interpolation function that maps ECDF quantiles to intensity values in the Klf14 training data set. These functions can be used together with intensity->quantile interpolation functions calculated for a new histology image to perform histogram matching colour correction * **'mean_klf14'**, **'std_klf14'**: mean and standard deviation of the **'val_r_klf14'**, **'val_g_klf14'**, **'val_b_klf14'** vectors There are also weight files for the pipeline trained with all the data, instead of the 10-fold cross-validation partition. These were not used for the paper, but could be useful for future experiments * `klf14_b6ntac_exp_0101_cnn_dmap_model.h5`: Keras weights for the **EDT CNN** (Histology to Euclidean Distance Transform regression) * `klf14_b6ntac_exp_0104_cnn_segmentation_correction_overlapping_scaled_contours_model.h5`: Keras weights for the **Correction CNN** (Segmentation Correction regression) * `klf14_b6ntac_exp_0102_cnn_contour_after_dmap_model.h5`: Keras weights for the **Contour CNN** (EDT to Contour detection) * `klf14_b6ntac_exp_0103_cnn_tissue_classifier_fcn_model.h5`: Keras weights for the **Tissue CNN** (Pixel-wise tissue classifier) ### `histology.7z` (29.1 GB) 165 H&E histology whole slides from Hamamatsu scanner (`.ndpi`). ### `klf14.7z` (2.3 GB) Mice metadata, training/testing data sets for the pipeline, intermediate files created during training, and neural network weights for multiple experiments. * `klf14_b6ntac_meta_info.csv`: Klf14 mice metadata * **Animal Identifier**, **id:** unique ID for each mouse * **ko_parent:** heterozygous parent of origin for the KO allele (father, PAT or mother, MAT) * **sex:** female or male * **genotype:** wild type (KLF14-KO:WT) or heterozygous (KLF14-KO:Het) * **BW:** body weight (g) * **SC:** subcutaneous depot weight (g) * **gWAT:** gonadal depot weight (g) * **Liver:** livel weight (g) * **cull_age:** age at time of culling (days) * **BW_alive:** body weight measured before culling * **BW_alive_date:** age at time of BW_alive measure * **mother:** unique ID for mouse's mother * **mother_genotype:** mouse's mother genotype * `klf14_b6ntac_training`: Directory with hand-traced segmentations of training histology windows. 131 windows sampled from 20 whole slides, plus hand-traced contours that were used for training DeepCytometer and compute population distributions. These segmentations were used for CNN training, but note that there's a cleaned-up version of these data below, and it was the cleaned-up version that was used for the paper experiments * `ndpifile_row_YYYYYY_col_XXXXXX[.tif/.xcf/.svg]`: * **ndpifile:** name of the whole slide file (e.g. `KLF14-B6NTAC 36.1c PAT 98-16 C1 - 2016-02-11 10.45.00`) * **row_YYYYYY:** Y-coordinate of the top-left corner of the sampling window, in pixels * **col_XXXXXX:** X-coordinate of the top-left corner of the sampling window, in pixels * **.tif:** TIFF file with the histology sampling window * **.xcf:** Gimp file with the histology and hand-traced contours (the contours were drawn in Gimp) * **.svg:** SVG (Scalable Vector Graphics) that contains the hand-traced contours in the XCF file * `klf14_b6ntac_training_v2`: Same as `klf14_b6ntac_training`, but the hand-traced data set was cleaned up to remove small contours of dubious cells, or cells that are fully overlapped by others * `klf14_b6ntac_training_non_overlap`: Directory with intermediate images to train the networks. These images are generated by script [`klf14_b6ntac_training_non_overlap`](https://github.com/MRC-Harwell/cytometer/blob/main/scripts/klf14_b6ntac_exp_0077_generate_non_overlap_training_images.py) * `klf14_b6ntac_training_augmented`: Directory with intermediate images used to train the networks (using augmentation to reduce overfitting). These images are generated by script [`klf14_b6ntac_exp_0078_generate_augmented_training_images.py`](https://github.com/MRC-Harwell/cytometer/blob/main/scripts/klf14_b6ntac_exp_0078_generate_augmented_training_images.py) * `klf14_b6ntac_seg`: Deprecated. Directory to store whole slide coarse segmentations in old experiments (e.g. `klf14_b6ntac_exp_0076_generate_training_images.py`). Of little interest for most users * `klf14_b6ntac_results`: Deprecated. Directory to store miscellanea output from some experiments. Of little interest for most users ## Casero et al. (2021). "Klf14 mouse white adipose tissue histology DeepZoom files and AIDA annotations for visualisation of DeepCytometer white adipocyte segmentations" (doi: 10.5281/zenodo.5149005) ### `aida_data_Klf14_v8_images.7z` (16.9 GB) Histology images converted to DeepZoom so that they can be visualised with [AIDA](https://github.com/alanaberdeen/AIDA). To use this, decompress this file and put the resulting `images` directory in your `AIDA/dist/data/` directory. ### `aida_data_Klf14_v8_annotations.7z` (18 GB) White adipocyte segmentations in AIDA annotation `.json` files (one contour per cell, one file per whole slide). Each slide has the following files: * `SLIDENAME.json`: Soft link to the annotations file that we want to associate to slide `SLIDENAME.ndpi`, e.g. `SLIDENAME` = `KLF14-B6NTAC-PAT-39.2d 454-16 B1 - 2016-03-17 12.16.06` * `SLIDENAME.lock`: Empty file used to tell the pipeline that `SLIDENAME.ndpi` has already been processed or is being currently processed * `SLIDENAME_coarse_mask.npz`: File with the coarse tissue segmentation of `SLIDENAME.ndpi` and the internal state of the pipeline (execution times, steps, etc) * `SLIDENAME_exp_0106_auto.json`: Annotations (all segmentations without filtering from the Auto algorithm, i.e. segmentation without object overlap). Contours are grouped by the tile they were processed in * `SLIDENAME_exp_0106_auto_aggregated.json`: Filtered annotations (non-white adipocytes removed) of the Auto algorithm. All contours aggregated into a single tile * `SLIDENAME_exp_0106_corrected.json`: Annotations (all segmentations without filtering from the Corrected algorithm, i.e. segmentation with object overlap). Contours are grouped by the tile they were processed in * `SLIDENAME_exp_0106_corrected_aggregated.json`: Filtered annotations (non-white adipocytes removed) of the Corrected algorithm. All contours aggregated into a single tile To use this, decompress this file and put the resulting `annotations` directory in your `AIDA/dist/data/` directory. </pre>
IMC Segmentation Pipeline results of example IMC data
<p>If you are working with these files, please cite them as follows:<br><br>Windhager, J., Zanotelli, V.R.T., Schulz, D. et al. An end-to-end workflow for multiplexed image processing and analysis. Nat Protoc (2023). <a href="https://doi.org/10.1038/s41596-023-00881-0">https://doi.org/10.1038/s41596-023-00881-0</a></p><p>This repository hosts the results of processing example imaging mass cytometry (IMC) data hosted at <a href="http://10.5281/zenodo.5949116">10.5281/zenodo.5949116</a> using the IMC Segmentation Pipeline available at <a href="https://github.com/BodenmillerGroup/ImcSegmentationPipeline">https://github.com/BodenmillerGroup/ImcSegmentationPipeline</a> (DOI: <a href="http://10.5281/zenodo.6402666">10.5281/zenodo.6402666</a>) v3.6. Please refer to <a href="https://github.com/BodenmillerGroup/steinbock">https://github.com/BodenmillerGroup/steinbock</a> as alternative processing framework and <a href="http://10.5281/zenodo.6043600">10.5281/zenodo.6043600</a> for the data generated by <i>steinbock</i>.</p><p>The following files are part of the <strong>analysis.zip</strong> folder when running the IMC Segmentation Pipeline:</p><ul><li><strong>cpinp</strong>: contains input files for the segmentation pipeline</li><li><strong>cpout</strong>: contains all final output files of the pipeline: <i>cell.csv</i> containing the single-cell features; <i>Experiment.csv</i> containing CellProfiler metadata; <i>Image.csv</i> containing acquisition metadata; <i>Object relationships.csv</i> containing an edge list indicating interacting cells; <i>panel.csv</i> containing channel information; <i>var_cell.csv</i> containing cell feature information; <i>var_Image.csv</i> containing acquisition feature information; <i>images </i>containing the hot pixel filtered multi-channel images and the channel order; <i>masks</i> containing the segmentation masks; <i>probabilities </i>containing the pixel probabilities.</li><li><strong>histocat</strong>: contains single channel .tiff files per acquisition for upload to histoCAT (<a href="https://bodenmillergroup.github.io/histoCAT/">https://bodenmillergroup.github.io/histoCAT/</a>)</li><li><strong>crops</strong>: contains upscaled image crops in .h5 format for ilastik (<a href="https://www.ilastik.org/">https://www.ilastik.org/</a>) training</li><li><strong>ometiff</strong>: contains .ome.tiff files per acquisition, .png files per panorama and additional metadata files per slide</li><li><strong>ilastik</strong>: multi channel images for ilastik pixel classification (<i>_ilastik.full</i>) and their channel order (<i>_ilastik.csv</i>); upscaled multi channel images for ilastik pixel prediction (<i>_ilastik_s2.h5</i>); upscaled 3 channel images containing ilastik pixel probabilities (<i>_ilastik_s2_Probabilities.tiff</i>).</li></ul><p>The remaining files are part of the root directory:</p><ul><li><strong>docs.zip: </strong>Documentation of the pipeline in markdown format</li><li>I<strong>MCWorkflow.ilp: </strong>Ilastik pixel classifier pre-trained on the example data</li><li><strong>resources.zip: </strong>The CellProfiler pipelines and CellProfiler plugins used for the analysis</li><li><strong>scripts.zip: </strong>Python notebooks used for pre-processing and downloading the example data</li><li><strong>src.zip: </strong>Scripts for the imcsegpipe python package</li></ul>
HiRISE DTMs generated using NASA's Ames Stereo Pipeline
<p>In an effort to better understand the surface roughness of Martian lava flows, we generated over 30 HiRISE DTMs using ISIS3 and ASP and extracted their roughness (Rodriguez Sanchez-Vahamonde and Neish, 2020). We have posted these DTMs for public use here. </p> <p>HiRISE stereo images typically have a spatial sampling of 25 - 50 centimeters, providing us with DTMs of 1 - 2 meters per pixel. We also converted the HiRISE stereo-pair ID for each product into its proper DTM ID using the NASA Planetary Data System product naming convention for HiRISE DTMs (<a href="https://www.uahirise.org/dtm/about.php">https://www.uahirise.org/dtm/about.php</a>; last accessed 18.09.2019).</p>
Towards an open pipeline for the detection of Critical Infrastructure from satellite imagery – A case study on electrical substations in The Netherlands
<p><strong>Abstract.</strong> Critical infrastructure (CI) are at risk of failure due to the increased frequency and magnitude of climate extremes related to climate change. It is thus essential to include them in a risk management framework to identify risk hotspots, develop risk management policies and support adaptation strategies to enhance their resilience. However, the lack of information on the exposure of CI prevents their incorporation in large-scale risk assessment studies. This study sets out to improve the representation of CI for risk assessment studies by building a neural network model to detect CI assets from optical remote sensing imagery. We present a pipeline that extracts CI from OpenStreetMaps, processes the imagery and assets' masks, and trains a Mask R-CNN model that allows for instance segmentation of CI at the asset level. This study provides an overview of the pipeline and tests it with the detection of electrical substations assets in the Netherlands. Several experiments are presented for different under-sampling percentages of the majority class (25%, 50% and 100%) and hyperparameters settings (batch size and learning rate). The best metrics achieved are an Average Precision at an Intersection over Union of 50% of 30.93 and a tile F-score of 89.88%. This allows us to confirm the feasibility of the method and invite disaster risk researchers to use this pipeline for other infrastructure types. We conclude by exploring the different avenues to improve the pipeline by addressing the class imbalance, Transfer Learning and Explainable AI.</p>
Simulated BCRseq data for the nf-core/airrflow pipeline benchmark
<p>Simulated BCRseq data for the nf-core/airrflow pipeline benchmark.</p> <p>The original repertoires simulated with ImmuneSim (sim_repertoire_orig_repA/repB/repC.tsv) and the clonally expanded repertoires (sim_repertoire_clonally_expandedrepA/repB/repC.tsv), as well as the fasta file formats from the clonally expanded repertoires with and without UMIs are also shared. The fasta files were used to simulate the sequencing reads with various degrees of sequencing errors.</p>
Using the traditional microscope for mineral grain orientation determination: A prototype image analysis pipeline for optic-axis mapping (POAM). Original dataset.
<p>The data repository contains data obtained with the microscope Nikon Eclipse LV100ND that was stitched with <a href="https://imagej.net/plugins/trakem2/">TrakEM2 software</a>. The files allow reproducing the results obtained and plot in <a href="https://doi.org/10.1111/jmi.13284">Acevedo et al. (2024)</a> <strong>"Using the traditional microscope for mineral grain orientation determination: A prototype image analysis pipeline for optic-axis mapping (POAM)."</strong> by Acevedo Zamora, M. A., Schrank, C. E., & Kamber, B. S.</p> <p>The prototype uses MatLab scripts (<a href="https://github.com/marcoaaz/AcevedoEtAl._2024a_POAM">AcevedoEtAl._2024a_POAM</a>) that were documented in the paper Supplementary Material 1. The metadata can be found in Supplementary Material 3 and follows the structure of this data repository. The user needs downloading and changing the paths to run the same scripts and reproduce the results.</p> <p>Note: After download, unzip and merge (copy-paste) the folders (parts 1, 2 and 3). Before merging, the containing folder should be re-named to 'paper 2_datasets' to match exactly the MatLab scripts and reproduce our work.</p> <p>The remaining questions should be addressed to Marco Acevedo (maaz.geologia@gmail.com ; marco.acevedozamora@qut.edu.au)</p> <p>Thanks.</p>
Supplemental Data from the article "The SmARTR pipeline: a modular workflow for the cinematic rendering of 3D scientific imaging data"
<h1><strong>Please, refer to <a href="https://github.com/MeVisLab/SmARTR-Networks">this GitHub repository</a> for additional info, updates, issue reports, and discussion<br></strong></h1> <p><strong>A collection of configuration files (SmARTR networks) published in "<a href="https://doi.org/10.1016/j.isci.2024.111475">The SmARTR Pipeline: a modular workflow for the cinematic rendering of 3D scientific imaging data</a>", enabling the creation of cinematic (photorealistic) renderings of 3D data in the FREE software <a href="https://www.mevislab.de/download">MeVisLab</a><br></strong></p> <ul> <li>Each folder in the archive contains one or more SmARTR network files, the scan and mask files required for the practical examples detailed in the <a href="https://www.cell.com/cms/10.1016/j.isci.2024.111475/attachment/8d79036b-acb6-4cda-a5ff-f56317691ebc/mmc1.pdf">Supplemental Data</a> of the article, and an additional folder with LUT presets.</li> </ul>
Datasets and Pipeline V1.0 from An Atlas of Plant Transposable Elements
<p>In this repository, we deposited support data for the article "An Atlas of Plant Transposable Elements", available at <a href="http://apte.cp.utfpr.edu.br/">http://apte.cp.utfpr.edu.br/</a>.</p> <p>Here, we included:</p> <p><strong>1.) Supplementary material data:</strong><br> A) SuppMat_1.xlsx: The genome assembly reference access from Ensembl Plants species used.<br> B) SuppMat_2.docx: A brief transposable elements annotation steps are used in this work.</p> <p><strong>2.) Code and software: </strong>all script code create, third-party software, how we are used it, are detailed using Arabidopsis thaliana genome as an example in the GitHub: <a href="https://github.com/alerpaschoal/apte_pipeline">https://github.com/alerpaschoal/apte_pipeline</a> under the MIT license (please see details in licence.txt file). For the third part-software, consult their terms.</p> <p>To report bugs, to ask for help, and to give any feedback, please contact Alexandre R. Paschoal (paschoal@utfpr.edu.br) or Douglas S. Domingues (douglas.domingues@unesp.br).</p>
PRJNA860062 Assigned Taxonomy and QIIME2 Pipeline
<p><strong>PRJNA860062 Assigned Taxonomy:</strong></p> <p>This upload comprises two datasets with the assigned taxonomy for sequence variants of BioProject PRNJA860062.</p> <ul> <li>PRJNA860062_ASVCounts_NCBItaxonomy.txt</li> <li>PRJNA860062_ASVCounts_SILVAtaxonomy.txt</li> </ul> <p>BioProject PRNJN860062 compares bacterial profiles of zebrafish larvae microbiota resulting from two different microbial colonization methods. The full description and sequence data for this project can be obtained from the Sequence Read Archive (<a href="https://www.ncbi.nlm.nih.gov/bioproject">https://www.ncbi.nlm.nih.gov/bioproject</a>). </p> <p>The dataset with the SILVA taxonomy can directly be obtained using the QIIME2 script included in this upload ('PRJNA860062_QIIME2Script.txt'). As previously noted by Lesack and Birol (2018), SILVA species annotations include nomenclature errors (<a href="https://doi.org/10.1101/441576">DOI: 10.1101/441576</a>). Therefore, the dataset with the NCBI taxonomy comprises a manually corrected taxonomy for BioProject PRNJA860062, based on the family to phylum level nomenclature of the NCBI taxonomy browser (<a href="https://www.ncbi.nlm.nih.gov/taxonomy">https://www.ncbi.nlm.nih.gov/taxonomy</a>).</p> <p>Both files are tab-delimited text files, include the domain to species level taxonomy in the first 7 columns, and include the number of assigned sequence variants (ASVs) per taxon in the final 6 colums, corresponding to BioSample SAMN29820940, SAMN29820941, SAMN29820942, SAMN29820943, SAMN29820944, and SAMN29820945.</p> <p> </p> <p><strong>QIIME 2 Pipeline:</strong></p> <p>The QIIME2 script that was used to obtain the assigned SILVA taxonomy BioProject PRNJA860062 is uploaded as:</p> <ul> <li>PRJNA860062_QIIME2Script.txt</li> </ul> <p>Input files that are required to run this script, including a manifest text file, sample metadata, and the reference sequences and taxonomy from the SILVA 138 small subunit (16S/18S) rRNA database Ref NR 99, are uploaded in the zipped file:</p> <ul> <li>PRJNA860062_InputFiles.zip</li> </ul> <p>FASTQ sequence data for BioSample SAMN29820940, SAMN29820941, SAMN29820942, SAMN29820943, SAMN29820944, and SAMN29820945, can be obtained from the Sequence Read Archive under BioProject PRNJA860062 (<a href="https://www.ncbi.nlm.nih.gov/bioproject">https://www.ncbi.nlm.nih.gov/bioproject</a>).</p> <p>All output files are uploaded in the zipped file:</p> <ul> <li>PRJNA860062_OutputFiles.zip</li> </ul> <p>Data provenance, including the versions of python (3.6.7) and python packages, can be acquired by dragging QIIME2 Visualizations (.qzv output files) into the QIIME2 viewing interface (<a href="http://view.qiime2.org">http://view.qiime2.org</a>).</p>
nNPipe: A neural network pipeline for automated analysis of morphologically diverse catalyst systems - Resources
<p>This dataset comprises of resources required to replicate the results described in "<em>nNPipe</em>: A neural network pipeline for automated analysis of morphologically diverse catalyst systems". <em>nNPipe </em>is a deep learning based method in which two deep convolutional neural networks are used for the automated analysis of 2048x2048 HRTEM images.</p> <p>The file contains:<br> - Relevant experimental images as well as ground truth for Pd/C and Au/Ge systems.<br> - A workflow file explaining the nNPipe workflow.<br> - Mathematica 12.1 code for the generation of computational models.<br> - MATLAB code for HRTEM multislice simulations using MULTEM, as well as code required to form respective training datasets.<br> - Weights and files required for training the YOLOv5x module.<br> - Weights and files required for training the SegNet module.<br> - Mathematica 12.1 code required for reconstruction of 2048x2048 binary segmented maps of HRTEM images. </p>
Scans dataset for evaluation of AR-based assembly pipeline of half-timber dry-stone structures
<p><strong>Dataset description</strong></p> <p><em>The dataset is realized by the EPFL Laboratories of IBOIS, and EESD on the occasion of joint collaboration with the common goal of exploring augmented reality (AR)-based fabrication for half-timber dry-stone construction. It is open sourced and free to be used for anyone's own research.</em></p> <p>The current dataset has been employed to evaluate the designed digital fabrication pipeline. It consists of raw point clouds and reconstructed models of two digitized prototypes of one-layered dry-stone structures with a 1.7 m length, ~0.6 height, and 0.7 m width, each with a different composition of building units. The first wall presents 40 mineral by-products uniquely from by-products of quarry processes involving sawing and water-cutting. In contrast, the second wall features 30 mineral scraps issued from all mixed quarry's operations of transformations.</p> <p>The collection contains both the reconstructed models of the as-built artifacts and the corresponding recorded model of the AR-guided assembly processes for the two walls. The recorded model contains meshes of the placed stones accordingly to the developed geometric planner. By comparing the misalignment between each stone of both models (e.g. Root Mean Square Error distances between two set of points) is possible to obtain metrics about the performance of the proposed AR-assembly method.</p> <p> </p> <p><strong>Dataset labeling guide</strong></p> <table summary="test2"> <thead> <tr> <th scope="row">label</th> <th scope="col">content</th> </tr> </thead> <tbody> <tr> <th scope="row">01/02</th> <td>First or second wall.</td> </tr> <tr> <th scope="row">rs</th> <td>Raw data: unprocessed scans.</td> </tr> <tr> <th scope="row">ab</th> <td>As-built model: model presenting all the registered stones.</td> </tr> <tr> <th scope="row">rc</th> <td>Recorded model: collection of tracked stones during the construction. Each stone has being placed following the AR system is recorded in this sub-set.</td> </tr> <tr> <th scope="row">ev</th> <td>Evaluation set: documentation and working files for the evaluation of the developed AR assembly system.</td> </tr> <tr> <th scope="row">colored_stone</th> <td>Evaluation set outcome: a graphical representation of the misalignment between each stone of the as-built model (ab) with the recorded model (rc).</td> </tr> </tbody> </table> <p> </p> <p><strong>Equipement specs</strong></p> <p>All the raw scans present in the dataset have been obtained from a FARO Freestyle 2 equipped with a Mobile PC for live point cloud processing.</p> <pre><code>@manual{farofreestyle2, title = {FARO Freestyle 2 and Mobile PC}, year = 2022, month = feb, note = {User Manual}, organization = {FARO Technologies Inc.}, url = {https://downloads.faro.com/index.php/s/sqcRBipgSy9GaEq?dir=undefined&openfile=138985} } </code></pre> <p> </p> <p><strong>For extra info</strong></p> <p>The open-sourced code for the developed AR assembly:</p> <pre><a href="https://doi.org/10.5281/zenodo.7181087">https://doi.org/10.5281/zenodo.7181087</a></pre> <p>The complete dataset of digitized mineral scraps:</p> <pre><a href="https://doi.org/10.5281/zenodo.7189478">https://doi.org/10.5281/zenodo.7189478</a></pre> <p> </p> <p><strong>Version notes</strong></p> <p>- The current dataset is published as linked documentation to a future publication, yet not reviewed.</p> <p><strong>Change log</strong></p> <p>- Typos</p> <p>- Correct order of authors</p>
Data for common data models to streamline metabolomics processing and annotation, and implementation in a Python pipeline
<p>This upload contains the HZV029 Plasma and HZV029 Two-Phase dataset for reviewers of the "Data for common data models to streamline metabolomics processing and annotation, and implementation in a Python pipeline" submission. </p> <p>Both datasets will be uploaded to metabolomics workbench and the upload completed before final publication of the manuscript. For the he HZV029 Plasma datasets only the final run is included for any sample (i.e., failed injections or other samples with data quality issues that were reran during acquisition were omitted).</p> <p>Also included in the upload is the source code for the MetDataModel and the pcpfm at the time of manuscript re-submission and the pcpfm itself. If you find this upload in the future, please check out the github repos for more updated versions:</p> <p>https://github.com/shuzhao-li-lab/PythonCentricPipelineForMetabolomics</p> <p>https://github.com/shuzhao-li-lab/metDataModel</p> <p>The github repo does not store the input the data for space reasons, they only have the notebooks. However, the .zip here has both the notebooks by themselves in the notebook subdirectory and a separate directory with the notebooks and the data used to generate all the figures and results in the manuscript.</p> <p><strong>Some information that is needed to rerun this analysis:</strong></p> <p>Sequence files are critical to the functioning of the pipeline. The sequence files for all analyses are provided under sequence_files.zip. These can be used to recapitulate the analysis by eitehr changing the filepath to each acquisition to where you put it on your sytem or by placing the sequence file in the same directory as the mzml or raw. In the latter case, the pipeline will search for filenames matching the sample names. The sequence files also store some sample metadata such as the type of sample a given acquisition is (unknown, pooled, qc, etc...)</p> <p>.raw to .mzML conversion works well on MacOS but may not work well on other systems. You will need to use the ability to specify your own conversion command or convert files outside of the pipeline. </p> <p>To replicate the results, you do need to have the annotation sources downloaded which can be done using the pipeline. MS2 annotation requires the files in the AcquireX directory which is MS2 acquisitions on pooled HZV029 plasma samples.</p> <p>For the comparison between MetaboAnalystR and the pcpfm, subsets of the datasets were used. These subsets and the sequence files are in Subsets_for_performance_testing.zip. The sequences are also in the sequence_files directory as well</p> <p>The notebooks reference data in the analysis folders. Copies of these files are located with the notebooks to ease reproduction of the exact results in the paper; however, to do so, you will need to change paths to this data in the notebook. This lets the notebooks be ran during a rerun without copying intermediates back and forth and it keeps the github repo clean.</p> <p><strong>Version History:</strong></p> <p>This version is after reviewer comments and is for resubmission.</p> <p> </p> <p><strong>Contributions:</strong></p> <p>Joshua M Mitchell implemented the pipeline and was first author on the manuscript. Shuzhao Li is the corresponding author on the manuscript. </p> <p>Maheshwor Thapa performed the experiments to collect the HZV029 data. Yuanye Chi helped with testing and documenting the pipeline. </p> <p>Jiangou (Jeff) Xia and Zhiqiang Pang provided the R portion of the analysis. </p>
Heritage Dataset around TAPI Pipeline, Afghanistan by EAMENA/ICONEM- Pilot Phase
This dataset contains 433 heritage sites documented by EAMENA and ICONEM along over 338 km of the TAPI (Turkmenistan-Afghanistan-Pakistan-India) gas pipeline project in Kandahar and Helmand provinces, Afghanistan, using remote sensing techniques.
Data for: Tang et al., Interpretable classification of Alzheimer's disease pathologies with a convolutional neural network pipeline. bioRxiv 2018.
<p>Datasets containing 63 whole slide images (WSIs) and their segmented 256x256 pixel tiles with approximately 80,000 tile-level amyloid-β pathology expert annotations.</p> <p><strong>Paper</strong>: "Interpretable classification of Alzheimer's disease pathologies with a convolutional neural network pipeline", bioRxiv 454793; DOI: <a href="https://doi.org/10.1101/454793">https://doi.org/10.1101/454793</a>.</p> <p><strong>Details:</strong> A total of 63 WSIs for 63 unique decedent cases spanning Alzheimer’s disease (AD) to non-AD and possessing a variety of CERAD scores. WSIs comprise three datasets as follows:</p> <ol> <li><em>Development (Phases I-II)</em>. 33 WSIs used for convolutional neural network (CNN) model development (29 training, 4 validation).</li> <li><em>Hold-out (Phase III)</em>. 10 WSIs selected by an expert neuropathologist as a held-out test set to assess the generalizability of the CNN model.</li> <li><em>CERAD-like hold-out</em>. 20 blinded WSIs collected solely for use in a CERAD-like scoring comparison study.</li> </ol> <p>Datasets 1 and 2 were color-normalized and segmented to 256x256 pixel image tiles for model training set (61,370 images), validation set (8,630 images), and hold-out test set (10,873 images). Dataset 3 was color-normalized but not segmented.</p> <p>Expert labels of plaques for Dataset 1 and 2 tiles are included in corresponding CSV files.</p> <p><strong>Slide source and preparation:</strong> All samples were retrieved from archives of the University of California, Davis Alzheimer’s Disease Center Brain Bank (<a href="https://www.ucdmc.ucdavis.edu/alzheimers/">https://www.ucdmc.ucdavis.edu/alzheimers/</a>). Archival samples analyzed in this study were 5 μm formalin fixed, paraffin embedded sections of the superior and middle temporal gyrus from human brain. The tissue had been previously stained with an amyloid-β antibody (4G8, recognizing residues 17-24, BioLegend, formerly Covance) that were first pretreated with formic acid to rid samples of endogenous protein. All slides were digitized using an Aperio AT2 up to 40x magnification.</p> <p><strong>Code:</strong> Please visit <a href="https://github.com/keiserlab/plaquebox-paper">https://github.com/keiserlab/plaquebox-paper</a></p> <p> </p>
Bioinformatic pipeline: Genomic diversity landscape of the honey bee gut microbiota
<p>This data-set describes the full bioinformatic pipeline used to analyze 54 metagenomic samples of the honey bee gut microbiota. Each sample was isolated from an individual honey bee, and all samples originate from two colonies of the Engel laboratory at the University of Lausanne, Switzerland. The full raw data-set is available from the sequence-read archive: SRP150166.</p> <p>A publication based on this analysis is currently under review, with the title: "Genomic diversity landscape of the honey bee gut microbiota", and an upload to Biorxiv is also underway.</p> <p>The data-set contains tar-balls for the different main workflows of the analysis. Dowload and unpack to view the contents (tar -zxvf filename.tar.gz). For each workflow, all directories contain README.txt files, describing the contents of the directory. Due to size constraints, some intermediate files have been omitted, and some workflows are demonstrated for a subset of the data. However, the full analysis can be reproduced from the raw data, using the provided scripts.</p> <p>Scripts are included within workflow directories, and are also provided as a separate tar-ball for convenience. All perl-scripts come with documentation, which can be viewed by typing: "perl script_name.pl -h". For R scripts, the usage is indicated as a comment in the top lines of each script. Note that many of the scripts require specific input-files to be present in the run-directory. Their usage is demonstrated within the workflow directories in bash-scripts (*.sh). Commands used for generating plots and some statistics are given within workflow directories in text-files "R.commands" when applicable.</p> <p>Aside from custom code, the pipeline also utilizes various open-source Software packages, which are detailed in the file "software_dependencies.txt". Note, while many of the scripts will run fast on any computer, some steps of the pipeline are computationally demanding, and will require significant computing time, as well as storage space. When scripts are known to be time-consuming, this is indicated in the script help message.</p> <p> </p> <p> </p> <p> </p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.