Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
3
datasets available to search
ShareScore release 0.9.0
Dataset results
3 results for “refseq mash”
Mash Sketch of RefSeq Bacterial Representative Genomes v217
<p>This was created to get a new mash reference that was current. The script to create this uses ncbi datasets and mash (https://github.com/UPHL-BioNGS/Grandeur/blob/main/bin/new_mash_ref.sh)<br> <br> This was created on April 27, 2023, and is RefSeq v217<br> <br> ```bash</p> <p>#/bin/bash<br> out=mash_db<br> <br> mkdir $out</p> <p>cd $out</p> <p>datasets summary genome taxon bacteria --reference --as-json-lines | \</p> <p>dataformat tsv genome --fields accession,assminfo-refseq-category,organism-name --elide-header | \</p> <p>grep representative | \</p> <p>tee representative_genomes.txt | \</p> <p>cut -f 1 > genome_ids.txt</p> <p>echo "$(date): Downloading genomes for ids"</p> <p>datasets download genome accession --inputfile genome_ids.txt --filename rep-genomes.zip</p> <p>echo "$(date): Decompressing zip file"</p> <p>unzip rep-genomes.zip</p> <p>echo "$(date): Creating file for mash"</p> <p>cat ncbi_dataset/data/*/*.fna | sed 's/ /_/g' | sed 's/,//g' > rep-genomes.fasta</p> <p>echo "$(date): Skeching rep-genomes.fasta"</p> <p>mash sketch -i -p 20 rep-genomes.fasta -o rep-genomes</p> <p>############################################################</p> <p>echo "$(date): File preparation is complete"</p> <p>ls -alh rep-genomes.fasta</p> <p>ls -alh rep-genomes.msh<br> <br> ```</p>
Mash Sketch of RefSeq Bacterial Reference Genomes
<p>The mash reference that can be downloaded from <a href="https://mash.readthedocs.io/en/latest/data.html">the mash documentaion</a> is for RefSeq version 70.</p> <p>I do not inherently have a problem with RefSeq version 70, but RefSeq is well past version 200 now. </p> <p>RefSeq updates four times year, and I needed an easy way to create and distribute a mash sketch file of the representative bacterial/prokaryotic genomes.<br><br>This is intended to be a place to hold the mash sketches from <a href="https://github.com/erinyoung/update_mash_dist">https://github.com/erinyoung/update_mash_dist</a>.<br><br>The mash sketch file from erinyoung/update_mash_dist requires git lfs to be installed when cloning the repository, which is cumbersome for some users.<br><br>The update requency is intended to mirror that of RefSeq (i.e. 4 time a year), but... is likely to be less frequent than that.<br><br>Don't hesitate to <a href="https://github.com/erinyoung/update_mash_dist/issues">submit an issue</a> if this needs to get updated.<br><br>I do have some prior zenodo repositories (https://zenodo.org/records/10519852 , https://zenodo.org/records/7887021 , and https://zenodo.org/records/7348463 ) which hold the same mash sketch reference, but the refseq version is in the title. I'd rather have one repository that gets updated rather than create new repositories each time.<br><br>This is how the mash reference file was created:<br><br></p> <pre><code># Step 1. Download Datasets and Dataformat </code></pre> <pre><code>wget https://ftp.ncbi.nlm.nih.gov/pub/datasets/command-line/v2/linux-amd64/datasets wget https://ftp.ncbi.nlm.nih.gov/pub/datasets/command-line/v2/linux-amd64/dataformat chmod +x datasets dataformat</code></pre> <pre><code> # Step 2. Download Mash <br></code></pre> <pre><code> wget https://github.com/marbl/Mash/releases/download/v2.3/mash-Linux64-v2.3.tar tar -xvf mash-Linux64-v2.3.tar </code></pre> <pre><code><br> # Step 3. Get a list of all the genomes # Note: this also changes how some of the names are represented datasets summary genome taxon bacteria --reference --as-json-lines | \ dataformat tsv genome --fields accession,organism-name --elide-header | \ sed 's/\[//g' | \ sed 's/\]//g' | \ sed 's/["'\'']//g' | \ sed 's/endosymbiont of /endosymbiont_of_/g' > \ ids.txt # Step 4. Download the reference files and sketch them # Note: Since this is done in Github Actions (GA), I need to keep everything below 30G. # The best way to do this is to download the process each reference file individually, and then combine it to the whole. # This obviously does not need to be followed if not under those same limitations. while read line do id=$(echo $line | awk '{print $1}') ge=$(echo $line | awk '{print $2}') if [ ! -n "$ge" ] ; then ge="unknown" ; fi sp=$(echo $line | awk '{print $3}') if [ ! -n "$sp" ] ; then sp="unknown" ; fi datasets download genome accession $id unzip ncbi_dataset.zip cp ncbi_dataset/data/*/*_genomic.fna ${ge}_${sp}_${id}.fasta if [ ! -f RefSeqSketches_${version}.msh ] then mash sketch ${ge}_${sp}_${id}.fasta -o RefSeqSketches_${version} else mash sketch ${ge}_${sp}_${id}.fasta -o ${ge}_${sp}_${id} mv RefSeqSketches_${version}.msh tmp.msh mash paste RefSeqSketches_${version} tmp.msh ${ge}_${sp}_${id}.msh rm tmp.msh ${ge}_${sp}_${id}.msh fi rm ${ge}_${sp}_${id}.fasta rm -rf ncbi_dataset/ rm ncbi_dataset.zip rm README.md rm md5sum.txt done < ids.txt</code></pre> <pre><br><br>To use</pre> <pre><code># download file wget <insert url for file> mask sketch sample.fasta RefSeqSketches_<version>.msh > mash_results.txt # These results are unsorted, so many find it useful to sort them. sort -gk3 mash_results.txt > sorted_mash_results.txt</code></pre> <p> <br>The should look like the following:</p> <pre><code>2024CK-00429-UT-M03999-240412_contigs.fa Streptococcus_pyogenes_GCF_900475035.1.fasta 0.0116661 0 643/1000 2024CK-00429-UT-M03999-240412_contigs.fa Streptococcus_dysgalactiae_GCF_016128095.1.fasta 0.0782587 0 107/1000 2024CK-00429-UT-M03999-240412_contigs.fa Streptococcus_canis_GCF_900636575.1.fasta 0.132399 2.34894e-153 32/1000 2024CK-00429-UT-M03999-240412_contigs.fa Streptococcus_agalactiae_GCF_001552035.1.fasta 0.164662 1.32611e-72 16/1000 2024CK-00429-UT-M03999-240412_contigs.fa Streptococcus_castoreus_GCF_000425025.1.fasta 0.174408 2.34302e-58 13/1000 2024CK-00429-UT-M03999-240412_contigs.fa Streptococcus_didelphis_GCF_000380005.1.fasta 0.182269 8.30736e-49 11/1000 2024CK-00429-UT-M03999-240412_contigs.fa Streptococcus_uberis_GCF_900475595.1.fasta 0.186761 5.62934e-44 10/1000 2024CK-00429-UT-M03999-240412_contigs.fa Streptococcus_iniae_GCF_000831485.1.fasta 0.191731 3.33152e-39 9/1000 2024CK-00429-UT-M03999-240412_contigs.fa Streptococcus_ictaluri_GCF_000188015.2.fasta 0.197292 1.75608e-34 8/1000 2024CK-00429-UT-M03999-240412_contigs.fa Streptococcus_phocae_GCF_001302265.1.fasta 0.203604 2.46548e-30 7/1000</code></pre> <pre> </pre>
Mash Sketch of RefSeq Bacterial Representative Genomes v215
<p>Prepared as described in https://github.com/UPHL-BioNGS/DesolationCanyon.</p> <p> </p> <p>This is the representative genomes of refseq 215 with their taxon name.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.