Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
334
datasets available to search
ShareScore release 0.7.1
Dataset results
334 results for “python”
Airline Satisfaction Survey Data: Data Science for Business with Python
<p>Companion dataset for the textbook entitled "Data Science for Business with Python"</p>
Python lab automation landscape catalog
<p>This version contains all the useful original data, presented in a simple web page. Some more polishing is still necessary before this is appropriate for wider dissemination or contributions, therefore the pre-1.0 version tag.</p>
Python file loading time comparison across filetypes
<p>A quick test of loading times of various file formats that can be used to store image data in B x N x M shape (B: number of bands, N, M: number of pixels on both axes). The Jupyter Notebook used to obtain these results is provided.</p> <p> </p> <p><strong>Methods</strong></p> <p>I used Python 3.6.15 with the following packages:</p> <ul> <li>astropy 4.1</li> <li>numpy 1.19.5</li> <li>h5py 3.1.0</li> <li>hdf5 1.10.6</li> <li>matplotlib 3.3.4</li> <li>tqdm 4.62.3</li> <li>_pickle (version packaged with Python 3.6.15)</li> </ul> <p>The tests were run on Fedora 35 with an Intel© Xeon© W-1250 CPU @ 3.30GHz × 6, 16 GB of RAM and the data stored on 3 disks (HGST WD Ultrastar HUS726T4TALE6L4) in a RAID 5 configuration.</p> <p> </p> <p>Images are generated in (3, 64, 64) shape with pixel values drawn from a normal distribution. Two tests are run, both on 1000 images in total. For the first, those images are saved individually to the four file formats in question: .fits, .npy, .h5 and .pkl and then read one by one, and the operation np.mean() is applied (to prevent any memcaching). In the second test, they are saved in batches of 64, resulting in 64 images of (3, 64, 64) per file. This yields the second plot.</p> <p> </p> <p><strong>Results</strong></p> <p>When saving images individually, the .pkl files are loaded the quests at 0.077 ms per file for loading + running np.mean. .npy files take 3.0 times longer, .h5 files 6.1 times longer and .fits files 7.7 times longer. This changes when loading the batched files. Loading + applying np.mean is then fastest with .npy files at 0.034 ms per (3, 64, 64) data unit (the loading file of the batched file divided by the batch size), then .fits 1.4 times longer, .pkl 2.0 times longer and .h5 at 2.1 times longer.</p> <p> </p> <p><strong>Conclusion</strong></p> <p>The .fits format widely used in astronomy has a long loading time for individual files, most likely due to the overhead caused by reading the header. It is however one of the fastest file formats when saving images in batches. This should therefore be considered when storing large numbers of images.</p>
HDF5 datasets and python scripts to generate figures in "Butterfly distribution of relativistic electrons driven by parallel propagating lower band whistler chorus waves"
<p>HDF5 datasets and python scripts to generate figures in "Butterfly distribution of relativistic electrons driven by parallel propagating lower band whistler chorus waves"</p> <p>RBW simulation datasets in HDF5 format:</p> <ul> <li>300pT.h5 The particle dataset to generate the figures.</li> </ul> <p>Python scripts to generate figures in the manuscript.</p> <p>- Environment: Python 3.6.7 :: Anaconda 4.4.0 (64-bit)</p> <p>- Required modules: matplotlib, numpy, h5py</p> <ul> <li>Figure1.py Generate figure 1.</li> <li>Figure2.py Generate figure 2.</li> <li>Figure3.py Generate figure 3.</li> <li>Figure4.py Generate figure 4.</li> <li>QLDe.py Calculate bounce averaged diffusion coefficients according to Shprits et al. (2006) (doi: https://doi.org/10.1029/ 2006JA011725).</li> </ul> <p> </p>
Dataset of Jupyter Notebooks from the paper "A Large-Scale Comparison of Python Code in Jupyter Notebooks and Scripts"
<pre>This archive contains the dataset of properly-licensed Jupyter notebooks from the MSR'22 paper "A Large-Scale Comparison of Python Code in Jupyter Notebooks and Scripts". The dataset contains 847,881 notebooks stored in the PostgreSQL dump file. You can find the details about the database in the README file. To transform the notebooks into this convenient format and to calcuate the structural metrics, we used our library called Matroskin, which can be found here: <a href="https://github.com/JetBrains-Research/Matroskin">https://github.com/JetBrains-Research/Matroskin</a>. </pre>
Exploratory Topic Modelling in Python Dataset - EHRI-3
<p>In the EHRI-3 project, we are investigating tools and methods that historical researchers and scholars can use to better understand, visualise, and interpret the material held by our partner archives. This dataset accompanies a tutorial exploring a technique called topic modelling in the context of a Holocaust-related historical collection.</p> <p>We were on the lookout for datasets that would be easily accessible and, for convenience, predominantly in English. One such dataset was the United States Holocaust Memorial Museum’s (USHMM) extensive collection of oral history testimonies, for which there are a considerable number of textual transcripts. The museum’s total collection consists of over 80,703 testimonies, 41,695 of which are available in English, with 2,894 of them listing a transcript.</p> <p>Since there is not yet a ready-to-download dataset that includes these transcripts, we had to construct our own. Using a web scraping tool, we managed to create a list of the links pointing to the metadata (including transcripts) of the testimonies that were of interest to us. After obtaining the transcript and other metadata of each of these testimonies, we were able to create our dataset and curate it to remove any unwanted entries. For example, we made sure to remove entries with restrictions on access or use. We also removed entries with transcripts that consisted only of some automatically generated headers and entries which turned out to be in languages other than English. The remaining 1,873 transcripts form the corpus of this tutorial — a small, but still decently sized dataset.</p> <p>The process that we followed to put together this dataset is detailed in the Jupyter Notebook accompanying this post, which can be found in this <a href="https://github.com/mdermentzi/ehri-topic-modelling-guide">Github repository</a>.</p> <p>In this Zenodo upload, the user can find two files, each of them containing a <a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_pickle.html">pickled pandas DataFrame</a> that was obtained at a different stage of the tutorial:</p> <p>"unrestricted_df.pkl" contains 1,946 entries of Oral Testimony transcripts and has five fields (RG_number, text, display_date, conditions_access, conditions_use)<br> "unrestricted_lemmatized_df.pkl" contains 1,873 entries of Oral Testimony transcripts and has six fields (RG_number, text, display_date, conditions_access, conditions_use, lemmas)</p> <p>Instructions on their intended use can be found in the accompanying Jupyter Notebook.</p> <p>Credits:</p> <p>The transcripts that form the corpus in this tutorial were obtained through the <a href="https://www.ushmm.org/">United States Holocaust Memorial Museum</a> (USHMM).</p>
Replication package for "An Exploratory Study on the Predominant Programming Paradigms in Python Code"
<p>This dataset includes scripts and data files used to generate all analysis and results from the paper. A README.md file is included for details on using the scripts - though all of the data the scripts generate should already be cached and none of the scripts actually need run.</p> <p>It also includes a spreadsheet containing the human judgements from Table 4 of the paper.</p> <p>Always current source for the scripts is available on GitHub: <a href="https://github.com/psybers/python-paradigms">https://github.com/psybers/python-paradigms</a></p>
SPAAM Summer School 2022: Introduction to Ancient Metagenomics - 3b2 Introduction to Python and Pandas
<p>Teaching data for practical session: "3b2 Introduction to Python and Pandas" of the 2022 SPAAM Summer School: Introduction to Ancient Metagenomics (Aug. 1-5 2022).</p> <p>See: <a href="https://spaam-community.github.io/wss-summer-school/#/2022/">https://spaam-community.github.io/wss-summer-school/#/2022/</a> or <a href="https://doi.org/10.5281/zenodo.6976711">https://doi.org/10.5281/zenodo.6976711</a> for slides.</p> <p>Once downloaded, run:</p> <pre><code>tar xvfz <session>.tar.gz</code></pre> <p> to decompress the data directory for the session.</p>
Python scripts / Jupyter Notebooks and data for training segmentation models on slide scans of diatom preparations from river Menne
<p>This archive contains the Jupyter Notebooks and data used for the deep learning experiments published in Kloster et al. 2022: Improving deep learning-based segmentation of diatoms in gigapixel-sized virtual slides by object-based tile positioning and object integrity constraint.</p> <p>The notebooks are numbered according to the order in which they are to execute. Please refer to the comments and documentation within the notebooks as well as to the manuscript for details. The data (image data, mask data & segmentation ground truth in COCO format for several different tiling strategies) is stored in separate subfolders corresponding with data usage (model training, validation, test) and tiling strategy. Please refer to the "readme" files for detailed information.</p> <p> </p> <p> </p>
Quantifying the Sensitivity and Unclonability of Optical Physical Unclonable Functions (DATA and Python codes)
<p>The data contain experimental and numerical challenge response pairs (CRPs) collected by experimental setup and using Puffraction homebuilt code. In the data set is available all CRPs referred to the paper entitled "Quantifying the Sensitivity and Unclonability of Optical Physical Unclonable Functions" by Giuseppe Emanuele Lio, Sara Nocentini, Lorenzo Pattelli, Eleonora Cara, Diederik Sybolt Wiersma, Ulrich R"uhrmair, and Francesco Riboli. </p> <p> </p> <p>The <strong><em>puffractio</em></strong> python code used to generate and process the numerical data is available at the following link <a href="https://github.com/lpattelli/puffractio.git">https://github.com/lpattelli/puffractio.git</a></p> <p> </p> <p>Please cite the following paper: </p> <p>Quantifying the Sensitivity and Unclonability of Optical Physical Unclonable Functions</p> <p><a href="https://onlinelibrary.wiley.com/action/doSearch?ContribAuthorRaw=Lio%2C+Giuseppe+Emanuele">Giuseppe Emanuele Lio</a>, <a href="https://onlinelibrary.wiley.com/action/doSearch?ContribAuthorRaw=Nocentini%2C+Sara">Sara Nocentini</a>, <a href="https://onlinelibrary.wiley.com/action/doSearch?ContribAuthorRaw=Pattelli%2C+Lorenzo">Lorenzo Pattelli</a>, <a href="https://onlinelibrary.wiley.com/action/doSearch?ContribAuthorRaw=Cara%2C+Eleonora">Eleonora Cara</a>, <a href="https://onlinelibrary.wiley.com/action/doSearch?ContribAuthorRaw=Wiersma%2C+Diederik+Sybolt">Diederik Sybolt Wiersma</a>, <a href="https://onlinelibrary.wiley.com/action/doSearch?ContribAuthorRaw=R%C3%BChrmair%2C+Ulrich">Ulrich Rührmair</a>, <a href="https://onlinelibrary.wiley.com/action/doSearch?ContribAuthorRaw=Riboli%2C+Francesco">Francesco Riboli</a></p> <p><a href="https://onlinelibrary.wiley.com/doi/full/10.1002/adpr.202200225">https://onlinelibrary.wiley.com/doi/full/10.1002/adpr.202200225</a></p>
Python scripts and datasets used in the article "Investigating the off-axis GRB afterglow scenario for extragalactic fast X-ray transients"
<p>This package includes datasets and python scripts used in the analysis and creation of figures in the A&A paper "Investigating the off-axis GRB afterglow scenario for extragalactic fast X-ray transients" (Wichern et al. 2024).</p>
Figure 4 in Natives bite back: depredation and mortality of invasive juvenile Burmese pythons (Python bivittatus) in the Greater Everglades Ecosystem
Figure 4. Photographic documentation indicative of mesomammal depredation on two telemetered juvenile Burmese pythons (Python bivittatus Kuhl, 1820) in Big Cypress National Preserve, Florida, USA. Left panel: A juvenile python's (MD1) radio transmitter recovered on 12 September 2021; the red circle encompasses sign of mastication. Right panel: Partially consumed juvenile python (MD3) carcass with transmitter discovered on 19 September 2021; partial Felidae tracks were located within two meters of the carcass. Images by the U.S. Geological Survey.
Figure 3 in Natives bite back: depredation and mortality of invasive juvenile Burmese pythons (Python bivittatus) in the Greater Everglades Ecosystem
Figure 3. Necropsy photographs of the juvenile Burmese python (Python bivittatus Kuhl, 1820) foraging death. The python's wounds were consistent in size with those that could be inflicted by the prey item (hispid cotton rat, Sigmodon hispidus Say & Ord, 1825) during efforts to capture and subdue. From the top left: (A) python with incision exposing the prey item in situ; (B) exterior wounding on dorsum of snake (white arrows); (C) posterior puncture wound showing proximity to the non-vascularized portion of the right lung (ID tag in orange above scale); (D) anterior puncture wound viewed from within the stomach lining. Subject was discovered in the field on 17 October 2021 in Big Cypress National Preserve, Florida, USA and represents the largest predator:prey size ratio in this size class at 1:1.06. Images by the U.S. Geological Survey.
Figure 5 in Natives bite back: depredation and mortality of invasive juvenile Burmese pythons (Python bivittatus) in the Greater Everglades Ecosystem
Figure 5. Photographic evidence of three telemetered juvenile Burmese pythons (Python bivittatus Kuhl, 1820) where mortality causes were only inconclusive or undetermined, in Big Cypress National Preserve, Florida, USA. Panel (A) intact transmitter of UM2 in situ found on 19 October 2022; (B) carcass orientation of UM3 found on flooded ground 23 October 2021; (C) partially consumed juvenile python (UM5) with transmitter discovered 29 October 2021 showing carcass compression and decay in matted ground cover; (D) python UM6 radio transmitter recovered 19 November 2021 with black and white hairs attached; (E) transmitter of UM7 suspended in open prairie grasses approx. 1.5 m above ground. Images by the U.S. Geological Survey.
Figure 2 in Natives bite back: depredation and mortality of invasive juvenile Burmese pythons (Python bivittatus) in the Greater Everglades Ecosystem
Figure 2. Florida cottonmouth (Agkistrodon conanti Gloyd, 1969) in-situ (left) that consumed a telemetered juvenile Burmese python (Python bivittatus Kuhl, 1820) and confirmed by radiography (right; used with editorial permission and further described in Bartoszek et al. 2021). Subject was discovered in the field on 31 May 2021 in Big Cypress National Preserve, Florida, USA. Images by U.S. Geological Survey (left) and Zoo Miami (right).
Figure 1 in Natives bite back: depredation and mortality of invasive juvenile Burmese pythons (Python bivittatus) in the Greater Everglades Ecosystem
Figure 1. Photographic evidence and representation of some of the variety of confirmed and potential causes of mortality found for invasive Burmese pythons (Python bivittatus Kuhl, 1820) in the Greater Everglades Ecosystem in 2021 in Big Cypress National Preserve, Florida, USA. From left to right: American Alligator (Alligator mississippiensis Daudin, 1802) depredations, mesomammal depredations (felid prints in muddy substrate; e.g., bobcat, Lynx rufus Schreber, 1777), Florida cottonmouth (Agkistrodon conanti Gloyd, 1969) depredations, potential avian depredations, mishandling/misidentification of appropriate prey (e.g., large hispid cotton rat, Sigmodon hispidus Say and Ord, 1825). Images by U.S. Geological Survey.
Pylint Results for Python Code Snippets on Stack Overflow
<p>This dataset contains valid pylint results for all Stack Overflow code snippets from SOTorrent that meet the following criteria</p> <ul> <li>Tagged with 'python'</li> <li>6 lines and above</li> <li>Contains basic python syntax (i.e. 'print', 'import', '(', '=')</li> <li>Produces a result when processed by Pylint</li> </ul>
API Database of Python frameworks & Labeled Issues
<p>PyLibAPIs.7z : contains public API data (MongoDB dump) for these frameworks:</p> <ul> <li>TensorFlow</li> <li>Keras</li> <li>Scikit-learn</li> <li>Pandas</li> <li>Flask</li> <li>Django</li> </ul> <p>Label.xlsx: contains issues and their labels</p> <p>Breaking Changes for All Frameworks.pdf: contains the breaking change distributions of all six frameworks</p>
Fig 2 in Predation On Sun Bears By Reticulated Python In East Kalimantan, Indonesian Borneo
Fig 2. Reticulated python being pulled out of an underground stream 1.5 month after it swallowed a radio-collared adult female Malay sun bear (photo courtesy M. van Nieuwstadt).
LangString Python Library
A Python library to handle langstrings and related datatypes.
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.