Find research datasets worth reusing

opencc-by-4.0Nov 2023View details →

Code and Source Data for "Knowledge-Guided Machine Learning can improve C cycle quantification in agroecosystems"

Datasets for code and Source Data for the study "Knowledge-Guided Machine Learning can improve C cycle quantification in agroecosystems" https://doi.org/10.1038/s41467-023-43860-5. All files belong to Licheng Liu and Zhenong Jin at University of Minnesota. deposit_code_v2.zip contains packaged codes and sample runs for KGML-ag-Carbon training, validation and implementations. Source Data.zip contains data for generating the figures inside the study.  Note: We used Pytorch 1.6.0 (<a href="https://pytorch.org/get-started/previous-versions/">https://pytorch.org/get-started/previous-versions/</a>, last access: 21 Oct 2023) and Python 3.7.11 (<a href="https://www.python.org/downloads/release/python-3711/">https://www.python.org/downloads/release/python-3711/</a>, last access: 21 Oct 2023) as the programming environment for model development. Statistical analysis, such as linear regression, was conducted using Statsmodels 0.14.0 (<a href="https://github.com/statsmodels/statsmodels/">https://github.com/statsmodels/statsmodels/</a>, last access: 21 Oct 2023) In order to use a GPU to speed-up the training process, we installed the CUDA Toolkit 10.1.243 (<a href="https://developer.nvidia.com/cuda-toolkit">https://developer.nvidia.com/cuda-toolkit</a>, last access: 21 Oct 2023).  To use the full kgml_lib function, please create a new environment with the same python and libs above.

opencc-by-4.0Nov 2023View details →

Determining non-significant bits on a C++ implementation of the LeNet-5 convolutional neural network to be used for storing error correcting codes to protect weights and biases. Robustness assessment of the network after integrating the proposed codes.

The architecture of the LeNet-5 convolutional neural network (CNN) was defined by LeCun in its paper "Gradient-based learning applied to document recognition" (<a href="https://ieeexplore.ieee.org/document/726791">https://ieeexplore.ieee.org/document/726791</a>) to classify images of hand written digits (MNIST dataset).This architecture has been customized to use Rectified Linear Unit (ReLU) as activation functions instead of Sigmoid.It consists of the following layers:<ul><li>conv1: Convolution 2D, 1 input channel (28x28), 3 output channels (28x28), kernel size 5, stride 1, padding 2.</li><li>relu1: Rectified Linear Unit (3@28x28).</li><li>max1: Subsampling buy max pooling (3@14x14).</li><li>conv2: Convolution 2D, 3 input channels (14x14), 6 output channels (14x14), kernel size 5, stride 1, padding 2.</li><li>relu2: Rectified Linear Unit (6@14x14).</li><li>max2: Subsampling buy max pooling (6@7x7).</li><li>fc1: Fully connected (294, 147)</li><li>fc2: Fully connected (147, 10)</li></ul>The fault hypotheses for this work include the occurrence of:<ul><li>S0/S1: multiple adjacent stuck-at-0 and stuck-at-1 faults to determine the least significant bits of weights and biases that could be used to store the proposed error correcting codes.</li><li>BF: single, double, and triple bit-flip faults to assess the robustness of the considered CNN</li></ul>In the memory cells containing all the parameters of the CNN:  <ul><li>w: weights (float32)</li><li>b: biases (float32)</li></ul>All the images (10000) from the MNIST dataset have been used as workload.The weights and biases of the LeNet-5 architecture have been protected using six different error correcting codes that have been deployed in the least significant bits of these elements.The parity check matrices (H = P I) that define these ECCs are:<ul><li>SEC(32, 26) (Hamming) under a classic policy (see methodology below):</li></ul>        11010010001000011101101000 100000        10101001000100011011010100 010000        01100100100010010110110010 001000        00011100010001001110001101 000100        00000011110000100001111011 000010        00000000001111100000000111 000001<ul><li>SEC(23, 18) (Hamming) under a conservative policy (see methodology below):</li></ul>        111100001111000000 10000        110011101000111000 01000        101011010100100110 00100        010110110010010101 00010        001101110001001011 00001<ul><li>SEC(13, 9) (Hamming) under an aggressive policy (see methodology below):</li></ul>        110111000 1000        101100110 0100        011010101 0010        111001011 0001<ul><li>DEC(32, 21) (low redundancy and reduced overhead DEC) under a classic policy (see methodology below):</li></ul>        111000011001010010000 10000000000        110110000011101000000 01000000000        101011000110000010001 00100000000        100101101000110001000 00010000000        011010101100100000100 00001000000        010101010100001001010 00000100000        001100110010010100100 00000010000        000011110001000110010 00000001000        000000001111001101001 00000000100        000000000000111100111 00000000010        000000000000000011111 00000000001<ul><li>DEC(28, 18) (low redundancy and reduced overhead DEC) under a conservative policy (see methodology below):</li></ul>        111111000000000000 1000000000        110100111100000000 0100000000        110000100011110000 0010000000        001110010011001100 0001000000        101100001010101010 0000100000        010001001101010110 0000010000        001011000101101001 0000001000        101000011000110101 0000000100        010001110000011011 0000000010        000010100110000111 0000000001<ul><li>DEC(17, 9) (low redundancy and reduced overhead DEC) under an aggressive policy (see methodology below):</li></ul>        111110000 10000000        111001100 01000000        110101010 00100000        101010110 00010000        101101001 00001000        100110101 00000100        100011011 00000010        110000111 00000001This dataset contains the raw data obtained from:<ul><li>running exhaustive fault injection campaigns for increasingly multiple stuck-at faults in the least significant bits of all weights and biases (simultaneously) and for all the images in the workload.</li><li>running statistical fault injection campaigns for single, double, and triple bit-flip faults, randomly targeting the considered locations and images in the workload.</li></ul><h3>Files information</h3><ul><li>no_ecc folder: Results obtained for the original (not protected) version of the CNN.<ul><li>golden_run.csv: Prediction obtained for all the images considered in the workload in the absence of faults (Golden Run). This is intended to act as oracle to determine the impact of injected faults.</li><li>sampling_SBF_10000.csv: Prediction obtained for running 10000 statistical fault injection experiments for single bit-flip faults.</li><li>sampling_DBF_10000.csv: Prediction obtained for running 10000 statistical fault injection experiments for double bit-flip faults.</li><li>sampling_TBF_10000.csv: Prediction obtained for running 10000 statistical fault injection experiments for triple bit-flip faults.</li><li>locating_sensitive_bits folder: Prediction obtained for all the images considered in the workload in presence of stuck-at-0/stuck-at-1 faults that simultaneously target the N least significant bits of all weights and biases. There is one file for each parameter of type of fault and range of targeted bits. Files for bits in the range [11, 0] are not included as they obtain eactly the same results as the Golden Run (faults do not alter the behaviour of the network).</li></ul></li><li>sec/classic, sec/conservative, and sec/aggressive folders: They contain the results obtained for the CNN protected by SEC(32, 26), SEC(23, 18), and SEC(13, 9), respectively.<ul><li>golden_run.csv: Prediction obtained for all the images considered in the workload in the absence of faults (Golden Run). This is intended to act as oracle to determine the impact of injected faults. It must be noted that this file could be different that the golden_run.csv file for the original version of the CNN, as deploying the ECC in the weights and biases may have affected the behaviour of the network.</li><li>sampling_SBF_10000.csv: Prediction obtained for running 10000 statistical fault injection experiments for single bit-flip faults. They should all be tolerated by the definition of the ECC.</li><li>sampling_DBF_10000.csv: Prediction obtained for running 10000 statistical fault injection experiments for double bit-flip faults. They could be more harmful than for the unprotected version of the CNN, as the ECC may erroneously flip correct bits.</li></ul></li><li>dec/classic, dec/conservative, and dec/aggressive folders: They contain the results obtained for the CNN protected by DEC(32, 21), DEC(28, 18), and DEC(17, 9), respectively.<ul><li>golden_run.csv: Prediction obtained for all the images considered in the workload in the absence of faults (Golden Run). This is intended to act as oracle to determine the impact of injected faults. It must be noted that this file could be different that the golden_run.csv file for the original version of the CNN, as deploying the ECC in the weights and biases may have affected the behaviour of the network.</li><li>sampling_DBF_10000.csv: Prediction obtained for running 10000 statistical fault injection experiments for double bit-flip faults. They should all be tolerated by the definition of the ECC.</li><li>sampling_TBF_10000.csv: Prediction obtained for running 10000 statistical fault injection experiments for triple bit-flip faults. They could be more harmful than for the unprotected version of the CNN, as the ECC may erroneously flip correct bits.</li></ul></li></ul><h3>Methodology information</h3>First, the CNN was used to classify all the images of the workload in the absence of faults to get a reference to determine the impact of faults. This is golden_run.csv file.To locate non-significant bits in weights and biases, fault injection experiments were executed targeting all elements of all parameters of the CNN using the following procedure:<ul><li>The initial mask targeted only the least significant bit</li><li>Until the mask targets all bits of the elements (32 bits as they are single-precision floating point values):<ul><li>Affect the bits (setting them to 0 or 1 in case of stuck-at-0 or stuck-at-1 faults) identified by the mask for all elements of all parameters.</li><li>Classify all the images of the workload in the presence of this fault. The obtained output was stored in a given .csv file.</li><li>Remove the fault from the CNN by restoring the affected bits to its previous value.</li><li>Add the next adjacent bit to the mask, so it targets an additional least significant bit.</li></ul></li></ul>The analysis of the obtained results may help in determining which bits can be used to store an ECC:<ul><li>which bits never affect the behaviour of the CNN, as the predicted classification is exactly the same than in the absence of faults.</li><li>which bits midly affect the behaviour of the CNN, as although the predicted classifications differ from those in the absence of faults, the accuracy of the network is barely affected.</li><li>which bits greatly affect the behaviour of the CNN, as the accuracy of the network is significantly affected.</li></ul>Accordingly, three different policies have been identified for deploying an ECC using these bits:<ul><li>Classic policy: The ECC protects as much bits as possible.</li><li>Conservative policy: The ECC protects all those bits that may affect the prediction of the network.</li><li>Aggressive policy: The ECC protects only those bits that significantly affect the accuracy of the network.</li></ul>After designing and deploying a single ECC and a double ECC for each of the identified policies, fault injection experiments were executed to verify their behaviour in the presence of faults.Single and double ECCs were tested against single and double bit-flip, respectively (all faults should be tolerated,) and double and triple bit-flips, respectively (a correct bit could be erroneously flipped.)Due to the heavy computational load of the decoders, statistical injection was used to run the required fault injection campaigns with a sample size (number of experiments) of 10000.Each experiment consisted in:<ul><li>Randomly selecting the image to process, and the parameter, element, and bits (mask) to be targeted by the fault.</li><li>Affecting the bits (inverting them) identified by the mask.</li><li>Classifying the selected image of the workload in the presence of this fault. The obtained output was stored in a given .csv file.</li><li>Removing the fault from the CNN by restoring the affected bits to its previous value.</li></ul><h3>List of variables (Name : Description (Possible values))</h3><ul><li>IMGID: Integer number identifying the considered image (1-9999).</li><li>TENSORID: Integer number identiying the parameter affected by the fault (0 - No fault, 1 - conv1.w, 2 - conv1.b, 3 - conv2.w, 4 - conv2.b, 5 - fc1.w, 6 - fc1.b, 7 - fc2.w, 8 - fc2.b).</li><li>ELEMID: Integer number identiying the element of the parameter affected by the fault (-1 - No fault, [0-2] - conv1.b, [0-74] - conv1.w, [0-5] - conv2.b, [0-149] - conv2.w, [0-146] - fc1.b, [0-43217] - fc1.w, [0-9] - fc2.b, [0-1469] - fc2.w).</li><li>MASK: 8-digit hexadecimal number identifying those bits affected by the fault ([00000000 - No fault, FFFFFFFF - all 32 bits faulty]).</li><li>FAULT: String identiying the type of fault (NF - No fault, BF - bit-flip, S0 - Stuck-at-0, S1 - Stuck-at-1).</li><li>SOFTMAX: 10 decimal numbers obtained after applying the softmax function to the provided output. They represent the probability of the image of belonging to the corresponding category for classification.</li><li>PRED: Integer number representing the category predicted for the processed image.</li><li>LABEL: integer number representing the actual category for the processed image.</li></ul>

dryad40/100

Data and code from: Spatial ecology of the Turks & Caicos boa, Chilabothrus c. chrysogaster Cope, 1871 (Serpentes: Boidae)

Obtaining ecological and natural history data from cryptic squamates can be challenging, but is crucial to understanding species' biology, particularly in the context of conservation. In the Greater Antilles, this challenge is especially apparent, particularly among the West Indian boas (genus Chilabothrus). Most species have had only minimal natural history study, with a few exceptions. The Turks & Caicos boa (C. chrysogaster) has been studied intensively for over 16 years on the small privately owned island of Big Ambergris Cay, Turks and Caicos Islands. We conducted a multi-year radio-tracking study on the species to generate information relevant to spatial habitat use and movement that will inform conservation decision-making in the face of increasing development pressure. We tracked a total of 19 female snakes using surgically implanted transmitters, enabling us to obtain between 16 and 40 location observations per boa over the lifetime of each transmitter. We estimated home ranges, the core space used by an animal, using range distributions, finding that females have a home range of 0.70 ha to 1.2 ha. We also estimated occurrence distributions, the use of space between specific time intervals, finding an average occurrence area of 1.62 ha. Several females overlapped in their spatial habitat use, and we observed female boas using two novel habitats for the species (iron shore wrack and red mangrove). This study provides valuable information on the spatial ecology of an endangered boa and will serve to inform conservation work that is currently underway.

opencc-zeroDec 2023View details →

openmit-licenseFeb 2024View details →

Code for 'Risks of unavoidable impacts on forests at 1.5C with and without overshoot'

<h2>Risks of unavoidable impacts on the Siberian and Amazon forests from AR6 WGIII's Illustrative Mitigation Pathways.</h2> Contains code and data for reproducing results from Munday et al., in review. <h3>Installing the conda environment</h3> To install the conda environment, run the following on your command line: <code>conda env create -f environment.yml</code> Notes on the notebooks <code>all_other_figures.ipynb</code> requires a good amount of memory to run (it runs fine on a decent spec. MacBook Pro), based on the quantity of data required for some of the plots. Some variables have squiggly underlines - don't worry about this. The notebooks will still run, this is just a consequence of using the <code>%%capture</code> keyword in some notebook cells, which makes sure hundreds of irrelevant warnings aren't output.

opencc-by-4.0Dec 2021View details →

Source data and code for: Existing fossil fuel extraction would warm the world beyond 1.5°C

Source data and code for the study, "Existing fossil fuel extraction would warm the world beyond 1.5°C." Datasets 1-4 include mine-level data collected for China (Dataset 1), India (Dataset 2), and five other countries (Dataset 3) that are among the world's top nine coal producers - the United States, Indonesia, Australia, South Africa, and Poland. Dataset 4 includes global and country-level output data from the 1,000-run Monte Carlo simulation. <Committed_Reserves_Monte_Carlo_Input_Data.zip> includes data and code to replicate the Monte Carlo simulation.

opencc-by-4.0Jan 2019View details →

Code and data for "Current fossil fuel infrastructure does not yet commit us to 1.5°C warming"

This package generates all of the model runs and plotting code for "Current infrastructure does not yet commit us to 1.5°C warming". See enclosed README file for dependencies and how to run.

opencc-by-4.0Mar 2023View details →

Supplementary data and code to "An assessment of quaternary structure functionality in homomer protein complexes" by G. Abrusan and C. Foguet, https://doi.org/10.1093/molbev/msad070

Scripts and high-level data to reproduce the figures and supplementary figures of "An assessment of quaternary structure functionality in homomer protein complexes" by G. Abrusan and C. Foguet, https://doi.org/10.1093/molbev/msad070

dryad40/100

Data and code from: Spatial ecology of the Turks & Caicos boa, Chilabothrus c. chrysogaster Cope, 1871 (Serpentes: Boidae)

Open the record for dataset details and reuse information.

publicDec 2023View details →

opencc-by-4.0Aug 2023View details →

[Dataset and Code] Axis convergence in C. elegans embryos

Embryos develop in a surrounding that guides key aspects of their development. For example, the anteroposterior (AP) body axis is always aligned with the geometric long axis of the surrounding eggshell in fruit flies and worms. The mechanisms that ensure convergence of the AP axis with the long axis of the eggshell remain unresolved. We investigate axis convergence in early C. elegans development, where the nascent AP axis, when misaligned, actively re-aligns to converge with the long axis of the egg.  Here, we identify two physical mechanisms that underlie axis convergence.  First, bulk cytoplasmic flows, driven by actomyosin cortical flows, can directly reposition the AP axis. Second, active forces generated within the pseudocleavage furrow, a transient actomyosin structure similar to a contractile ring, can drive a mechanical re-orientation such that it becomes positioned perpendicular to the long axis of the egg. This in turn ensures AP axis convergence. Numerical simulations, together with experiments that either abolish the pseudocleavage furrow or change the shape of the egg, demonstrate that the pseudocleavage furrow-dependent mechanism is the major driver of axis convergence. We conclude that active force generation within the actomyosin cortical layer drives axis convergence in the early nematode.

opencc-by-4.0Nov 2023View details →

Data and code for replication of: "Global warming and heat extremes to exacerbate inflationary pressures. M. Kotz, F. Kuik, E. Liz, C. Nickel. Nature Communications Earth & Environment (2023)."

This repository contains secondary data and code necessary to reproduce the results of the manuscript:Global warming and heat extremes to exacerbate inflationary pressures.M. Kotz, F. Kuik, E. Liz, C. Nickel. Nature Communications Earth & Environment (2023). For further information please contact: maxkotz@pik-potsdam.de This document contains:1. An outline of the data included in the repository.2. An outline of the code included in the repository. See the README for further details. Credit and thanks go to Miles Parker, Chiara Osbat and Emanuele Franceschi for compiling the inflation data which is used in this study. Inflation data provided here has been anonymised (countries shuffled and names replaced by random letter combinations) to enable reproduction of our results, while limiting further use. Moreover, inflation in terms of the change in the logarithm of prices is included, whereas the level of price indices are excluded. For full inflation data please see the forthcoming publication by Miles Parker, Chiara Osbat,and Emanuele Franceschi (contact Miles.Parker@ecb.europa.eu for further enquiries into the raw inflation data).

opencc-by-4.0Mar 2024View details →

Genomics polymorphisms of Staphylococcus aureus strain NCTC 8325 in the lab stock maintained at TUM (WT), after 30 passes in BHI media (D) and after 30 passes detecting 4 -fold MIC increase to isocyanide -code I16- 3 biological replicates (A,B,C), and 3 independent colonies sequenced per replicate at the end of the experiment.

Genomics polymorphisms of Staphylococcus aureus strain NCTC 8325 in the lab stock maintained at TUM (WT), after 30 passes in BHI media (D) and after 30 passes detecting 4 -fold MIC increase to isocyanide -code I16-  3 biological replicates (A,B,C), and 3 independent colonies sequenced per replicate at the end of the experiment. Determined from Illumina shotgun genomic sequencing datasets, mapping and analyses vs the reference genome of the strain https://www.ncbi.nlm.nih.gov/nuccore/NC_007795.1/

opencc-by-4.0Sep 2022View details →

Third-order momentum advection on the quasi-hexagonal C-grid on the sphere: Data and source code

This upload contains data and source code accompanying the paper submitted to JAMES (Journal of advances in modeling Earth systems) under the title 'Third-order momentum advection on the quasi-hexagonal C-grid on the sphere' See README files for further details.

opencc-by-4.0Jun 2019View details →

Preprocessed C# Source Codes for Machine Learning

The dataset comes from the HackerRank site, 329,937 C# source codes of 22 tasks were collected and all verified by unit tests. During the download process, source codes received only a unique serial number instead of the user name who solved the task and stored inside the 'task_name/origin' folder. After collecting the data, a new database was created, which included cleaned-up versions of the source codes ('task_name/cleaned' folders contains). Finally, a third set of data was extracted from this cleaned-up version, where a delimiter was inserted before and after each elementary expression to support easy processing and analysis processes ('task_name/reduced' folders contains). Inside the 'task_name' folder three csv files, which contain the equality checking result. The compressed folder also contains a vector space (and related files) made from the reduced data set. These four files are directly in the main folder.

opencc-zeroAug 2021View details →

Codes and source data files for: Proximity labeling identifies LOTUS domain proteins that promote the formation of perinuclear germ granules in C. elegans

The germ line produces gametes that transmit genetic and epigenetic information to the next generation. Maintenance of germ cells and development of gametes require germ granules—well-conserved membraneless and RNA-rich organelles. The composition of germ granules is elusive owing to their dynamic nature and their exclusive expression in the germ line. Using C. elegans germ granule, called P granule, as a model system, we employed a proximity-based labeling method in combination with mass spectrometry to comprehensively define its protein components. This set of experiments identified over 200 proteins, many of which contain intrinsically disordered regions. An RNAi-based screen identified factors that are essential for P granule assembly, notably EGGD-1 and EGGD-2, two putative LOTUS-domain proteins. Loss of eggd-1 and eggd-2 results in separation of P granules from the nuclear envelope, germline atrophy and reduced fertility. We show that intrinsically disordered regions of EGGD-1 are required to anchor EGGD-1 to the nuclear periphery while its LOTUS domains are required to promote perinuclear localization of P granules. Together, our work expands the repertoire of P granule constituents and provides new insights into the role of LOTUS-domain proteins in germ granule organization.

opencc-zeroJan 2023View details →

History and practice of 8 U.S. Code §1103 notes, Sec. 102(c): Improvement of barriers at border

To expedite the construction of barriers along the U.S.-Mexico border, an extensive array of laws has been waived. This authority is codified in 8 U.S. Code §1103 notes, Sec. 102(c) - Improvement of Barriers at Border. In 1996 the Endangered Species Act of 1973 and the National Environmental Policy Act of 1969 were waived directly by Congress and in 2005 Congress delegated authority to the Secretary of Homeland Security to waive any law necessary with the REAL ID Act. Covering environmental protections, historic preservation, procurement procedures, religious freedoms, and more, waiver determinations facilitate the extensive and more rapid construction of border barriers. This dataset documents the history of this aspect of U.S. law as debated, as legislated, as proclaimed, and as litigated. This dataset is designed to accompany the following academic paper: Madsen, Kenneth D. 2022. Institutionalising the Exception: Homeland Security Section 102(c) Waivers and the Construction of Border Barriers. Geopolitics. https://doi.org/10.1080/14650045.2022.2126766

publicJan 2023View details →

History and practice of 8 U.S. Code §1103 notes, Sec. 102(c): Improvement of barriers at border

Open the record for dataset details and reuse information.