Skip to main content
Powered by ShareScore

Find research datasets worth reusing

Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.

1,899

datasets available to search

ShareScore release 0.9.0

Reset

Dataset results

1,899 results for “Running”

Learn how ShareScore rates datasets ↗
zenodo52/100

Gravity Spy Machine Learning Classifications of LIGO Glitches from Observing Runs O1, O2, O3a, and O3b

<p>This data set contains all classifications that the Gravity Spy Machine Learning model for LIGO glitches from the first three observing runs (<a href="https://doi.org/10.7935/K57P8W9D">O1</a>, <a href="https://doi.org/10.7935/CA75-FM95">O2</a> and O3, where O3 is split into <a href="https://doi.org/10.7935/nfnt-hm34">O3a</a> and <a href="https://doi.org/10.7935/pr1e-j706">O3b</a>). Gravity Spy classified all noise events identified by the <a href="https://doi.org/10.1016/j.softx.2020.100620">Omicron trigger pipeline</a> in which Omicron identified that the signal-to-noise ratio was above 7.5 and the peak frequency of the noise event was between 10 Hz and 2048 Hz. To classify noise events, Gravity Spy made <a href="https://en.wikipedia.org/wiki/Constant-Q_transform">Omega scans</a> of every glitch consisting of 4 different durations, which helps capture the morphology of noise events that are both short and long in duration.</p> <p>There are <a href="https://doi.org/10.1088/1361-6382/aa5cea">22 classes</a> used for O1 and O2 data (including No_Glitch and None_of_the_Above), while there are <a href="https://doi.org/10.1088/1361-6382/ac1ccb">two additional classes</a> used to classify O3 data (while None_of_the_Above was removed).</p> <p>For O1 and O2, the glitch classes were: 1080Lines, 1400Ripples, Air_Compressor, Blip, Chirp, Extremely_Loud, Helix, Koi_Fish, Light_Modulation, Low_Frequency_Burst, Low_Frequency_Lines, No_Glitch, None_of_the_Above, Paired_Doves, Power_Line, Repeating_Blips, Scattered_Light, Scratchy, Tomte, Violin_Mode, Wandering_Line, Whistle</p> <p>For O3, the glitch classes were: 1080Lines, 1400Ripples, Air_Compressor, Blip, <strong>Blip_Low_Frequency</strong>, Chirp, Extremely_Loud, <strong>Fast_Scattering</strong>, Helix, Koi_Fish, Light_Modulation, Low_Frequency_Burst, Low_Frequency_Lines, No_Glitch, None_of_the_Above, Paired_Doves, Power_Line, Repeating_Blips, Scattered_Light, Scratchy, Tomte, Violin_Mode, Wandering_Line, Whistle</p> <p>The data set is described in <a href="https://doi.org/10.1088/1361-6382/acb633"><strong>Glanzer </strong><em>et al</em><strong>. (2023)</strong></a>, which we ask to be cited in any publications using this data release. Example code using the data can be found in this <a href="https://colab.research.google.com/drive/19q_lItODPk7qw_sohlHyWPnAbY0FZyt8?usp=sharing"><strong>Colab notebook</strong></a>.</p> <p>If you would like to download the Omega scans associated with each glitch, then you can use the gravitational-wave data-analysis tool <a href="https://gwpy.github.io/docs/stable/">GWpy</a>. If you would like to use this tool, please install anaconda if you have not already and create a virtual environment using the following command</p> <pre><code class="language-bash">conda create --name gravityspy-py38 -c conda-forge python=3.8 gwpy pandas psycopg2 sqlalchemy</code></pre> <p>After downloading one of the CSV files for a specific era and interferometer, please run the following Python script if you would like to download the data associated with the metadata in the CSV file. We recommend not trying to download too many images at one time. For example, the script below will read data on Hanford glitches from O2 that were classified by Gravity Spy and filter for only glitches that were labelled as Blips with 90% confidence or higher, and then download the first 4 rows of the filtered table.</p> <pre><code class="language-python">from gwpy.table import GravitySpyTable H1_O2 = GravitySpyTable.read('H1_O2.csv') H1_O2[(H1_O2["ml_label"] == "Blip") &amp; (H1_O2["ml_confidence"] &gt; 0.9)] H1_O2[0:4].download(nproc=1)</code></pre> <p>Each of the columns in the CSV files are taken from various different inputs:&nbsp;</p> <p>[&lsquo;event_time&rsquo;, &lsquo;ifo&rsquo;, &lsquo;peak_time&rsquo;, &lsquo;peak_time_ns&rsquo;, &lsquo;start_time&rsquo;, &lsquo;start_time_ns&rsquo;, &lsquo;duration&rsquo;, &lsquo;peak_frequency&rsquo;, &lsquo;central_freq&rsquo;, &lsquo;bandwidth&rsquo;, &lsquo;channel&rsquo;, &lsquo;amplitude&rsquo;, &lsquo;snr&rsquo;, &lsquo;q_value&rsquo;] contain metadata about the signal from the <a href="https://virgo.docs.ligo.org/virgoapp/Omicron/">Omicron pipeline</a>.&nbsp;</p> <p>[&lsquo;gravityspy_id&rsquo;] is the unique identifier for each glitch in the dataset.&nbsp;</p> <p>[&lsquo;1400Ripples&rsquo;, &lsquo;1080Lines&rsquo;, &lsquo;Air_Compressor&rsquo;, &lsquo;Blip&rsquo;, &lsquo;Chirp&rsquo;, &lsquo;Extremely_Loud&rsquo;, &lsquo;Helix&rsquo;, &lsquo;Koi_Fish&rsquo;, &lsquo;Light_Modulation&rsquo;, &lsquo;Low_Frequency_Burst&rsquo;, &lsquo;Low_Frequency_Lines&rsquo;, &lsquo;No_Glitch&rsquo;, &lsquo;None_of_the_Above&rsquo;, &lsquo;Paired_Doves&rsquo;, &lsquo;Power_Line&rsquo;, &lsquo;Repeating_Blips&rsquo;, &lsquo;Scattered_Light&rsquo;, &lsquo;Scratchy&rsquo;, &lsquo;Tomte&rsquo;, &lsquo;Violin_Mode&rsquo;, &lsquo;Wandering_Line&rsquo;, &lsquo;Whistle&rsquo;] contain the machine learning confidence for a glitch being in a particular Gravity Spy class (the confidence in all these columns should sum to unity). These use the original 22 classes in all cases.</p> <p>[&lsquo;ml_label&rsquo;, &lsquo;ml_confidence&rsquo;] provide the machine-learning predicted label for each glitch, and the machine learning confidence in its classification.&nbsp;</p> <p>[&lsquo;url1&rsquo;, &lsquo;url2&rsquo;, &lsquo;url3&rsquo;, &lsquo;url4&rsquo;] are the links to the publicly-available <a href="https://gwdetchar.readthedocs.io/en/stable/omega/">Omega scans</a> for each glitch. &lsquo;url1&rsquo; shows the glitch for a duration of 0.5 seconds, &lsquo;url2&rsquo; for 1 seconds, &lsquo;url3&rsquo; for 2 seconds, and &lsquo;url4&rsquo; for 4 seconds.</p> <p>For the most recently uploaded training set used in Gravity Spy machine learning algorithms, please see <a href="https://zenodo.org/record/1486046#.YZfcar3MJqs">Gravity Spy Training Set</a> on Zenodo.&nbsp;</p> <p><br> For detailed information on the training set used for the original Gravity Spy machine learning paper, please see <a href="https://zenodo.org/record/1476156#.YZfchL3MJqs">Machine learning for Gravity Spy: Glitch classification and dataset</a> on Zenodo.</p>

opencc-by-4.0Nov 2021View details →
zenodo48/100

Background data: Untangling the effects of multiple human stressors and their impacts on fish assemblages in European running waters

<p>This dataset presents some backkground data from the EFI+ database. Related work addresses human stressors and their impacts on fish assemblages at pan-European scale by analysing single and multiple stressors and their interactions. Based on an extensive dataset with 3105 fish sampling sites, patterns of stressors, their combination and nature of interactions, i.e. synergistic, antagonistic and additive were investigated. </p> <p>Data were derived within the EU-project "Improvement and Spatial extension of the European Fish Index (EFI+)". EFI+, an EU FP6 research project from 2007-2009 was designed to gain new knowledge and to further develop and improve new biological assessment methods to meet needs of the Water Framework Directive (WFD). </p> <p>Background data are available for boxplots and barplots shown in the related research article in STOTEN.</p>

opencc-by-nc-nd-4.0May 2017View details →
zenodo48/100

Preindustrial Control (PIC) Run for OSU-UVic2.9.10 (MOBI2.2) Input Data

<p>Input data required for a simulation of the preindustrial control (PIC) simulation with the OSU version of the University of Victoria climate model (version 2.9) with the Model of Ocean Biogeochemistry and Isotopes (MOBI2.2).</p>

opencc-by-4.0May 2024View details →
zenodo48/100

Last Glacial Maximum (LGM) Run for UVic2.9.10 (MOBI2.2) Input Data

<p>Input data required for a simulation of the Last Glacial Maximum (LGM) simulation with the OSU version of the University of Victoria climate model (version 2.9.10) with the Model of Ocean Biogeochemistry and Isotopes (MOBI2.2).</p>

opencc-by-4.0May 2024View details →
zenodo48/100

Run-to-failure data set of ball bearings subjected to time-varying load and speed conditions

<p>This data set consist of experimental data collected during 17 run-to-failure experiments on ball bearings subjected to time-varying load and speed conditions. No defect was initiated in the bearings before the experiments. A detailed description file is enclosed.</p> <p>Version 2024-04-02: All experiments B01 through B17 are uploaded. Furthermore, Figure 1 of the description file has been updated.</p> <p>Please also cite our paper, when using this data set: Javanmardi, A., Aimiyekagbon, O. K., Bender, A. ., Kimotho, J. K., Sextro, W., &amp; H&uuml;llermeier, E. (2024). Remaining Useful Lifetime Estimation of Bearings Operating under Time-Varying Conditions. <em>PHM Society European Conference</em>, <em>8</em>(1), 9. https://doi.org/10.36001/phme.2024.v8i1.4101</p>

opencc-by-4.0Mar 2024View details →
zenodo48/100

A high-resolution, multi-decadal, free-running, hydrodynamic simulation of the East Australia Current System using the Regional Ocean Modeling System (Version 3.0, 1994-2019)

<p>The data is from a Regional Ocean Modelling System free-running, hydrodynamic simulation of the East Australian Current System. The model has a horizontal resolution of 2.5-6 km in the cross-shore direction and 5 km in the alongshore direction, and 30 vertical s-levels. The model domain covers the southeastern Australia oceanic region from 25.1-41.5&deg;S and 147.1-162.2&deg;E, and the grid is orientated 20 degrees clockwise to be predominantly orientated alongshore. The time period covered is 02 Jan 1994 to 28 Feb 2019. The model outputs provided are daily averages of the following variables: Two-dimensional variables: Sea surface height (zeta), barotropic cross-grid velocity (u) and barotropic along-grid velocity (v). Three-dimensional variables: Temperature (temp), salinity (salt), density (rho), cross-grid velocity (u), along-grid velocity (v) and vertical velocity (w), temperature time rate of change (temp_rate), temperature horizontal advection term (temp_hadv), temperature vertical advection term (temp_vadv), temperature horizontal diffusion term (temp_hdiff), temperature vertical diffusion term (temp_vdiff). In this version, the heat budget terms (temp_rate, temp_hadv, temp_vadv, temp_hdiff and temp_vdiff) are set to be zeros on the land.</p> <p>&nbsp;</p> <p>This model is part of the <a href="../records/8294716"><strong>South East Australian Coastal Ocean Forecast System (SEA-COFS)</strong></a> suite of models.</p>

opencc-by-4.0Jul 2024View details →
zenodo48/100

Raw Data of Pilot Plant Runs for CONSENS Project (Case Study 1)

<p>In&nbsp;case study one of the CONSENS project, two aromatic substances were coupled by a lithiation reaction, which is a prominent example in pharmaceutical industry. The two aromatic reactants (Aniline and <em>o</em>-FNB) were mixed with Lithium-base (LiHMDS)&nbsp;in a continuous modular plant&nbsp;to produce the desired product (Li-NDPA) and a salt (LiF). The salt precipitates which leads to the formation of particles. The feed streams were&nbsp;subject to&nbsp;variation to drive the plant to its optimum.&nbsp;</p> <p>The uploaded data comprises the results from four days during continuous plant operation time. Each day is denoted from day 1-4 and represents the dates 2017-09-26,&nbsp;2017-09-28,&nbsp;2017-10-10,&nbsp;2017-10-17.</p> <p>In the following the contents of the files are explained.</p> <p><strong>NIR_data_AQ15_raw.zip: </strong>Contains Bruker binary files (0-Files)&nbsp;of NIR spectrometer at AQ15 (Location is located subsequently to NMR spectrometer)</p> <p><strong>NMR_spectra_raw.zip:&nbsp;</strong>Contains Spinsolve files (Binarys of&nbsp;FID and Spectrum, DX-Files) of NMR spectrometer. The use of DX-files files is not recommended.</p> <p><strong>PCS_data_csv.zip</strong>: Contains csv-files of the process control system (PCS) including data of mass flow controlers (*_Bilanz.csv),&nbsp;filling level (*_FillLe.csv), pressures (*_pres), temperatures (*_Temp), position of valves (*_Valves). Relevant labels are: BP13 = LiHMDS&nbsp;storage tank, BP12 = aniline storage tank, BP12 = <em>o</em>-FNB storage tank, CM003 and CM004 = tubular reactors, T0041 and T005 = Temperature at reactor exits, P009 and P003 = Pressure at reactor inlets, P006 = Pressure at reactor exits.</p> <p><strong>housing_data_NMR.csv</strong>: Contrains data of NMR enclosure of all four days. Each columns from left to right represent timestamps, bypass pressure (bar), bypass temperature (&deg;C), Gasalarm (logical), bypass actual flowrate (g&nbsp;min<sup>-1</sup>),&nbsp; bypass flowrate setpoint (g&nbsp;min<sup>-1</sup>), bypass density (kg&nbsp;m<sup>3</sup>)</p> <p><strong>matlab_variables_explanation.xlsx</strong>: Explanation of variables used in matlab structure &quot;data_validation_run&quot;.</p> <p><strong>data_validation_run.mat</strong>: Matlab structure containing most relevant process data including NMR results, NIR results, housing data of NMR, and process control system data.</p>

opencc-by-4.0Sep 2018View details →
zenodo48/100

Atmospheric climate model output of the COSMO-CLM2 regional climate model hindcast run over Antarctica (1987-2016)

<p>The dataset contains monthly output of a&nbsp;COSMO-CLM&sup2; (COSMO-CLM coupled to the Community Land Model) atmospheric hindcast simulation over Antarctica which is described&nbsp;and evaluated in the following paper:&nbsp;</p> <p>Souverijns, N., Gossart, A., Demuzere, M., Lenaerts, J.T.M., Medley, B., Gorodetskaya, I.V., Vanden Broucke, S., van Lipzig, N.P.M., 2019. A new Regional Climate Model for POLAR-CORDEX: Evaluation of a 30-year Hindcast with COSMO-CLM&sup2; over Antarctica. Journal of Geophysical Research: Atmospheres, 124, 1405-1427. (doi:10.1029/2018JD028862)</p> <p>Details of the model simulation:<br> - COSMO-CLM version&nbsp;5.0_clm6<br> - Community Land Model version 4.5<br> - Horizontal resolution: 0.25&deg;x0.25&deg;<br> - Vertical resolution: 40 levels<br> - Time period: 1987-2016 (excluding&nbsp;4 years of spin-up)<br> - Driving model: ERA-Interim<br> &nbsp;</p> <p>The data provided here has a monthly time resolution and contains the monthly average of all variables except denoted otherwise below. As such, each file consists of 360 time steps.<br> - AEVAP_S: Surface evaporation [kg m-2] (summed value for each month)<br> - ALB: Surface albedo [-] (only for austral summer months)<br> - ALHFL_S: Surface latent heat flux [W m-2]<br> - ALWD_S: Downward longwave radiation at the surface [W m-2]<br> - ALWU_S: Upward longwave radiation at the surface [W m-2]<br> - ASHFL_S: Surface sensible heat flux [W m-2]<br> - ASOB_S: Surface net downward shortwave radiation [W m-2]<br> - ASWDIFD_S: Diffuse downward shortwave radiation at the surface [W m-2]<br> - ASWDIFU_S: Diffuse upward shortwave radiation at the surface [W m-2]<br> - ASWDIR_S: Direct downward shortwave radiation at the surface [W m-2]<br> - ATHB_S: Surface net downward longwave radiation at the surface [W m-2]<br> - P: Pressure at 40 vertical levels [Pa]<br> - QV: Specific humidity at 40 vertical levels [kg kg-1]<br> - RH2M: Relative humidity at 2 meter [%]<br> - SNOW_GSP: Surface snowfall amount [kg m-2]&nbsp;(summed value for each month)<br> - T2M: Temperature at 2 meter [K]<br> - T: Temperature at 40 vertical levels [K]<br> - WS10M: Wind speed at 10 meter [m s-1]<br> - WS: Wind speed at 40 vertical levels [m s-1]</p>

opencc-by-4.0Jan 2019View details →
zenodo48/100

Masks for ISIMIP3 Agriculture (GGCMI phase 3) model runs

<p>This dataset consists of a netCDF file with a number of layers at half-degree global resolution. Each layer is a binary map representing whether each gridcell is included (1 if yes, 0 if no) in one or more input datasets used in the ISIMIP3 Agriculture (GGCMI phase 3) model runs. Individual-dataset masks:</p> <ul> <li>has_soil&nbsp;indicates&nbsp;inclusion in the&nbsp;<a href="https://data.isimip.org/10.48364/ISIMIP.942125">ISIMIP3 soil input dataset (Volkholz &amp; M&uuml;ller, 2020)</a>&nbsp;(ignoring &quot;gravel,&quot; which has some missing cells).</li> <li>has_cropcals indicates inclusion in the <a href="https://zenodo.org/record/5062513">J&auml;germeyr et al. (publication in prep.)</a>&nbsp;crop calendar dataset (ignoring second-season rice, which is not grown in all gridcells).</li> <li>has_lu indicates inclusion in all 15 area maps in the historical land use area dataset <a href="https://protocol.isimip.org/protocol/ISIMIP3b/index.html#socioeconomic-forcing">prepared for ISIMIP3</a> (landuse-totals_histsoc_annual_1850_2014.nc).</li> <li>has_crops indicates inclusion in all 15 area maps in the historical 15-crop dataset <a href="https://protocol.isimip.org/protocol/ISIMIP3b/index.html#socioeconomic-forcing">prepared for ISIMIP3</a> (landuse-15crops_histsoc_annual_1850_2014.nc). Note that there are two gridcells that are missing from this&nbsp;</li> <li>has_fertilizer indicates inclusion in every fertilizer_application_histsoc*.nc file in the&nbsp;<a href="http://doi.org/10.5281/zenodo.4954582">fertilizer and manure dataset prepared by Heinke et al. (2021) for GGCMI3</a>.</li> <li>has_all is a composite mask indicating inclusion in all of the above.</li> </ul> <p>Also included are a figure showing the masks and the MATLAB script used to generate the data and figure.</p> <ul> </ul>

opencc-by-4.0Jun 2021View details →
zenodo48/100

Multivariate Time Series data of Fatigued and Non-Fatigued Running from Inertial Measurement Units

<p>The data captured came from mounting a single Shimmer3&nbsp;IMU on the lumbar of 19 recreational runners. The participants were all regular runners and injury free. The study protocol was reviewed and approved by the human research ethics committee at University College Dublin.<br><br>The data was collected in three segments; in the first, the participant completed a 400m run at a comfortable pace; the second segment consisted of a beep test which acted as the fatiguing protocol for this study; and the last segment where the runner was required to complete the 400m run at their comfortable pace, this time in their fatigued state. The beep test requires the runner to continuously run between two points 20m apart following an audio which produces `beeps' indicating when the person should begin running from one end to the other. The test eventually requires the runner to increase their pace as the interval between the `beeps' reduces as the test progresses. The fatiguing protocol ends when the runner is unable to keep up the increase in pace. The runs were all done on an outdoor running track. The sensor captured acceleration, angular velocity and magnetometer data throughout the three stages of the trials at a sampling rate of 256Hz. The data included here are segmented strides from the two 400m runs of&nbsp;each of the 19 participants. The labels on the data represent the participant number and whether it was a fatigued stride ('F') or a not fatigued stride ('NF').<br>The data used from the sensors includes data from the accelerometer in three directions (X, Y, Z) and the gyroscope in three directions (X, Y, Z). The direction of each of the axis is relative to the sensor. Two extra signals, magnitude acceleration and magnitude gyroscope were derived from the component signals and included in the analysis.</p><p>Kindly cite one of the following papers when using this data:</p><p>B. Kathirgamanathan, B. Caulfield and P. Cunningham, "Towards Globalised Models for Exercise Classification using Inertial Measurement Units," 2023 IEEE 19th International Conference on Body Sensor Networks (BSN), Boston, MA, USA, 2023, pp. 1–4, doi: 10.1109/BSN58485.2023.10331612</p><p>B. Kathirgamanathan, T. Nguyen, G. Ifrim, B. Caulfield, P. Cunningham. Explaining Fatigue in Runners using Time Series Analysis on Wearable Sensor Data, XKDD 2023: 5th International Workshop on eXplainable Knowledge Discovery in Data Mining, ECML PKDD, 2023, <a href="http://xkdd2023.isti.cnr.it/papers/223.pdf">http://xkdd2023.isti.cnr.it/papers/223.pdf</a></p>

opencc-by-4.0Jun 2023View details →
zenodo48/100

Large Ensemble Dataset for Discovering Global Peak Water Limit of Future Groundwater Withdrawals Using 900 GCAM Runs

<h2><strong>Global Groundwater Withdrawals Peak&nbsp;Over the 21st Century&nbsp;</strong></h2> <p>The large ensemble dataset contains groundwater related model outputs from 900 scenarios modeled using <a href="http://jgcri.github.io/gcam-doc/toc.html">Global Change Analysis Model (GCAM)</a>. The scenario ensemble&nbsp;members include five Shared Socioeconomic Pathways (SSPs), four Representative Concentration Pathways (RCPs), five global climate model outputs, three groundwater depletion limits, two surface water storage expansion regimes, and two historical groundwater depletion trends.</p> <h3><strong>Journal Article</strong></h3> <p>Niazi, H., Wild, T.B., Turner, S.W.D., Graham, N.T., Hejazi, M., Msangi, S., Kim, S., Lamontagne, J.R., &amp; Zhao, M. (2024).&nbsp;<a href="https://rdcu.be/dFpb5">Global peak water limit of future groundwater withdrawals</a>.&nbsp;<em>Nature Sustainability, 7</em>(4), 413&ndash;422.&nbsp;<a href="https://doi.org/10.1038/s41893-024-01306-w" rel="nofollow">https://doi.org/10.1038/s41893-024-01306-w</a></p> <p>Read full-text here: <a href="https://rdcu.be/dFpb5">https://rdcu.be/dFpb5</a>&nbsp;</p> <h3><strong>Data Repository&nbsp;</strong></h3> <p>This <em><strong>data</strong></em> repository is to be used in combination with the&nbsp;<em><strong>main</strong></em>&nbsp;<a href="https://github.com/JGCRI/niazi-etal_2024_nature-sustainability">meta-repository</a> containing all scripts and files for reproducing the experiment as well as the analysis and post-processing of the model outputs.</p> <p>Scripts and smaller files are provided in the <a href="https://github.com/JGCRI/niazi-etal_202X_xyz">GitHub meta-repository</a> whereas larger files are provided in this data repository. Please complete the repository by placing the files as described hereunder. Please find the GitHub meta-repository here: <a href="https://github.com/JGCRI/niazi-etal_2024_nature-sustainability">https://github.com/JGCRI/niazi-etal_2024_nature-sustainability</a></p> <p>Descriptions of files:</p> <ol> <li><em><strong>gcam-5.7z</strong></em> contains the GCAM version used to simulate&nbsp;900 scenarios of plausible futures. The model folder contains all necessary input files to reproduce the simulations. <ul> <li>The model is to be used in combination with the <a href="https://github.com/JGCRI/niazi-etal_2024_nature-sustainability">meta-repository</a>&nbsp;to setup batch runs on cluster.</li> <li>Please navigate to <a href="https://github.com/JGCRI/niazi-etal_202X_xyz/tree/main/model">model/</a> folder for&nbsp;other scenario-specific and model setup folders and files. <em><strong>gcam-5</strong></em>&nbsp;is to be extracted in the same directory (./<em>model/gcam-5/</em>).&nbsp;</li> <li>For the first-time users of GCAM, please follow&nbsp;guidance on <a href="http://jgcri.github.io/gcam-doc/toc.html">GCAM wiki</a>&nbsp;to setup GCAM or for background knowledge.&nbsp;</li> </ul> </li> <li><em><strong>crop_yeild.7z</strong></em>: This file contains inputs related to&nbsp;climate impacts on crop yields. This is to be downloaded and extracted&nbsp;in the&nbsp;<a href="https://github.com/JGCRI/niazi-etal_202X_xyz/tree/main/model/combined_impacts">model/combined_impacts/</a>&nbsp;folder.&nbsp;</li> <li><em><strong>outputs-all.7z: </strong></em>Key model outputs queried and collated from 900 GCAM runs are explained hereunder.&nbsp;The files could be downloaded individually (.csv&nbsp;files)&nbsp;or all at once in .7z format (<a href="../api/files/80b237d3-b22f-499f-8b8e-76c3846720a0/outputs-all.7z">outputs-all.7z</a>). These files are to be placed in the <a href="https://github.com/JGCRI/niazi-etal_202X_xyz/tree/main/model/outputs">model/outputs</a>&nbsp;folder of the <a href="https://github.com/JGCRI/niazi-etal_2024_nature-sustainability">meta-repository</a>.&nbsp; <ul> <li><em><strong>ag_prod_all_GW_scenarios.csv</strong></em>&nbsp;- Agricultural production across all scenario for 2050 and 2100 (tonnes)</li> <li><em><strong>prices_water_withdrawal_all.csv</strong> -&nbsp;</em>Water prices across all scenarios and years ($/km<sup>3</sup>)</li> <li><em><strong>global_irrigated_prod_by_crop.csv</strong></em>&nbsp;-&nbsp;All irrigated agricultural production for each crop across and scenarios all years (tonnes)</li> <li><em><strong>surface_water_production_all.csv</strong></em>&nbsp;- Runoff across all scenarios and years (km<sup>3</sup>)</li> <li><em><strong>groundwater_production_FINAL.csv</strong></em>&nbsp;- Groundwater withdrawals across all scenarios and years (km<sup>3</sup>)</li> <li><em><strong>water_withdrawals_desal_all.csv</strong></em>&nbsp;- Water withdrawals from desalination plants across all scenarios and years (km<sup>3</sup>)</li> </ul> </li> </ol> <h3><strong>Short introduction to the study</strong></h3> <p>Using 900 GCAM runs, this study finds that global groundwater withdrawals are expected to peak around mid-century, followed by a decline through 21st century, exposing about half of the population living in one-third of basins to groundwater stress, with cost and availability of surface water storage being the most significant driver of future groundwater withdrawals. This first-ever robust, quantitative confirmation of the peak-and-decline pattern for groundwater, previously only known for fossil fuels and minerals, raises concerns for basins heavily dependent on groundwater.</p> <p>Niazi, H., Wild, T.B., Turner, S.W.D., Graham, N.T., Hejazi, M., Msangi, S., Kim, S., Lamontagne, J.R., &amp; Zhao, M. (2024).&nbsp;<a href="https://rdcu.be/dFpb5">Global peak water limit of future groundwater withdrawals</a>.&nbsp;<em>Nature Sustainability, 7</em>(4), 413&ndash;422.&nbsp;<a href="https://doi.org/10.1038/s41893-024-01306-w" rel="nofollow">https://doi.org/10.1038/s41893-024-01306-w</a></p> <p>Read full-text here: <a href="https://rdcu.be/dFpb5">https://rdcu.be/dFpb5</a></p> <h3><strong>Contact&nbsp;</strong></h3> <p>Please reach out to Hassan Niazi at&nbsp;<a href="mailto:hassan.niazi@pnnl.gov">hassan.niazi@pnnl.gov</a> for any questions.&nbsp;</p>

opencc-by-4.0Apr 2022View details →
edi48/100

Monitoring adult spring-run Chinook salmon throughout the adult lifespan on Deer and Mill Creek

The California Department of Fish and Wildlife (CDFW) collects data on adult salmonids on Deer Creek and Mill Creek. Data is collected annually via redd surveys on Mill Creek and holding surveys on Deer Creek; video camera systems on both creeks collect data on upstream passage 24 hours a day, 7 days a week from February to August. Data from this monitoring is used to estimate adult escapement (upstream passage) abundance and timing, spawner abundance, and other important metrics for adult salmonids in the watershed. These data will also be used to inform the development of a juvenile production estimate (JPE) for spring-run Chinook salmon in the Sacramento River Watershed as required by Condition of Approval 7.5.2 of Incidental Take Permit No. 2081-2019-006-00 (ITP) issued by CDFW to California Department of Water Resources (DWR) for the long-term operation of the State Water Project.

openCC (other)Jun 2024View details →
edi48/100

Limno run codes, dates, and locations associated with vertical lake profile data collected in the McMurdo Dry Valleys, Antarctica (1991-2025, ongoing)

This data package provides a summary of "limno runs" performed each season in lakes located throughout the McMurdo Dry Valleys of Antarctica as part of the McMurdo Dry Valleys Long Term Ecological Research program. Each limno run is identified by unique code that allows one to compile a complete set of limnological data from a particular lake and location at specific time points. There are usually two or three limno runs performed per lake per austral summer field season, although the number of runs may vary by lake and by season depending on site access, site conditions, and other external factors.

openCC (other)Dec 2025View details →
zenodo44/100

Hydraulic scale model experiments on the two-dimensional run-up of impulse wave trains on steep to vertical slopes

<p>This dataset includes the experimental data and videos, which were generated during the study on the run-up of impulse wave trains at the Laboratory of Hydraulics, Hydrology and Glaciology (VAW), ETH Zurich.</p>

opencc-by-4.0Jan 2020View details →
zenodo44/100

MAIC-2 simulation run_t39

<p>Visualization of the time-dependent H<sub>2</sub>O glaciation of Mars, simulated by MAIC-2 experiment run_t39 by&nbsp;<a href="https://doi.org/10.5281/zenodo.3698542">Greve et al. (2012)</a> [only the part from 7.5 million years ago until today shown]. North up, vertical exaggeration factor 750. Bottom panel: Obliquity over time.</p>

opencc-by-4.0May 2012View details →
zenodo44/100

Test data for running snakePipes : ATAC-seq workflow

<p><strong>Test files for running snakePipes workflows</strong></p> <p><strong>snakePipes</strong> are pipelines built using snakemake and python for the analysis of epigenomic datasets. Please refer to <a href="https://snakepipes.readthedocs.io/en/latest/">this link</a>&nbsp;for further information on snakePipes.</p> <p>This folder contains test files that can be used to run the ATAC-seq workflow under snakePipes. To test the workflow, follow the following steps :&nbsp;</p> <ul> <li>Download or prepare genome fasta, indices and annotations for fruit fly (<strong>dm6</strong>) genome.</li> <li>Download and install snakePipes via `conda create -n snakePipes -c mpi-ie -c bioconda -c conda-forge snakePipes`</li> <li>Update <a href="https://snakepipes.readthedocs.io/en/latest/content/running_snakePipes.html#genome-configuration-file">Genome configuration file</a>&nbsp;with path to indices and annotations.</li> <li>Move to this repository and run the example <strong>command.sh</strong></li> </ul>

opencc-by-4.0Mar 2020View details →
zenodo44/100

Test data for running snakePipes : mRNA-seq workflow

<p><strong>Test files for running snakePipes workflows</strong></p> <p><strong>snakePipes</strong> are pipelines built using snakemake and python for the analysis of epigenomic datasets. Please refer to <a href="https://snakepipes.readthedocs.io/en/latest/">this link</a>&nbsp;for further information on snakePipes.</p> <p>This folder contains test files that can be used to run the mRNA-seq workflow under snakePipes. To test the workflow, follow the following steps :&nbsp;</p> <ul> <li>Download or prepare genome fasta, indices and annotations for mouse (<strong>GRCm38</strong>) genome.</li> <li>Download and install snakePipes via `conda create -n snakePipes -c mpi-ie -c bioconda -c conda-forge snakePipes`</li> <li>Update <a href="https://snakepipes.readthedocs.io/en/latest/content/running_snakePipes.html#genome-configuration-file">Genome configuration file</a>&nbsp;with path to indices and annotations.</li> <li>Move to this repository and run the example <strong>command.sh</strong></li> </ul>

opencc-by-4.0Mar 2020View details →
zenodo44/100

Block-wise sparse matrix-vector product dataset and convolutional neural nets for estimating the run time and energy consumption of the sparse matrix-vector product

<p><strong>Introduction</strong></p> <p><strong>SpMV-CNN</strong> is a set of Convolutional Neural Networks (CNNs) that provide&nbsp;accurate estimations of the performance and energy consumption of the SpMV kernel. The proposed CNN-based models use a block-wise approach to make the CNN&nbsp;architecture independent of the matrix size. These models cat be trained to estimate run time as well as total, package and DRAM energy consumption at different processor frequencies.</p> <p><strong>Prerequisites</strong></p> <p><strong>SpMV-CNN</strong> requires Python3 with the following packages:</p> <pre><code>keras==2.1.6 tensorflow==1.8.0 h5py==2.7.1 matplotlib==2.1.1 scikit-learn==0.19.1 </code></pre> <p><strong>Obtaining the dataset</strong></p> <p>The execution time and energy consumption data corresponding to the SpMV&nbsp;operation on a set of sparse matrices from the SuiteSparse Matrix Collection have been obtained on an Intel Xeon E5-2630 core running at frequencies 1.2,&nbsp;1.6, 2.0, 2.4 GHz. The energy consumption measurements are obtained via the Intel RAPL interface and gathered at three different levels (total, package and DRAM, where total = package + DRAM) for this specific processor.</p> <p>The <code>spmv-cnn-dataset.tgz</code> archive contains the whole dataset, including&nbsp;the following HDF5 files:</p> <pre><code>$ tree . |-- test | |-- f_1200000_b250 | | |-- output_2cubes_sphere_1200000.h5 | | |-- output_apache2_1200000.h5 | | |-- output_bcsstk36_1200000.h5 | | |-- output_cfd1_1200000.h5 | | |-- output_cfd2_1200000.h5 | | |-- output_ct20stif_1200000.h5 | | |-- output_denormal_1200000.h5 | | |-- output_Dubcova2_1200000.h5 | | |-- output_Dubcova3_1200000.h5 | | |-- output_ecology2_1200000.h5 | | |-- output_gyro_1200000.h5 | | |-- output_gyro_k_1200000.h5 | | |-- output_msc10848_1200000.h5 | | |-- output_msc23052_1200000.h5 | | |-- output_nasasrb_1200000.h5 | | |-- output_nd3k_1200000.h5 | | |-- output_offshore_1200000.h5 | | |-- output_oilpan_1200000.h5 | | |-- output_olafu_1200000.h5 | | |-- output_parabolic_fem_1200000.h5 | | |-- output_qa8fm_1200000.h5 | | |-- output_raefsky4_1200000.h5 | | |-- output_s3dkq4m2_1200000.h5 | | |-- output_s3dkt3m2_1200000.h5 | | |-- output_ship_001_1200000.h5 | | |-- output_ship_003_1200000.h5 | | |-- output_shipsec1_1200000.h5 | | |-- output_shipsec5_1200000.h5 | | |-- output_shipsec8_1200000.h5 | | |-- output_smt_1200000.h5 | | |-- output_thermomech_dM_1200000.h5 | | |-- output_thread_1200000.h5 | | `-- output_vanbody_1200000.h5 | |-- f_1600000_b250 | | |-- output_2cubes_sphere_1600000.h5 | | |—- ... | | `-- output_vanbody_1600000.h5 | |-- f_2000000_b250 | | |-- output_2cubes_sphere_2000000.h5 | | |—- ... | | `-- output_vanbody_2000000.h5 | `-- f_2400000_b250 | |-- output_2cubes_sphere_2400000.h5 | |—- ... | `-- output_vanbody_2400000.h5 |-- test_pagerank | |-- f_1200000_b250 | | |-- output_adaptive_1200000.h5 | | |-- output_cit-HepPh_1200000.h5 | | |-- output_delaunay_n22_1200000.h5 | | |-- output_email-Enron_1200000.h5 | | |-- output_email-EuAll_1200000.h5 | | |-- output_europe_osm_1200000.h5 | | |-- output_hugebubbles-00020_1200000.h5 | | |-- output_rgg_n_2_24_s0_1200000.h5 | | |-- output_road_usa_1200000.h5 | | |-- output_Stanford_1200000.h5 | | |-- output_wb-edu_1200000.h5 | | |-- output_web-BerkStan_1200000.h5 | | |-- output_web-Google_1200000.h5 | | |-- output_web-NotreDame_1200000.h5 | | |-- output_wiki-Talk_1200000.h5 | | `-- output_wiki-Vote_1200000.h5 | |-- f_1600000_b250 | | |-- output_adaptive_1600000.h5 | | |—- ... | | `-- output_wiki-Vote_1600000.h5 | |-- f_2000000_b250 | | |-- output_adaptive_2000000.h5 | | |—- ... | | `-- output_wiki-Vote_2000000.h5 | `-- f_2400000_b250 | |-- output_adaptive_2400000.h5 | |—- ... | `-- output_wiki-Vote_2400000.h5 `-- train |-- merged_energy_train_shuffle_f1200000_250.h5 |-- merged_energy_train_shuffle_f1600000_250.h5 |-- merged_energy_train_shuffle_f2000000_250.h5 `-- merged_energy_train_shuffle_f2400000_250.h5 </code></pre> <p>The matrices contained in the merged training files (<code>merged_energy_train_shuffle_fXX00000_250.h5</code>) are the following:</p> <pre><code>$ tree . |-- output_af_0_k101_1200000.h5 |-- output_af_1_k101_1200000.h5 |-- output_af_2_k101_1200000.h5 |-- output_af_3_k101_1200000.h5 |-- output_af_4_k101_1200000.h5 |-- output_af_5_k101_1200000.h5 |-- output_af_shell10_1200000.h5 |-- output_af_shell1_1200000.h5 |-- output_af_shell2_1200000.h5 |-- output_af_shell3_1200000.h5 |-- output_af_shell4_1200000.h5 |-- output_af_shell5_1200000.h5 |-- output_af_shell6_1200000.h5 |-- output_af_shell7_1200000.h5 |-- output_af_shell8_1200000.h5 |-- output_af_shell9_1200000.h5 |-- output_atmosmodd_1200000.h5 |-- output_atmosmodj_1200000.h5 |-- output_atmosmodl_1200000.h5 |-- output_audikw_1_1200000.h5 |-- output_BenElechi1_1200000.h5 |-- output_bmw3_2_1200000.h5 |-- output_bmw7st_1_1200000.h5 |-- output_bmwcra_1_1200000.h5 |-- output_bone010_1200000.h5 |-- output_boneS01_1200000.h5 |-- output_boneS10_1200000.h5 |-- output_bundle_adj_1200000.h5 |-- output_cage14_1200000.h5 |-- output_cage15_1200000.h5 |-- output_circuit5M_1200000.h5 |-- output_circuit5M_dc_1200000.h5 |-- output_CO_1200000.h5 |-- output_consph_1200000.h5 |-- output_CoupCons3D_1200000.h5 |-- output_crankseg_1_1200000.h5 |-- output_crankseg_2_1200000.h5 |-- output_CurlCurl_2_1200000.h5 |-- output_CurlCurl_3_1200000.h5 |-- output_CurlCurl_4_1200000.h5 |-- output_dielFilterV2real_1200000.h5 |-- output_dielFilterV3real_1200000.h5 |-- output_Emilia_923_1200000.h5 |-- output_ESOC_1200000.h5 |-- output_F1_1200000.h5 |-- output_F2_1200000.h5 |-- output_Fault_639_1200000.h5 |-- output_Freescale1_1200000.h5 |-- output_Freescale2_1200000.h5 |-- output_FullChip_1200000.h5 |-- output_G3_circuit_1200000.h5 |-- output_Ga10As10H30_1200000.h5 |-- output_Ga19As19H42_1200000.h5 |-- output_Ga3As3H12_1200000.h5 |-- output_Ga41As41H72_1200000.h5 |-- output_Ge87H76_1200000.h5 |-- output_Ge99H100_1200000.h5 |-- output_Geo_1438_1200000.h5 |-- output_gsm_106857_1200000.h5 |-- output_Hardesty3_1200000.h5 |-- output_hood_1200000.h5 |-- output_Hook_1498_1200000.h5 |-- output_human_gene1_1200000.h5 |-- output_human_gene2_1200000.h5 |-- output_inline_1_1200000.h5 |-- output_JP_1200000.h5 |-- output_kkt_power_1200000.h5 |-- output_ldoor_1200000.h5 |-- output_Long_Coup_dt0_1200000.h5 |-- output_Long_Coup_dt6_1200000.h5 |-- output_mat_104_10000_1200000.h5 |-- output_mat_104_1000_1200000.h5 |-- output_mat_104_5000_1200000.h5 |-- output_mat_112_10000_1200000.h5 |-- output_mat_112_1000_1200000.h5 |-- output_mat_112_5000_1200000.h5 |-- output_mat_120_10000_1200000.h5 |-- output_mat_120_1000_1200000.h5 |-- output_mat_120_5000_1200000.h5 |-- output_mat_128_10000_1200000.h5 |-- output_mat_128_1000_1200000.h5 |-- output_mat_128_5000_1200000.h5 |-- output_mat_16_10000_1200000.h5 |-- output_mat_16_1000_1200000.h5 |-- output_mat_16_5000_1200000.h5 |-- output_mat_24_10000_1200000.h5 |-- output_mat_24_1000_1200000.h5 |-- output_mat_24_5000_1200000.h5 |-- output_mat_32_10000_1200000.h5 |-- output_mat_32_1000_1200000.h5 |-- output_mat_32_5000_1200000.h5 |-- output_mat_40_10000_1200000.h5 |-- output_mat_40_1000_1200000.h5 |-- output_mat_40_5000_1200000.h5 |-- output_mat_48_10000_1200000.h5 |-- output_mat_48_1000_1200000.h5 |-- output_mat_48_5000_1200000.h5 |-- output_mat_56_10000_1200000.h5 |-- output_mat_56_1000_1200000.h5 |-- output_mat_56_5000_1200000.h5 |-- output_mat_64_10000_1200000.h5 |-- output_mat_64_1000_1200000.h5 |-- output_mat_64_5000_1200000.h5 |-- output_mat_72_10000_1200000.h5 |-- output_mat_72_1000_1200000.h5 |-- output_mat_72_5000_1200000.h5 |-- output_mat_80_10000_1200000.h5 |-- output_mat_80_1000_1200000.h5 |-- output_mat_80_5000_1200000.h5 |-- output_mat_8_10000_1200000.h5 |-- output_mat_8_1000_1200000.h5 |-- output_mat_8_5000_1200000.h5 |-- output_mat_88_10000_1200000.h5 |-- output_mat_88_1000_1200000.h5 |-- output_mat_88_5000_1200000.h5 |-- output_mat_96_10000_1200000.h5 |-- output_mat_96_1000_1200000.h5 |-- output_mat_96_5000_1200000.h5 |-- output_memchip_1200000.h5 |-- output_ML_Laplace_1200000.h5 |-- output_mouse_gene_1200000.h5 |-- output_msdoor_1200000.h5 |-- output_m_t1_1200000.h5 |-- output_nd12k_1200000.h5 |-- output_nd24k_1200000.h5 |-- output_nd6k_1200000.h5 |-- output_nlpkkt120_1200000.h5 |-- output_nlpkkt80_1200000.h5 |-- output_PFlow_742_1200000.h5 |-- output_pwtk_1200000.h5 |-- output_rajat31_1200000.h5 |-- output_RM07R_1200000.h5 |-- output_Rucci1_1200000.h5 |-- output_Serena_1200000.h5 |-- output_Si34H36_1200000.h5 |-- output_Si41Ge41H72_1200000.h5 |-- output_Si87H76_1200000.h5 |-- output_SiO2_1200000.h5 |-- output_sls_1200000.h5 |-- output_StocF-1465_1200000.h5 |-- output_TEM152078_1200000.h5 |-- output_TEM181302_1200000.h5 |-- output_thermal2_1200000.h5 |-- output_tmt_sym_1200000.h5 |-- output_torso1_1200000.h5 |-- output_Transport_1200000.h5 |-- output_TSOPF_FS_b300_c2_1200000.h5 |-- output_TSOPF_FS_b300_c3_1200000.h5 |-- output_TSOPF_RS_b2383_1200000.h5 |-- output_TSOPF_RS_b2383_c1_1200000.h5 |-- output_TSOPF_RS_b678_c2_1200000.h5 `-- output_x104_1200000.h5 f_1600000_b250 |-- output_af_0_k101_1600000.h5 |—- ... `-- output_x104_1600000.h5 f_2000000_b250 |-- output_af_0_k101_2000000.h5 |—- ... `-- output_x104_2000000.h5 f_2400000_b250 |-- output_af_0_k101_2400000.h5 |—- ... `-- output_x104_2400000.h5 </code></pre> <p><strong>Creating your own dataset</strong></p> <p>If you wish to create your own training/testing dataset on a different target&nbsp;architecture you need to take the following steps:</p> <ol> <li> <p>Build the SpMV driver:</p> <ol> <li> <p>Go to <code>cd SpMV-driver/src</code></p> </li> <li> <p>Edit makefile and set the PAPI and HDF5&nbsp;install prefixes.</p> </li> <li> <p>Build the driver via <code>make.</code></p> </li> </ol> </li> <li> <p>Run the SpMV driver:&nbsp;</p> <p><code>./driver &lt;arg0&gt; &lt;arg1&gt; ...</code></p> <p>List of driver arguments:</p> <pre><code>matrix = audikw_1.rb # Input matrix in rb format reps = 10000 # Number of repetitions of the operation to avoid overhead block_size_ini = 250 # Minimum block size block_size_end = 1000 # Maximum block size increment = 250 # Increment between block sizes base = 0 # Starting nnz of the matrix freq = [2400000, 2000000, 1600000, 1200000] # Operating frequency sym = 1 # If 1 the matrix is symmetric. If 0 the matrix is no-symmetric.</code></pre> <p>Example:</p> <p><code>numactl --membind 0 taskset -c 0 ./src/driver audikw_1.rb 10000 250 1000 250 0 2400000</code></p> <p>Note that <code>numactl</code> and <code>taskset</code> utilities are used to guarantee both NUMA and&nbsp;process-to-core affinity.</p> </li> <li> <p>Generating the dataset:</p> <ol> <li> <p>Edit the <code>SpMV-driver/run_all.sh</code> and uncomment the line <code>matrices =</code> in order to launch the driver for Train_symmetric / Train_noSymmetric /&nbsp;Test_symmetric / Test_noSymmetric matrices.</p> </li> <li> <p>Edit the 3rd parameter in the command SpMV-driver/run_driver.sh: 1 for&nbsp;symmetric matrices, 2 for unsymmetric matrices.</p> </li> <li> <p>Edit the command in SpMV-driver/run_driver.sh to select the input parameters&nbsp;of the driver as explained before.</p> </li> <li> <p>Run <code>SpMV-driver/run_all.sh</code> to obtain <code>hdf5</code> files that will create the&nbsp;dataset.</p> </li> </ol> </li> <li> <p>Merging the dataset:</p> <p>Run the script</p> <p><code>python3 SpMV-driver/merge_train_matrices.py /path/to/hdf5/matrix/files /output/path</code></p> <p>to obtain a single <code>hdf5</code> file containing all data from individual <code>hdf5</code> files&nbsp;obtained in the previous step. This merged file is the training dataset.</p> </li> </ol> <p><strong>Hyperparameter search</strong></p> <p>The script <code>spmv_cnn_hyperas.py</code> performs the hyperparameter search via the&nbsp;Hyperas tool. This script requires the hdf5 file dataset in the directory&nbsp;<code>dataset/train/</code> and produces both a <code>best_model_*.json</code> and <code>best_run_*.json&nbsp;</code>files in the&nbsp;<code>results/models/</code> directory containing the model structure and&nbsp;hyperparameters of the best performing configuration.</p> <p>This script can be invoked in the following way:</p> <p><code>python3 spmv_cnn_hyper.py 2400000 Time</code></p> <p>where <code>2400000</code> is the operating processor frequency (2.4 GHz) at which the&nbsp;dataset was generated and <code>Time</code> the modeled metric. According to the labels in&nbsp;the dataset, the hyperparameter search can also be performed with the <code>Energy</code>,&nbsp;<code>EPKG</code> and <code>EDRAM</code> metrics, corresponding to the energy measured by the Intel&nbsp;RAPL counters from our Intel Xeon Haswell core. In our case, however, we only&nbsp;search hyperparameters for the <code>Time</code> and <code>Energy</code> metrics at 2.4 GHz. Other&nbsp;metrics and frequencies inherit the best performing model and settings from the&nbsp;previous configuration.</p> <p><strong>Training</strong></p> <p>The script <code>spmv_cnn_train.py</code> performs the training on the best performing&nbsp;models obtained on the previous step. For that, it uses both the&nbsp;<code>best_model_*.json</code> and <code>best_run_*.json</code> files obtained in the hyperparameter&nbsp;search.</p> <p>This script can be invoked in the following way:</p> <p><code>python3 spmv_cnn_train.py 2400000 Time</code></p> <p>where <code>2400000</code> is the operating processor frequency (2.4 GHz) and <code>Time</code> the&nbsp;modeled metric. The training should be performed per metric and frequency. The&nbsp;training produces a file that contains the trained weights, so the model is&nbsp;ready for performing inference (testing).</p> <p><strong>Testing</strong></p> <p>The script <code>spmv_cnn_test.py</code> performs the test on the set of testing matrices&nbsp;involved in the SpMV operation.</p> <p>This script can be invoked in the following way:</p> <p><code>python3 spmv_cnn_test.py 2400000 Time</code></p> <p>where <code>2400000</code> is the operating processor frequency (2.4 GHz) and <code>Time</code> the&nbsp;modeled metric. The test should be performed per metric and frequency. The&nbsp;training produces two files in the <code>results/tests/</code> directory:</p> <ul> <li><code>Pred_*.txt</code>: This file contains the real measurements and the predictions&nbsp;obtained by the CNN for the individual vpos blocks of the testing matrices.</li> <li><code>Test_*.txt</code>: This file summarizes the information of <code>Pred_*.txt</code> file,&nbsp;showing the average relative error among the blocks of each test matrix and the&nbsp;total relative error, which is computed by summing up the real measurements and&nbsp;the predictions for all the blocks of a same matrix and computing the relative&nbsp;error upon those values.</li> </ul> <p><em>Note that this testing step and the two previous steps (hyperparameter search&nbsp;and training) can be performed at once using the <code>run.sh</code> script.</em></p> <p><strong>References</strong></p> <p>Publications describing <strong>SpMV-CNN-Model</strong>:</p> <ul> <li>Barreda, M., Dolz, M.F., Casta&ntilde;o, M.A. et al. Performance modeling of the&nbsp;sparse matrix&ndash;vector product via convolutional neural networks. J Supercomputing (2020). <a href="https://doi.org/10.1007/s11227-020-03186-1">https://doi.org/10.1007/s11227-020-03186-1</a></li> </ul> <p><strong>Acknowledgments</strong></p> <p>The <strong>SpMV-CNN-Model</strong> research has been partially supported by:</p> <ul> <li> <p>Project TIN2017-82972-R <strong>&ldquo;Agorithmic Techniques for Energy-Aware and&nbsp;Error-Resilient High Performance Computing&rdquo;</strong> funded by the Spanish Ministry of Economy and Competitiveness (2018-2020).</p> </li> <li> <p>Project CDEIGENT/2017/04 <strong>&ldquo;High Performance Computing for Neural Networks&rdquo;&nbsp;</strong>funded by the Valencian Government.</p> </li> <li> <p>Project UJI-A2019-11 <strong>&ldquo;Energy-Aware High Performance Computing for Deep&nbsp;Neural Networks&rdquo;</strong> funded by the Universitat Jaume I.</p> </li> </ul>

opengpl-2.0-or-laterJul 2020View details →
zenodo44/100

Example Cloud-Resolving Model Output (Dec 2013 run)

<p>This dataset contains a selection of&nbsp;hourly, domain-mean quantities&nbsp;from the large-ensemble of realistic cloud-resolving model experiments described here: https://acp.copernicus.org/articles/20/6291/2020/</p>

opencc-by-4.0Dec 2020View details →
zenodo44/100

Raw and processed GO term data to support running GCEA analyses using ensemble-based nulls, as described in the manuscript, 'Overcoming bias in gene category enrichment analyses of brain-wide transcriptomic data'.

<p>Data to support a toolbox for performing gene category enrichment analyses, including against ensembles of null phenotypes.</p> <p>Descriptions of how these data files can be used for this purpose are in the documentation for the toolbox, at https://github.com/benfulcher/GCEA_FalsePositives</p>

opencc-by-4.0Jan 2021View details →

ScienceDex guides

Understand access before you commit

These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.

Compare curated datasets

Allen Brain Atlas

Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.

allen-brain-atlas
neuroscienceopenDocumentation, web resources, and API references are available online.
Last verified 2026-04-30Open record

Annotated Behaviour and Observability Dataset (ABODe)

ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.

abode-home-cage
behavioral-neuroscienceopenThe DataShare record exposes download links for annotations, documentation, license text, and the zipped per-snippet data directory.
Last verified 2026-04-30Open record

DANDI Archive for NWB datasets

DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.

dandi-nwb
electrophysiologyopenPublished Dandiset metadata and archive endpoints are available through the production DANDI API.
Last verified 2026-04-30Open record

International Brain Laboratory public data

The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.

ibl
behavioral-neuroscienceopenPublic sessions can be searched and loaded from the IBL public data server through ONE.
Last verified 2026-04-29Open record

OpenNeuro

OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.

openneuro
neuroscienceopenPublished datasets are available on demand over the internet.
Last verified 2026-04-29Open record