Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
65
datasets available to search
ShareScore release 0.9.0
Dataset results
65 results for “Energy Production”
1600 years of modelled energy production and demand for European Countries (Norway, France, Italy, Spain, and Sweden)
<h3>Citation</h3> <p>When using this dataset, please cite the following paper: van der Most et al. Temporally compounding energy droughts in European electricity systems with hydropower, 10 January 2024, PREPRINT (Version 1) available at Research Square [https://doi.org/10.21203/rs.3.rs-3796061/v1].</p> <h3>Description</h3> <p>This dataset contains daily renewable energy production and demand data used in the study "Temporally compounding energy droughts in European electricity systems with hydropower". The dataset includes production data for various renewable energy sources (offshore wind, onshore wind, solar photovoltaics, run-of-river, and hydropower reservoir inflow) and electricity demand. It was generated wit the use of 1600 years of climate model data and a daily renewable electricity production and demand modelling framework. The study focuses on five European countries with significant hydropower capacities: Norway, France, Italy, Spain, and Sweden.</p> <h3>Content</h3> <ul> <li> <p><strong>Energy Production Data</strong>:</p> <ul> <li>Offshore and Onshore Wind Power: Derived from 10 m wind speed data extrapolated to hub height, using power law equations and cubic power curves.</li> <li>Solar Photovoltaics (PV): Based on solar irradiance and temperature-dependent cell efficiency calculations.</li> <li>Hydropower: Includes inflow data for run-of-river and reservoir hydropower systems modelled with routed runoff data</li> <li>Hydropower dispatch is modelled at the national level using a linear optimization approach that aims to minimize the difference between demand and the sum of all renewable energy production over a year, directing the solution to following the load curves.</li> </ul> </li> <li> <p><strong>Energy Demand Data</strong>:</p> <ul> <li>Daily load data from ENTSO-E tranparancy fitted using a logistic smooth transmission regression approach to national mean, population-weighted daily near-surface temperatures from ERA5 reanalysis data.</li> <li>Demand curves account for weekdays and weekends but exclude cultural and socio-economic factors such as holidays.</li> </ul> </li> </ul> <h3>Methodology</h3> <p>The dataset is generated using the KNMI Large Ensemble Time Slice (KNMI-LENTIS) dataset, which includes 160 sets of 10-year physical climate model simulations of present-day climate (2000-2009). The simulations are conducted with the EC-Earth3 global climate model. The energy production and demand data are modeled to assess the impact of meteorological drivers on energy systems, with a focus on identifying periods of high residual loads (energy droughts). The model set-up has been validated with the use of ERA5 data in previous work. </p> <h3>Usage</h3> <p>This dataset is intended for researchers and policymakers interested in studying the impact of climate variability on renewable energy systems. It provides insights into how different meteorological conditions can lead to energy droughts and offers a basis for developing strategies to enhance the resilience of energy systems.</p> <p> </p>
Production line dataset for task scheduling and energy optimization - Demand Response Participation
<p>Using the previous dataset at <<a href="https://zenodo.org/record/4106746">https://zenodo.org/record/4106746</a>> it was simulated an announcement of a demand response program at period 757, describing a demand response event from period 937 (Friday at 21:00h) to 960 (Friday at 23:00h) , where each period represents five minutes. The demand response program imposed a limit consumption, during its event, of 2.5 kWh. The announcement of the demand response allowed the use of the proposed solution to limit the energy consumption. For that, the algorithm described in section 3.3 was executed at period 769 (Friday at 7:00h).</p> <p>The API can be found at <<a href="http://www.gecad.isep.ipp.pt/api/spear/%3E">http://www.gecad.isep.ipp.pt/api/spear/</a>></p> <p>File Description:</p> <ul> <li>Input_JSON_Demand_Response_Optimization - JSON input data for the demand response participation</li> <li>Output_JSON_Demand_Response_Optimization - JSON output data for the demand response participation</li> <li>Output_Statistics_Demand_Response_Optimization - Excel output demand response participation statistics</li> <li>Comparison_Output_Statistics_Demand_Response - Excel output statistics comparing the before and after the demand response participation</li> </ul>
Production line dataset for task scheduling and energy optimization - Schedule Optimization
<p>The case study of this dataset uses real production data, provided by a textile company that manufactures hang tags. Their working schedule is from 7h00 of Monday to 23h00 of Saturday. This dataset uses a period of 5 minutes for all task durations and energy data. The case study considers a six-day period from 7h00 of Monday to 23h00 of Saturday. The scheduling algorithm was used for three machines that share the same cell.<br> <br> The API can be found at <http://www.gecad.isep.ipp.pt/api/spear/><br> <br> File Description:</p> <ul> <li>Input_JSON_Schedule_Optimization - JSON input data for the schedule optimization</li> <li>Output_JSON_Schedule_Optimization - JSON output data for the schedule optimization</li> <li>Output_Statistics_Schedule_Optimization - Excel output schedule optimization statistics</li> </ul>
Working time, energy throughput and value added embodied in production, consumption and trade by subsectors for the US, the EU, China and rest of the world (2011)
<p>This repository contains the data needed to reproduce the results in:</p> <p>Pérez-Sánchez, L., Velasco-Fernández, R., Giampietro, M., The international division of labor and embodied working time in trade for the US, the EU and China, Ecological Economics. <a href="http://doi.org/10.1016/j.ecolecon.2020.106909">https://doi.org/10.1016/j.ecolecon.2020.1069097</a></p> <p>Sources of data are specified in the dataset (under tab "references")</p> <p> </p>
Wind energy production in forests conflicts with tree - roosting bats
<p>Many countries are investing heavily in wind power generation,<sup>1</sup> triggering a high demand for suitable land. As a result, wind energy facilities are increasingly being installed in forests,<sup>2,3</sup> despite the fact that forests are crucial for the protection of terrestrial biodiversity.<sup>4</sup> This green-green dilemma is particularly evident for bats, as most species at risk of colliding with wind turbines roost in trees.<sup>2</sup> With some of these species reported to be declining,<sup>5-8</sup> we see an urgent need to understand how bats respond to wind turbines in forested areas, especially in Europe where all bat species are legally protected. We used miniaturized global positioning system (GPS) units to study how European common noctule bats (<em>Nyctalus noctula</em>), a species that is highly vulnerable at turbines,<sup>9</sup> respond to wind turbines in forests. Data from 60 tagged common noctules yielded a total of 8129 positions, of which 2.3% were recorded at distances <100 m from the nearest turbine. Bats were particularly active at turbines <500 m near roosts, which may require such turbines to be shut down more frequently at times of high bat activity to reduce collision risk. Beyond roosts, bats avoided turbines over several kilometers, supporting earlier findings on habitat loss for forest-associated bats.<sup>10</sup> This habitat loss should be compensated by developing parts of the forest as refugia for bats. Our study highlights that it can be particularly challenging to generate wind energy in forested areas in an ecologically sustainable manner with minimal impact on forests and the wildlife that inhabit them.</p>
Historical and modelled renewable energy production for India
<p>This archive contains all the datasets produced for the paper:<br><br><span>Hunt, K. M. R.</span>, & <span>Bloomfield, H. C.</span> (<span>2024</span>). <span>Quantifying renewable energy potential and realized capacity in India: Opportunities and challenges</span>. <em>Meteorological Applications</em>, <span>31</span>(<span>3</span>), e2196. <a href="https://doi.org/10.1002/met.2196">https://doi.org/10.1002/met.2196</a></p> <p> </p> <table style="border-collapse: collapse; width: 99.9642%;"><colgroup><col style="width: 31.0476%;"><col style="width: 17.1785%;"><col style="width: 37.7477%;"><col style="width: 14.0133%;"></colgroup> <tbody> <tr> <td><strong>Data Description </strong></td> <td><strong>Figure/Table</strong></td> <td><strong> File Name</strong></td> <td><strong>Dates Valid</strong></td> </tr> <tr> <td>Installed capacity by type in each state</td> <td>Table 1</td> <td>installed-by-state-oct2022.csv</td> <td>Oct 2022</td> </tr> <tr> <td>All-India installed capacity by type</td> <td>Figure 2</td> <td>tabulated-installed-by-date.csv</td> <td>2017–2023</td> </tr> <tr> <td>Hourly wind capacity factor</td> <td>Figure 4</td> <td>wind capacity factor.zip</td> <td>1979–2022</td> </tr> <tr> <td>Hourly solar capacity factor</td> <td>Figure 6</td> <td>solar capacity factor.zip </td> <td>1979–2022</td> </tr> <tr> <td>Present-day installation locations</td> <td>Figure 11</td> <td>OSM[hydropower,wind_turbine,solar]_ installations.geojson</td> <td>Mar 2022</td> </tr> <tr> <td>Gridded 1°×1° estimate of installed wind/solar capacity</td> <td>Figure 12a/13a</td> <td>CEA_1x1_gridded_installed_[wind,solar]_cap.nc</td> <td>May 2021</td> </tr> <tr> <td>Gridded 1°×1° estimate of installed wind capacity</td> <td>Figure 12b</td> <td>TWP_1x1_gridded_installed_wind_cap.nc</td> <td>May 2021</td> </tr> <tr> <td>Gridded 1°×1° estimate of installed solar capacity</td> <td>Figure 13b</td> <td>K21_1x1_gridded installed solar cap.nc</td> <td>Sep 2018</td> </tr> <tr> <td>Reported daily wind/solar/hydro production</td> <td>Figure 14/S3</td> <td>POSOCO_reported_[wind,solar,hydro]_MU_ daily.csv</td> <td>2012–2023</td> </tr> <tr> <td>Modelled ‘historical’ production</td> <td>Figure 14/16/S4a/b</td> <td>modelled-historical-[daily,hourly]-renewable output.nc</td> <td>1979–2022</td> </tr> <tr> <td>Recommended locations for new wind/solar installations</td> <td>Figure 17</td> <td>areas-for-exploration.nc</td> <td>--</td> </tr> </tbody> </table> <p> </p> <p> </p>
Hybridization of Fossil- and CO2-Based Routes for Ethylene Production using Renewable Energy
<p>Dataset associated with the publication "Hybridization of Fossil- and CO<sub>2</sub>-Based Routes for Ethylene Production using Renewable Energy" by Iasonas Ioannou, Sebastiano C. D'Angelo, Antonio J. Martín, Javier Pérez-Ramírez, and Gonzalo Guillén-Gosálbez, available at <a href="https://doi.org/10.1002/cssc.202001312">https://doi.org/10.1002/cssc.202001312</a>. The dataset includes the numeric data associated with most of the scenarios described in the main manuscript and in the Supporting Information (SI), as well as the tables presented in the main manuscript and in the SI converted in a machine-readable format.</p> <p>The structure of the dataset is here elucidated sheet by sheet:</p> <ul> <li><strong>MS-Results</strong>: numerical values associated with the economic and environmental results included in both the main manuscript and the SI, for all the considered scenarios. The results include the total price for the assessed scenarios, with and without externalities, with uncertainty ranges, as well as the environmental results for human health, ecosystems, resources, and global warming potential (GWP).</li> <li><strong>MS-Tables</strong>: table reported in the main manuscript associated with the price and breakeven point of four assessed scenarios dependent on different CO<sub>2</sub> source assumptions.</li> <li><strong>SI-Tables-Economics</strong>: tables reported in the SI associated with the economic assessment of all the scenarios.</li> <li><strong>SI-Tables-LCI</strong>: tables reported in the SI associated with the environmental assessment of all the scenarios.</li> <li><strong>SI-Tables-AdditionalResults</strong>: tables reported in the SI associated with additional results presented in the work.</li> </ul>
Block-wise sparse matrix-vector product dataset and convolutional neural nets for estimating the run time and energy consumption of the sparse matrix-vector product
<p><strong>Introduction</strong></p> <p><strong>SpMV-CNN</strong> is a set of Convolutional Neural Networks (CNNs) that provide accurate estimations of the performance and energy consumption of the SpMV kernel. The proposed CNN-based models use a block-wise approach to make the CNN architecture independent of the matrix size. These models cat be trained to estimate run time as well as total, package and DRAM energy consumption at different processor frequencies.</p> <p><strong>Prerequisites</strong></p> <p><strong>SpMV-CNN</strong> requires Python3 with the following packages:</p> <pre><code>keras==2.1.6 tensorflow==1.8.0 h5py==2.7.1 matplotlib==2.1.1 scikit-learn==0.19.1 </code></pre> <p><strong>Obtaining the dataset</strong></p> <p>The execution time and energy consumption data corresponding to the SpMV operation on a set of sparse matrices from the SuiteSparse Matrix Collection have been obtained on an Intel Xeon E5-2630 core running at frequencies 1.2, 1.6, 2.0, 2.4 GHz. The energy consumption measurements are obtained via the Intel RAPL interface and gathered at three different levels (total, package and DRAM, where total = package + DRAM) for this specific processor.</p> <p>The <code>spmv-cnn-dataset.tgz</code> archive contains the whole dataset, including the following HDF5 files:</p> <pre><code>$ tree . |-- test | |-- f_1200000_b250 | | |-- output_2cubes_sphere_1200000.h5 | | |-- output_apache2_1200000.h5 | | |-- output_bcsstk36_1200000.h5 | | |-- output_cfd1_1200000.h5 | | |-- output_cfd2_1200000.h5 | | |-- output_ct20stif_1200000.h5 | | |-- output_denormal_1200000.h5 | | |-- output_Dubcova2_1200000.h5 | | |-- output_Dubcova3_1200000.h5 | | |-- output_ecology2_1200000.h5 | | |-- output_gyro_1200000.h5 | | |-- output_gyro_k_1200000.h5 | | |-- output_msc10848_1200000.h5 | | |-- output_msc23052_1200000.h5 | | |-- output_nasasrb_1200000.h5 | | |-- output_nd3k_1200000.h5 | | |-- output_offshore_1200000.h5 | | |-- output_oilpan_1200000.h5 | | |-- output_olafu_1200000.h5 | | |-- output_parabolic_fem_1200000.h5 | | |-- output_qa8fm_1200000.h5 | | |-- output_raefsky4_1200000.h5 | | |-- output_s3dkq4m2_1200000.h5 | | |-- output_s3dkt3m2_1200000.h5 | | |-- output_ship_001_1200000.h5 | | |-- output_ship_003_1200000.h5 | | |-- output_shipsec1_1200000.h5 | | |-- output_shipsec5_1200000.h5 | | |-- output_shipsec8_1200000.h5 | | |-- output_smt_1200000.h5 | | |-- output_thermomech_dM_1200000.h5 | | |-- output_thread_1200000.h5 | | `-- output_vanbody_1200000.h5 | |-- f_1600000_b250 | | |-- output_2cubes_sphere_1600000.h5 | | |—- ... | | `-- output_vanbody_1600000.h5 | |-- f_2000000_b250 | | |-- output_2cubes_sphere_2000000.h5 | | |—- ... | | `-- output_vanbody_2000000.h5 | `-- f_2400000_b250 | |-- output_2cubes_sphere_2400000.h5 | |—- ... | `-- output_vanbody_2400000.h5 |-- test_pagerank | |-- f_1200000_b250 | | |-- output_adaptive_1200000.h5 | | |-- output_cit-HepPh_1200000.h5 | | |-- output_delaunay_n22_1200000.h5 | | |-- output_email-Enron_1200000.h5 | | |-- output_email-EuAll_1200000.h5 | | |-- output_europe_osm_1200000.h5 | | |-- output_hugebubbles-00020_1200000.h5 | | |-- output_rgg_n_2_24_s0_1200000.h5 | | |-- output_road_usa_1200000.h5 | | |-- output_Stanford_1200000.h5 | | |-- output_wb-edu_1200000.h5 | | |-- output_web-BerkStan_1200000.h5 | | |-- output_web-Google_1200000.h5 | | |-- output_web-NotreDame_1200000.h5 | | |-- output_wiki-Talk_1200000.h5 | | `-- output_wiki-Vote_1200000.h5 | |-- f_1600000_b250 | | |-- output_adaptive_1600000.h5 | | |—- ... | | `-- output_wiki-Vote_1600000.h5 | |-- f_2000000_b250 | | |-- output_adaptive_2000000.h5 | | |—- ... | | `-- output_wiki-Vote_2000000.h5 | `-- f_2400000_b250 | |-- output_adaptive_2400000.h5 | |—- ... | `-- output_wiki-Vote_2400000.h5 `-- train |-- merged_energy_train_shuffle_f1200000_250.h5 |-- merged_energy_train_shuffle_f1600000_250.h5 |-- merged_energy_train_shuffle_f2000000_250.h5 `-- merged_energy_train_shuffle_f2400000_250.h5 </code></pre> <p>The matrices contained in the merged training files (<code>merged_energy_train_shuffle_fXX00000_250.h5</code>) are the following:</p> <pre><code>$ tree . |-- output_af_0_k101_1200000.h5 |-- output_af_1_k101_1200000.h5 |-- output_af_2_k101_1200000.h5 |-- output_af_3_k101_1200000.h5 |-- output_af_4_k101_1200000.h5 |-- output_af_5_k101_1200000.h5 |-- output_af_shell10_1200000.h5 |-- output_af_shell1_1200000.h5 |-- output_af_shell2_1200000.h5 |-- output_af_shell3_1200000.h5 |-- output_af_shell4_1200000.h5 |-- output_af_shell5_1200000.h5 |-- output_af_shell6_1200000.h5 |-- output_af_shell7_1200000.h5 |-- output_af_shell8_1200000.h5 |-- output_af_shell9_1200000.h5 |-- output_atmosmodd_1200000.h5 |-- output_atmosmodj_1200000.h5 |-- output_atmosmodl_1200000.h5 |-- output_audikw_1_1200000.h5 |-- output_BenElechi1_1200000.h5 |-- output_bmw3_2_1200000.h5 |-- output_bmw7st_1_1200000.h5 |-- output_bmwcra_1_1200000.h5 |-- output_bone010_1200000.h5 |-- output_boneS01_1200000.h5 |-- output_boneS10_1200000.h5 |-- output_bundle_adj_1200000.h5 |-- output_cage14_1200000.h5 |-- output_cage15_1200000.h5 |-- output_circuit5M_1200000.h5 |-- output_circuit5M_dc_1200000.h5 |-- output_CO_1200000.h5 |-- output_consph_1200000.h5 |-- output_CoupCons3D_1200000.h5 |-- output_crankseg_1_1200000.h5 |-- output_crankseg_2_1200000.h5 |-- output_CurlCurl_2_1200000.h5 |-- output_CurlCurl_3_1200000.h5 |-- output_CurlCurl_4_1200000.h5 |-- output_dielFilterV2real_1200000.h5 |-- output_dielFilterV3real_1200000.h5 |-- output_Emilia_923_1200000.h5 |-- output_ESOC_1200000.h5 |-- output_F1_1200000.h5 |-- output_F2_1200000.h5 |-- output_Fault_639_1200000.h5 |-- output_Freescale1_1200000.h5 |-- output_Freescale2_1200000.h5 |-- output_FullChip_1200000.h5 |-- output_G3_circuit_1200000.h5 |-- output_Ga10As10H30_1200000.h5 |-- output_Ga19As19H42_1200000.h5 |-- output_Ga3As3H12_1200000.h5 |-- output_Ga41As41H72_1200000.h5 |-- output_Ge87H76_1200000.h5 |-- output_Ge99H100_1200000.h5 |-- output_Geo_1438_1200000.h5 |-- output_gsm_106857_1200000.h5 |-- output_Hardesty3_1200000.h5 |-- output_hood_1200000.h5 |-- output_Hook_1498_1200000.h5 |-- output_human_gene1_1200000.h5 |-- output_human_gene2_1200000.h5 |-- output_inline_1_1200000.h5 |-- output_JP_1200000.h5 |-- output_kkt_power_1200000.h5 |-- output_ldoor_1200000.h5 |-- output_Long_Coup_dt0_1200000.h5 |-- output_Long_Coup_dt6_1200000.h5 |-- output_mat_104_10000_1200000.h5 |-- output_mat_104_1000_1200000.h5 |-- output_mat_104_5000_1200000.h5 |-- output_mat_112_10000_1200000.h5 |-- output_mat_112_1000_1200000.h5 |-- output_mat_112_5000_1200000.h5 |-- output_mat_120_10000_1200000.h5 |-- output_mat_120_1000_1200000.h5 |-- output_mat_120_5000_1200000.h5 |-- output_mat_128_10000_1200000.h5 |-- output_mat_128_1000_1200000.h5 |-- output_mat_128_5000_1200000.h5 |-- output_mat_16_10000_1200000.h5 |-- output_mat_16_1000_1200000.h5 |-- output_mat_16_5000_1200000.h5 |-- output_mat_24_10000_1200000.h5 |-- output_mat_24_1000_1200000.h5 |-- output_mat_24_5000_1200000.h5 |-- output_mat_32_10000_1200000.h5 |-- output_mat_32_1000_1200000.h5 |-- output_mat_32_5000_1200000.h5 |-- output_mat_40_10000_1200000.h5 |-- output_mat_40_1000_1200000.h5 |-- output_mat_40_5000_1200000.h5 |-- output_mat_48_10000_1200000.h5 |-- output_mat_48_1000_1200000.h5 |-- output_mat_48_5000_1200000.h5 |-- output_mat_56_10000_1200000.h5 |-- output_mat_56_1000_1200000.h5 |-- output_mat_56_5000_1200000.h5 |-- output_mat_64_10000_1200000.h5 |-- output_mat_64_1000_1200000.h5 |-- output_mat_64_5000_1200000.h5 |-- output_mat_72_10000_1200000.h5 |-- output_mat_72_1000_1200000.h5 |-- output_mat_72_5000_1200000.h5 |-- output_mat_80_10000_1200000.h5 |-- output_mat_80_1000_1200000.h5 |-- output_mat_80_5000_1200000.h5 |-- output_mat_8_10000_1200000.h5 |-- output_mat_8_1000_1200000.h5 |-- output_mat_8_5000_1200000.h5 |-- output_mat_88_10000_1200000.h5 |-- output_mat_88_1000_1200000.h5 |-- output_mat_88_5000_1200000.h5 |-- output_mat_96_10000_1200000.h5 |-- output_mat_96_1000_1200000.h5 |-- output_mat_96_5000_1200000.h5 |-- output_memchip_1200000.h5 |-- output_ML_Laplace_1200000.h5 |-- output_mouse_gene_1200000.h5 |-- output_msdoor_1200000.h5 |-- output_m_t1_1200000.h5 |-- output_nd12k_1200000.h5 |-- output_nd24k_1200000.h5 |-- output_nd6k_1200000.h5 |-- output_nlpkkt120_1200000.h5 |-- output_nlpkkt80_1200000.h5 |-- output_PFlow_742_1200000.h5 |-- output_pwtk_1200000.h5 |-- output_rajat31_1200000.h5 |-- output_RM07R_1200000.h5 |-- output_Rucci1_1200000.h5 |-- output_Serena_1200000.h5 |-- output_Si34H36_1200000.h5 |-- output_Si41Ge41H72_1200000.h5 |-- output_Si87H76_1200000.h5 |-- output_SiO2_1200000.h5 |-- output_sls_1200000.h5 |-- output_StocF-1465_1200000.h5 |-- output_TEM152078_1200000.h5 |-- output_TEM181302_1200000.h5 |-- output_thermal2_1200000.h5 |-- output_tmt_sym_1200000.h5 |-- output_torso1_1200000.h5 |-- output_Transport_1200000.h5 |-- output_TSOPF_FS_b300_c2_1200000.h5 |-- output_TSOPF_FS_b300_c3_1200000.h5 |-- output_TSOPF_RS_b2383_1200000.h5 |-- output_TSOPF_RS_b2383_c1_1200000.h5 |-- output_TSOPF_RS_b678_c2_1200000.h5 `-- output_x104_1200000.h5 f_1600000_b250 |-- output_af_0_k101_1600000.h5 |—- ... `-- output_x104_1600000.h5 f_2000000_b250 |-- output_af_0_k101_2000000.h5 |—- ... `-- output_x104_2000000.h5 f_2400000_b250 |-- output_af_0_k101_2400000.h5 |—- ... `-- output_x104_2400000.h5 </code></pre> <p><strong>Creating your own dataset</strong></p> <p>If you wish to create your own training/testing dataset on a different target architecture you need to take the following steps:</p> <ol> <li> <p>Build the SpMV driver:</p> <ol> <li> <p>Go to <code>cd SpMV-driver/src</code></p> </li> <li> <p>Edit makefile and set the PAPI and HDF5 install prefixes.</p> </li> <li> <p>Build the driver via <code>make.</code></p> </li> </ol> </li> <li> <p>Run the SpMV driver: </p> <p><code>./driver <arg0> <arg1> ...</code></p> <p>List of driver arguments:</p> <pre><code>matrix = audikw_1.rb # Input matrix in rb format reps = 10000 # Number of repetitions of the operation to avoid overhead block_size_ini = 250 # Minimum block size block_size_end = 1000 # Maximum block size increment = 250 # Increment between block sizes base = 0 # Starting nnz of the matrix freq = [2400000, 2000000, 1600000, 1200000] # Operating frequency sym = 1 # If 1 the matrix is symmetric. If 0 the matrix is no-symmetric.</code></pre> <p>Example:</p> <p><code>numactl --membind 0 taskset -c 0 ./src/driver audikw_1.rb 10000 250 1000 250 0 2400000</code></p> <p>Note that <code>numactl</code> and <code>taskset</code> utilities are used to guarantee both NUMA and process-to-core affinity.</p> </li> <li> <p>Generating the dataset:</p> <ol> <li> <p>Edit the <code>SpMV-driver/run_all.sh</code> and uncomment the line <code>matrices =</code> in order to launch the driver for Train_symmetric / Train_noSymmetric / Test_symmetric / Test_noSymmetric matrices.</p> </li> <li> <p>Edit the 3rd parameter in the command SpMV-driver/run_driver.sh: 1 for symmetric matrices, 2 for unsymmetric matrices.</p> </li> <li> <p>Edit the command in SpMV-driver/run_driver.sh to select the input parameters of the driver as explained before.</p> </li> <li> <p>Run <code>SpMV-driver/run_all.sh</code> to obtain <code>hdf5</code> files that will create the dataset.</p> </li> </ol> </li> <li> <p>Merging the dataset:</p> <p>Run the script</p> <p><code>python3 SpMV-driver/merge_train_matrices.py /path/to/hdf5/matrix/files /output/path</code></p> <p>to obtain a single <code>hdf5</code> file containing all data from individual <code>hdf5</code> files obtained in the previous step. This merged file is the training dataset.</p> </li> </ol> <p><strong>Hyperparameter search</strong></p> <p>The script <code>spmv_cnn_hyperas.py</code> performs the hyperparameter search via the Hyperas tool. This script requires the hdf5 file dataset in the directory <code>dataset/train/</code> and produces both a <code>best_model_*.json</code> and <code>best_run_*.json </code>files in the <code>results/models/</code> directory containing the model structure and hyperparameters of the best performing configuration.</p> <p>This script can be invoked in the following way:</p> <p><code>python3 spmv_cnn_hyper.py 2400000 Time</code></p> <p>where <code>2400000</code> is the operating processor frequency (2.4 GHz) at which the dataset was generated and <code>Time</code> the modeled metric. According to the labels in the dataset, the hyperparameter search can also be performed with the <code>Energy</code>, <code>EPKG</code> and <code>EDRAM</code> metrics, corresponding to the energy measured by the Intel RAPL counters from our Intel Xeon Haswell core. In our case, however, we only search hyperparameters for the <code>Time</code> and <code>Energy</code> metrics at 2.4 GHz. Other metrics and frequencies inherit the best performing model and settings from the previous configuration.</p> <p><strong>Training</strong></p> <p>The script <code>spmv_cnn_train.py</code> performs the training on the best performing models obtained on the previous step. For that, it uses both the <code>best_model_*.json</code> and <code>best_run_*.json</code> files obtained in the hyperparameter search.</p> <p>This script can be invoked in the following way:</p> <p><code>python3 spmv_cnn_train.py 2400000 Time</code></p> <p>where <code>2400000</code> is the operating processor frequency (2.4 GHz) and <code>Time</code> the modeled metric. The training should be performed per metric and frequency. The training produces a file that contains the trained weights, so the model is ready for performing inference (testing).</p> <p><strong>Testing</strong></p> <p>The script <code>spmv_cnn_test.py</code> performs the test on the set of testing matrices involved in the SpMV operation.</p> <p>This script can be invoked in the following way:</p> <p><code>python3 spmv_cnn_test.py 2400000 Time</code></p> <p>where <code>2400000</code> is the operating processor frequency (2.4 GHz) and <code>Time</code> the modeled metric. The test should be performed per metric and frequency. The training produces two files in the <code>results/tests/</code> directory:</p> <ul> <li><code>Pred_*.txt</code>: This file contains the real measurements and the predictions obtained by the CNN for the individual vpos blocks of the testing matrices.</li> <li><code>Test_*.txt</code>: This file summarizes the information of <code>Pred_*.txt</code> file, showing the average relative error among the blocks of each test matrix and the total relative error, which is computed by summing up the real measurements and the predictions for all the blocks of a same matrix and computing the relative error upon those values.</li> </ul> <p><em>Note that this testing step and the two previous steps (hyperparameter search and training) can be performed at once using the <code>run.sh</code> script.</em></p> <p><strong>References</strong></p> <p>Publications describing <strong>SpMV-CNN-Model</strong>:</p> <ul> <li>Barreda, M., Dolz, M.F., Castaño, M.A. et al. Performance modeling of the sparse matrix–vector product via convolutional neural networks. J Supercomputing (2020). <a href="https://doi.org/10.1007/s11227-020-03186-1">https://doi.org/10.1007/s11227-020-03186-1</a></li> </ul> <p><strong>Acknowledgments</strong></p> <p>The <strong>SpMV-CNN-Model</strong> research has been partially supported by:</p> <ul> <li> <p>Project TIN2017-82972-R <strong>“Agorithmic Techniques for Energy-Aware and Error-Resilient High Performance Computing”</strong> funded by the Spanish Ministry of Economy and Competitiveness (2018-2020).</p> </li> <li> <p>Project CDEIGENT/2017/04 <strong>“High Performance Computing for Neural Networks” </strong>funded by the Valencian Government.</p> </li> <li> <p>Project UJI-A2019-11 <strong>“Energy-Aware High Performance Computing for Deep Neural Networks”</strong> funded by the Universitat Jaume I.</p> </li> </ul>
Datasets associated with Agostini, S., Houlbreque, F., Biscéré, T., Harvey, B. P., Heitzman, J. M., Takimoto, R., et al. (2020). Greater mitochondrial energy production provides resistance to ocean acidification in 'winning' hermatypic corals. Front. Mar. Sci. 7. doi:10.3389/fmars.2020.600836.
<p>Datasets associated with Agostini, S., Houlbreque, F., Biscéré, T., Harvey, B. P., Heitzman, J. M., Takimoto, R., et al. (2020). Greater mitochondrial energy production provides resistance to ocean acidification in ‘winning’ hermatypic corals. Front. Mar. Sci. 7. doi:10.3389/fmars.2020.600836.</p>
Data accompanying the manuscript "Assessing the Probability of Extremely Low Wind Energy Production in Europe at Sub-seasonal to Seasonal Time Scales"
<p>This dataset contains time series of wind energy production aggregated over France and Europe, obtained from a 1000-year climate simulation from the CESM model (version 1.2.2, Hurrel et al. 2013), coupled to a simple energy model to compute grid-point capacity factor from surface wind. Wind power is then computed by multiplying the capacity factor by the installed capacity, taken from 5 e-Highway scenarios (X5, X7, X10, X13 and X16), and integrated over the regions of interest. More details about the climate simulation, wind energy model and installed capacity scenarios can be found in the associated manuscript, "Assessing the Probability of Extremely Low Wind Energy Production in Europe at Sub-seasonal to Seasonal Time Scales" (Cozian et al. 2023).</p><p>The data is organized into 10 files for France and 10 files for Europe. In each case, the 10 files correspond to 10 batches of 100 years each, with 3-hourly output. Each file contains 5 time series corresponding to the 5 installed capacity scenarios.</p><h4>References</h4><ul><li>Hurrell J W, Holland M M, Gent P R, Ghan S, Kay J E, Kushner P J, Lamarque J F, Large W G, Lawrence D, Lindsay K, Lipscomb W H, Long M C, Mahowald N, Marsh D R, Neale R B, Rasch P, Vavrus S, Vertenstein M, Bader D, Collins W D, Hack J J, Kiehl J and Marshall S (2013). The community earth system model: A framework for collaborative research. Bulletin of the American Meteorological Society, 94, 1339–1360. <a href="https://doi.org/10.1175/BAMS-D-12-00121.1">https://doi.org/10.1175/BAMS-D-12-00121.1</a></li><li>e-Highway 2050 (2015). Europe's future secure and sustainable electricity infrastructure. <a href="https://docs.entsoe.eu/baltic-conf/bites/www.e-highway2050.eu/results">https://docs.entsoe.eu/baltic-conf/bites/www.e-highway2050.eu/results</a></li><li>Cozian B, Herbert C and Bouchet F (2023). Assessing the Probability of Extremely Low Wind Energy Production in Europe at Sub-seasonal to Seasonal Time Scales. <a href="https://doi.org/10.48550/arXiv.2311.13526">https://doi.org/10.48550/arXiv.2311.13526</a></li></ul>
Research data supporting "Impact of global heterogeneity of renewable energy supply on heavy industrial production and green value chains"
<p>Research data supporting the peer-reviewed article "Impact of global heterogeneity of renewable energy supply on heavy industrial production and green value chains" by the same authors.</p>
Joint Optimization of Production and Maintenance for Cost-effective Manufacturing and Demand Response Participation Dataset - Energy Cost Optimization with Energy Selling
<p>Using the previous dataset at <<a href="https://zenodo.org/record/4106746">https://zenodo.org/record/4106746</a>> an energy cost optimization considering the presence of an energy buyer is proposed to validate the scheduler’s ability to maximize profits while also minimizing energy costs. The scenario considers an added sales value corresponding to 50% of the buying. For this scenario, the genetic algorithm was executed for 2 hours, with 1 and 0 for the optimization weights total cost and machine occupancy deviation, respectively.</p> <p> </p> <p>File Description:</p> <ul> <li>Input_JSON_Energy_Cost_Energy_Selling_Optimization - JSON input data for the energy cost optimization with energy selling</li> <li>Output_JSON_Energy_Cost_Energy_Selling_Optimization - JSON output data for the energy cost optimization with energy selling</li> <li>Output_Statistics_Energy_Cost_Energy_Selling_Optimization - Excel output energy cost optimization with energy selling statistics</li> </ul>
Energy consumption data from an office building, waterpark and warehouse in Slovenia and energy production data from a PV Plan (900KW)
<p>The first dataset included 9-month hourly energy data from a waterpark, a warehouse and high-rise office buildings. The second dataset includes 10-year hourly energy production data from a PV plant (900KW). Both datasets refer to Ljubljana, Slovenia. </p>
ECMWF ERA interim derived atmospheric mass, moisture and energy budget products
<p>As observations and atmospheric reanalyses have improved, the diagnostics that can be computed with confidence also increase. Accordingly, a new formulation of the energetics of the atmosphere is laid out, with a view to advancing diagnostic studies of Earth's energy budget and flows. It is utilized to produce assessments of the vertically integrated divergences in both the atmosphere and ocean. Careful conservation of mass is required, with special attention given to the hydrological cycle and redistribution of mass associated with precipitation and evaporation, and a new method for ensuring this is developed. It guarantees that the atmospheric divergence is associated with moisture and precipitation, unlike previous methods. A new term, identified as associated with the enthalpy of precipitation, is included in a preliminary way. It is sensitive to the formulation, and the use of temperature in degrees Celsius instead of Kelvin greatly reduces errors and produces the extra term with values up to about 65 W/m2. New results for 2000 to 2017 are presented for the vertical-mean and annual-mean diabatic atmospheric heating, atmospheric moistening, and total atmospheric energy divergence. Results for the atmospheric divergence are combined with top-of-atmosphere radiation observations to deduce total surface energy fluxes.</p> <p>These data files are monthly and span from 1979 to 2017, smoothed at T-106 resolution. The data format is NetCDF. A full dataset description is available at https://journals.ametsoc.org/view/journals/clim/31/16/jcli-d-17-0838.1.xml</p>
Energy efficiency on Philips Lightings products for outdoor lighting
<p>The dataset is a compilation of specifications and performance metrics for different lighting products from Philips Lighting catalogs. It spans various products across different technology types and years, which suggests a focus on the evolution and comparison of lighting efficiency over time.</p> <p>Here are some key points about the dataset:</p> <p>- **Product Information**: Each entry in the `Nombre` column provides specific details about a Philips Lighting product, likely including the model and technical specifications.</p> <p>- **Technology Classification**: The `Tecno` column classifies each product according to its lighting technology, such as LED, CDM, SOX, etc. This allows for analysis across different types of lighting technologies.</p> <p>- **Energy Consumption and Efficiency**: The dataset includes data on energy consumption (`Consumo`) and efficiency (`Efi(lm/w)` and `Efi2`). These metrics are crucial for understanding the energy cost of running the lights and for analyzing improvements in energy efficiency over time. Efi is the calculated energy efficiency from the catalogue data and the Efi2 is the reported energy eficiency.</p> <p>- **Light Output and Quality**: The `Lumens` and `CCT` columns provide information on the brightness and color temperature of the lighting products. This is valuable for assessing the quality and suitability of the light for various applications.</p> <p>- **Economic Considerations**: The `Precio` column, while not filled in for all entries, would give insights into the economic aspect of the lighting products, potentially allowing for cost-benefit analysis.</p> <p>- **Temporal Trends**: The `Año` column indicates the year associated with the product, which can be used to track changes and advancements in lighting technology over time.</p> <p>- **Product Longevity**: The `Vida` column, although unspecified in the dataset preview, would generally relate to the lifespan of the lighting product, an important factor in both consumer choice and sustainability considerations.</p> <p>In summary, this dataset serves as a resource for analyzing Philips Lighting products' performance over time, understanding trends in lighting technology efficiency, and potentially assisting in strategic decisions related to product development, marketing, and sustainability efforts.</p>
Upcycling food ingredients from orange by-products by hot air-microwave drying. Impact on energy consumption.
<p>Currently industrial citrus by-products represent a relevant environmental issue. The main aim of this work was the chemical characterization of the different bioactive compounds obtained after hot air-microwave drying (HAD+MW) of orange by-products, and their further conversion into three <strong>upcycled </strong>ingredients with health-related benefits: aqueous extract, ethanolic extract and <strong>dietary fibre</strong>. Total phenolics, antioxidant capacity, individual phenolic acids, flavonoids, limonin and carotenoids were monitored during blanching and colour extraction steps by analysing fresh by-products and process co-products: an aqueous extract rich in polyphenols and an ethanolic extract rich in carotenoids. After drying, the resulting fibre was characterized in terms of chemical composition, soluble and insoluble dietary fibre content and particle size. Technological properties and colour were compared to those of commercial citrus fibre. Energy and time consumption were compared with conventional hot air drying (HAD). Most polyphenols (50-65 %) and limonin (70 %) were extracted during the blanching step. 86 % of carotenoids were removed by soaking in ethanol. The orange fibre obtained had 71.9 g DF/ 100 g and antioxidant properties (205 mg TE/ Kg<sub>dm</sub>). Whiteness, water retention capacity and oil retention capacity were similar to commercial citrus fibre. HAD+MW reduced drying time and energy consumption by up to 50 % compared to HAD.</p>
Balancing profitability of energy production, societal impacts and biodiversity in offshore wind farm design
<p>Dataset related to the article: Virtanen, E.A., Lappalainen, J., Nurmi, M., Viitasalo, M., Tikanmäki, M., Heinonen, J., Atlaskin, E., Kallasvuo, M., Tikkanen, H., Moilanen, A. (2022) Balancing profitability of energy production, societal impacts and biodiversity in offshore wind farm design. Renewable and Sustainable Energy Reviews 158, 112087.</p> <p>Dataset includes suitability maps for offshore windfarms, where priority values are scaled between 0-1 (note the reversed value scale): analysis solution (A) economy, (B) society, (C) biodiversity, (D) restrictions, (E) A+B+C without restrictions and (F) A+B+C with restrictions. Dataset includes also the conflict map (and R script), where each three main solutions (A, B, C) are mapped onto an RGB color composite map. </p> <p>Additional details can be found from the published article: <a href="https://doi.org/10.1016/j.rser.2022.112087">https://doi.org/10.1016/j.rser.2022.112087</a></p>
Polynomial chaos to efficiently compute the annual energy production in wind farm layout optimization
<p>Data for the Wind Energy Science paper "Polynomial chaos to efficiently compute the annual energy production in wind farm layout optimization".</p> <p>The data includes a file describing the wind direction distribution. The i<sup>th</sup> probability value corresponds to the probability of the wind coming between direction i and i+1.</p> <p>The other data files, corresponding to the wind farm layouts, provide the x,y coordinates of the wind turbines. </p>
Production Line energy and material flow dataset
<p>File dumpXXXXXX-1122 recounts production of 16 pieces, with<br> workplan "production with no fuse".<br> File dumpXXXXXX-1302 recounts production of 16 pieces, with workplan :production with left<br> fuse"</p>
The LSST Dark Energy Science Collaboration (DESC) Science Requirements Document v1 Released Data Products
<p>This tarball includes software and data products associated with the DESC Science Requirements Document (SRD) v1. See the "Executive Summary and User Guide" in the enclosed PDF of the DESC SRD for instructions on how to use and cite those products. The DESC SRD is described on <a href="https://arxiv.org/abs/1809.01669">arXiv</a> as follows:</p> <p>The Large Synoptic Survey Telescope (LSST) Dark Energy Science Collaboration (DESC) will use five cosmological probes: galaxy clusters, large scale structure, supernovae, strong lensing, and weak lensing. The Science Requirements Document (SRD) quantifies the expected dark energy constraining power of these probes individually and together, with conservative assumptions about analysis methodology and follow-up observational resources based on our current understanding and the expected evolution within the field in the coming years. We then define requirements on analysis pipelines that will enable us to achieve our goal of carrying out a dark energy analysis consistent with the Dark Energy Task Force definition of a Stage IV dark energy experiment.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.