Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
239
datasets available to search
ShareScore release 0.7.1
Dataset results
239 results for “Energy system”
Case study result data set for the submitted article "Implications of hydrogen import prices for the German energy system in a model-comparison experiment"
<p>The data set contains result data for the German energy system in a long term scenario (scenario year 2045) as described in the publication "Implications of hydrogen import prices for the German energy system in a model-comparison experiment". The results have been generated with the models REMod of Fraunhofer Institute for Solar Energy Systems ISE, Enertile of Fraunhofer Institute for Systems and Innovation Research ISI, and SCOPE SD of Fraunhofer Institute for Energy Economics and Energy System Technology IEE.</p><p><strong>Abbreviations:</strong></p><ul><li>BEV - Battery Electric Vehicles</li><li>CC - Combined Cycle</li><li>CCGT - Combined Cycle Gas Turbine</li><li>CHP - Combined heat and power</li><li>CO2 - Carbon dioxide</li><li>con - consumption</li><li>FC - Fuel Cell</li><li>FCEV - Fuel Cell Electric Vehicle</li><li>gen - generation</li><li>H2 - Hydrogen</li><li>HT - High temperature</li><li>ICE - Internal Combustion Engine</li><li>LDV - Light-Duty Vehicle</li><li>LT - Low temperature</li><li>med - medium</li><li>OC - Open Cycle</li><li>OCGT - Open Cycle Gas Turbine</li><li>PHEV - Plug-In Hybrid Vehicles</li><li>PS - Pumped Storage</li><li>PV - Photovoltaics</li><li>ST - Steam turbine</li><li>SynFuel - Synthetic fuel</li><li>w/ - with</li><li>w/o - without</li><li>yr - year</li></ul>
A complete energy community dataset with photovoltaic generation, battery energy storage systems and electric vehicles (v1.5)
<p>This dataset represents a complete European energy community based on actual data. In this scenario, a community of 250 households was built using real energy consumption and solar generation data obtained in homes throughout Europe. In total, 200 community members were assigned solar generation, while 150 were assigned a battery storage system. From the acquired sample, new profiles were created and randomly assigned to each end-user while also receiving two electric cars with information on their capacity, state-of-charge, and usage. Furthermore, it is provided the electric vehicle chargers’ information on their location, type, and cost of operation.</p> <p> </p> <p>Version 1.5 update: <span>on the Sheet EVs, lines 29 (Capacity kW), 30 (Charge kW), and 31 (Discharge kW) were updated to the correct values.</span></p> <p> </p> <p>This work has been published in Elsevier's Data in Brief journal:<br><em> Ricardo Faia, Calvin Goncalves, Luis Gomes, Zita Vale<br> Dataset of an energy community with prosumer consumption, photovoltaic generation, battery storage, and electric vehicles<br> Data in Brief, 2023, 109218, ISSN 2352-3409<br> <a href="https://doi.org/10.1016/j.dib.2023.109218.">https://doi.org/10.1016/j.dib.2023.109218</a><br> (<a href="https://www.sciencedirect.com/science/article/pii/S2352340923003372)">https://www.sciencedirect.com/science/article/pii/S2352340923003372)</a></em></p> <p> </p> <p>We would be grateful if you could acknowledge the use of this dataset in your publications. Please use the Data in Brief publication to cite this work.</p> <p> </p> <p>Reference data used to create this dataset:</p> <ul> <li>Filtered energy profiles and renewable energy production profiles: <a href="../record/6778401">https://zenodo.org/record/6778401</a></li> </ul> <ul> <li>Battery storage systems and electric vehicles: <a href="../record/4737293">https://zenodo.org/record/4737293</a></li> </ul>
Data from: Separating the impact of individual land surface properties on the terrestrial surface energy budget in both the coupled and uncoupled land–atmosphere system
Open the record for dataset details and reuse information.
Latch-based control of energy output in spring actuated systems
Open the record for dataset details and reuse information.
Tracking wildlife energy dynamics with unoccupied aircraft systems and 3-dimensional photogrammetry
Open the record for dataset details and reuse information.
Data from: Towards a better understanding of avian collisions in wind energy facilities using automatic detection systems
Open the record for dataset details and reuse information.
Comparing power-system- and user-oriented battery electric vehicle charging representation and its implications on energy system modeling
<p>This supplementary material includes data and code for the research described in the paper "Comparing power-system- and user-oriented battery electric vehicle charging representation and its implications on energy system modeling". The code containts an interface between the output files of the agent-based simulation model CURRENT and the energy system optimization model REMix as well as some scripts for analyzing REMix results. The data folder contains input data for REMix, the complete list of all model runs analyzed in the paper in the GAMS format .gdx as well as Excel files containing annual results of the sensitivity runs and respective pivot tables and figures for respective analysis.</p>
Monitoring of cow location in barn by an open source low cost low energy Bluetooth tag system
<p>Supplementary materials for: Bloch, V., Pastell, M., 2020. Monitoring of Cow Location in a Barn by an Open-Source, Low-Cost, Low-Energy Bluetooth Tag System. Sensors 20, 3841. <a href="https://doi.org/10.3390/s20143841">https://doi.org/10.3390/s20143841</a></p>
Data from: Linking size spectrum, energy flux and trophic multifunctionality in soil food webs of tropical land-use systems
1. Many ecosystem functions depend on the structure of food webs, which heavily relies on the body size spectrum of the community. Despite that, little is known on how the size spectrum of soil animals responds to agricultural practices in tropical land-use systems and how these responses affect ecosystem functioning. 2. We studied land-use induced changes in belowground communities in tropical lowland ecosystems in Sumatra (Jambi province, Indonesia), a hotspot of tropical rainforest conversion to rubber and oil palm plantations. The study included ca. 30,000 measured individuals from 33 high-order taxa of meso- and macrofauna spanning eight orders of magnitude in body mass. Using individual body masses we calculated the metabolism of trophic guilds and used food-web models to calculate energy fluxes and infer ecosystem functions, such as decomposition, herbivory, primary and intraguild predation. 3. Land-use change was associated with reduced abundance and taxonomic diversity of soil invertebrates, but strong increase in total biomass and moderate changes in total energy flux. These changes were due to increased biomass of large-sized decomposers in soil, in particular earthworms, with their share in community metabolism increasing from 11% in rainforest to 59-76% in jungle rubber, and rubber and oil palm plantations. Decomposition, i.e. the energy flux to decomposers, stayed unchanged, but herbivory, primary and intraguild predation decreased by an order of magnitude in plantation systems. Intraguild predation was very important, being responsible for 38% of the energy flux in rainforest according to our model. 4. Conversion of rainforest into monoculture plantations is associated by an uneven loss of size classes and trophic levels of soil invertebrates resulting in sequestration of energy in large-sized primary consumers and restricted flux of energy to higher trophic levels. Pronounced differences between rainforest and jungle rubber reflect sensitivity of rainforest soil animal communities to moderate land-use changes. Soil communities in plantation systems sustained high total energy flux despite reduced biodiversity. The high energy flux into large decomposers but low energy fluxes to other trophic guilds suggests that trophic multifunctionality of belowground communities is compromised in plantation systems.
Data bundle for egon-data: A transparent and reproducible data processing pipeline for energy system modeling
<p><strong>egon-data</strong> provides a transparent and reproducible open data based data processing pipeline for generating data models suitable for energy system modeling. The data is customized for the requirements of the research project <strong>eGon</strong>. The research project aims to develop tools for an open and cross-sectoral planning of transmission and distribution grids. For further information please visit the eGon <a href="https://ego-n.org/">project website</a> or its <a href="https://github.com/openego/eGon-data">Github repository.</a></p> <p>egon-data retrieves and processes data from several different external input sources. As not all data dependencies can be downloaded automatically from external sources we provide a data bundle to be downloaded by egon-data.</p> <p>The following data sets are part of the available data bundle:</p> <ol> <li> <p><strong>climate_zones_germany</strong></p> <ul> <li> <p>Climate zones in Germany</p> </li> <li> <p>source: Own representation based on DWD TRY climate zones</p> </li> <li> <p>License: Attribution 4.0 International (CC BY 4.0)</p> </li> </ul> </li> <li> <p><strong>cutouts</strong></p> <ul> <li> <p>Weather data from Europe in 2011. Source: ERA5</p> </li> </ul> </li> <li> <p><strong>demand_regio_backup</strong></p> <ul> <li> <p>Electricity and heat demands</p> </li> </ul> </li> <li> <p><strong>emobility</strong></p> <ul> <li> <p>Data on eMobility mit_trip_data:<br>motorized individual travel - individual trips of electric vehicles (EV) generated with a modified version of simBEV v0.1.3 (https://github.com/rl-institut/simbev/tree/1f87c716d14ccc4a658b8d2b01fd12b88a4334d5). simBEV generates driving profiles for BEVs and PHEVs based upon MID data (BMVI) per RegioStaR7 region type (BBSR).</p> </li> <li> <p>Reiner Lemoine Institut, June 2022</p> </li> <li> <p>License: Attribution 4.0 International (CC BY 4.0)</p> </li> </ul> </li> <li> <p><strong>entsoe</strong></p> <ul> <li> <p> </p> </li> </ul> </li> <li> <p><strong>gas_data</strong></p> <ul> <li> <p>CH4 infrastructure</p> </li> <li> <p>Biogas demand</p> </li> <li> <p>CH4 demand</p> </li> <li> <p>Source: SciGRID_gas</p> </li> </ul> </li> <li> <p><strong>geothermal_potential</strong></p> <ul> <li> <p>Spatial distribution of deep geothermal potentials in Germany</p> </li> <li> <p>source: <a href="https://doi.org/10.3390/en11020332">Assessment and Public Reporting of Geothermal Resources in Germany: Review and Outlook</a></p> </li> <li> <p>License: Attribution 4.0 International (CC BY 4.0)</p> </li> </ul> </li> <li> <p><strong>household_electricity_demand_profiles</strong></p> <ul> <li> <p>Annual profiles in hourly resolution of electricity demand of private households for different household types (singles, couples, other) with varying number of elderly and children.<br>The profiles were created using a bottom-up load profile generator by Fraunhofer IEE developed in the Bachelor's thesis "Auswirkungen verschiedener Haushaltslastprofile auf PV-Batterie-Systeme" by Jonas Haack, Fachhochschule Flensburg, December 2012.<br>The columns are named as follows: "<HH_TYPE_PREFIX>a<PROFILE_ID>", e.g. P2a0000 is the first profile of a couple's household with 2 children. See publication below for the list of prefixes. Values are given in Wh.<br>A related conference paper can be obtained here: http://publica.fraunhofer.de/documents/N-374761.html</p> </li> <li> <p>License: Attribution 4.0 International (CC BY 4.0)</p> </li> </ul> </li> <li> <p><strong>household_heat_demand_profiles</strong></p> <ul> <li> <p>Sample heat time series including hot water and space heating for single- and multi-familiy houses. The profiles were created using the loadprofile generator by Fraunhofer IEE developed in the Master's thesis "Synthesis of a heat and electrical load profile for single and multi-family houses used for subsequent performance tests of a multi-component energy system", Simon Ruben Drauz, RWTH Aachen University, March 2016</p> </li> <li> <p>License: Attribution 4.0 International (CC BY 4.0)</p> </li> </ul> </li> <li> <p><strong>hydrogen_network</strong></p> <ul> <li> <p>Planned H2 infrastructure</p> </li> <li> <p>Forecast H2 demand</p> </li> <li> <p>Source: fnb-gas</p> </li> </ul> </li> <li> <p><strong>hydrogen_storage_potential_saltstructures</strong></p> <ul> <li> <p>The data are taken from figure 7.1 in Donadei, S., et al., (2020), p. 7-5..</p> </li> <li> <p>Source: Flach lagernde Salze, (c) BGR Hannover, 2021.<br>Datenquelle: InSpEE-Salzstrukturen, (c) BGR, Hannover, 2015. &<br>Donadei, S., Horváth, B., Horváth, P.-L., Keppliner, J., Schneider, G.-S., &<br>Zander-Schiebenhöfer, D. (2020). Teilprojekt Bewertungskriterien und<br>Potenzialabschätzung. BGR. Informationssystem Salz: Planungsgrundlagen,<br>Auswahlkriterien und Potenzialabschätzung für die Errichtung von Salzkavernen<br>zur Speicherung von Erneuerbaren Energien (Wasserstoff und Druckluft) –<br>Doppelsalinare und flach lagernde Salzschichten: InSpEE-DS. Sachbericht.<br>Hannover: BGR.</p> </li> <li> <p>License: The original data are licensed under the GeoNutzV, see <a href="https://sg.geodatenzentrum.de/web_public/gdz/lizenz/geonutzv.pdf">https://sg.geodatenzentrum.de/web_public/gdz/lizenz/geonutzv.pdf</a></p> </li> </ul> </li> <li> <p><strong>industrial_gas_demand</strong></p> </li> <li> <p><strong>industrial_sites</strong></p> <ul> <li> <p>Information about industrial sites with DSM-potential in Germany from a Master's thesis by Danielle Schmidt. The data set includes own information on the coordinates of every industrial site.</p> </li> <li> <p>source: Schmidt, Danielle. (2019). Supplementary material to the masters thesis: NUTS-3 Regionalization of Industrial Load Shifting Potential in Germany using a Time-Resolved Model [Data set]. Zenodo. https://doi.org/10.5281/zenodo.3613767</p> </li> <li> <p>License: Attribution 4.0 International (CC BY 4.0)</p> </li> </ul> </li> <li> <p><strong>mastr_geocoding</strong></p> </li> <li> <p><strong>nep2035_version2021</strong></p> <ul> <li> <p>Data extracted from the German grid development plan - power</p> </li> <li> <p>source: Netzentwicklungsplan Strom 2035 (2021), erster Entwurf | Übertragungsnetzbetreiber (M) CC-BY-4.0</p> </li> <li> <p>License: Attribution 4.0 International (CC BY 4.0)</p> </li> </ul> </li> <li> <p><strong>pipeline_classification_gas</strong></p> <ul> <li> <p>Parameters for the classification of gas pipelines</p> </li> <li> <p>source: Single parameters extracted from <a href="https://www.econstor.eu/bitstream/10419/173388/1/1011162628.pdf">Electricity, Heat and Gas Sector Data for Modelling the German System</a></p> </li> <li> <p>License: Attribution 4.0 International (CC BY 4.0)</p> </li> </ul> </li> <li> <p><strong>pypsa_eur</strong></p> </li> <li> <p><strong>regions_dynamic_line_rating</strong></p> <ul> <li> <p>German regions suitable to model dynamic line rating</p> </li> <li> <p>source: Own representation based on <a href="https://www.transnetbw.de/files/pdf/netzentwicklung/netzplanungsgrundsaetze/UENB_PlGrS_Juli2020.pdf">Grundsätze für die Ausbauplanung des Deutschen Übertragungsnetze (2020)</a></p> </li> <li> <p>License: Attribution 4.0 International (CC BY 4.0)</p> </li> </ul> </li> <li> <p><strong>re_potential_areas</strong></p> <ul> <li> <p>Eligible areas for wind turbines and ground-mounted PV systems.</p> </li> <li> <p>Reiner Lemoine Institut, January 2022</p> </li> <li> <p>License: Attribution 4.0 International (CC BY 4.0)</p> </li> </ul> </li> <li> <p><strong>wind_offshore_status2019</strong></p> <ul> <li> <p> </p> </li> </ul> </li> <li> <p><strong>WZ_definition</strong></p> <ul> <li> <p>Definitions of industrial and commercial branches</p> </li> <li> <p>source: <a href="https://www.destatis.de/static/DE/dokumente/klassifikation-wz-2008-3100100089004.pdf">Klassifikation der Wirtschaftszweige (WZ 2008)</a></p> </li> <li> <p>Extract from Terms of Use: © Statistisches Bundesamt, Wiesbaden 2008 Vervielfältigung und Verbreitung, auch auszugsweise, mit Quellenangabe gestattet.</p> </li> </ul> </li> <li> <p><strong>zensus_households</strong><strong> </strong></p> <ul> <li> <p>Dataset describing the amount of people living by a certain types of family-types, age-classes,sex and size of household in Germany in state-resolution.</p> </li> <li> <p>source: Data retrieved from <a href="https://ergebnisse2011.zensus2022.de/datenbank/online">Zensus Datenbank</a> by performing these steps:</p> <ul> <li> <p>Search for: "1000A-2029"</p> </li> <li> <p>or choose topic: "Bevölkerung kompakt"</p> </li> <li> <p>Choose table code: "1000A-2029" with title "Personen: Alter (11 Altersklassen)/Geschlecht/Größe desprivaten Haushalts - Typ des privaten Haushalts (nach Familien/Lebensform)"</p> </li> <li> <p>Change setting "GEOLK1" to "Bundesländer (16)" higher resolution "Landkreise und kreisfreie Städte (412)" only accessible after registration.</p> </li> </ul> </li> <li> <p>Extract from Terms of Use: © Statistische Ämter des Bundes und der Länder 2021, Vervielfältigung und Verbreitung, auch auszugsweise, mit Quellennachweis gestattet.</p> </li> </ul> </li> <li> <p><strong>zensus_population</strong></p> </li> <li> <p><strong>district_heating_shares_egon.csv</strong></p> </li> </ol>
Data for "Solving the OH + glyoxal problem: A complete theoretical description of post transition state energy deposition in activated systems."
Open the record for dataset details and reuse information.
Distributed Multi-objective Optimization in Cyber-Physical Energy Systems
<p>The data includes results for a distributed multi-objective optimization in Cyber-Physical Energy Systems. The respective implementation for the scenarios can be found here: https://github.com/Digitalized-Energy-Systems/MOO-CPES/releases/tag/Distributed_Multi-objective_Optimization_in_Cyber-Physical_Energy_Systems<br>In this case, a multi-agent system exists for the optimization in which agents represent chp units or wind plants. For the optimization using the agents, the agents have to fulfill a target schedule, with contains of the sum of all unit schedules. Regarding the target schedule, three objectives are considered: minimizing the difference between the<br>produced power in sum and the given target schedule, minimizing the emissions and minimizing the uncertainties.</p>
A comparative study between air cooling and liquid cooling thermal management systems for a high-energy lithium-ion battery module
Open the record for dataset details and reuse information.
AWESOME Energy System Model data and model
<p><span>This repository collects all the necessary items defined to setup and to run the Energy System optimization model for the AWESOME project.</span></p> <p><span>Detailed specifications of the adopted model (OSeMOSYS) and data are descripted in Deliverable D2.4 document: "</span><span>Future Energy Scenarios".</span></p> <p><span>In particular, the repository provides all essential data and scripts to define the energy model defined for projecting the energy scenarios developed for the AWESOME project. The document reports the development of an open-source energy system optimization model of the energy supply chain for the spatial domain useful for the AWESOME project (i.e. including Egypt, Ethiopia, and Sudan). The model is then used to explore different pathways of future energy scenarios in terms of energy demand and infrastructure evolution and their economic and environmental impacts. The future sectoral energy demand scenarios are developed based on the Socio-economic Pathways (SSPs) and the outcomes of D2.1 (Demographic projections), using a multi-sectoral optimal resource allocation economic model. </span></p> <p>This record contains:</p> <p>- The Deliverable D2.4, where the optimization model and the calculation of the energy system scenarios under different climatic scenarios are presented.</p> <p>- The results for each implemented scenario, in terms of installed capacity and energy generation of energy technologies (.tif files and excel files), for the Nile River Basin and at the national level for each focus country (Ethiopia, Sudan and Egypt).</p> <p>- Description of the data (excel file and pdf file)</p>
Multiple Parameter Replica Exchange Gaussian Accelerated Molecular Dynamics for Enhanced Sampling and Free Energy Calculation of Biomolecular Systems
Open the record for dataset details and reuse information.
Modelling and Comparing Converter Architectures and Energy Harvesting ICs for Battery-Free Systems
<p>Artifacts containing measurement scripts, data sets, and simulations for the paper "Modelling and Comparing Converter Architectures and Energy Harvesting ICs for Battery-Free Systems" (currently submitted and under review).</p>
Key results and plot files for the paper "Diversity of biomass usage pathways to achieve emissions targets in the European energy system"
<p>Key results and plot files for the paper:</p> <p>Millinger, M., Hedenus, F., Zeyen, E. <em>et al.</em> Diversity of biomass usage pathways to achieve emissions targets in the European energy system. <em>Nat Energy</em> <strong>10</strong>, 226–242 (2025). https://doi.org/10.1038/s41560-024-01693-6</p>
Supplementary material for "Comprehensive framework for dynamic energy assessment of building systems using IFC graphs and Modelica"
<p>The provided files include the following:</p> <ul> <li>The IFC model of the demo building used in Section 2.</li> <li>The parsed space boundaries graph.</li> <li>The resulting fragmented space boundaries graph.</li> </ul>
Understanding the Energy Consumption of Cloud-native Software Systems
<p>These artifacts contain the dataset generated in the paper "Understanding the Energy Consumption of Cloud-native Software Systems". The dataset contains resource utilisation, power consumption, estimated power consumption and load metrics for a cloud-native software system. The test setup is consists of 6 machines running an OpenStack cluster. This OpenStack cluster is running 12 virtual machines that are in turn hosting a Kubernetes cluster. Performance and (estimated) power consumption are measured on all levels of the system.</p> <p>The Bare-Metal (BM) and Virtual Machines (VM) are identified by their IP addresses. For BM the IP mapping is as follows:</p> <ul> <li>192.168.1.109 - i3-04</li> <li>192.168.1.110 - i3-02</li> <li>192.168.1.111 - i5-02</li> <li>192.168.1.112 - i3-01</li> <li>192.168.1.113 - i5-01</li> <li>192.168.1.114 - i5-04</li> </ul> <p>For VM the IP mapping is:</p> <ul> <li>192.168.1.190 - kubernetes-agent-7</li> <li>192.168.1.191 - kubernetes-agent-6</li> <li>192.168.1.192 - kubernetes-agent-10</li> <li>192.168.1.194 - kubernetes-master</li> <li>192.168.1.196 - kubernetes-agent-3</li> <li>192.168.1.197 - kubernetes-agent-1</li> <li>192.168.1.198 - kubernetes-agent-2</li> <li>192.168.1.201 - kubernetes-agent-4</li> <li>192.168.1.203 - kubernetes-agent-9</li> <li>192.168.1.204 - kubernetes-agent-5</li> <li>192.168.1.207 - kubernetes-agent-8</li> <li>192.168.1.209 - kubernetes-agent-11</li> </ul> <p>The VMs are deployed on the BMs as follows:</p> <ul> <li>i3-01 <ul> <li>kubernetes-agent-2</li> <li>kubernetes-agent-8</li> </ul> </li> <li>i3-02 <ul> <li>kubernetes-agent-1</li> <li>kubernetes-agent-6</li> </ul> </li> <li>i3-04 <ul> <li>kubernetes-agent-10</li> <li>kubernetes-agent-11</li> </ul> </li> <li>i5-01 <ul> <li>kubernetes-agent-3</li> <li>kubernetes-agent-5</li> </ul> </li> <li>i5-02 <ul> <li>kubernetes-master</li> <li>kubernetes-agent-7</li> </ul> </li> <li>i5-04 <ul> <li>kubernetes-agent-4</li> <li>kubernetes-agent-9</li> </ul> </li> </ul> <p>Note that the following BMs are excluded from the experiments as they have other roles in the cluster and do not run workloads:</p> <ul> <li>i3-03 (Juju - deploying OpenStack on BM nodes)</li> <li>i3-05 (ProxMox - external observability tools that do not run in the cluster)</li> <li>i5-03 (MAAS - provisioning of BM nodes)</li> </ul> <h1>Data Sets</h1> <p>The artifacts consists of 3 separate data sets: <code>constant</code>, <code>direct</code> and <code>linear</code>, corresponding to the respective load profile applied to the SUT as discussed in the paper. Each data set contains the same metrics, but the system is put under a different load.</p> <p>Every experiment is repeated 3 times. All 3 repetitions are included in the data set. The data is collected using Prometheus and stored in JSON files.</p> <p>For the <code>constant</code> and <code>linear</code> data sets, load is applied by deploying the "OpenTelemetry demo application", and sending automated user requests to the application. For the <code>direct</code> dataset, this application is not used and instead Kubernetes pods are created that apply a constant load to the cluster.</p> <p>The timestamps of the experiments are as follows:</p> <table> <tbody> <tr> <th>Run</th> <th>Iteration</th> <th>Start</th> <th>End</th> </tr> </tbody> <tbody> <tr> <td>Constant - 0 users</td> <td>1</td> <td>02-05-2024 10:08</td> <td>02-05-2024 10:52</td> </tr> <tr> <td> </td> <td>2</td> <td>06-05-2024 13:20</td> <td>06-05-2024 13:58</td> </tr> <tr> <td> </td> <td>3</td> <td>07-05-2024 09:20</td> <td>07-05-2024 09:54</td> </tr> <tr> <td>Constant - 50 users</td> <td>1</td> <td>02-05-2024 13:26</td> <td>02-05-2024 13:57</td> </tr> <tr> <td> </td> <td>2</td> <td>15-05-2024 14:06</td> <td>15-05-2024 14:38</td> </tr> <tr> <td> </td> <td>3</td> <td>07-05-2024 09:56</td> <td>07-05-2024 10:39</td> </tr> <tr> <td>Constant - 100 users</td> <td>1</td> <td>02-05-2024 14:00</td> <td>02-05-2024 14:40</td> </tr> <tr> <td> </td> <td>2</td> <td>06-05-2024 14:40</td> <td>06-05-2024 15:30</td> </tr> <tr> <td> </td> <td>3</td> <td>07-05-2024 10:42</td> <td>07-05-2024 11:33</td> </tr> <tr> <td>Constant - 200 users</td> <td>1</td> <td>16-05-2024 09:37</td> <td>16-05-2024 10:09</td> </tr> <tr> <td> </td> <td>2</td> <td>06-05-2024 15:32</td> <td>06-05-2024 16:13</td> </tr> <tr> <td> </td> <td>3</td> <td>07-05-2024 11:34</td> <td>07-05-2024 12:11</td> </tr> <tr> <td>Constant - 400 users</td> <td>1</td> <td>02-05-2024 15:24</td> <td>02-05-2024 15:55</td> </tr> <tr> <td> </td> <td>2</td> <td>06-05-2024 16:14</td> <td>06-05-2024 16:49</td> </tr> <tr> <td> </td> <td>3</td> <td>07-05-2024 12:13</td> <td>07-05-2024 13:08</td> </tr> <tr> <td>Linear</td> <td>1</td> <td>22-05-2024 15:13</td> <td>22-05-2024 15:56</td> </tr> <tr> <td> </td> <td>2</td> <td>22-05-2024 16:05</td> <td>22-05-2024 16:48</td> </tr> <tr> <td> </td> <td>3</td> <td>23-05-2024 09:58</td> <td>23-05-2024 10:41</td> </tr> <tr> <td>Direct</td> <td>1</td> <td>2024-05-23 13:01:28</td> <td>2024-05-23 14:00:05</td> </tr> <tr> <td> </td> <td>2</td> <td>2024-05-23 14:33:23</td> <td>2024-05-23 15:32:01</td> </tr> <tr> <td> </td> <td>3</td> <td>2024-05-23 15:39:36</td> <td>2024-05-23 16:38:12</td> </tr> </tbody> </table> <p>A complete log of the experiments can be found in <code>experiments_log.txt</code>.</p> <h2>Constant</h2> <p>The constant data set contains the metrics for the system under a constant load. The load is generated by a Locust script that sends requests to the system. Data collection starts 1 minute after the desired number of concurrent users is reached and requests have stabilised. This experiment is performed for the following constant number of concurrent users:</p> <ul> <li>0 users</li> <li>50 users</li> <li>100 users</li> <li>200 users</li> <li>400 users</li> </ul> <p>Note that the <code>0 users</code> data does not contain the <code>report_*.html</code> and <code>request_*.csv</code> files, as these are generated by Locust and Locust is not run for the <code>0 users</code> scenario.</p> <p>Furthermore, note that Horizontal Pod Autoscaling is <em>not</em> enabled for this data set.</p> <h2>Linear</h2> <p>The linear dataset contains the metrics for the system under a linearly scaling load. The load is generated through Locust. It starts at 0 users, and scales up to 100 users at a rate of 1 user per 20 seconds. After the load reaches 100 users, another 10 minutes of data is recorded. Every dataset is 45 minutes long, with 1.7 minutes of no load, 33.3 minutes of scaling up, and 10 minutes of max load.</p> <p>Note that Horizontal Pod Autoscaling <em>is</em> enabled for this data set.</p> <h2>Direct</h2> <p>The direct data set compliments the linear dataset. While the linear dataset provides a realistic load with things like networking factors being taken into account, the linear dataset can bottleneck on things like networking and the request client, so CPU usage is not maxed out. The direct dataset scales up linearly by applying direct CPU load to the Kubernetes pods without any application simulating a real usecase. The dataset works by deploying Kubernetes pods that max out immediately on exactly 200mCPU of load. The experiment starts with 1 pod, and 2 pods are added every 90 seconds up till 77 pods (the maximum number of pods the cluster allows to be scheduled). The first 1.5 minutes is no load, then 57 minutes to scale up, and then another 1.5 minutes at max load.</p> <p>Note that the <code>direct</code> data set does not contain the <code>app_*.json</code> files, as this data set does not deploy an application and therefore no application specific metrics are collected. Instead, a <code>script_log.txt</code> is provided that explains when and how the direct load was scaled up.</p> <h1>Files</h1> <p>A table for each file in the data set can be found below, including the unit of the metric and a description of what data is collected in that file. The format of all JSON files is the API response format used by Prometheus. More information on this topic can be found here: <a title="https://prometheus.io/docs/prometheus/latest/querying/api/" href="https://prometheus.io/docs/prometheus/latest/querying/api/">https://prometheus.io/docs/prometheus/latest/querying/api/</a>. All timestamps are in the CEST timezone.</p> <table> <tbody> <tr> <th>Filename</th> <th>Unit</th> <th>Description</th> </tr> </tbody> <tbody> <tr> <td>app_ads_ad_requests_total.json</td> <td>Total Count</td> <td>Total requests received by the ad microservice</td> </tr> <tr> <td>app_currency_counter_total.json</td> <td>Total Count</td> <td>Total currency that circulated through the system</td> </tr> <tr> <td>app_frontend_requests_total.json</td> <td>Total Count</td> <td>Total requests received by the frontend service</td> </tr> <tr> <td>app_payment_transactions_total.json</td> <td>Total Count</td> <td>Total transactions made to the transaction microservice</td> </tr> <tr> <td>app_recommendations_counter_total.json</td> <td>Total Count</td> <td>Total recommendations made by the recommendation microservice</td> </tr> <tr> <td>container_blkio_device_usage_total.json</td> <td>Total Bytes</td> <td>Total bytes used by blkio devices for pods</td> </tr> <tr> <td>container_cpu_usage_seconds_total.json</td> <td>Total Seconds</td> <td>Cumulative cpu time consumed by the pod</td> </tr> <tr> <td>container_cpu_user_seconds_total.json</td> <td>Total Seconds</td> <td>Cumulative user cpu time consumed by the pod</td> </tr> <tr> <td>container_fs_reads_bytes_total.json</td> <td>Total Bytes</td> <td>Cumulative count of bytes read by the pod</td> </tr> <tr> <td>container_fs_writes_bytes_total.json</td> <td>Total Bytes</td> <td>Cumulative count of bytes written by the pod</td> </tr> <tr> <td>container_memory_rss.json</td> <td>Bytes</td> <td>Resident Set Size of the pod</td> </tr> <tr> <td>kepler_container_bpf_cpu_time_ms_total.json</td> <td>Milliseconds</td> <td>CPU time for the pod as measured through a Kepler BPF program</td> </tr> <tr> <td>kepler_container_core_joules_total.json</td> <td>Total Joules</td> <td>Total energy consumption of CPU cores used by a pod</td> </tr> <tr> <td>kepler_container_dram_joules_total.json</td> <td>Total Joules</td> <td>Total energy consumption of DRAM used by a pod</td> </tr> <tr> <td>kepler_container_joules_total.json</td> <td>Total Joules</td> <td>Aggregated total energy consumption of a pod</td> </tr> <tr> <td>kepler_container_package_joules_total.json</td> <td>Total Joules</td> <td>Cumulative energy consumed by all cores and uncore components of a pod</td> </tr> <tr> <td>kepler_node_core_joules_total.json</td> <td>Total Joules</td> <td>Aggregation of core_joules of all pods running on a Kubernetes node</td> </tr> <tr> <td>kepler_node_dram_joules_total.json</td> <td>Total Joules</td> <td>Aggregation of dram_joules of all pods running on a Kubernetes node</td> </tr> <tr> <td>kepler_node_package_joules_total.json</td> <td>Total Joules</td> <td>Aggregation of package_joules of all pods running on a Kubernetes node</td> </tr> <tr> <td>node_cpu_scaling_frequency_hertz.json</td> <td>Hertz</td> <td>Current scaled cpu thread frequency of a machine (BM or VM)</td> </tr> <tr> <td>node_cpu_seconds_total.json</td> <td>Total Seconds</td> <td>Total number of seconds the CPU worked on a machine (BM or VM)</td> </tr> <tr> <td>node_disk_read_time_seconds_total.json</td> <td>Total Seconds</td> <td>Total number of seconds spent reading disk on a machine (BM or VM)</td> </tr> <tr> <td>node_disk_write_time_seconds_total.json</td> <td>Total Seconds</td> <td>Total number of seconds spent writing disk on a machine (BM or VM)</td> </tr> <tr> <td>node_hwmon_temp_celsius.json</td> <td>Celsius</td> <td>Temperature of the machine (BM) as reported by its monitoring hardware</td> </tr> <tr> <td>node_load1.json</td> <td>Load Average</td> <td>Load on the machine (BM or VM) averaged over 1 minute</td> </tr> <tr> <td>node_load5.json</td> <td>Load Average</td> <td>Load on the machine (BM or VM) averaged over 5 minutes</td> </tr> <tr> <td>node_load15.json</td> <td>Load Average</td> <td>Load on the machine (BM or VM) averaged over 15 minutes</td> </tr> <tr> <td>node_memory_Active_bytes.json</td> <td>Bytes</td> <td>Active number of bytes in memory on the machine (BM or VM)</td> </tr> <tr> <td>node_memory_Committed_AS_bytes.json</td> <td>Bytes</td> <td>Committed number of bytes in memory on the machine (BM or VM)</td> </tr> <tr> <td>node_rapl_core_joules_total.json</td> <td>Total Joules</td> <td>Total energy consumption of CPU cores by a machine, estimated by RAPL</td> </tr> <tr> <td>node_rapl_dram_joules_total.json</td> <td>Total Joules</td> <td>Total energy consumption of DRAM by a machine, estimated by RAPL</td> </tr> <tr> <td>node_rapl_package_joules_total.json</td> <td>Total Joules</td> <td>Total energy consumption of the machine package, estimated by RAPL</td> </tr> <tr> <td>node_rapl_psys_joules_total.json</td> <td>Total Joules</td> <td>Total energy consumption of the machine psys, estimated by RAPL</td> </tr> <tr> <td>power_consumption.json</td> <td>Watt</td> <td>Energy consumption as measured by the physical power plugs</td> </tr> <tr> <td>report_*.html</td> <td>-</td> <td>HTML report describing details of locust actions during the experiment</td> </tr> <tr> <td>requests_*.csv</td> <td>-</td> <td>Request summary per endpoint generated by locust</td> </tr> <tr> <td>scaph_host_power_microwatts.json</td> <td>Microwatt</td> <td>Power consumption of the whole machine as estimated by Scaphandre</td> </tr> <tr> <td>scaph_process_cpu_usage_percentage.json</td> <td>Percentage</td> <td>Per-process CPU usage as a percentage of total machine CPU</td> </tr> <tr> <td>scaph_process_memory_bytes.json</td> <td>Bytes</td> <td>Per-process memory usage</td> </tr> <tr> <td>scaph_process_power_consumption_microwatts.json</td> <td>Microwatt</td> <td>Per-process energy consumption as estimated by Scaphandre</td> </tr> <tr> <td>script.log</td> <td>-</td> <td>Log for the direct experiments for scaling up the pods</td> </tr> </tbody> </table> <h1>Scripts</h1> <div> <div>All scripts used to query this data from the prometheus endpoint and to generate the results in the associated paper are included in the scripts directory. To run a script, the script must be placed in the same directory as the data it is operated on (e.g. /scripts/constant/power_estimation.ipynb has to be in /constant).</div> </div>
Case studies used in the test of the MCDA-MSS for energy systems analysis, described according to its 156 features
<p>Case studies used in the test of the MCDA-MSS, described according to its 156 features.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.