Find research datasets worth reusing

Characteristics of negative cloud-to-ground lightning flashes associated with their peak currents of first return strokes

These are the data that were used to draw the figures in an submitted manuscript ("Characteristics of negative cloud-to-ground lightning flashes associated with their peak currents of first return strokes").

An evaluation of cloud-precipitation structures in mixed-phase stratocumuli over the southern ocean in kilometer-scale ICON simulations during CAPRICORN

The repository contains ICON simulated outputs (Cntrl_ICON.nc.gz, No_Gr_ICON.nc.gz, No_Ice_ICON.nc.gz, 100m_Vert_ICON.nc.gz, and 50m_Vert_ICON.nc.gz) and two km mean and ensemble forward simulated results (Cntrl_PAMTRA.gz, No_Gr_PAMTRA.gz, No_Ice_PAMTRA.gz, 100m_Vert_PAMTRA.gz, and 50m_Vert_PAMTRA.gz) for the two day period (00:00 UTC on March 26, 2016 to 00:00 UTC on March 28, 2016). The prefix in the filenames corresponds to the experiments described in the manuscript.

data of precipitation cloud system information for April-June 2018-2022 in South China

The three-dimensional structural information of precipitation cloud systems in South China for April-June 2018-2022 is recorded in this data, including the rain cells': id, latitude, longitude, DSD, precipitation contour, radar reflectivity factor, rainstorm top height, melting layer height and so on.

Cost Management in Cloud Computing: a study on financial waste and correction strategies

This project explores the identification and mitigation of financial wastage in Cloud Computing (CC) operations by professionals in the field, examining the alignment of their strategies with FinOps guidelines. Through a survey distributed via social media and messaging apps to CC professionals, the study engaged 28 participants from various companies, sizes, and sectors. It revealed that these professionals have encountered cost-related issues in cloud applications, particularly financial wastage. The investigation aimed to understand the prevalence of financial wastage in CC and evaluate the effectiveness of the adopted practices in addressing these inefficiencies within the framework of FinOps principles.

opencc-by-nc-nd-1.0Jun 2024View details →

DNS dataset for modelling homogeneous ignition processes of clustering solid particle clouds in isotropic turbulence

<h2>Abstract</h2> This dataset is being published to enable the development of models for igniting and combusting solid particles in isotropic turbulence using flamelet tabulated chemistry. This dataset is generated using the forced homogeneous isotropic turbulence in order to investigate the effect of active turbulent forces on the ignition of particles, and it is used as the supplementary material for the manuscript "Modeling homogeneous ignition processes of clustering solid particle clouds in isotropic turbulence", which was accepted for publication in the special issue of Fuel Journal for the proceeding of the 4th international Oxyflame workshop. The particles are chosen to have near unity Stokes numbers, which promote particle clustering to study the ignition phenomenon for particle clusters, which is common during ignition and combustion of particle clouds in large industrial solid fuel-powered burners. Since the ignition process of clustering solid particle clouds is transient, different time instances during the ignition process are presented to facilitate the modelling effort for the transient ignition behaviour of the particles. The dataset consists of gas-phase data, particle data and the most important routines required to process the dataset.    This database is a valuable resource for users developing solid fuel ignition and combustion in turbulent conditions.  It should be noted that this dataset is a reduced version of the full dataset in order to size limitations in the sharing platforms. More information and full dataset can be provided upon request. For more information, contact: p.farmand@itv.rwth-aachen.de <h2>Technical details</h2> The data provided in this dataset contains gas phase data, particle data, and some post-processing scripts for visualization of the data. The data is generated in forced homogeneous isotropic turbulence (HIT) with an initial preferential concentration of the particles in a hot atmosphere to study the impact of particle clustering on ignition. Simulations were performed within a region with the physical size of 12.8mm * 12.8mm * 12.8mm with periodic boundary conditions in all directions. The domain size is discretized with a three-dimensional cartesian mesh with a resolution of Δx = 50 μm. A forced isotropic turbulent field with Re_λ=30 and the Kolmogorov length scale η =100 microns has been chosen. The dispersed phase consists of 10,000 particles of Colombian coal with D_p= 20 microns and T0=300K and with an apparent density of 700kg/m3. Non-reactive particles are first randomly distributed in the box filled with air with 20% oxygen and an initial gas temperature of T = 1500K, which is relevant to practical PCC applications. These conditions lead to an initial Stokes number of around 5, for which a clustering behaviour in particle cloud motion is expected. The employed forced isotropic turbulence ensures maintaining the same turbulence statistics during non-reactive and reactive simulations, as summarised in the following table: <table> <tbody> <tr> <td> time [ms] </td> <td> Re_λ </td> <td> Re_Turb </td> <td> η[m] </td> <td> l_t[m] </td> <td> t_η[ms] </td> <td> t_l[ms] </td> <td> St </td> </tr> <tr> <td>0</td> <td>30.7</td> <td>141.8</td> <td>1.04e-4</td> <td>4.27e-3</td> <td>4.47e-2</td> <td>5.33e-1</td> <td>6.199</td> </tr> <tr> <td>0.46</td> <td>30.9</td> <td>143.9</td> <td>9.94e-5</td> <td>4.13e-3</td> <td>4.16e-2</td> <td>4.99e-1</td> <td>6.207</td> </tr> <tr> <td>0.5</td> <td>30.5</td> <td>140.1</td> <td>9.96e-5</td> <td>4.06e-3</td> <td>4.27e-2</td> <td>5.05e-1</td> <td>6.333</td> </tr> <tr> <td>0.55</td> <td>29.4</td> <td>129.8</td> <td>1.01e-4</td> <td>3.88e-3</td> <td>4.34e-2</td> <td>4.94e-1</td> <td>6.032</td> </tr> <tr> <td>0.6</td> <td>28.5</td> <td>122.1</td> <td>1.02e-4</td> <td>3.76e-3</td> <td>4.46e-2</td> <td>4.92e-1</td> <td>5.775</td> </tr> <tr> <td>0.65</td> <td>27.7</td> <td>115.2</td> <td>1.04e-4</td> <td>3.66e-3</td> <td>4.48e-2</td> <td>4.81e-1</td> <td>5.365</td> </tr> </tbody> </table> η and t_η are the respective Kolmogorov length and time scales, and l_t and t_l correspond to the integral length and time scales.   <h3>Gas phase and particle data:</h3> The gas phase data has HDF5 formation, which contains selected scalars relevant to model developments. Since the ignition process is a transient process, two different time instances, t=0.5ms with the maximum number of ignited regions during the ignition process and t=0.65ms at the end of the ignition process, are chosen. Before starting the reactive simulations, a non-reactive simulation for 20.5ms was performed to form the particle clusters, and then the reactive simulation was started. Therefore, the data.out_2.100E-02.h5 corresponds to t = 0.5ms and data.out_2.115E-02.h5 corresponds to t = 0.65ms. Each HDF5 dataset has the following structure: Group Flow: <ul> <li>U, V, W</li> </ul> Group scalar:  <ul> <li>CO, CO2, H2O, C2H2, O2, N2, OH, progress variable (PV), temperature(T), enthalpy(h), heat capacity (Cp), density(RHO), pressure(P), mixture fraction(Z), dissipation rate, heat conductivity</li> </ul> Group C_dot: <ul> <li>Molar production rate of CO, CO2, O2, OH</li> </ul> Group ST: <ul> <li>Mass fraction rate of OH</li> </ul> Group var (data required for subfilter analysis and PDF modelling): <ul> <li>RHO * h</li> <li>RHO * h * h</li> <li>RHO * PV</li> <li>RHO * PV * PV</li> <li>RHO * RHO</li> <li>RHO * RHO * RHO</li> <li>RHO * T</li> <li>RHO * T * T</li> <li>RHO * Z</li> <li>RHO * Z * Z</li> </ul> These gas phase data can be used to study the Eulerian field, which is impacted by the particles through the source terms, which are obtained from the Lagrangian framework. For the particle data, two CSV files containing information about each particle's position and temperature are provided.  In each CSV file, which corresponds to the same time instance as the gas phase data, this structure can be found: <ul> <li>Points_0, Points_1, Points_2: particle position x,y,z</li> <li>coal_defaultT: particle temperature</li> </ul> <h3>Scripts:</h3> Different scripts are provided for a reader to enable the first-time use of the data, such as visualising the gas phase quantities, calculating the conditional mean and statistical analysis, and clustering analysis for the particles. Here is a brief information regarding the scripts: <ul> <li>hdf5_2D_Con_Mean_Plots.m: This script can be used for loading the HDF5 file and visualising the field, joint PDF, and joint correlation of different quantities with respect to different parameters. This script requires another module to calculate the conditional mean of the data.</li> <li>Compute_ConditionalMean_histograms_2D.m: This module calculates the conditional average of a 2D array.</li> <li>clustering_voronoi_3D.m: This script can analyse the particle data and calculate the clustering limit based on the Voronoi algorithm. It can also filter the clustered particles and separate different clusters using the nearest neighbour and DB-SCAN methods.</li> </ul>

opencc-zeroJun 2024View details →

Cloud Computing Badge

The Cloud Computing Badge can be used for presentations, courses, ...

Photos data for "Revealing Halos Concealed by CIrrus Clouds"

Original photo data which are shown on the figures  in the manuscript, "Revealing Hlaos Concealed by Cirrus Clouds".

Innovative 3D Cone Point Cloud Fitting via Landweber Iteration

Cone surface fitting is essential in various fields, including computer graphics, computer vision, and robotics.  However, factors such as noise, initial parameter selection, and the varying distribution of point clouds can significantly impact fitting accuracy and stability.  To address these challenges, we propose a novel optimization method based on Landweber iteration.  This method solves an objective function that includes the cone vertex.  Initially, Landweber iteration is used to calculate high-precision initial values for the conical surface by leveraging all point cloud data.  Subsequently, the RANSAC algorithm filters the point cloud data based on these initial values, eliminating points that do not meet the distance threshold.  Finally, an error equation is formulated, and the cone surface parameters are further refined using an optimization algorithm based on Landweber iteration.  Simulation experiments validate the feasibility and robustness of our approach, demonstrating superior accuracy and stability in 3D cone surface fitting.

Cloud-resolving model for machine learning buoyant cloudy updraught

Atmospheric model data from simulations carried out using the Unified Model nesting suite u-cj161. This repository includes scripts to perform all the required steps. Retrieval of the model output files from the archive: ml_cape_pdf_1_retrieve_lam_files.py  Coarse-graining of the 1.5 km atmospheric model data to a series of different "global model" resolutions by calculating mean profiles of key thermodynamic variables and the calculation of the fraction of that coarse-volume in which the 1.5 km pixels meet the criterion of being bouyant cloudy updraughts: ml_cape_pdf_2d_coarse_grain_bcu.py  Subsampling to rebalance the data set since so much of the raw data consists of examples of no convectie activity: ml_cape_pdf_3e_sub_sample_bcu.py  There was too much data to process all the raw data in one go, so it was done is several steps. There is then a need to join multiple files together: ml_cape_pdf_3g_stitch_together_bcu_files.py  The data is on the Unified Model L70 grid (counting from the bottom up), but the goal is to deploy the machine-learnt algorithm on the E3SM grid (72 levels counting from the top down), so some regirddign is required: ml_cape_pdf_3h_regrid_um_to_e3sm.py  Training of 1 two-headed 1d CNN is done using: ml_cape_pdf_4c_bcu_cnn.py  Evaluation against witheld data is done using: ml_cape_pdf_5b_validate_BCU.py

Deep Gradient Reinforcement learning for Music Improvisation in cloud computing framework

The improvised music is further rendered in the MIDI format. The Bach Chorales dataset with six different attributes relevant to musical compositions is employed in implementing the present research. The model was set up in a containerised cloud environment and controlled for smooth load distribution. Five different parameters, such as pitch frequency (PF), standard pitch delay (SPD), average distance between peaks (ADP), note duration gradient (NDG) and pitch class gradient (PCG) are leveraged to assess the quality of the improvised music.

Supplemental Data for "Adaptive Container Service: a New Paradigm for Robust and Optimized Bioinformatics Workflow Deployment in the Cloud."

All supplemental data for "Adaptive Container Service: a New Paradigm for Robust and Optimized Bioinformatics Workflow Deployment in the Cloud." Abstract: We propose Adaptive Container Service (ACS), a new paradigm for deploying bioinformatics workflows in cloud computing environments. By encapsulating the entire workflow within a single virtual container, combined with automatic workflow checkpointing and dynamic migration to appropriately scaled containers, ACS-based deployment demonstrates several key advantages over alternative strategies: it enables optimal resource provision to any workflow that comprise of multiple applications with diverse computing needs; it provides protection against application-agnostic out-of-memory (OOM) errors or spot instance interruptions; and it reduces efforts required for workflow development, optimization, and management because it runs workflows with minimal or no code modifications. Proof-of-concept experiments show that ACS avoided both under- and over-provisioning in monolithic single-container deployment. Despite being deployed as a single container, it achieved comparable resource utilization efficiency as optimized Nextflow-managed, multi-modular workflows. Analysis of over 18,000 workflow runs demonstrated that ACS can effectively reduce workflow failures by two-thirds. These findings suggest that ACS frees developers from navigating the complexity of deploying robust workflows and rightsizing compute resources in the cloud, leading to significant reduction in workflow development time and savings in cloud computing costs. Contains the following directories: Fig2-bbtools: running metrics for BBTools Fig3-rna-seq: running metrics for RNA-Seq Fig4-Job_records: meta data and running metrics of 18,000+ jobs

Galaxy job runtime measurements with Encrypted and plain storage volumes on Cloud deployments.

Storage volume performance measurement on Cloud environment using Galaxy and Mapping tools: Bowtie2, STAR and Salmon. Galaxy job runtime for encrypted and not ecrypted storage volumes are reported. Scripts and Documentation on GitHub.

Cloud Vertical Structure

Active cloud observations from A-Train’s CloudSat and CALIPSO satellites offer opportunities to examine the vertical structure of hydrometeor layers. We use the 2B-CLDCLASS-LIDAR merged CloudSat-CALIPSO product to examine global aspects of hydrometeor vertical stratification. We group the data into major cloud vertical structure (CVS) classes based on our interpretation of how clouds in three standard atmospheric layers overlap and provide their global frequency of occurrence.  Reference : Oreopoulos, L., N. Cho, and D. Lee (2017), New insights about cloud vertical structure from CloudSat and CALIPSO observations, J. Geophys. Res. Atmos., 122, 9280–9300, doi:10.1002/2017JD026629.

Evaluating Cloud Properties Across New Zealand: Ceilometer Network Data

A network of 16 Vaisala CL31 ceilometers operated by MetService at airports across New Zealand had data available and were processed using ALCF with a cloud threshold mask of 6e-6. Nine sites were in Te Ika-a-Māui / North Island, six in Te Waipounamu / South Island and one site on Chatham Island. Metadata for the 16 sites are available in the Processed Ceilometer Medata.csv file.

Improving Volcanic SO2 Cloud Modeling Through Data Fusion and Trajectory Analysis: A Case Study of 2022 Hunga Tonga Eruption

Dataset Overview: This dataset comprises approximately 500 clusters of aggregated observational data collected from January 16 to 20 during the ascending (ASC) and descending (DES) periods. We grouped a large number of observation points into these clusters and calculated trajectories from the center of each cluster. The choice of 500 clusters was driven by pragmatic considerations, aiming for a balance between computational feasibility and the level of detail needed for our analysis. Data Unit Description: The "mass" values in this dataset for each cluster are calculated by multiplying the mass per unit area (g/m2) of individual data points by the area covered by each point, thus providing the total mass in grams (g). The "heights" are presented in units of kilometers (km), representing the observed top heights of each cluster.

CONUS and sub-regional monthly cloudiness supporting for the trend analysis: "CONUS Cloud Pattern Change 1980-2020" Vo T.T., Hu L., Xue L., Chen S. (2024)

The dataset supporting for the publication: "CONUS Cloud Pattern Change 1980-2020" Vo T.T., Hu L., Xue L., Chen S. (2024), Journal of Climate.  <ul> <li>The data format is in tabular form (.csv, comma delimited). Each row is the monthly aggregated for each sub-region  More specifically, the description of each column with associated with its unit formatted in the table are listed as follows: <ul> <li>datetime: time of the observation (formatted as month/day/year)</li> <li>original_time_series: original cloud coverage before processing using the trend analysis mentioned in the paper (unit: percentage, %)</li> <li>enso_neutral: cloud coverage removing the ENSO effect using linear regression method followed by Gu and Adler 2011 (unit: percentage, %)</li> <li>cloud_coverage: cloud coverage removing the ENSO effect and seasonality and remainder (refer to the cloud coverage used in the manuscript) using Seasonal Decomposition of Time Series by Loess (STL) method (unit: percentage, %)</li> <li>region_name_list: name of the sub-region</li> <li>cloud_type: certain cloud type (all clouds refers to total clouds)</li> </ul> </li> </ul>   References:  Gu, G., & Adler, R. F. (2011). Precipitation and temperature variations on the interannual time scale: Assessing the impact of ENSO and volcanic eruptions. Journal of Climate, 24(9), 2258–2270. https://doi.org/10.1175/2010JCLI3727.1 Cleveland, R. B., Cleveland, W. S., & Terpenning, I. (1990). STL: A Seasonal-Trend Decomposition Procedure Based on Loess. Journal of Official Statistics, 6(1), 3. http://ezproxy.montevallo.edu:2048/login?url=https://www.proquest.com/scholarly-journals/stl-seasonal-trend-decomposition-procedure-based/docview/1266805989/se-2?accountid=12538

Dataset for "Aerosol mixing state, new particle formation, and cloud droplet number concentration in an urban environment "

Inverted HTDMA data, parcel model input, and parcel model output associated with the manuscript. HTDMA data are from the TRACER campaign and inverted using the LSQ1 or LSQ2 method described in Petters (2021). Model input is a 4 lognormal aerosol mode representation of the aerosol size distribution during TRACER, obtained by fitting measured size distributions obtained from mobility-based and optical-based sensors. Hygroscopicity is assigned for each mode based on observed size-resolved hygroscopicity from the HTDMA measurements. Model output refers to parcel model simulations using the pyrcel model (Rothenberg and Iompar). Petters, M. D.: Revisiting matrix-based inversion of scanning mobility particle sizer (SMPS) and humidified tandem differential mobility analyzer (HTDMA) data, Atmos. Meas. Tech., 14, 7909–7928, https://doi.org/10.5194/amt-14-7909-2021, 2021.  Rothenberg, D. and lompar: darothen/pyrcel: Pyrcel v1.3.2, , https://doi.org/10.5281/zenodo.8378595, 2023.563

Data to support 'Deforestation amplifies climate change effects on warming and cloud level rise in African montane forest'

This respository contains output data to support the manuscript titled 'Deforestation amplifies climate change effects on warming and cloud level rise in African montane forest' by Temesgen Alemayehu Abera, Janne Heiskanen Eduardo Eiji Maeda, Mohammed Ahmed Muhammed, Netra Bhandari, Ville Vakkari, Binyam Tesfaw Hailu, Petri K.E. Pellikka, Andreas Hemp, Pieter G. van Zyl, and Dirk Zeuss

Section 5.2 "Task Area 2: Technical infrastructure and cloud resources" Figure 9

Figure 9. In the NFDI4BIOIMAGE Cloud, provisional S3 storage and core SaaS offerings will be provided by platform operators for all participants. Service operators may deploy and manage services in the IaaS cloud or in Kubernetes. from NFDI Grant Application, "National Research Data Infrastructure for Microscopy and Bioimage Analysis" (NFDI4BIOIMAGE)