Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
2,170
datasets available to search
ShareScore release 0.7.1
Dataset results
2,170 results for “cloud”
ICON-LEM Ny-Ålesund low-level clouds polar night and polar day 2021/2022
<h3>Low-level clouds during the polar night and polar day simulated in ICON-LEM for Ny-Ålesund </h3> <p>This data set was created using the ICON-LEM model with ca. 600m resolution and a diagnostic tool "microphysical wrapper". It contains the meteogram output of the Ny-Ålesund column (Svalbard) and the microphysical process rates. The data was created for the polar night (Nov 2021- Feb 2022) and polar day (May - Aug 2022). Clouds are classified as low-level if their cloud top height (CTH) is below 2.5 km and the distance between any cloud with CTH above 2.5 km is at least 500 m higher. The data set was first used and described in the <em>publication: </em></p> <p>T. Kiszler, D. Ori, V. Schemann<em>. </em>(preprint) Microphysical processes involving the vapour phase dominate in simulated low-level Arctic clouds. <em>Atmospheric Physics and Chemistry, </em>https://doi.org/10.5194/egusphere-2023-2986<em><br></em></p> <p>This data is related to the repository <a href="https://github.com/TracyMcBean/Kiszler_et_al_2023_microphysics">https://github.com/TracyMcBean/Kiszler_et_al_2023_microphysics</a></p> <p><em>File description:</em></p> <p>*_PN is polar night data</p> <p>*_PD is polar day data</p> <p>LLC_<em>meteo_<yyyymm>_ICONv1</em>_v6.nc : Contains the meteogram variables (thermodynamics, surface variables, hydrometeors)</p> <p>LLC_wrapper_mass_<yyyymm>_ICONv1_v6.nc : Contains hydrometeors masses after diagnostic run of a microphysical wrapper</p> <p>LLC_wrapper_tend_<yyyymm>_ICONv1_v6.nc : Contains the mircophysical process rates showing the mass change per timestep </p> <p>low_cloud_times_v6_*.csv : Contains the date and time when a low-level cloud was detected</p>
Benchmarking on Microservices Configurations and the Impact on the Performance in Cloud Native Environments
<p><strong>The peer reviewed publication for this dataset has been published in LCN 2022, 47th Annual IEEE Conference on Local Computer Networks. Please cite this paper when referring to the dataset: https://www.eurecom.fr/publication/6971.</strong></p> <p>Cloud-native and containerization have changed the way to develop and deploy applications. Cloud-native rethinks the application architecture by embracing a microservice approach, where each microservice is packaged into containers to run in a centralized or an edge cloud. When deploying the container running the micro-service, the tenant has to specify the needed computing resources to run their workload in terms of the amount of CPU and memory limit. However, it is not straightforward for a tenant to know in advance the computing amount that allows running the microservice optimally. This will have an impact not only on the service performances but also on the infrastructure provider, particularly if the resource overprovisioning approach is used. To overcome this issue, we conduct an experimental study aiming to detect if a tenant's configuration allows running its service optimally. We run several experiments on a cloud-native platform, using different types of applications under different resource configurations. The obtained results are presented in the accepted IEEE LCN paper (https://www.eurecom.fr/publication/6971) and are shared in this dataset.</p> <p>The datasets are collected for 3 types of applications: Web servers written in python and Golang, RabbitMQ data broker and the OpenAirInterface 5G Core network function AMF (Access and Mobility Management Function).</p> <p><br> </p> <p><strong>Web Servers:</strong></p> <p><strong>files: </strong>golang-web-server-performance.csv, python-web-server-performance.csv</p> <p>We used Golang and Python-based web servers for the test. Each request to the web server returns a video of a size 43 MB. For testing we used ApacheBench, a command-line program used for benchmarking HTTP web servers. ApacheBench allows parallel requests from multiple clients. For each web server instance we send a number of requests ranging from 100 to 1000 and a concurrency level between 1 and 100, representing the number of parallel clients performing the requests.</p> <p>The information available in the dataset are as follows:</p> <p><strong>time:</strong> timestamp of collection of metrics.</p> <p><strong>ram_limit:</strong> the memory allocated to the container in megabytes.</p> <p><strong>cpu_limit:</strong> the CPU allocated to the container.</p> <p><strong>ram_usage:</strong> the amount of memory used by the container at the time of the metrics collection in byte.</p> <p><strong>cpu_usage:</strong> the amount of CPU used by the container at the time of the metrics collection.</p> <p><strong>n:</strong> the number of requests sent to the container.</p> <p><strong>c:</strong> the concurrency level in the requests.</p> <p><strong>lat50:</strong> the least response time for the best 50% requests in microseconds.</p> <p><strong>lat66:</strong> the least response time for the best 66% requests in microseconds.</p> <p><strong>lat75:</strong> the least response time for the best 75% requests in microseconds.</p> <p><strong>lat80:</strong> the least response time for the best 80% requests in microseconds.</p> <p><strong>lat90:</strong> the least response time for the best 90% requests in microseconds.</p> <p><strong>lat95:</strong> the least response time for the best 95% requests in microseconds.</p> <p><strong>lat98:</strong> the least response time for the best 98% requests in microseconds.</p> <p><strong>lat99:</strong> the least response time for the best 99% requests in microseconds.</p> <p><strong>lat100:</strong> the least response time in microseconds.</p> <p> </p> <p><strong>5G Core network’s AMF:</strong></p> <p><strong>file: </strong>amf-performance.csv</p> <p>For testing we use my5G-RANTester, a tool for emulating control and data planes of the UE and gNB (5G base station). The number of simultaneous registration requests that are sent to each instance of the AMF varies between 10 and 400.</p> <p>The information available in the dataset are as follows:</p> <p><strong>time:</strong> timestamp of collection of metrics.</p> <p><strong>ram_limit:</strong> the memory allocated to the container in megabytes.</p> <p><strong>cpu_limit:</strong> the CPU allocated to the container.</p> <p><strong>ram_usage:</strong> the amount of memory used by the container at the time of the metrics collection in byte.</p> <p><strong>cpu_usage:</strong> the amount of CPU used by the container at the time of the metrics collection.</p> <p><strong>n:</strong> the number of parallel registration requests sent to the AMF.</p> <p><strong>mean:</strong> the mean registration time for all the registration requests in microseconds.</p> <p><strong>lat50:</strong> the median registration time for registration requests in microseconds.</p> <p><strong>lat75: </strong>the least registration time for the best 75% registration requests in microseconds.</p> <p><strong>lat80:</strong> the least registration time for the best 80% registration requests in microseconds.</p> <p><strong>lat90:</strong> the least registration time for the best 90% registration requests in microseconds.</p> <p><strong>lat95:</strong> the least registration time for the best 95% registration requests in microseconds.</p> <p><strong>lat98:</strong> the least registration time for the best 98% registration requests in microseconds.</p> <p><strong>lat99:</strong> the least registration time for the best 99% registration requests in microseconds.</p> <p><strong>lat100:</strong> the least registration time in microseconds.</p> <p> </p> <p><strong>RabbitMQ data broker:</strong></p> <p><strong>file: </strong>rabbitmq-performance.csv</p> <p>For testing we used RabbitMQ PerfTest which is a throughput testing tool that simulates basic workloads and provides the throughput and the time that a message takes to be consumed by a consumer. For each deployed RabbitMQ server we used a number of producers and consumers that ranges from 50 to 500. Each producer sends messages to the broker with a rate of 100 messages per second for a period of time of 90 seconds.</p> <p>The information available in the dataset are as follows:</p> <p><strong>time:</strong> timestamp of collection of metrics.</p> <p><strong>ram_limit:</strong> the memory allocated to the container in megabytes.</p> <p><strong>cpu_limit:</strong> the CPU allocated to the container.</p> <p><strong>ram_usage:</strong> the amount of memory used by the container at the time of the metrics collection in byte.</p> <p><strong>cpu_usage:</strong> the amount of CPU used by the container at the time of the metrics collection.</p> <p><strong>n:</strong> the number of producers sending messages to the RabbitMQ server.</p> <p><strong>Min:</strong> the minimum consumption time for the producer messages.</p> <p><strong>lat50:</strong> the median consumption time for the producer messages.</p> <p><strong>lat75:</strong> the least consumption time for the best 75% messages in microseconds.</p> <p><strong>lat95:</strong> the least consumption time for the best 95% messages in microseconds.</p> <p><strong>lat99:</strong> the least consumption time for the best 99% messages in microseconds.</p>
Dataset for paper "Ejecta cloud distributions for the statistical analysis of impact cratering events onto asteroids' surfaces: a sensitivity analysis"
<p>Dataset for the paper "Ejecta cloud distributions for the statistical analysis of impact cratering events onto asteroids' surfaces: a sensitivity analysis" published in Icarus.</p>
Composited land surface temperature of the greater Phoenix, Arizona, USA metropolitan area and surrounding Sonoran desert derived from cloud-free, summer (June, July, and August) Landsat imagery: 1985-2020
This project calculates land surface temperature (LST) from remotely sensed imagery. The intent is to extend the previous version of the LST data for the CAP LTER study area in central Arizona, USA to include 2020 and update the products so that they are based on a composite of images from each year (all available cloud-free acquisitions from June, July, and August) in the analysis to reduce the potential for outlier images or pixels to impact analyses. The aim is to make updated LST data accessible to stakeholders and researchers studying the greater Phoenix, Arizona, USA metropolitan area. LST is calculated from cloud-free Landsat 5 and 8 imagery (30m resolution) from summer months (June, July, and August) in 1985, 1990, 1995, 2000, 2005, 2010, 2015, and 2020. All images are cropped to the CAP LTER study area boundary.
Point cloud data from terrestrial laser scanning for stem volume modelling of Scots pine trees
<p>Stem volume is a key forest inventory attribute characterizing growth and yield of individual trees and forest stands. Three-dimensional information from terrestrial laser scanning (TLS) can be used to reconstruct tree stems and provide information on stem volume as well as stem shape. We collected diameter at breast height and height information with traditional field measurements as well as preprocessed TLS point cloud data on 230 Scots pine trees (<em>Pinus sylvestris L.</em>) from southern Finland. The data set here includes three-dimensional information on Scots pine tree stems derived from TLS point clouds. The usage of this data set can include, but is not limited to, development of point cloud processing algorithms for single tree stem reconstruction and investigations of of stem volume modelling for Scot pine. </p> <p>This data set includes two files: Scots_pines.txt includes DBH and height information based on field measurements from the 230 Scots pine trees. File includes the following columns: treeID, DBH, and h, where DBH is presented in cm and h (i.e. tree height) in m. Stem_points.zip, on the other hand, includes 230 laz-files where figure in the name of the laz-file refers to the tree ID in Scots_pines.txt-file. Laz-files include three columns that describe x, y, and z, coordinates (in meters) of stem points in a local coordinate system extracted from the normalized TLS point clouds (i.e. z coordinate describes height above ground).</p>
INTERACT-II (INTERcomparison of Aerosol and Cloud Tracking - II)
<p>Following the previous efforts of INTERACT (INTERcomparison of Aerosol and Cloud Tracking), the INTERACT-II campaign used multi-wavelength Raman lidar measurements to assess the performance of an automatic compact micro-pulse lidar (MiniMPL) and two ceilometers (CL51 and CS135) in providing reliable information about optical and geometric atmospheric aerosol properties. The campaign took place at the CNR-IMAA Atmospheric Observatory (760 ma.s.l.; 40.60<sup>∘</sup> N, 15.72<sup>∘</sup> E) in the framework of ACTRIS-2 (Aerosol Clouds Trace gases Research InfraStructure) H2020 project. Co-located simultaneous measurements involving a MiniMPL, two ceilometers and two EARLINET multi-wavelength Raman lidars were performed from July to December 2016.</p> <p>All the data from the CIAO lidars, the MiniMPL and from theCHM15k, CS135 and the CT25K ceilometers, operating collocated and simultaneously during the INTERACT-II campaign, are provided here. Additional files for the correction of the MiniMPL incomplere overlap are also provided.</p> <p>The results of the campaign are described in detail in Madonna et al., 2018 (<a href="https://amt.copernicus.org/articles/11/2459/2018/">https://amt.copernicus.org/articles/11/2459/201</a>8/).</p>
Cloud Condensation Nuclei number concentrations over the Southern Ocean during the austral summer of 2016/2017 on board the Antarctic Circumnavigation Expedition (ACE).
<p><strong>Dataset abstract</strong></p> <p>Cloud Condensation Nuclei (CCN) are a subclass of atmospheric aerosol particles, which can be activated to cloud droplets at a certain supersaturation, with respect to water. Due to their abundance, these particles can affect micro-physical properties of clouds, while acting as CCN. It was found that CCN are relevant for the Earth’s radiation budget, by affecting cloud albedo and lifetime. When giving a number concentration of CCN, also the supersaturation at which it was measured has to be given.</p> <p>With additional information on particle number size distribution, the hypothetical diameter of particle activation (critical diameter) was derived. Further, the particle hygroscopicity parameter (kappa) was calculated using the critical diameter. Values of kappa can be a proxy for bulk chemical composition of the sampled CCN population.</p> <p>Our dataset gives CCN number concentrations measured by a CCN counter (type CCN-100 by DMT, Boulder, US) operated at five different levels of supersaturation (0.15%, 0.2%, 0.3%, 0.5%, 1%) during the Antarctic Circumnavigation Expedition (ACE) cruise over the Southern Ocean, as part of the ACE-SPACE project. Temporal coverage is from December 20, 2016 to March 19, 2017. We give 5-minute averaged and quality controlled CCN number concentrations, critical diameter and kappa values.</p> <p><strong>Dataset contents</strong></p> <ul> <li>ACESPACE_cloud_condensation_nuclei_number_concentration_SS015.csv, data file, comma-separated values</li> <li>ACESPACE_cloud_condensation_nuclei_number_concentration_SS020.csv, data file, comma-separated values</li> <li>ACESPACE_cloud_condensation_nuclei_number_concentration_SS030.csv, data file, comma-separated values</li> <li>ACESPACE_cloud_condensation_nuclei_number_concentration_SS050.csv, data file, comma-separated values</li> <li>ACESPACE_cloud_condensation_nuclei_number_concentration_SS100.csv, data file, comma-separated values</li> <li>ACESPACE_cloud_condensation_nuclei_critical_diameter_SS015.csv, data file, comma-separated values</li> <li>ACESPACE_cloud_condensation_nuclei_critical_diameter_SS020.csv, data file, comma-separated values</li> <li>ACESPACE_cloud_condensation_nuclei_critical_diameter_SS030.csv, data file, comma-separated values</li> <li>ACESPACE_cloud_condensation_nuclei_critical_diameter_SS050.csv, data file, comma-separated values</li> <li>ACESPACE_cloud_condensation_nuclei_critical_diameter_SS100.csv, data file, comma-separated values</li> <li>ACESPACE_cloud_condensation_nuclei_hygroscopicity_parameter_SS015.csv, data file, comma-separated values</li> <li>ACESPACE_cloud_condensation_nuclei_hygroscopicity_parameter_SS020.csv, data file, comma-separated values</li> <li>ACESPACE_cloud_condensation_nuclei_hygroscopicity_parameter_SS030.csv, data file, comma-separated values</li> <li>ACESPACE_cloud_condensation_nuclei_hygroscopicity_parameter_SS050.csv, data file, comma-separated values</li> <li>ACESPACE_cloud_condensation_nuclei_hygroscopicity_parameter_SS100.csv, data file, comma-separated values</li> <li>data_file_header_number_concentration.txt, metadata, text</li> <li>data_file_header_critical_diameter.txt, metadata, text</li> <li>data_file_header_hygroscopicity_parameter.txt, metadata, text</li> <li>change_log.txt, metadata, text</li> <li>README.txt, metadata, text</li> </ul> <p>The files listed above contain Cloud Condensation Nuclei (CCN) number concentration (N_CCN), critical diameter (D_crit) and particle hygroscopicity parameter (KAPPA) values for the Antarctic Circumnavigation Expedition from in-situ measurements. Each file contains only N_CCN, D_crit or KAPPA values for one of the five measured levels of supersaturation (SS), e.g., N_CCN at SS=0.15% in ACESPACE_cloud_condensation_nuclei_number_concentration_SS015.csv or N_CCN at SS=0.2% in ACESPACE_cloud_condensation_nuclei_number_concentration_SS020.csv etc. In addition, for each N_CCN value the respective temperature of the CCNCs measurement column (T_col) is given. Values are from 1 Hz measurements and averaged to represent 5-minute intervals.</p> <p>For every given value of CCN number concentration, the respective supersaturation level is given, although files only contain values for one level only. Additionally, longitude and latitude for the ship’s position at the start time of the averaging period are given.</p> <p>For latitude and longitude nan values are given, in cases where positioning data was not available for the given time period. There are no nan values for CCN number concentration included, in a way that only quality assured data is given.</p> <p><strong>Change log</strong></p> <p>v1.1 - data files updated</p> <ul> <li>change dataset title to specify ACE cruise</li> <li>change time resolution to 5 minutes</li> <li>addition of critical diameter data</li> <li>addition of hygroscopicity parameter data</li> <li>create separate data_file_headers</li> <li>add change log</li> </ul> <p>v1.0 - initial release of dataset</p>
LAUTx - Individual Tree Point Clouds From Austrian Forest Inventory Plots
<p>This dataset contains manually segmented tree point clouds from Personal Laser Scanning (PLS) data, and additionally automatic segmented trees from the same point clouds. The raw point cloud data has been published in LAUT - Terrestrial and Personal laser scanner data from Austrian forest Inventory plots (<a href="https://doi.org/10.5281/zenodo.3698956">https://doi.org/10.5281/zenodo.3698956</a>) and six of those plots were processed for this data. Purpose of this data is to serve as benchmarking for automatic tree segmentation algorithms.</p>
A Data-driven Analysis of a Cloud Data Center: Statistical Characterization of Workload, Energy and Temperature
<p>A characterization of cloud data center logs, analyzing its workload, energy and thermal characteristics. For more details of the dataset, please read the following paper: <a href="http://hpc.ec.tuwien.ac.at/files/UCC_23_data_center_analysis.pdf">http://hpc.ec.tuwien.ac.at/files/UCC_23_data_center_analysis.pdf.</a></p><p> </p><p>If you use the dataset, please cite the following work:</p><p>Shashikant Ilager, Adel N. Toosi, Mayank Raj Jha, Ivona Brandic, Rajkumar Buyya, "A Data-driven Analysis of a Cloud Data Center: Statistical Characterization of Workload, Energy and Temperature", In Proceedings of the 16th IEEE/ACM International Conference on Utility and Cloud Computing (UCC2023), Messina, Italy, December 4-7, 2023.</p>
Single column 1D radiative transfer simulations during PS106 including low-level-stratus clouds in the central Arctic
<p>The collection of datasets published contain the input parameters and output simulations from a single column 1D radiative transfer simulations using the <strong>R</strong>apid <strong>R</strong>adiative <strong>T</strong>ransfer <strong>M</strong>odel for <strong>G</strong>eneral Circulation Model (GCM) applications (RRTMG).</p><p>The data set contains simulations for the PS106 research cruise conducted in 2017 in the Central Arctic. The simulations are based on remote sensing data which were processed with the Cloudnet algorithm to derive cloud macro - and microphyiscal products. The atmospheric profiles of temperature, pressure, and ozone are from ERA5 (European Centre for Medium-Range Weather Forecasts (ECMWF) Re-Analysis) and values of surface albedo from CERES (Clouds and the Earth's Radiant Energy System) SYN1deg Ed. 4.1.</p>
Processing of 3-D Polygon Mesh Model and Radio Propagation Simulations in a Cave: Surface Reconstruction from Point Cloud, Simplification of the Mesh, and Ray Tracing
<p><strong>ABOUT</strong></p><p>This repository includes mesh data from cave geometry scanning and processing, and radio propagation data from ray tracing simulations.</p><p>The geometry data is obtained with laser scanning in a cave in Slovenija. </p><p>The geometry processing includes (i) 3-D shape reconstruction - surface reconstruction from point cloud data and (ii) simplification - reduction of the geometric complexity of the 3-D mesh model. </p><p>The radio propagation data is obtained using CloudRT [1] ray-tracing simulator. </p><p>The obtained propagation-related quantities include information about the propagation mechanism, interactions with the geometry, received power, delay, azimuth and elevation angles of arrival and departure, and path loss. </p><p> </p><p><strong>AUTHORS</strong></p><p>Teodora Kocevska, Andrej Hrovat, Tomaž Javornik</p><p>Department of Communication Systems</p><p>Jožef Stefan Institute, SI-1000 Ljubljana, Slovenia</p><p>teodora.kocevska@ijs.si</p><p> </p><p><strong>GEOMETRY PROCESSING</strong></p><p>The cave segment used for the propagation calculations is selected from a point cloud obtained in a cave in Litia, Slovenia. The point cloud is obtained with 3-D laser scanning of the environment. The selected segment is approx. 58 m long. Several parameter configurations were considered for 3-D shape reconstruction, including Poisson surface reconstruction with octree depths of 8, 10, and 12. Geometries that represent the cave shape and have different levels of complexity were created and studied. In the simplification process, one and two-stage simplification was explored using the Quadric Edge Collapse Decimation approach. </p><p> </p><p><strong>RADIO SETUP</strong></p><p>The transmitter (Tx) is fixed at the entrance of the cave and the receiver (Rx) is moved along the cave in 40 positions with a step of 1 m.</p><p>Omnidirectional antennas at the Tx and Rx sites and vertical polarization are considered. The antenna is mounted 1.5 m above the ground.</p><p>The start frequency is 3.5 GHz, the end frequency is 3.6 GHz and the step is 10 MHz. Direct propagation and first-order reflection are considered. </p><p>The cave geometry is represented by a triangular mesh, and the material of the cave is wet earth. The material electromagnetic properties are selected according to the specifications presented in [2].</p><p> </p><p><strong>FOLDER STRUCTURE</strong></p><p>The folder structure is:</p><p> - Polygon_Mesh_Models</p><p> <i># 3-D environment models with varying </i>levels<i> of geometry complexity</i></p><p> - Reconstruction_Segmen1_Poisson_Surface_Reconstruction</p><p> - Simplification_Segment1_Quadric_Edge_Collapse_Decimation</p><p> - Propagation_Data</p><p> <i># Propagation quantities of all rays between a transmitter and receiver</i></p><p> - AllRay_PropData</p><p> - PathLoss</p><p> - readme.txt</p><p> - RayTracing_EnvironmentModel</p><p> <i> # Final environment model used for ray tracing simulations</i></p><p> - Cave_MeshModel.json</p><p> - Cave_MeshModel.skb</p><p> - Cave_MeshModel.skp</p><p> - RayTracing_MaterialProperties</p><p> <i># Properties of the materials in the environment</i></p><p> - materials.json</p><p> - materials.mtl</p><p> - readme.txt</p><p> - Cave_Length.txt</p><p> <i># Length between selected locations in the environment</i></p><p> - Cave_Segment1_visual.png</p><p> <i> # Visualization of the environment segment used for propagation calculation</i></p><p> - readme.txt</p><p> <i># Overall description </i></p><p><strong>REFERENCES</strong></p><p>[1] D. He, B. Ai, K. Guan, L. Wang, Z. Zhong, and T. Kürner, "The Design and Applications of High-Performance Ray-Tracing Simulation Platform for 5G and Beyond Wireless Communications: A Tutorial," in IEEE Communications Surveys & Tutorials, vol. 21, no. 1, pp. 10-27, First quarter 2019, doi: 10.1109/COMST.2018.2865724.</p><p>[2] R. sector of International Telecommunication Union (ITU-R), "Effects of building materials and structures on radio wave propagation above about 100 MHz," International Telecommunication Union, ITU-R Recommendation P.2040-2, 2021.</p><p> </p><p><strong>ACKNOWLEDGEMENT</strong></p><p>This work was supported by the Slovenian Research Agency under grant <strong>J2-3048</strong>.</p><p> </p>
Dataset for the paper "Anomaly Detection in Large-Scale Cloud Systems: An Industry Case and Dataset"
<p>We present a large-scale anomaly detection dataset collected from IBM Cloud's Console over approximately 4.5 months. This high-dimensional dataset captures telemetry data from multiple data centers, specifically designed to aid researchers in developing and benchmarking anomaly detection methods in large-scale cloud environments. It contains 39,365 entries, each representing a 5-minute interval, with 117,448 features/attributes, as interval_start is used as the index. The dataset includes detailed information on request counts, HTTP response codes, and various aggregated statistics. The dataset also includes labeled anomaly events identified through IBM's internal monitoring tools, providing a comprehensive resource for real-world anomaly detection research and evaluation.</p> <p><strong>File Descriptions</strong></p> <ul> <li><code>location_downtime.csv</code> - Details planned and unplanned downtimes for IBM Cloud data centers, including start and end times in ISO 8601 format.</li> <li><code>unpivoted_data.parquet</code> - Contains raw telemetry data with 413 million+ rows, covering details like location, HTTP status codes, request types, and aggregated statistics (min, max, median response times).</li> <li><code>anomaly_windows.csv</code> - Ground truth for anomalies, listing start and end times of recorded anomalies, categorized by source (Issue Tracker, Instant Messenger, Test Log).</li> <li><code>pivoted_data_all.parquet</code> - Pivoted version of the telemetry dataset with 39,365 rows and 117,449 columns, including aggregated statistics across multiple metrics and intervals.</li> <li><code>demo/demo.[ipynb|html]</code>: This demo file provides examples of how to access data in the Parquet files, available in Jupyter Notebook (<code>.ipynb</code>) and HTML (<code>.html</code>) formats, respectively.</li> </ul> <p>Further details of the dataset can be found in <strong>Appendix B: Dataset Characteristics</strong> of the <a href="https://arxiv.org/abs/2411.09047">paper</a> titled <strong><em>"Anomaly Detection in Large-Scale Cloud Systems: An Industry Case and Dataset."</em></strong> Sample code for training anomaly detectors using this data is provided in <a href="https://doi.org/10.5281/zenodo.14598119" target="_blank" rel="noopener">this package</a>.</p> <p> </p> <p>When using the dataset, please cite it as follows:</p> <pre><code>@misc{islam2024anomaly,</code><br><code> title={Anomaly Detection in Large-Scale Cloud Systems: An Industry Case and Dataset}, </code><br><code> author={Mohammad Saiful Islam and Mohamed Sami Rakha and William Pourmajidi and Janakan Sivaloganathan and John Steinbacher and Andriy Miranskyy},</code><br><code> year={2024},</code><br><code> eprint={2411.09047},</code><br><code> archivePrefix={arXiv},</code><br><code> url={https://arxiv.org/abs/2411.09047}</code><br><code>}</code></pre> <p> </p>
New Challenges in Point Cloud Visual Quality Assessment: A Systematic Review (Dataset)
<p>This dataset is a collection of annotated information on the scientific papers screened and analyzed for the systematic review of the literature in Point Cloud Visual Quality Assessment. </p> <p>The data is structured as follows:</p> <ul> <li>General information <ul> <li>Document title</li> <li>Authors</li> <li>Year of publication</li> <li>Venue (Conference or Journal title)</li> <li>Citations (number)</li> <li>URL/DOI</li> </ul> </li> </ul> <ul> <li>About the content <br> <ul> <li>Content Type: Point clouds (PC), Colored Point clouds (CPC), Meshes, Dynamic Point Clouds (DPC)</li> <li>Content source: Source of the content used in a subjective QA test or the evaluation of one or more QA metrics</li> </ul> </li> </ul> <ul> <li>About metric benchmarks <ul> <li>Subjective Ground-truth Data: Dataset(s) Source of the subjective scores used as ground-truth in a QA metric benchmark</li> <li>Assessed Metrics: Types of metrics assessed in a benchmark (JPEG standards, IQM, NR, State-of-the-art, others)</li> <li>Performance Measures: PLCC, SROCC, KRCC, RMSE, OR, others</li> </ul> </li> </ul> <ul> <li>About Objective QA metrics <ul> <li>Metric: Name given to the metric introduced in this paper</li> <li>Base: 3D-based or Projection-based</li> <li>Categories: Categories that characterize the approach of the proposed metric (Feature-based, Learning-Based, Perceptual-based, IQM, others) </li> <li>Reference: Full-Reference (FR), Reduced-Reference (RR) or No-Reference (NR)</li> </ul> </li> </ul> <ul> <li>About Subjective QA experiments <ul> <li>Display: Type of display (2D, 3D, AR, MR, VR) and interaction approach (passive, interactive, 3DoF, 6DoF) used in the described experiment.</li> <li>Rendering: Type of rendering used to display the stimuli (Points, Squares, Cubes, Surface)</li> <li>Lab/Remote: The experiment was run in one or more lab environments, or remotely (Lab, Cross-Lab, Remote)</li> <li>Rating: Subjective rating methodology used in the experiment (ACR, DSIS, PWC, others)</li> <li>Dataset: Name of the new subjective dataset if the experiment's results were published.</li> <li>Observers: Number of observers </li> <li>Distortion type: Types of distortions applied to the stimuli and assessed in the experiment</li> </ul> </li> </ul>
LigPCDS: Labeled Dataset of X-ray Protein Ligand Images in 3D Point Cloud and Validated Deep Learning Models
<p>The difference electron density from X-ray protein crystallography was used to create the first dataset of labeled ligand images in 3D point clouds, named <strong>LigPCDS</strong>. The dataset contain 244,226 entries of free organic ligands containing 3D representations labeled with two major labeling approaches: SP-based and AtomSymbol-based.</p> <p> </p> <p>The data from free organic molecules (non-covalent ligands) was retrieved from the Research Collaboratory for Structural Bioinformatics Protein Data Bank (RCSB PDB) in december 2019 with resolutions ranging from 1.5 to 2.2 Å. The ligand images (blobs) were interpolated from their calculated difference electron density map in a 3D grid-like bounding box, around their atomic positions, and stored in point clouds. These ligand grid representations were further processed to retrive the final ligands representation in 3D point clouds using a mask of the shape of the ligand. A grid spacing of 0.5 Å gave the best results. The density value of the grid points was used as feature. The labeling approach used the structure of the ligands to propose vocabularies of chemical classes based on the chemical atoms themselves and their cyclic substructures. These structure annotations were applied pointwise to the ligand 3D representations using an atomic sphere model. Four proposed vocabularies were validated by successfully training good performance deep learning models for the semantic segmentation of a stratified dataset from LigPCDS, using 78902 entries.</p> <p>The four validated deep learning models are: (i) the LigandRegion, composed by generic atoms of any type; (ii) the AtomCycle, composed by generic atoms outside cycles and generic cycles; (iii) the AtomC347CA56, composed by generic atoms outside cycles, not aromatic cycles of size 3 to 7 and aromatic cycles of size 5 and 6; and (iv) the AtomSymbolGroups, composed by the atoms symbols with groupings. The mean accuracy of these models in their cross-validation was between 49.7% <span lang="EN-GB">[-19.4,20.</span><span lang="EN-GB">2]</span> and 77.4% <span lang="EN-GB">[-11.7,12.1]</span> in terms of Intersection over Union (mIoU) metric and between 62.4% <span lang="EN-GB">[-18.8,19.</span><span lang="EN-GB">7]</span> and 87.0% <span lang="EN-GB">[-8.4,8.8]</span> in F1-score (mF1), confidence interval between squared brackets. The models i, ii and iii and the used labeled representations in 3D point cloud are contained in the SP-based record; and model iv and its used labeled representations are contained in the AtomSymbol-based record.</p> <p>The dataset and validated models may be used to tackle problems regarding known and unknown ligand building to drug discovery and fragment screening pipelines. </p> <p>The code used to create and validated the LigPCDS is available at the following repository: https://github.com/danielatrivella/np3_ligand</p> <p>This repository also contains the NP³ Blob Label application for ligand building using the validated deep learning models from LigPCDS.</p>
Cloud Storage Use in Europe
<p>Percentage of individuals who use internet storage space to save documents, pictures, music, video or other files. Based on Eurostat data with approximation.</p> <p>For more reusable visualization with German, Dutch and Hungarian titles please refer to <a href="https://figshare.com/account/projects/129944/articles/20455224">https://figshare.com/account/projects/129944/articles/20455224</a></p>
Extended datasets from MM-IMDB and Ads-Parallelity dataset with the features from Google Cloud Vision API
<p>This is extended datasets from MM-IMDB [<a href="https://openreview.net/forum?id=S12_nquOe">Arevalo+ ICLRW'17</a>], Ads-Parallelity [<a href="https://arxiv.org/abs/1807.08205">Zhang+ BMVC'18</a>] dataset with the features from Google Cloud Vision API. These datasets are stored in jsonl (JSON Lines) format.</p> <p><strong>Abstract (from our paper):</strong></p> <p>There is increasing interest in the use of multimodal data in various web applications, such as digital advertising and e-commerce. Typical methods for extracting important information from multimodal data rely on a mid-fusion architecture that combines the feature representations from multiple encoders. However, as the number of modalities increases, several potential problems with the mid-fusion model structure arise, such as an increase in the dimensionality of the concatenated multimodal features and missing modalities. To address these problems, we propose a new concept that considers multimodal inputs as a set of sequences, namely, deep multimodal sequence sets (DM<sup>2</sup>S<sup>2</sup>). Our set-aware concept consists of three components that capture the relationships among multiple modalities: (a) a BERT-based encoder to handle the inter- and intra-order of elements in the sequences, (b) intra-modality residual attention (IntraMRA) to capture the importance of the elements in a modality, and (c) inter-modality residual attention (InterMRA) to enhance the importance of elements with modality-level granularity further. Our concept exhibits performance that is comparable to or better than the previous set-aware models. Furthermore, we demonstrate that the visualization of the learned InterMRA and IntraMRA weights can provide an interpretation of the prediction results.</p> <p><strong>Dataset (MM-IMDB and Ads-Parallelity):</strong></p> <p>We extended two multimodal datasets, namely, MM-IMDB [<a href="https://openreview.net/forum?id=S12_nquOe">Arevalo+ ICLRW'17</a>], Ads-Parallelity [<a href="https://arxiv.org/abs/1807.08205">Zhang+ BMVC'18</a>] for the empirical experiments. The MM-IMDB dataset contains 25,925 movies with multiple labels (genres). We used the original split provided in the dataset and reported the F1 scores (micro, macro, and samples) of the test set. The Ads-Parallelity dataset contains 670 images and slogans from persuasive advertisements to understand the implicit relationship (parallel and non-parallel) between these two modalities. A binary classification task is used to predict whether the text and image in the same ad convey the same message.</p> <p>We transformed the following multimodal information (i.e., visual, textual, and categorical data) into textual tokens and fed these into our proposed model. We used the <a href="https://cloud.google.com/vision">Google Cloud Vision API</a> for the visual features to obtain the following four pieces of information as tokens: (1) text from the OCR, (2) category labels from the label detection, (3) object tags from the object detection, and (4) the number of faces from the facial detection. We input the labels and object detection results as a sequence in order of confidence, as obtained from the API. We describe the visual, textual, and categorical features of each dataset below.</p> <p><em><strong>MM-IMDB</strong></em>: We used the title and plot of movies as the textual features, and the aforementioned API results based on poster images as visual features.</p> <p><em><strong>Ads-Parallelity</strong></em>: We used the same API-based visual features as in MM-IMDB. Furthermore, we used textual and categorical features consisting of textual inputs of transcriptions and messages, and categorical inputs of natural and text concrete images.</p>
X-Shooting ULLYSES: Massive Stars at low metallicity - II. DR1: Advanced optical data products for the Magellanic Clouds
<p>Xshooter optical spectroscopic data of Magellanic Clouds targets observed by the ESO Large Program X-Shooting ULLYSES: Massive Stars at low metallicity (PI: Vink; Porgram ID: 106.2011Z). <br><br>This version is identical to the previous version but includes the LMC and SMC atlases and the static calibration files with the new flux models (all arms) and spline anchor points (only UVB).</p>
Landsat bands (cloud free), tree cover (2000, 2010), bare-ground and surface water occurrence at 250 m based on GlobalForestWatch and USGS
<p>Landsat bands (cloud free) and tree cover (2000) based on Hansen et al. (2013), global surface water occurrence based on Pekel at al. (2016), and tree cover and bare-ground cover (2010) based the USGS land cover mapping projects (University of Maryland, Department of Geographical Sciences and USGS). All layers resampled to spatial resolution 1/480 d.d. (about 250 m) using gdalwarp with "average" resampling. Antarctica is not included. Original layers are available at 30 m resolution.</p> <p>If you discover a bug, artifact or inconsistency in the maps, or if you have a question please use some of the following channels:</p> <ul> <li>Technical issues and questions about the code: <a href="https://gitlab.com/openlandmap/global-layers/issues">https://gitlab.com/openlandmap/global-layers/issues</a> </li> <li>General questions and comments: <a href="https://disqus.com/home/forums/landgis/">https://disqus.com/home/forums/landgis/</a></li> </ul> <p>All files internally compressed using "COMPRESS=DEFLATE" creation option in GDAL. File naming convention:</p> <ul> <li>lcv = theme: land cover,</li> <li>bareground = variable: occurrence of bareground,</li> <li>landsat.usgs = determination method: Landsat landcover at 30 m resolution project (https://landcover.usgs.gov/glc/),</li> <li>p = probability or fraction,</li> <li>250m = spatial resolution / block support: 250 m,</li> <li>s0..0cm = vertical reference: land surface,</li> <li>2010..2010 = time reference: year 2010,</li> <li>v1.0 = version number: 1.0,</li> </ul>
Data for the 'Evaluation of global simulations of aerosol particle and cloud condensation nuclei number, with implications for cloud droplet formation'
<p>All numerical data used in the manuscript <strong>“Evaluation of global simulations of aerosol particle number and cloud condensation nuclei, and implications for cloud droplet formation” </strong>by G. S. Fanourgakis et al. ACP (2019) are categorized and provided in a number of files. All files are in the hdf format. A readme file is also provided.</p> <p>These data files have been created by G. S. Fanourgakis (fanourg@uoc.gr)</p> <p>Details on the data are provided in Fanourgakis et al. Atmos. Chem. Phys. 2019 https://doi.org/10.5194/acp-2018-1340 (e-mail to <a href="mailto:mariak@uoc.gr">mariak@uoc.gr</a> ; <a href="mailto:athanasios.nenes@epfl.ch">athanasios.nenes@epfl.ch</a> )</p> <p>For an in-depth understanding of the description below, a study of the above mentioned manuscript is required.</p> <p>(A) Station model results</p> <p>The station results can be found in files with filenames of the form:</p> <p>station $MODEL.nc</p> <p>The “$MODEL” (as well as all names starting with “$”) indicates a variable, and more specifically one of the models participated in the present study. The values of this variable are tabulated in Table 1 in the readme file.</p> <p>In each file a number of computational results are provided by the specified model for all nine (9) stations that provided observational data. The name of the variable is formed as:</p> <p>st $STATION $FIELDhour st $STATION $FIELD month</p> <p>where all possible values of the variables $STATION and $FIELD are tabulated in Tables 2 and 3 in the readme file, respectively. The extension _hour denotes that hourly values for the field are provided, while the extension _month the monthly average of this quantity. For example, the variable</p> <p>st Finokalia CCN02 hour</p> <p>found in the file station_TM4-ECPL.nc, contains the hourly values of the CCN<sub>0<em>.</em>2 </sub>at the Finokalia station as computed by the TM4-ECPL model. In a similar way, in the file station_EMAC.nc, the variable below gives the monthly values of dust at Vavihill as computed with the EMAC model.</p> <p>st Vavihill DU month</p> <p>Notice also that in all files hourly and monthly data are provided for the time period from 1-1-2011 up to 31-12-2015 (60 months and 43,824 hours)</p> <p>(B) Station observational results</p> <p>There is one file that contains all observational data from Schmale et al., SCIENTIFIC DATA | 4:170003 | DOI: 10.1038/sdata.2017.3, 2017 (<a href="mailto:julia.schmale@psi.ch">julia.schmale@psi.ch</a>) and the data that were computed based on the observations (i.e. number of cloud droplets) (contact person: athanasios.nenes@epfl.ch). The file is</p> <p>station observations.nc</p> <p>while the following fields are contained in there:</p> <p>st $STATION $FIELDhour</p> <p>st $STATION $FIELD month</p> <p>The values of variables are given in the Tables 2 and 3 in the readme file. The time period covered is from 1-1-2011 up to 31-12-2015. Notice that due to the lack of observations a lot of data are missing. For missing observational data the value -9999.999 is given. Contact person for the observational data is Julia Schmale (julia.schmale@psi.ch).</p> <p>(C) Station Multi-model Median</p> <p>Monthly averages of the models can be found in the file</p> <p>station MMM.nc</p> <p>The following fields can be found in the file</p> <p>st $STATION $FIELD month median</p> <p>st $STATION$FIELD month quart25</p> <p>st $STATION$FIELD month quart75</p> <p>where the values of the variables $STATION and $FIELD can be found in Tables 2 and 3, respectively. The extension median corresponds to the multi-model median, while the quart25 and quart75 to the 25 % and 75 % quartiles, respectively.</p> <p>(D) Global model results</p> <p>In the following single file can be found for each of the models the surface distribution of various fields.</p> <p>results global models year2011.nc</p> <p>They correspond to the annual mean of the year 2011. The resolution of the grid is 1<sup>◦ </sup>× 1<sup>◦</sup>. The file contains the following variables:</p> <p>$FIELD $MODEL</p> <p>The $FIELD and $MODEL can be found in Tables 3 and 1, respectively.</p> <p>(E) Global average results</p> <p>In the file</p> <p>surface_ global_average_year2011.nc</p> <p>can be found in 5<sup>◦</sup>×5<sup>◦ </sup>resolution, the Multi-model median of surface distribution of the various fields denoted in Table 3 and their corresponding diversity. The names of the variables are formed as:</p> <p>med $FIELD</p> <p>div $FIELD</p> <p>where, ‘med’ stands for median and ‘div’ for diversity calculated as standard deviation divided by the mean of the model results.</p> <p>Tables and details on the fields provided are given in the readme file.</p>
Hong Kong Annotated Airborne LiDAR Point Clouds
<p>The annotated point clouds were generated to train the weakly supervised semantic segmentation algorithm Semantic Query Network (SQN) to classify point clouds <sup>[1]</sup>. The dataset covers 16 tiles of airborne LiDAR data in an area of 7.2 km2 in Shatin, Hong Kong, China. 11 tiles were used for training, while 5 tiles were used for validation. There are multiple types of construction in the dataset including high-rise residential buildings, low-rise village houses, and large public buildings. Green spaces are mainly composed of wood areas in open spaces (e.g., in parks and hills) and planted trees in residential gardens and nearby roads. Point clouds are classified in ground, buildings, and trees.</p> <p>The LiDAR data is owned by the Hong Kong government. Please visit the Spatial Data Portal, Survey Division, CEDD (https://sdportal.cedd.gov.hk/#/en/) for more details.</p> <p> </p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.