Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
30
datasets available to search
ShareScore release 0.9.0
Dataset results
30 results for “Time series prediction”
Towards understanding the importance of time-series features in automated algorithm performance prediction
<p><strong>merged_feature_importance.csv</strong> - CSV with feature importance values with different meta-models, forecasting algorithms, and feature importance methods computed on 30 different train/test splits.</p> <p><strong>Catch22.csv</strong> - Catch22 features (raw time-series)</p> <p><strong>Catch22Log.csv</strong> - Catch22 features (log time-series)</p> <p><strong>Catch22Diff.csv</strong> - Catch22 features (differenced time-series)</p> <p><strong>TSFresh.csv</strong> - TSFresh features (raw time-series)</p> <p><strong>TSFreshLog.csv</strong> - TSFresh features (log time-series)</p> <p><strong>TSFreshDiff.csv</strong> - TSFresh features (differenced time-series)</p> <p><strong>mape.csv</strong> - sMAPE performance for all forecasting algoirthms</p>
Validation of Time Series Technique for Prediction of Conformational States of Amino Acids
<p>Validation of Time Series Technique for Prediction of Conformational States of Amino Acids</p> <p>- a project for fulfillment of M.Sc (Master of Science) in Bioinformatics.</p>
Predicting time series of vegetation leaf area index across North America based on climate variables for land surface modeling using attention-enhanced LSTM
<p>We developed an attention-enhanced long and short memory (AELSTM) model for predicting vegetation LAI time series based on climatic data. The developed AELSTM model establishes the relationships between the time series of vegetation LAI and climatic variables. </p>
High-resolution wall-to-wall time series predictions of seasonal maize area and yield for Rwanda over 2019-2023
<p>This is the companion dataset to publication {TBD}. It contains 1) seasonal composites of predicted maize cover and yield at 10 m resolution in Rwanda for two annual agricultural seasons over five years, 2) scripts for the end-to-end machine learning pipeline that produces these data products, and 3) data or references needed as inputs to the pipeline. </p> <h2>1) Maize cover and yield seasonal composites</h2> <p>The data are provided here as netCDF4 files with four dimensions for x, y, band, and season. They can also be accessed as Google Earth ImageCollections at: </p> <ul> <li>https://code.earthengine.google.com/?asset=projects/b2p-geospatial/assets/lulc_classifier_composite</li> <li>https://code.earthengine.google.com/?asset=projects/b2p-geospatial/assets/maize_yield_composite </li> </ul> <h3>Land cover and maize classification</h3> <p>The land cover classification file is found at <code>data/composites/lulc_classifier_Rwanda_2019to2023.nc</code>.</p> <p>The land cover classification images contain 3 bands/variables: <em>maizeProb</em>, the raw predicted probability of the pixel being maize given by the gradient boosted tree model; <em>majorityClass</em>, the categorical land cover class with the highest predicted probability among any of the nine classes in the respective pixel; and <em>optimalClass</em>, the categorical land cover class adjusted to agree with national statistics for expected maize area.</p> <p>The land cover classes map to the raster values as follows: </p> <div> <div> <pre>{<br> 1: 'maize',<br> 2: 'nonmaize_annual',<br> 3: 'nonmaize_perennial',<br> 4: 'scrub_shrub_land',<br> 5: 'forest',<br> 6: 'flooded_vegetation',<br> 7: 'water',<br> 8: 'structure',<br> 9: 'bare'<br>}</pre> </div> </div> <p>The dataset includes 5 years (2019-2023) and 10 seasons - the available time period at time of publication. In Rwanda, maize is typically planted and harvested during two distinct agricultural seasons per year: Season A from September to February and Season B from March to June. Therefore the seasons in the data are: 2019_Season_A, 2019_Season_B, 2020_Season_A, 2020_Season_B, 2021_Season_A, 2021_Season_B, 2022_Season_A, 2022_Season_B, 2023_Season_A, 2023_Season_B.</p> <h3>Maize yield</h3> <p>The maize yield file is found at <code>data/composites/maize_yield_Rwanda_2019to2023.nc</code>.</p> <p>Each of the images in the yield composites has 3 bands/variables also: <em>maizeYield</em>, the model's output of continuous predicted yield (kg/ha) in each pixel regardless of land class; <em>maizeYield_majorityClass</em>, predicted maize yield masked to the majority class land classification; and <em>maizeYieldAdj_optimalClass</em>, where the raw predicted yields were masked to the optimal maize classification land cover layer and normalized to national statistics. </p> <p>The dataset includes the same seasons as the classification product; see above for a description.</p> <h2>2) End-to-end machine learning pipeline</h2> <p>All earth observation imagery, analysis, and outputs unless otherwise stated were hosted in the Google Earth Engine (GEE) environment and developed with the Earth Engine Python API in Python v3.10. To set up a local conda environment use the <code>scripts/environment.yml</code> file. The user must have <a href="https://cloud.google.com/storage">Google Cloud Storage (GCS)</a> and <a href="https://cloud.google.com/earth-engine">Google Earth Engine (GEE)</a> accounts. The pipeline, at this scale, will incur some processing and storage fees, although Google offers a free trial to all new users and the total cost of the high-resolution wall-to-wall predictions is nominal (~$20 for one season). </p> <p>The scripts needed to perform the pipeline are located in the <code>scripts</code> folder. </p> <p>The files contained in the <code>scripts/helpers</code> directory will be called by various subsequent scripts and do not to be run interactively by the user. </p> <p>Follow the script in the order described below. The user should pause after running each script and confirm that all outputs were created and loaded to GCS before continuing the pipeline; for some steps this may take hours to days depending on processing speed. </p> <h3>Google Cloud Storage and Earth Engine set-up</h3> <p>Users should specify the names of the bucket and asset project that were chosen during set up of their GCS and GEE environments in the <em>Objects</em> section of <code>scripts/helpers/maize_pipeline_0_workspace.py</code>.</p> <h3>Pipeline set-up</h3> <p>In <code>scripts/pipeline_setup</code>, you will find the following scripts to perform data preparation of inputs into model building and prediction. </p> <ul> <li><code>maize_pipeline_1_clean_training_data.py</code> - Cleans and merges all available crop label and yield data for model training and validation</li> <li><code>maize_pipeline_2_dwnld_data_training.py</code> - Downloads satellite-derived and auxiliary features at training data points for model building</li> <li><code>maize_pipeline_3_dwnld_data_inference.py</code> - Downloads satellite-derived and auxiliary features at every 10 m pixel in Rwanda on a district-wise basis for prediction</li> </ul> <h3>Land cover and maize classification</h3> <p>In <code>scripts/maize_classification</code>, you will find the following scripts to perform model building, prediction, and post-processing for the classificaton of land cover type and maize cover.</p> <ul> <li><code>maize_classifier_1_feature_selection.py</code> - Selects features subset for land cover classification with mutual information score or variable importance</li> <li><code>maize_classifier_2_build_model.py</code> - Builds gradient boosted tree model for land cover classification from training data</li> <li><code>maize_classifier_3_prediction.py</code> - Applies model for land cover classification to every 10 m pixel in Rwanda by season and district</li> <li><code>maize_classifier_4_postprocess.py</code> - Mosaics district-wise predictions and normalizes maize cover predictions to national agricultural statistics</li> </ul> <h3>Maize yield</h3> <p>In <code>scripts/maize_yield</code>, you will find the following scripts to perform modeling building, prediction, and post-processing for maize yield estimation. </p> <ul> <li><code>maize_yield_1_build_model.py</code> - Builds gradient boosted tree model and performs bias correction for maize yield estimation from training data</li> <li><code>maize_yield_2_prediction.py</code> - Applies model for maize yield estimation to every 10 m pixel in Rwanda by season and district</li> <li><code>maize_yield_3_postprocess.py</code> - Mosaics district-wise predictions and normalizes maize yield predictions to national agricultural statistics</li> </ul> <p>If you are running the entire pipeline with refreshed training data and model building, run each of these scripts, in order. By default, the script will run all A and B seasons from 2019A to current. Otherwise, if you just wish to re-run or update seasonal predictions from the existing classification or yield model run <code>maize_pipeline_3_dwnld_data_inference.py</code> to download the seasonal feature data across Rwanda and <code>maize_classifier_3_prediction.py</code>and <code>maize_classifier_4_postprocess.py</code> for classification predictions or <code>maize_yield_2_prediction.py</code> and <code>maize_yield_3_postprocess.py</code> for yield predictions, making sure to specify which season(s) are of interest in each script. However to do this, you also need to have a copy of the previously built models in your GCS (provided at <code>data/models</code>). </p> <h2>3) Input data into machine learning pipeline</h2> <p>A description of datasets that must be sourced outside of the GEE platform is provided below. When available, the primary data source is also included in the directory <code>data/baselayers</code>. All other data, including Sentinel-2 imagery, auxiliary data, and other existing global land cover classificaiton products are hosted on GEE and called by the scripts directly. All datasets last accessed on 12 March 2024.</p> <h3>Administrative and geological boundaries</h3> <ul> <li>World Countries - Downloaded from <a href="https://datacatalog.worldbank.org/search/dataset/0038272/World-Bank-Official-Boundaries">The World Bank Official Boundaries</a> and included here at <code>data/baselayers/World_Countries</code>.</li> <li>Rwanda district boundaries - Downloaded from <a href="https://datacatalog.worldbank.org/search/dataset/0041453/Rwanda-Admin-Boundaries-and-Villages">The World Bank Rwanda Admin Boundaries And Villages</a> and included here at <code>data/baselayers/WB_NISR_2018</code>. This should be loaded into a FeatureCollection GEE asset named <em>districts_fc</em> for use in the pipeline. </li> <li>Rwanda agro-ecological zones - Downloaded from <a href="https://doi.org/10.1371/journal.pone.0149239">Nzeyimana, Hartemink & Geissen (2016)</a> and included here at <code>data/baselayers/MINAGRI_AEZ_1980</code>. This should be loaded into a FeatureCollection GEE asset named <em>aez_rwanda</em> for use in the pipeline. </li> </ul> <h3>Global land cover classification product</h3> <ul> <li>Microsoft/Impact Observatory LULC - Although the <a href="https://planetarycomputer.microsoft.com/dataset/io-lulc-9-class">10m Annual Land Use Land Cover (9-class) V1</a> product contains data from 2017-2022, only the LULC map from the year 2021 was used, provided here at <code>data/baselayers/impactobs_lulc_rwa_2021.tif</code>. This should be loaded into an ImageCollection GEE asset named <em>impact_obs_lulc</em> for use in the pipeline.</li> </ul> <p>(The others - Dynamic World and ESA's WorldCover - are hosted on GEE directly.)</p> <h3>Land cover labels and maize yield crop cuttings</h3> <ul> <li>One Acre Fund - Contact authors to request access as this dataset is not hosted publicly. </li> <li>RTI International - The original source of this data (Radiant MLHub) has been discontinued, but users may be able to access it via <a href="https://beta.source.coop/repositories/rti/rwanda-crop-type/">Source Cooperative</a>. The data is also included here at <code>data/baselayers/rti_rwanda_crop_type_labels</code>. </li> <li>Crop Harvest - Downloaded from <a href="../records/7257688">Tseng et al. (2021, v13)</a> and included here at <code>data/baselayers/CropHarvest</code>. These data points were ultimately not used in the training data, but are provided here for others that may find this dataset useful in their context.</li> </ul> <h3>Rwanda national agricultural surveys</h3> <ul> <li>National Institute of Statisitcs Rwanda (NISR) - Downloaded from <a href="https://statistics.gov.rw/datasource/seasonal-agricultural-survey">NISR Seasonal Agricultural Survey</a> and existing seasons included here at <code>data/baselayers/NISR_Seasonal_Ag_Surveys</code>. For each subsequent season, the user will have to download the spreadsheet of survey results from the NISR webpage (linked) and add the respective season to the <code>get_nisr_data</code> function in the <code>helpers/maize_pipeline_0_helpers_postprocess.py</code> script to clean and read in the data for use in the pipeline. </li> </ul> <p> </p>
Synthetic dataset and prediction files for the paper "Denoising of Geodetic Time Series Using Spatiotemporal Graph Neural Networks: Application to Slow Slip Event Extraction", by Costantino et al. (2024)
<p>Synthetic database used for training and evaluation of SSEdenoiser</p>
Dataset: Bridging Time-series Image Phenotyping and Functional-Structural Plant Modeling to Predict Adventitious Root System Architecture
<p>Dataset for Bridging Time-series Image Phenotyping and Functional-Structural Plant Modeling to Predict Adventitious Root System Architecture manuscript submitted to Plant Phenomics. The dataset contains raw and processed root architecture images, RhizoVision trait outputs, and the associated R scripts for statistical analysis and model parameterization.</p>
Data from: Multi-decadal time series of remotely sensed vegetation improves prediction of soil carbon in a subtropical grassland
Open the record for dataset details and reuse information.
The supporting data for the paper "Synergistic Enhancement of LSTM Time Series Prediction via Companion Strategy and Decay Operator-Improved Aquila Optimization"
<p>数据生成程序</p> <p>该数据集是使用 The Investor's Exchange API 生成的,脚本会定期获取标准普尔 500 指数中所有公司的历史股价。详细说明和脚本可以在 GitHub 存储库中找到。该数据每5年更新一次,最近一次更新于2018年2月。</p> <p> </p> <p>数据处理方法和步骤</p> <p>数据处理的主要步骤包括:</p> <p> </p> <p>数据采集:使用 API 获取每只股票的历史数据,存储在.csv文件中。</p> <p>数据清理:删除重复条目,纠正格式错误,确保数据完整性。</p> <p>数据合并:将单个股票数据合并到一个大.csv文件中,以便于使用。</p> <p>数据验证:通过检查时间序列的连续性和完整性来验证数据的准确性。</p> <p>使用的设备和工具</p> <p>数据采集工具:Python 脚本</p> <p>数据处理工具:用于数据清洗和处理的 Pandas 库</p> <p>数据存储:CSV文件格式</p> <p>时间和地理范围</p> <p>时间范围:数据涵盖过去 5 年的历史股票价格,最新更新于 2018 年 2 月。</p> <p>地理范围:数据涵盖标准普尔500指数中的所有公司,主要是美国市场数据。</p> <p>时间和空间分辨率</p> <p>时间分辨率:每日数据,每个交易日一条记录。</p> <p>空间分辨率:无地理空间分辨率;数据按公司分组。</p> <p>表格数据</p> <p>条目总数:条目总数取决于标准普尔500指数中的公司数量和总交易日数。</p> <p>行标题和列标题:</p> <p>日期:交易日期格式为yy-mm-dd</p> <p>开盘价:开盘价(美元)</p> <p>最高价:当日最高价(美元)</p> <p>最低价:当日最低价格(美元)</p> <p>收盘价:收盘价(美元)</p> <p>交易量:成交股数</p> <p>名称:以股票代码的名义</p> <p>缺失数据</p> <p>数据集在某些交易日可能缺少数据,主要是由于非交易日(例如节假日)或API数据采集过程中的临时网络问题。这些缺失的数据通常不会影响整体分析结果。</p> <p> </p> <p>数据错误</p> <p>由于数据源是第三方 API,因此数据错误的可能性很低。如果发现错误,通常是由于 API 数据采集过程中的临时网络问题造成的。数据清理过程旨在最大限度地减少和纠正这些错误。</p> <p> </p> <p>数据文件说明</p> <p>数据文件类型:</p> <p>all_stocks_5yr.csv:包含所有股票的合并数据文件。</p> <p>individual_stocks_5yr文件夹:包含每个股票的单个.csv文件。</p> <p>文件内容和格式:文件采用 CSV 格式,每个文件包含日期、开盘价、最高价、最低价、收盘价、成交量和股票名称列。</p> <p>文件大小:文件大小取决于特定股票的交易数据量,通常从几MB到几十MB不等。</p> <p>文件格式说明</p> <p>数据以通用的 CSV 格式存储,可以使用 Excel、Notepad++ 或任何支持 CSV 文件的工具打开和查看。对于进一步的数据处理和分析,可以使用 Python Pandas 库。</p> <p> </p> <p>总结</p> <p>该数据集提供过去5年标准普尔500指数中所有公司的历史股价数据,包括开盘价、最高价、最低价、收盘价、交易量等详细信息。它适用于各种财务数据分析和建模应用。数据通过 API 获取并处理,以确保准确性和完整性。</p>
Ground-Based Doppler Orbitography and Radiopositioning Integrated by Satellite (DORIS) IDS Earth Orientation Parameters (EOP) Time Series Product from NASA CDDIS
Doppler Orbitography and Radiopositioning Integrated by Satellite (DORIS) Earth Orientation Parameters Time Series Product from the NASA Crustal Dynamics Data Information System (CDDIS). DORIS is a dual-frequency Doppler system consisting of a receiver flying aboard a satellite and a globally distributed network of ground beacons. The DORIS receiver on-board the orbiting satellite tracks the dual-frequency radio signals transmitted by the network of ground beacons and generates the DORIS data. A measurement is made of either the Doppler shift or absolute phase as the satellite’s orbit moves over the ground-based beacon. DORIS data records contain a time-tagged range-rate measurement with associated ancillary information. DORIS observations from a global network can be utilized for a variety of products. Analysis Centers (ACs) of the International DORIS Service (IDS) retrieve DORIS data on a regular basis to compute various DORIS products from data generated by the DORIS beacons supporting the IDS network, including the time series of Earth orientation parameters (EOPs). The IDS Analysis Center Coordinator combines these solutions to produce an official IDS EOP product. The EOP time series are available in text format.
Long-term prediction of nonlinear time series
This paper is about applying recurrent least squares support vector machines (LS-SVM) on three ESTSP08 competition datasets. Least squares support vector machines are used as nonlinear models in order to avoid local minima problems. Then prediction task is re-formulated as function approximation task. Recurrent LS-SVM uses nonlinear autoregressive exogenous (NARX) model to build nonlinear regressor, by estimating in each iteration the next output value, given the past output and input measurements.
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.