Find research datasets worth reusing

opencc-by-4.0Oct 2024View details →

Time series of chaotic systems

<div> <div> </div> </div> <div> Long time series of chaotic systems, all three-dimensional. Can be used in short- and long-term forecasting, reconstruction, etc. Codes in GitHub: https://github.com/Zheng-Meng/Dynamics-Reconstruction-ML. We used the dataset in dynamics reconstruction from sparse observations with no training on target systems: Zhai, Zheng-Meng, Jun-Yin Huang, Benjamin D. Stern, and Ying-Cheng Lai. "Reconstructing dynamics from sparse observations with no training on target system." arXiv preprint arXiv:2410.21222 (2024). In addition, two folders with additional data, data_response, which is generated by dysts (https://github.com/williamgilpin/dysts) and data_nonautonomous, are provided for further evaluation of the dynamics reconstruction framework.   </div>

opencc-by-4.0Nov 2023View details →

Time-series global 30 m wetland maps from 2000 to 2022

Basic informationA novel global 30 m wetland annual dataset with fine classification system is generated, covering the period of 2000-2022 and containing 8 wetland subcategories (permanent water, swamp, marsh, flooded flats, saline, mangrove forest, salt marshes, and tidal flats). Notes:The GWL_FCS30 annual maps are divided into 961 5°×5° geographical tiles, and each tile contains 23 bands which denotes the tidal flat maps in 2000, 2001, 2002,..., 2021, 2022.Usage Policy:If you plan to use our data in a scientific analysis paper, we strongly recommend contacting us in advance to seek opinions, and consider our contributions in the acknowledgments or as co-authors.Citations:Zhang, X., Liu, L., Zhao, T., Chen, X., Lin, S., Wang, J., Mi, J., and Liu, W.: GWL_FCS30: a global 30 m wetland map with a fine classification system using multi-sourced and time-series remote sensing imagery in 2020, Earth Syst. Sci. Data, 15, 265–293, <a href="https://doi.org/10.5194/essd-15-265-2023">https://doi.org/10.5194/essd-15-265-2023</a>, 2023

opencc-by-4.0Mar 2018View details →

A global collection of paleoclimate proxy time series over the Common Era

*Dec 2023 Proxy Database Update: This database has been updated based on a more stringent proxy screening procedure. All screening calculations and decisions are documented in the accompanying Julia code Pluto notebook file (see also the HTML file preview of the code). This screening has resulted in the removal of 183 proxy time series from the original file.*Original Data Descriptor:This collection of paleoclimate proxy data currently includes 2591 tree ring chronologies, 197 coral and sclerosponge records, 153 ice core isotope records, 26 speleothem isotope records, 10 lake sediment records, and 1 marine sediment record, for a total of 2,978 records. The proxy records have been collected with a focus on the past 2000 years.This dataset has been collated through collaborative efforts with the Last Millennium Reanalysis project (Hakim et al. 2016) and provides the basis for the reconstructions presented in Steiger et al. (2018). Though the majority of data records are culled from PAGES2k Consortium (2017) and Breitenmoser et al. (2014), some data was taken from the NOAA NCEI's World Data Center for Paleoclimatology archive. Additionally, data was solicited from multiple investigators such as Eric Steig, Stephanie Hayman, Sylke Draschba, Henning Kuhnert, Andy Baker, and others amounting to approximately 60 coral, ice core, speleothem, and lake sediment datasets, though all files in this archive can be freely shared. Datasets were selected whose resolution was at least 25 years, temporal duration was greater than 40 years, and consisted of proxies that had established proxy system models (forward models). The data files are in Matlab format and the variables here include the proxy names ('lmr2k_names'), the proxy data ('lmr2k_data'), the proxy type ('archive'), the proxy measurement ('msrmt'), the proxy latitudes and longitudes ('p_lat' and 'p_lon'), and the years of the proxies ('year'). Seasonally resolved proxies have been averaged from April to the following calendar year March.Updates will be made to this proxy collection as more proxy data become available. Breitenmoser, P., et al. "Forward modelling of tree-ring width and comparison with a global network of tree-ring chronologies." Climate of the Past 10.2 (2014): 437.PAGES2k Consortium. "A global multiproxy database for temperature reconstructions of the common era." Scientific Data 4, 170088 (2017).Hakim, Gregory J., et al. "The last millennium climate reanalysis project: Framework and first results." Journal of Geophysical Research: Atmospheres 121.12 (2016): 6745-6764.Steiger, N. J. et al. "A reconstruction of global hydroclimate and dynamical variables over the Common Era." Scientific Data 5:180086 (2018).

R code and supplementary data for : "A framework for mapping conservation agricultural fields using time-series optical and radar imagery"

Source code and cover crop maps for the paper "A framework for mapping conservation cropland using optical and radar time series imagery." (Zhou et al., 2025) https://doi.org/10.1016/j.rse.2025.114858   The entire workflow consists of these steps: 1. Obtain satellite data from Google Earth Engine platform. script path: (<a href="https://code.earthengine.google.com/?scriptPath=users%2Fyuez9466%2FCApractice%3ANDVI">https://code.earthengine.google.com/?scriptPath=users%2Fyuez9466%2FCApractice%3ANDVI</a>). You need to obtain the NDVI, NBR2, Sentinel-1 Radar dataset and Precipitation data for your research area and seltected time interval. Download .csv data from Google Cloud, then convert the format of the data for following calculations.(see 1_import_transfer_data.R) 2. Obtain the annual crop types in your study area, either through agricultural census data or remote sensing predictions (not mentioned in this paper), calculate organic carbon input based on the crop types. Extracting seasons based on time-series NDVI values using phenofit package. (see 2_NDVI_Smooth_Divide_seasons.R) 3. Calculating the length of the cover crop growing season and periods of bare soil, also get the nessasary covariates for tillage model meanwhile. (see 3_CC_BS_length_add_Tillage.R) 4. Build a tillage model. (see 4_Build_Tillage_model) Build your own conservation agriculture fields model.

Predicting time series of vegetation leaf area index across North America based on climate variables for land surface modeling using attention-enhanced LSTM

We developed an attention-enhanced long and short memory (AELSTM) model for predicting vegetation LAI time series based on climatic data. The developed AELSTM model establishes the relationships between the time series of vegetation LAI and climatic variables.

Sliding velocity, water discharge, water pressure, and rainfall time series at Argentière Glacier between 2019 and 2021

Files Description: ================================== cavitometer_2019-2021.dat: ================================== Contains 30-min sampled values of recorded sliding velocities at the cavitometer. Column 1 = Date Column 2 = Velocity (mm/hour) ================================ water_discharge_2019-2021.dat: ================================ Contains 15-min sampled values of recorded water discharge at the glacier outlet. Column 1 = Date Column 2 = Water discharge (m3/s) ================================ water_pressure_2019-2021.dat: ================================ Contains 30-min sampled values of recorded water pressure at the borehole BH2. Column 1 = Date Column 2 = Water pressure (bar) ================================ rainfall_2019-2021.dat: ================================ Contains 30-min sampled values of recorded rainfall at the meteo station. Column 1 = Date Column 2 = Rainfall (mm w.eq./hour)

opencc-zeroJan 2024View details →

dryad32/100

Time-series of groundwater recharge, Tiber Riber Basin, Italy from 801 CE to the present day

Groundwater, essential for water availability, sanitation, and achieving Sustainable Development Goals, is shaped by climate dynamics and complex hydrogeological structures. Here, we provide a time-series of groundwater recharge from 801 CE to the present day in the Tiber River Basin, Italy, using historical records and hydrological modelling. Groundwater drought occurred in 36% of the Medieval Climatic Anomaly (801-1249) years, 12% of the Little Ice Age (1250-1849) years, and 26% of the Modern Warming Period (1850-2020) years. Importantly, a predominant warm phase of the Atlantic Multidecadal Oscillation, aligned with solar maxima, coincided with prolonged dry spells during both the medieval and modern periods, inducing a reduction in recharge rates due to hydrological memory effects. This study enhances understanding of climate-water interactions, offering a comprehensive view of groundwater dynamics in central Mediterranean and highlighting the importance of the past for sustainable future strategies. Leveraging this understanding can address water scarcity and enhance basin resilience.

Agricultural land use (raster) : National-scale crop type maps for Germany from combined time series of Sentinel-1, Sentinel-2 and Landsat data (2017 to 2021)

The dataset contains maps of the main classes of agricultural land use (dominant crop types and other land use types) in Germany, which are produced annually at the Thünen Institute beginning with the year 2017 on the basis of satellite data. The maps cover the entire open landscape, i.e., the agriculturally used area (UAA) and e.g., uncultivated areas. The map was derived from time series of Sentinel-1, Sentinel-2, Landsat 8 and additional environmental data. Map production is based on the methods described in <a href="https://doi.org/10.1016/j.rse.2021.112831">Blickensdörfer et al. (2022)</a>. All optical satellite data were managed, pre-processed and structured in an analysis-ready data (ARD) cube using the open-source software <a href="https://force-eo.readthedocs.io/en/latest/">FORCE </a>- Framework for Operational Radiometric Correction for Environmental monitoring (Frantz, D., 2019), in which SAR and environmental data were integrated. The map extent covers all areas in Germany that are defined in the respective year as cropland, grassland, small woody features, heathland, peatland or unvegetated areas according to ATKIS Basis-DLM (Geobasisdaten: © GeoBasis-DE / BKG, 2020).  Version v201: Post-processing of the maps included a sieve filter as well as a ruleset for the reduction of non-plausible areas using the Basis-DLM and the digital terrain model of Germany (Geobasisdaten: © GeoBasis-DE / BKG, 2015). Version v202: Additional post-processing was performed to detect and mask additional non-plausible areas that were not adequately covered by the first post-processing (e.g., areas with sparse vegetation, montane forests) based on the &bdquo;Ökosystematlas Deutschland“ (© Statistisches Bundesamt, Deutschland, 2024). As a consequence, the current version includes a new class “Small woody features on other land”. Furthermore, the class "permanent grassland" was refined. Each pixel that was classified as "cultivated grassland" in at least five years (between 2017 and 2022) was translated to "permanent grassland" in the annual maps. The maps are available as cloud optimized GeoTiffs, which makes downloading the full dataset optional. All data can directly be accessed in QGIS, R, Python or any supported software of your choice using the provided URL to the datasets (right click on the respective data set --> “copy link address”). By doing so the entire map area or only the regions of interest can be accessed. QGIS legend files for data visualization can be downloaded separately. Class-specific accuracies for each year are provided in the respective tables. We provide this dataset "as is" without any warranty regarding the accuracy or completeness and exclude all liability.    References: Blickensdörfer, L., Schwieder, M., Pflugmacher, D., Nendel, C., Erasmi, S., & Hostert, P. (2022). Mapping of crop types and crop sequences with combined time series of Sentinel-1, Sentinel-2 and Landsat 8 data for Germany. Remote Sensing of Environment, 269, 112831. BKG, Bundesamt für Kartographie und Geodäsie (2015). Digitales Geländemodell Gitterweite 10 m. DGM10. https://sg.geodatenzentrum.de/web_public/gdz/dokumentation/deu/dgm10.pdf (last accessed: 28. April 2022). BKG, Bundesamt für Kartographie und Geodäsie (2020). Digitales Basis-Landschaftsmodell. https://sg.geodatenzentrum.de/web_public/gdz/dokumentation/deu/basis-dlm.pdf (last accessed: 28. April 2022). Frantz, D. (2019). FORCE—Landsat + Sentinel-2 Analysis Ready Data and Beyond. Remote Sensing, 11, 1124. Statistisches Bundesamt, Deutschland (2024). Ökosystematlas Deutschland https://oekosystematlas-ugr.destatis.de/ (last accessed: 08.02.2024). ___________________________________________________________________________ National-scale crop type maps for Germany from combined time series of Sentinel-1, Sentinel-2 and Landsat data (2017 to 2021) © 2024 by Schwieder, Marcel; Tetteh, Gideon Okpoti; Blickensdörfer, Lukas; Gocht, Alexander; Erasmi, Stefan;  licensed under CC BY 4.0.  Funding was provided by the German Federal Ministry of Food and Agriculture as part of the joint project “Monitoring der biologischen Vielfalt in Agrarlandschaften” (<a href="https://www.agrarmonitoring-monvia.de/en/">MonViA</a>, Monitoring of biodiversity in agricultural landscapes). The study was financially supported by the European Environment Agency and the European Union’s Horizon Europe Research and Innovation programme under Grant Agreement No 101060423 (LAMASUS).

Future electricity demand time series for European Countries from 2023 to 2100

This dataset represents the future time series of electricity demand for European countries from 2023 to 2100, aligning with the findings presented in our paper 'Future Electricity Demand for Europe: Unraveling the Dynamics of the Temperature Response Function,' published in Applied Energy. To cite this dataset, please cite the published paper <a title="Persistent link using digital object identifier" href="https://doi.org/10.1016/j.apenergy.2024.123387" target="_blank" rel="noreferrer noopener">https://doi.org/10.1016/j.apenergy.2024.123387</a> This dataset includes electricity demand data for 36 European countries, with each year being presented as a distinct .CSV file. Data for all years in each country are then compressed in a single .ZIP file.  The column explanation is as below: <ul> <li>'country_code': the country code in 2 digits</li> <li>'year': the projection year</li> <li>'month': month of the year</li> <li>'day': day of the month</li> <li>'S0_RCP26_r1': time series data for electricity demand corresponding to Scenario S0 under the climate model ensemble realization r1, within the context of the Representative Concentration Pathway (RCP) 2.6.</li> <li>'S0_RCP26_r2': time series data for electricity demand corresponding to Scenario S0 under the climate model ensemble realization r2, within the context of the Representative Concentration Pathway (RCP) 2.6.</li> <li>'S0_RCP45_r1': time series data for electricity demand corresponding to Scenario S0 under the climate model ensemble realization r1, within the context of the Representative Concentration Pathway (RCP) 4.5.</li> <li>'S0_RCP45_r2': time series data for electricity demand corresponding to Scenario S0 under the climate model ensemble realization r2, within the context of the Representative Concentration Pathway (RCP) 4.5.</li> <li>'S0_RCP85_r1': time series data for electricity demand corresponding to Scenario S0 under the climate model ensemble realization r1, within the context of the Representative Concentration Pathway (RCP) 8.5.</li> <li>'S0_RCP85_r2': time series data for electricity demand corresponding to Scenario S0 under the climate model ensemble realization r1, within the context of the Representative Concentration Pathway (RCP) 8.5.</li> <li>'S1_RCP26_r1': time series data for electricity demand corresponding to Scenario S1 under the climate model ensemble realization r1, within the context of the Representative Concentration Pathway (RCP) 2.6.</li> <li>'S1_RCP26_r2': time series data for electricity demand corresponding to Scenario S1 under the climate model ensemble realization r2, within the context of the Representative Concentration Pathway (RCP) 2.6.</li> <li>'S1_RCP45_r1': time series data for electricity demand corresponding to Scenario S1 under the climate model ensemble realization r1, within the context of the Representative Concentration Pathway (RCP) 4.5.</li> <li>'S1_RCP45_r2': time series data for electricity demand corresponding to Scenario S1 under the climate model ensemble realization r2, within the context of the Representative Concentration Pathway (RCP) 4.5.</li> <li>'S1_RCP85_r1': time series data for electricity demand corresponding to Scenario S1 under the climate model ensemble realization r1, within the context of the Representative Concentration Pathway (RCP) 8.5.</li> <li>'S1_RCP85_r2': time series data for electricity demand corresponding to Scenario S1 under the climate model ensemble realization r1, within the context of the Representative Concentration Pathway (RCP) 8.5.</li> <li>'S2_RCP26_r1': time series data for electricity demand corresponding to Scenario S2 under the climate model ensemble realization r1, within the context of the Representative Concentration Pathway (RCP) 2.6.</li> <li>'S2_RCP26_r2': time series data for electricity demand corresponding to Scenario S2 under the climate model ensemble realization r2, within the context of the Representative Concentration Pathway (RCP) 2.6.</li> <li>'S2_RCP45_r1': time series data for electricity demand corresponding to Scenario S2 under the climate model ensemble realization r1, within the context of the Representative Concentration Pathway (RCP) 4.5.</li> <li>'S2_RCP45_r2': time series data for electricity demand corresponding to Scenario S2 under the climate model ensemble realization r2, within the context of the Representative Concentration Pathway (RCP) 4.5.</li> <li>'S2_RCP85_r1': time series data for electricity demand corresponding to Scenario S2 under the climate model ensemble realization r1, within the context of the Representative Concentration Pathway (RCP) 8.5.</li> <li>'S2_RCP85_r2': time series data for electricity demand corresponding to Scenario S2 under the climate model ensemble realization r1, within the context of the Representative Concentration Pathway (RCP) 8.5.</li> <li>'S3_RCP26_r1': time series data for electricity demand corresponding to Scenario S3 under the climate model ensemble realization r1, within the context of the Representative Concentration Pathway (RCP) 2.6.</li> <li>'S3_RCP26_r2': time series data for electricity demand corresponding to Scenario S3 under the climate model ensemble realization r2, within the context of the Representative Concentration Pathway (RCP) 2.6.</li> <li>'S3_RCP45_r1': time series data for electricity demand corresponding to Scenario S3 under the climate model ensemble realization r1, within the context of the Representative Concentration Pathway (RCP) 4.5.</li> <li>'S3_RCP45_r2': time series data for electricity demand corresponding to Scenario S3 under the climate model ensemble realization r2, within the context of the Representative Concentration Pathway (RCP) 4.5.</li> <li>'S3_RCP85_r1': time series data for electricity demand corresponding to Scenario S3 under the climate model ensemble realization r1, within the context of the Representative Concentration Pathway (RCP) 8.5.</li> <li>'S3_RCP85_r2': time series data for electricity demand corresponding to Scenario S3 under the climate model ensemble realization r1, within the context of the Representative Concentration Pathway (RCP) 8.5.</li> <li>'S4_RCP26_r1': time series data for electricity demand corresponding to Scenario S4 under the climate model ensemble realization r1, within the context of the Representative Concentration Pathway (RCP) 2.6.</li> <li>'S4_RCP26_r2': time series data for electricity demand corresponding to Scenario S4 under the climate model ensemble realization r2, within the context of the Representative Concentration Pathway (RCP) 2.6.</li> <li>'S4_RCP45_r1': time series data for electricity demand corresponding to Scenario S4 under the climate model ensemble realization r1, within the context of the Representative Concentration Pathway (RCP) 4.5.</li> <li>'S4_RCP45_r2': time series data for electricity demand corresponding to Scenario S4 under the climate model ensemble realization r2, within the context of the Representative Concentration Pathway (RCP) 4.5.</li> <li>'S4_RCP85_r1': time series data for electricity demand corresponding to Scenario S4 under the climate model ensemble realization r1, within the context of the Representative Concentration Pathway (RCP) 8.5.</li> <li>'S4_RCP85_r2': time series data for electricity demand corresponding to Scenario S4 under the climate model ensemble realization r1, within the context of the Representative Concentration Pathway (RCP) 8.5.</li> </ul>

opencc-by-4.0May 2018View details →

AVHRR NDVI Time Series

Vegetation seasonality assessment through remote sensing data is crucial to understand ecosystem responses to climatic variations and human activities at large-scales. Whereas the study of the timing of phenological events showed significant advances, their recurrence patterns at different periodicities has not been widely study, especially at global scale. In this work, we describe vegetation oscillations by a novel quantitative approach based on the spectral analysis of Normalized Difference Vegetation Index (NDVI) time series. A new set of global periodicity indicators permitted to identify different seasonal patterns regarding the intra-annual cycles (the number, amplitude, and stability) and to evaluate the existence of pluri-annual cycles, even in those regions with noisy or low NDVI. Most of vegetated land surface (93.18%) showed one intra-annual cycle whereas double and triple cycles were found in 5.58% of the land surface, mainly in tropical and arid regions along with agricultural areas. In only 1.24% of the pixels, the seasonality was not statistically significant. The highest values of amplitude and stability were found at high latitudes in the northern hemisphere whereas lowest values corresponded to tropical and arid regions, with the latter showing more pluri-annual cycles. The indicator maps compiled in this work provide highly relevant and practical information to advance in assessing global vegetation dynamics in the context of global change.

opencc-by-nc-4.0Mar 2024View details →

CoUDlabs_TA_USFD_02_ANNULAR_Regueiro. Time series analysis of sewer temperatures in an annular flume to estimate sediment accumulation

This dataset contains the results of the experimental campaign and how data were collected on the the <a href="https://co-udlabs.eu/">Co-UDlabs</a> Transnational Access project: Temperature time series analysis for predicting sedimentation in sewer systems, by Regueiro-Picallo et al. The Transnational Access project was funded under the European Union’s Horizon 2020 research and innovation programme under grant agreement No 101008626. The experiments were designed to further develop an innovative methodology for measuring sediment bed deposits in UDS based on temperature data analysis (<a href="https://doi.org/10.1039/D2EW00820C" target="_blank" rel="noopener">Regueiro-Picallo et al., 2023</a>; <a href="https://doi.org/10.1039/D3EW00825H" target="_blank" rel="noopener">Regueiro-Picallo et al., 2024</a>). Particularly, the aim of these experiments was to study the influence of the sewer hydrodynamics for estimating the sediment accumulation using temperature-based methods. Therefore, the main tasks of this project were the design of a temperature-control system for the ANNULAR flume to reproduce sewer temperature patterns, and the development of an experimental campaign including different flow, sediment depth and temperature pattern conditions. The purpose of these measurements is to prove the use of simplified systems to monitor sediment accumulation in sewer systems. The data are described so that others can use and reproduce.

Long time series (2001-2018) of daily evapotranspiration in China generated based on SEBAL: Part 2

The dataset named SEBAL evapotranspiration in China (SEBAL ET)  characterized the daily evapotranspiration (in millimeter) of vegetation in China from 2001 to 2018, the spatial resolution is 1 km × 1km and the temporal resolution is 1 day with the coordinate system of GCS_WGS_1984. The products were generated using Surface Energy Balance Algorithm of Land (SEBAL) and multi-sources remote sensing data, including MOD43A1 daily surface albedo, MOD11A1 daily surface temperature and MOD13 vegetation indices (obtained from NASA: https://ladsweb.modaps.eosdis.nasa.gov/search/), the meteorological data obtained from GMAO (https://gmao.gsfc.nasa.gov/research/highlights/2013-2015.php), the input variables were all aggregated of resampled to 1 km × 1km. The products were evaluated using the eight flux towers observation data for point validation and water balance method for regional validation and showed R value of 0.79 and 0.98, respectively, which indicated the products have a great performance.  SEBAL ET can be used for several geoscience studies, especially for global change, water resources mangement and agricultural drought monitoring, etc.

Long time series (2001-2018) of daily evapotranspiration in China generated based on SEBAL: Part 1

opencc-by-4.0Mar 2024View details →

Long-time series vSAG 37-F6

Supplementary data to the paper "Time series data provide insights into the evolution and abundance of one of the most abundant viruses in the marine virosphere: the uncultured pelagiphages vSAG 37-F6" <div>Includes the reads for the 1 and 7-years time series</div> <div> </div>

High-resolution wall-to-wall time series predictions of seasonal maize area and yield for Rwanda over 2019-2023

This is the companion dataset to publication {TBD}. It contains 1) seasonal composites of predicted maize cover and yield at 10 m resolution in Rwanda for two annual agricultural seasons over five years, 2) scripts for the end-to-end machine learning pipeline that produces these data products, and 3) data or references needed as inputs to the pipeline.  <h2>1) Maize cover and yield seasonal composites</h2> The data are provided here as netCDF4 files with four dimensions for x, y, band, and season. They can also be accessed as Google Earth ImageCollections at:  <ul> <li>https://code.earthengine.google.com/?asset=projects/b2p-geospatial/assets/lulc_classifier_composite</li> <li>https://code.earthengine.google.com/?asset=projects/b2p-geospatial/assets/maize_yield_composite </li> </ul> <h3>Land cover and maize classification</h3> The land cover classification file is found at <code>data/composites/lulc_classifier_Rwanda_2019to2023.nc</code>. The land cover classification images contain 3 bands/variables: maizeProb, the raw predicted probability of the pixel being maize given by the gradient boosted tree model; majorityClass, the categorical land cover class with the highest predicted probability among any of the nine classes in the respective pixel; and optimalClass, the categorical land cover class adjusted to agree with national statistics for expected maize area. The land cover classes map to the raster values as follows:  <div> <div> <pre>{ 1: 'maize', 2: 'nonmaize_annual', 3: 'nonmaize_perennial', 4: 'scrub_shrub_land', 5: 'forest', 6: 'flooded_vegetation', 7: 'water', 8: 'structure', 9: 'bare' }</pre> </div> </div> The dataset includes 5 years (2019-2023) and 10 seasons - the available time period at time of publication. In Rwanda, maize is typically planted and harvested during two distinct agricultural seasons per year: Season A from September to February and Season B from March to June. Therefore the seasons in the data are: 2019_Season_A, 2019_Season_B, 2020_Season_A, 2020_Season_B, 2021_Season_A, 2021_Season_B, 2022_Season_A, 2022_Season_B, 2023_Season_A, 2023_Season_B. <h3>Maize yield</h3> The maize yield file is found at <code>data/composites/maize_yield_Rwanda_2019to2023.nc</code>. Each of the images in the yield composites has 3 bands/variables also: maizeYield, the model's output of continuous predicted yield (kg/ha) in each pixel regardless of land class; maizeYield_majorityClass, predicted maize yield masked to the majority class land classification; and maizeYieldAdj_optimalClass, where the raw predicted yields were masked to the optimal maize classification land cover layer and normalized to national statistics.  The dataset includes the same seasons as the classification product; see above for a description. <h2>2) End-to-end machine learning pipeline</h2> All earth observation imagery, analysis, and outputs unless otherwise stated were hosted in the Google Earth Engine (GEE) environment and developed with the Earth Engine Python API in Python v3.10. To set up a local conda environment use the <code>scripts/environment.yml</code> file. The user must have <a href="https://cloud.google.com/storage">Google Cloud Storage (GCS)</a> and <a href="https://cloud.google.com/earth-engine">Google Earth Engine (GEE)</a> accounts. The pipeline, at this scale, will incur some processing and storage fees, although Google offers a free trial to all new users and the total cost of the high-resolution wall-to-wall predictions is nominal (~$20 for one season).  The scripts needed to perform the pipeline are located in the <code>scripts</code> folder.  The files contained in the <code>scripts/helpers</code> directory will be called by various subsequent scripts and do not to be run interactively by the user.  Follow the script in the order described below. The user should pause after running each script and confirm that all outputs were created and loaded to GCS before continuing the pipeline; for some steps this may take hours to days depending on processing speed.  <h3>Google Cloud Storage and Earth Engine set-up</h3> Users should specify the names of the bucket and asset project that were chosen during set up of their GCS and GEE environments in the Objects section of <code>scripts/helpers/maize_pipeline_0_workspace.py</code>. <h3>Pipeline set-up</h3> In <code>scripts/pipeline_setup</code>, you will find the following scripts to perform data preparation of inputs into model building and prediction.  <ul> <li><code>maize_pipeline_1_clean_training_data.py</code> - Cleans and merges all available crop label and yield data for model training and validation</li> <li><code>maize_pipeline_2_dwnld_data_training.py</code> - Downloads satellite-derived and auxiliary features at training data points for model building</li> <li><code>maize_pipeline_3_dwnld_data_inference.py</code> - Downloads satellite-derived and auxiliary features at every 10 m pixel in Rwanda on a district-wise basis for prediction</li> </ul> <h3>Land cover and maize classification</h3> In <code>scripts/maize_classification</code>, you will find the following scripts to perform model building, prediction, and post-processing for the classificaton of land cover type and maize cover. <ul> <li><code>maize_classifier_1_feature_selection.py</code> - Selects features subset for land cover classification with mutual information score or variable importance</li> <li><code>maize_classifier_2_build_model.py</code> - Builds gradient boosted tree model for land cover classification from training data</li> <li><code>maize_classifier_3_prediction.py</code> - Applies model for land cover classification to every 10 m pixel in Rwanda by season and district</li> <li><code>maize_classifier_4_postprocess.py</code> - Mosaics district-wise predictions and normalizes maize cover predictions to national agricultural statistics</li> </ul> <h3>Maize yield</h3> In <code>scripts/maize_yield</code>, you will find the following scripts to perform modeling building, prediction, and post-processing for maize yield estimation.  <ul> <li><code>maize_yield_1_build_model.py</code> - Builds gradient boosted tree model and performs bias correction for maize yield estimation from training data</li> <li><code>maize_yield_2_prediction.py</code> - Applies model for maize yield estimation to every 10 m pixel in Rwanda by season and district</li> <li><code>maize_yield_3_postprocess.py</code> - Mosaics district-wise predictions and normalizes maize yield predictions to national agricultural statistics</li> </ul> If you are running the entire pipeline with refreshed training data and model building, run each of these scripts, in order. By default, the script will run all A and B seasons from 2019A to current. Otherwise, if you just wish to re-run or update seasonal predictions from the existing classification or yield model run <code>maize_pipeline_3_dwnld_data_inference.py</code> to download the seasonal feature data across Rwanda and <code>maize_classifier_3_prediction.py</code>and <code>maize_classifier_4_postprocess.py</code> for classification predictions or <code>maize_yield_2_prediction.py</code> and <code>maize_yield_3_postprocess.py</code> for yield predictions, making sure to specify which season(s) are of interest in each script. However to do this, you also need to have a copy of the previously built models in your GCS (provided at <code>data/models</code>).  <h2>3) Input data into machine learning pipeline</h2> A description of datasets that must be sourced outside of the GEE platform is provided below. When available, the primary data source is also included in the directory <code>data/baselayers</code>. All other data, including Sentinel-2 imagery, auxiliary data, and other existing global land cover classificaiton products are hosted on GEE and called by the scripts directly. All datasets last accessed on 12 March 2024. <h3>Administrative and geological boundaries</h3> <ul> <li>World Countries - Downloaded from <a href="https://datacatalog.worldbank.org/search/dataset/0038272/World-Bank-Official-Boundaries">The World Bank Official Boundaries</a> and included here at <code>data/baselayers/World_Countries</code>.</li> <li>Rwanda district boundaries - Downloaded from <a href="https://datacatalog.worldbank.org/search/dataset/0041453/Rwanda-Admin-Boundaries-and-Villages">The World Bank Rwanda Admin Boundaries And Villages</a> and included here at <code>data/baselayers/WB_NISR_2018</code>. This should be loaded into a FeatureCollection GEE asset named districts_fc for use in the pipeline. </li> <li>Rwanda agro-ecological zones - Downloaded from <a href="https://doi.org/10.1371/journal.pone.0149239">Nzeyimana, Hartemink & Geissen (2016)</a> and included here at <code>data/baselayers/MINAGRI_AEZ_1980</code>. This should be loaded into a FeatureCollection GEE asset named aez_rwanda for use in the pipeline. </li> </ul> <h3>Global land cover classification product</h3> <ul> <li>Microsoft/Impact Observatory LULC - Although the <a href="https://planetarycomputer.microsoft.com/dataset/io-lulc-9-class">10m Annual Land Use Land Cover (9-class) V1</a> product contains data from 2017-2022, only the LULC map from the year 2021 was used, provided here at <code>data/baselayers/impactobs_lulc_rwa_2021.tif</code>. This should be loaded into an ImageCollection GEE asset named impact_obs_lulc for use in the pipeline.</li> </ul> (The others - Dynamic World and ESA's WorldCover - are hosted on GEE directly.) <h3>Land cover labels and maize yield crop cuttings</h3> <ul> <li>One Acre Fund - Contact authors to request access as this dataset is not hosted publicly. </li> <li>RTI International - The original source of this data (Radiant MLHub) has been discontinued, but users may be able to access it via <a href="https://beta.source.coop/repositories/rti/rwanda-crop-type/">Source Cooperative</a>. The data is also included here at <code>data/baselayers/rti_rwanda_crop_type_labels</code>. </li> <li>Crop Harvest - Downloaded from <a href="../records/7257688">Tseng et al. (2021, v13)</a> and included here at <code>data/baselayers/CropHarvest</code>. These data points were ultimately not used in the training data, but are provided here for others that may find this dataset useful in their context.</li> </ul> <h3>Rwanda national agricultural surveys</h3> <ul> <li>National Institute of Statisitcs Rwanda (NISR) - Downloaded from <a href="https://statistics.gov.rw/datasource/seasonal-agricultural-survey">NISR Seasonal Agricultural Survey</a> and existing seasons included here at <code>data/baselayers/NISR_Seasonal_Ag_Surveys</code>. For each subsequent season, the user will have to download the spreadsheet of survey results from the NISR webpage (linked) and add the respective season to the <code>get_nisr_data</code> function in the <code>helpers/maize_pipeline_0_helpers_postprocess.py</code> script to clean and read in the data for use in the pipeline. </li> </ul>

opencc-by-4.0Apr 2024View details →

Time Series Analysis with Python datasets

Datasets used in the course <a href="https://github.com/FilippoMB/python-time-series-handbook">Time Series Analysis with Python</a>.

opencc-by-4.0Apr 2024View details →

Time-series analysis of rhenium(I) organometallic covalent binding to a model protein for drug development: Raw Diffraction Images (38 week soak). Zenodo

The synchrotron raw diffraction images obtained at 38 weeks and wavelength 0.976 Å, illustrates the covalent coordination of the rhenium(I) tricarbonyl fragment to the His and Asp amino acid residues as well as movement along the solvent channels as described in the publication titled "Time-series analysis of rhenium(I) organometallic covalent binding to a model protein for drug development", written by Jacobs, Helliwell & Brink, IUCrJ, 2024, https://doi.org/10.1107/S2052252524002598. The raw diffraction images for the DLS data sets are made available at the Zenodo research data archive, as specified in the publication.

openNov 2024View details →

Global Landside Clustering of Aquaculture Ponds Distribution Acquired from Dense Time-Series Sentinel-2 Images by Google Earth Engine

This dataset reveals the global distribution pattern of landside clustering aquaculture ponds (LCAP) from a spatial perspective for the first time. It was derived from 4,015,054 tiles of the 10-m Sentinel-2 time-series images collected throughout 2020. The total area of global LCAP was estimated at 55,337.03 km2. Accuracy verification revealed that the Omission Error and Commission Error of the data is 7.51% and 16.69% respectively. We provide this dataset in ESRI shapefile format (.zip), which can be opened by ArcGIS. We invite you to download and utilize this dataset and recommend citing the following two references.

opencc-by-4.0Dec 2021View details →

Dataset for Spatial Heterogeneity of Uplift Pattern in the Western European Alps Revealed by InSAR Time Series Analysis

ZIP file with InSAR raw and smoothed final velocity solution values