zenodoopen

CY-Bench: A comprehensive benchmark dataset for subnational crop yield forecasting

<h1>CY-Bench: A comprehensive benchmark dataset for sub-national crop yield forecasting</h1> <h2> Overview</h2> CY-Bench is a dataset and benchmark for subnational crop yield forecasting, with coverage of major crop growing countries of the world for maize and wheat. By subnational, we mean the administrative level where yield statistics are published. When statistics are available for multiple levels, we pick the highest resolution. The dataset combines sub-national yield statistics with relevant predictors, such as growing-season weather indicators, remote sensing indicators, evapotranspiration, soil moisture indicators, and static soil properties. CY-Bench has been designed and curated by agricultural experts, climate scientists, and machine learning researchers from the <a href="https://www.agml.org/">AgML Community</a>, with the aim of facilitating model intercomparison across the diverse agricultural systems around the globe in conditions as close as possible to real-world operationalization. Ultimately, by lowering the barrier to entry for ML researchers in this crucial application area, CY-Bench will facilitate the development of improved crop forecasting tools that can be used to support decision-makers in food security planning worldwide. * Crops : Wheat & Maize * Spatial Coverage : Wheat (29 countries), Maize (38).   See <a href="https://github.com/WUR-AI/AgML-CY-Bench/blob/main/notebooks/cybench_summary.ipynb">CY-Bench Summary</a> for the list of countries. * Temporal Coverage : Varies. See <a href="https://github.com/WUR-AI/AgML-CY-Bench/blob/main/notebooks/cybench_summary.ipynb">CY-Bench Summary</a>. <h2>Data </h2> <h3>Data format</h3> The benchmark data is organized as a collection of CSV files (with the exception of location information, see below), with each file representing a specific category of variable for a particular country. Each CSV file is named according to the category and the country it pertains to, facilitating easy identification and retrieval. The data within each CSV file is structured in tabular format, where rows represent observations and columns represent different predictors related to a category of variable. <h3>Data content</h3> All data files are provided as .csv. <table style="width: 100%;"> <tbody> <tr> <td style="width: 10.3122%;">Data</td> <td style="width: 21.9489%;">Description</td> <td style="width: 18.069%;">Variables (units)</td> <td style="width: 13.8137%;">Temporal Resolution</td> <td style="width: 35.8562%;">Data Source (Reference)</td> </tr> <tr> <td style="width: 10.3122%;">crop_calendar</td> <td style="width: 21.9489%;">start and end of growing season</td> <td style="width: 18.069%;">sos (day of the year), eos (day of the year)</td> <td style="width: 13.8137%;">static</td> <td style="width: 35.8562%;">World Cereal (Franch et al, 2022)</td> </tr> <tr> <td style="width: 10.3122%;">crop_mask</td> <td style="width: 21.9489%;">crop area fraction</td> <td style="width: 18.069%;">crop_area (km2), crop_area_percentage (%)</td> <td style="width: 13.8137%;">static</td> <td style="width: 35.8562%;">WorldCereal (Van Tricht et al., 2023; EC-JRC, 2024)</td> </tr> <tr> <td style="width: 10.3122%;">fpar</td> <td style="width: 21.9489%;">fraction of absorbed photosynthetically active radiation</td> <td style="width: 18.069%;">fpar (%)</td> <td style="width: 13.8137%;">Dekadal (3 times a month; 1-10, 11-20, 21-31)</td> <td style="width: 35.8562%;">European Commission's Joint Research Centre (EC-JRC, 2024)</td> </tr> <tr> <td style="width: 10.3122%;">ndvi</td> <td style="width: 21.9489%;">normalized difference vegetation index</td> <td style="width: 18.069%;">-</td> <td style="width: 13.8137%;">approximately weekly</td> <td style="width: 35.8562%;">MOD09CMG (Vermote, 2015)</td> </tr> <tr> <td style="width: 10.3122%;">meteo</td> <td style="width: 21.9489%;">temperature, precipitation (prec), radiation, potential evapotranspiration (et0), climatic water balance (= prec - et0)  </td> <td style="width: 18.069%;">tmin (C), tmax (C), tavg (C), prec (mm0, et0 (mm), cwb (mm), rad (J m-2 day-1)</td> <td style="width: 13.8137%;">daily</td> <td style="width: 35.8562%;">AgERA5 (Boogaard et al, 2022)</td> </tr> <tr> <td style="width: 10.3122%;">soil_moisture</td> <td style="width: 21.9489%;">surface soil moisture, rootzone soil moisture</td> <td style="width: 18.069%;">ssm (kg m-2), rsm (kg m-2)</td> <td style="width: 13.8137%;">daily</td> <td style="width: 35.8562%;">GLDAS (Rodell et al, 2004)</td> </tr> <tr> <td style="width: 10.3122%;">soil</td> <td style="width: 21.9489%;">available water capacity, bulk density, drainage class</td> <td style="width: 18.069%;">awc (c m-1), bulk_density (kg dm-3), drainage class (category)</td> <td style="width: 13.8137%;">static</td> <td style="width: 35.8562%;">WISE Soil database (Batjes, 2016)</td> </tr> <tr> <td style="width: 10.3122%;">location</td> <td style="width: 21.9489%;">centroid</td> <td style="width: 18.069%;">latitude, logitude, region_area (km2)</td> <td style="width: 13.8137%;">static</td> <td style="width: 35.8562%;"> </td> </tr> <tr> <td style="width: 10.3122%;">yield</td> <td style="width: 21.9489%;">end-of-season yield</td> <td style="width: 18.069%;">yield (t ha-1)</td> <td style="width: 13.8137%;">yearly</td> <td style="width: 35.8562%;">Various country or region specific sources (see crop_statistics_... in https://github.com/WUR-AI/AgML-CY-Bench/tree/main/data_preparation)</td> </tr> </tbody> </table> <h3> </h3> <h3>Folder structure</h3> <ol> <li>cybench-data: The CY-Bench dataset has been structure at first level by crop type and subsequently by country. For each country, the folder name follows the ISO 3166-1 alpha-2 two-character code. A separate .csv is available for each predictor data and crop calendar as shown below. The csv files are named to reflect the corresponding country and crop type e.g. **variable_croptype_country.csv**. ``` CY-Bench │ └─── maize │   │ │   └─── AO │   │   -- crop_calendar_maize_AO.csv │   │   -- crop_mask_maize_AO.csv │   │   -- fpar_maize_AO.cs │   │   -- location_maize_AO.csv │   │   -- meteo_maize_AO.csv │   │   -- ndvi_maize_AO.csv │   │   -- soil_maize_AO.csv │   │   -- soil_moisture_maize_AO.csv │   │   -- yield_maize_AO.csv │   │  │   └─── AR │       -- crop_calendar_maize_AR.csv │       -- crop_mask_maize_AR.csv │       -- fpar_maize_AR.csv │       -- ... │    └─── wheat │   │ │   └─── AR │   │   -- crop_calendar_wheat_AR.csv │   │   -- crop_mask_wheat_AR.csv │   │   -- fpar_wheat_AR.csv │   │   ... ``` <h3>Example : CSV data content for maize in country X</h3> ``` X └─── crop_calendar_maize_X.csv │   -- crop_name (name of the crop) │   -- adm_id (unique identifier for a subnational unit) │   -- sos (start of crop season) │   -- eos (end of crop season) │ └─── crop_mask_maize_X.csv │   -- crop_name │   -- adm_id  │   -- crop_area │   -- crop_area_percentage │    └─── fpar_maize_X.csv │   -- crop_name │   -- adm_id  │   -- date (in the format YYYYMMdd) │   -- fpar │ └─── location_maize_X.csv │   -- crop_name │   -- adm_id  │   -- latitude │   -- longitude │   -- region_area │ └─── meteo_maize_X.csv │   -- crop_name │   -- adm_id  │   -- date (in the format YYYYMMdd) │   -- tmin (minimum temperature) │   -- tmax (maximum temperature) │   -- prec (precipitation) │   -- rad (radiation) │   -- tavg (average temperature) │   -- et0 (evapotranspiration) │   -- vpd (vapor pressure deficit) │   -- cwb (crop water balance) │  └─── ndvi_maize_X.csv │   -- crop_name │   -- adm_id │   -- date (in the format YYYYMMdd) │   -- ndvi   │    └─── soil_maize_X.csv │   -- crop_name │   -- adm_id │   -- awc (available water capacity) │   -- bulk_density │   -- drainage_class │    └─── soil_moisture_maize_X.csv │   -- crop_name │   -- adm_id │   -- date (in the format YYYYMMdd) │   -- ssm (surface soil moisture) │   -- rsm () │    └─── yield_maize_X.csv │   -- crop_name │   -- country_code │   -- adm_id │   -- harvest_year │   -- yield │   -- harvest_area │   -- production </li> <li>centroids.zip and polygons.zip include shapes or geometries as centroids ( x and y coordinates) and polygons (multipolygons) of administrative regions respectively. They are organized as follows: centroids │   └─── AO │   │   -- AO.cpg │   │   -- AO.dbf │   │   -- AO.prj │   │   -- AO.shp │   │   -- AO.shx │   └─── AR │   │   -- AR.cpg │   │   -- AR.dbf │   │   -- AR.prj │   │   -- AR.shp │   │   -- AR.shx ... polygons │   └─── AO │   │   -- AO.cpg │   │   -- AO.dbf │   │   -- AO.prj │   │   -- AO.shp │   │   -- AO.shx │   └─── AR │   │   -- AR.cpg │   │   -- AR.dbf │   │   -- AR.prj │   │   -- AR.shp │   │   -- AR.shx ...</li> </ol> <h3>Data access</h3> The full dataset can be downloaded directly from Zenodo or using the ```zenodo_get``` library <h2> License and citation</h2> We kindly ask all users of CY-Bench to properly respect licensing and citation conditions of the datasets included.   <h2>Version Notes</h2> 1.0 is the dataset submitted to NeurIPS Datasets and Benchmarks Track. The paper and discussions are here: https://openreview.net/forum?id=jkJDNG468g#discussion 1.1 and 1.2 fix some issues with column names and mismatches in adm_id between yield data and input data. 1.3 includes location information in the form of centroids and polygons of admin regions. 1.4 updates the fpar data for 2023. fpar data was incomplete for 2023 in earlier versions (due to unavailability in the data source itself). 1.5 fixes an issue in crop calendar 1.6 fixes an issue in ndvi time series 1.7 updates storage precision to 3 decimal places to reduce data size 1.8 filter out invalid yield values 1.9 Add vpd. Add location. ET0 obtained from AgERA5 (was AQUASTAT-FAO ). Use AgERA5 2.0 (was AgERA5 1.1) 1.10 Add region_are to location*.csv. Add crop_mask_*.csv. Fix error in yield Australia.

ShareScore

32/100

Overall dataset sharing score

Score breakdown

These five areas show where the dataset supports — or may limit — practical reuse.

Stewardship: 4
Harmonization: 8
Access: 8
Reuse readiness: 8
Engagement: 4