Proportions of Phytoplankton Functional Groups (PFT) retrieved using a Self-Organizing Map in the North Atlantic

<p>Concentrations of diagnostic pigments are retrieved from satellite data (Chl-a + Rrs at 4 wavelengths + SST) using a SOM trained on a global in-situ HPLC dataset (see El Hourany et al. 2019). Pigments are converted to PFTs using empirical coefficients. Three sets of coefficients are used and the results are averaged (see El Hourany et al. 2024). The PFT data is expressed as the proportion of each group in the total community abundance:<br><code>Proportion_i = (alpha_i * Pig_i) / sum_j(alpha_j * Pig_j)</code><br>There are seven PFT groups (diatoms, dinoflagellates, haptophytes, green algae, cryptophytes, pelagophytes, prokaryotes).</p> <p>The satellite input Chl-a data is included as well. It was retrieved from the Globcolour portal (<a href="https://hermes.acri.fr/">https://hermes.acri.fr/</a>) in 2022. The CHL-1 product for case 1 waters is used. It uses the AVW merging method: single-sensor level-2 Chl-a data is merged from multiple sensors (SeaWiFS, MERIS, MODIS-Aqua, VIIRS-NPP/JPSS1, OLCI-A/B).</p> <p>The data spans from 2002 to 2020, at a daily resolution. It is available on a regular latitude/longitude grid, at a resolution of 1/24&deg; (approximately 4 km) in a window of bounds 15&deg;N&minus;55&deg;N ; 82&deg;W&minus;40&deg;W.</p> <h2><strong>Storage</strong></h2> <p>All variables are stored in the same daily file: <code>PFT_SOM-DAP_GLOB_4km_daily/[year]/[month]/PFT_[year][month][day]_GLOB_4.nc</code>. The PFT proportions are stored as percentages (ranging between 0 and 100). Files are NetCDF4, and the metadata follow CF conventions.</p> <p>The PFT variables are stored using linear packing (see <a href="https://nco.sourceforge.net/nco.html#Linear-Packing">https://nco.sourceforge.net/nco.html#Linear-Packing</a>) as 16-bits unsigned integers (<code>NC_USHORT</code>), with a scale factor of 1.54e-3. This means values are discretized between 0 and ~100.92 (<code>=(2**16 - 2) * 1.54e-3</code>), with a discretization step of 1.54e-3.<br>As for the PFT variables, these values are in percent points.</p> <h2><strong>References</strong></h2> <ul> <li>El Hourany, R., Abboud-Abi Saab, M., Faour, G., Aumont, O., Cr&eacute;pon, M., Thiria, S.<br>&ldquo;Estimation of secondary phytoplankton pigments from satellite observations using Self-Organizing Maps (SOMs)&rdquo;,<br><em>J. Geophys. Res. Oceans</em> 124, 1357&ndash;1378, <a href="https://doi.org/10.1029/2018jc014450">https://doi.org/10.1029/2018jc014450</a>, <strong>2019</strong></li> <li> <div>El Hourany, R, Pierella Karlusich J., Zinger L., Loisel H., Levy M., and Bowler C.<br>&ldquo;Linking Satellites to Genes with Machine Learning to Estimate Phytoplankton Community Structure from Space&rdquo;<em>,<br>Ocean Science</em> 20 no. 1, 217&minus;39. <a href="https://doi.org/10.5194/os-20-217-2024">https://doi.org/10.5194/os-20-217-2024</a>, <strong>2024</strong></div> </li> </ul> <h2><strong>Changelog</strong></h2> <h3>v1.2</h3> <ul> <li>[2024-01-24] Finished re-arranging data and checked validity. 553 missing days / source files (about 6% of total data).</li> <li>[2023-12-01] Fix SST projection at PFT generation. Data fully re-generated.</li> </ul> <h3>v1.1</h3> <ul> <li>[2023-08-30] Store PFT data using linear packing.&nbsp;</li> </ul> <h3>v1.0</h3> <ul> <li>[2023-08-03] those data are reorganised, and metadata is added, using the script <a href="https://gitlab.in2p3.fr/clementhaeck/submeso-color/-/blob/develop/Compute/arrange_pft_data.py?ref_type=heads">https://gitlab.in2p3.fr/clementhaeck/submeso-color/-/blob/develop/Compute/arrange_pft_data.py?ref_type=heads</a></li> <li>[2023-07] data generated with the SOM were stored on `spirit:/data/lollier/PFT`</li> </ul>