Data and Processing from "Carbon-centric dynamics of Earth's marine phytoplankton"
<div><strong>Brief Summary:</strong></div> <div>This documentation is for associated data and code for: </div> <div>A. Stoer, K. Fennel, Carbon-centric dynamics of Earth's marine phytoplankton. Proceedings of the National Academy of Sciences (2024).</div> <div> </div> <div>To cite this software and data, please use:</div> <div> <div>A. Stoer, K. Fennel, Data and processing from "Carbon-centric dynamics of Earth's marine phytoplankton". Zenodo. <a href="https://doi.org/10.5281/zenodo.10949682" target="_blank" rel="noopener">https://doi.org/10.5281/zenodo.10949682</a>. Deposited 1 October 2024.</div> </div> <div> </div> <div><strong>List of folders and subfolders and what they contain:</strong></div> <div> <ol> <li>raw data: Contains raw data used in the analysis. This folder does not contain the satellite imagery, which will need to be downloaded from the NASA Ocean Color website (https://oceancolor.gsfc.nasa.gov/). <ol> <li>bgc-argo float data (subfolder): Includes Argo data from its original source or put into a similar Argo format</li> <li>global region data (subfolder): Includes data used to subset the Argo profiles into each 10deg lat region and basin.</li> <li>graff et al 2015 data (subfolder): Include the data digitized from Graff et al.'s Fig. 2.</li> </ol> </li> <li>processed data: data processing by this study (Stoer and Fennel, 2024) <ol> <li>processed bgc-argo data (subfolder): A binned processed file is present for each Argo float used in the analysis. Note these files include those describe in Table S1 (these are later processed in "3_stock_bloom_calc.py")</li> <li>processed satellite data (subfolder): includes a 10-deg latitude averaged for each satellite image processed (called "chl_sat_df_merged.csv"). This is later used to calculate a satellite chlorophyll-a climatology in "3_stock_bloom_calc.py".</li> <li>processed chla-irrad data (subfolder): includes the quality-controlled light diffuse attenuation data coupled with the chlorophyll-a fluorescence data to calculate slope factor corrections (the file is called "processed chla-irrad data.csv").</li> <li>processed topography data (subfolder): includes smoothed topography data (file named "ETOPO_2022_v1_60s_N90W180_surface_mod.tiff").</li> </ol> </li> <li>software: <ol> <li>0_ftp_argo_data_download.py: This program downloads the Argo data from the Global Data Assembly Center's FTP. Running this program will provide new Argo float profiles. However, there will be new floats and profiles present if downloaded. This will not match the historical record of Argo floats used in this analysis but could be useful for replicating this analysis when more data becomes available. The historical record of BGC-Argo floats are present in "/raw data/bgc-argo float data/" path. If you wish to downloaded other float data, see Gordon et al. (2020), Hamilton and Leidos (2017) and the data from the misclab website (https://misclab.umeoce.maine.edu/floats/).</li> <li>1_argo_data_processing.py: This program quality-controls and bins the biogeochemical data into a consistent format. This includes corrections and checks, like the spike/noise test or the non-photochemical quenching correction.</li> <li>2_sat_data_processing.py: this program processes the satellite data downloaded from the NASA Ocean Color website.</li> <li>3_stock_bloom_calc.py: this is the main program used to described the results of the study. The program takes the processed Argo data and groups it into regions and calculates slope factors, phytoplankton carbon & chlorophyll-a, global stocks, and bloom metrics.</li> <li>4_stock_calc_longhurst_province.py: This program repeats the global stocks calculations performed in "3_stock_bloom_calc.py" but bases the grouping on Longhurst Biogeochemical Provinces.</li> </ol> </li> </ol> </div> <div><strong>How to Replicate this Analysis:</strong></div> <div>Each program should be run in the order listed above. Path names where the data files have been downloaded will need to be updated in the code.</div> <div> </div> <div>To use the exact same Sprof files as used in the paper, skip running "0_ftp_argo_data_download.py" and start with "1_argo_data_processing.py" instead. Use the float data from the folder "bgc-argo float data". The program "0_ftp_argo_data_download.py" downloads the latest data from Argo database, so it is useful for updating the analysis. The program "1_argo_data_processing.py" may also be skipped to save time and the processed BGC-Argo float data may be used instead (see folder named "processed bgc-argo data"). </div> <div> </div> <div>Similarly, the program "2_sat_data_processing.py" may also be skipped, which otherwise can take multiple hours to process. The raw data is available from the NASA Ocean Color website (<a href="https://oceancolor.gsfc.nasa.gov/">https://oceancolor.gsfc.nasa.gov/</a>). The processed data from "2_sat_data_processing.py" is available so this step may be skipped to save time as well.</div> <div> </div> <div>The program "3_stock_bloom_calc.py" will require running "ocean_toolbox.py" (see below) in another tab. The portion of the program that involves QC for the irradiance profiles has been commented out to save processing time, and the pre-processed data used in the study has been linked instead (see folder "processed light data"). Similarly, pre-processed topography data is present in this repository. The original Earth Topography data can be accessed at <a href="https://www.ncei.noaa.gov/products/etopo-global-relief-model">https://www.ncei.noaa.gov/products/etopo-global-relief-model.</a></div> <p> </p> <p>A version of "3_stock_bloom_calc.py" using Longhurst provinces is available for exploring alternative groupings and their effects on stock calculations. See the program named "4_stock_calc_longhurst_province.py". You will need to download the Longhurst biogeochemical provinces from <a href="https://www.marineregions.org/">https://www.marineregions.org/</a>.</p> <p>To explore the effects of different slope factors, averaging methods, bbp spectral slopes, etc, the user will likely want to make changes to "3_stock_bloom_calc.py". Please do not hesitate to contact the correponding author (Adam Stoer) for guidance or questions.</p> <p><strong>ocean_toolbox.py:</strong></p> <p>import statsmodels.formula.api as smf<br>import os<br>import matplotlib.pyplot as plt<br>import numpy as np<br>from uncertainties import unumpy as unp<br>from scipy import stats</p> <p>def file_grab(root,find,start): #grabs files by file extensions and location<br> filelst = []<br> for subdir, dirs, files in os.walk(root):<br> for file in files:<br> filepath = subdir + os.sep + file<br> if filepath.endswith(find):<br> if filepath.startswith(start):<br> filelst.append(filepath)<br> return filelst</p> <p>def sep_bbp(data, name_z, name_chla, name_bbp):<br> <br> '''<br> data: Pandas Dataframe containing the profile data<br> name_z: name of the depth variable in data<br> name_chla: name of the chlorophyll-a variable in data<br> name_bbp: name of the particle backscattering variable in data <br> <br> returns: the data variable with particle backscattering partitioned into <br> phytoplankton (bbpphy) and non-algal particle components (bbpnap).<br> '''<br> #name_chla = 'chla'<br> #name_z = 'depth'<br> #name_bbp = 'bbp470'<br> dcm = data[data.loc[:,name_chla]==data.loc[:,name_chla].max()][name_z].values[0] # Find depth of deep chla maximum<br> part_prof = data[(data.loc[:,name_bbp]<np.median(data.loc[:,name_bbp]))] # find median bbp of profile<br> <br> mod = smf.quantreg('bbp470 ~ ' + str(name_z), <br> part_prof).fit(q=0.01) # Find model to 1 percentile<br> y_pred = mod.predict(part_prof.loc[:,name_z]) # Create predicted bbp_nap<br> <br> part_prof.loc[:,'bbp_back'] = y_pred.values # Predicted bbp NAP from linear trend<br> z_lim = part_prof.loc[(part_prof.loc[:,'bbp_back'].div(part_prof.loc[:,name_bbp])>=1), name_z].min() <br> <br> # Find depth where bbp NAP and bbp intersect<br> data.loc[data[name_z]>=z_lim, 'bbp_back'] = data.loc[data[name_z]>=z_lim, name_bbp].tolist()<br> data.loc[data[name_z]<z_lim,'bbp_back'] = data.loc[data[name_z]==z_lim, name_bbp].values[0] #data.loc[data[name_z]<z_lim, name_z].mul(lr.slope).add(lr.intercept)<br> <br> <br> data.loc[:,'bbpphy'] = data.loc[:, name_bbp].sub(data.loc[:,'bbp_back']) # Subtract bbp NAP from bbp for bbp from phytoplankton<br> data.loc[(data['bbpphy']<0)|(data['depth']>z_lim),'bbpphy'] = 0 # Subtract bbp NAP from bbp for bbp from phytoplankton</p> <p> return data['bbpphy'], z_lim</p> <p>def bbp_to_cphy(bbp_data, sf):<br> <br> '''<br> data: Pandas Dataframe containing the profile data<br> name_bbp: name of the particulate backscattering variable in data<br> name_bbp_err: name of particulate backscattering error variable in data<br> <br> returns: the data variable with particle backscattering [/m] converted into<br> phytoplankton carbon [mg/m^3].<br> '''<br> <br> cphy_data = bbp_data.mul(sf) </p> <p> return cphy_data</p>
ShareScore
40/100
Overall dataset sharing score
Score breakdown
These five areas show where the dataset supports — or may limit — practical reuse.
- Stewardship
- 8
- Harmonization
- 4
- Access
- 16
- Reuse readiness
- 8
- Engagement
- 4