Radiative-transfer dataset for "Distilling machine learning's added value: Pareto fronts in atmospheric applications"

<p>This dataset goes with the journal paper "Distilling machine learning's added value: Pareto fronts in atmospheric applications" by T. Beucler, A. Grundner, S. Shamekh, P. Ukkonen, M. Chantry, and R. Lagerquist.</p> <p>Subdirectory "training" contains unnormalized (in physical units) training data.&nbsp; Subdirectories "validation" and "testing" contain unnormalized validation and testing data.&nbsp; Subdirectory "training/for_pareto_paper_2024/simple" contains training data from the simple (clear-sky) dataset discussed in the paper; subdirectory "training/for_pareto_paper_2024/complex" contains training data from the complex (multi-cloud) dataset discussed in the paper.&nbsp; Subdirectories "validation/for_pareto_paper_2024/simple" and "validation/for_pareto_paper_2024/complex" are analogous but for the validation data; subdirectories "testing/for_pareto_paper_2024/simple" and "testing/for_pareto_paper_2024/complex" are analogous but for the testing data.</p> <p>Subdirectories beginning with "normalized_predictors" -- "normalized_predictors/training", "normalized_predictors/validation", "normalized_predictors/testing", "normalized_predictors/training/for_pareto_paper_2024/simple", "normalized_predictors/training/for_pareto_paper_2024/complex", etc. -- are analogous to the above but containing normalized predictors (in z-scores rather than physical units).</p> <p>Every file -- after unzipping, so that the extension is ".nc" rather than ".nc.gz" -- can be read by `example_io.read_file` in the ml4rt library (https://github.com/thunderhoser/ml4rt).</p>