Model outputs for the study "Guidance in Radiology Report Summarization: An Empirical Evaluation and Error Analysis"

<p>This resources provides pre-processed input data, model checkpoints and model outputs for&nbsp;experiments on the&nbsp;OpenI dataset in below&nbsp;study.&nbsp;</p> <blockquote> <p>Jan Trienes, Paul Youssef, J&ouml;rg Schl&ouml;tterer, and Christin Seifert. 2023. <a href="https://arxiv.org/abs/2307.12803">Guidance in Radiology Report Summarization: An Empirical Evaluation and Error Analysis</a>. In Proceedings of the 16th International Natural Language Generation Conference (INLG), Prague, Czech Republic. Association for Computational Linguistics.</p> </blockquote> <p>For more information please refer to the accompanying paper and code repository (<a href="https://github.com/jantrienes/inlg2023-radsum">https://github.com/jantrienes/inlg2023-radsum</a>).</p> <p><strong>The data is structured as follows:</strong></p> <ul> <li><code>data/preprocessed/</code>&nbsp;includes the dataset(s)&nbsp;for each model</li> <li><code>output/</code>&nbsp;includes one folder for each experiment/model run. The first part&nbsp;of each output path&nbsp;indicates the dataset that was used at inference.</li> <li>For a mapping between model IDs and results in the paper, see below table. All models were also trained <em>with the background section as input.&nbsp;</em>These are available in directories&nbsp;with the&nbsp;<code>-bg-</code> qualifier.&nbsp;</li> </ul> <table> <thead> <tr> <th>Model name in paper</th> <th>Output directory</th> </tr> </thead> <tbody> <tr> <td><em>Results from Table 2</em></td> </tr> <tr> <td>OracleExt</td> <td>openi-unguided/oracle</td> </tr> <tr> <td>BertExt (Liu and Lapata, 2019)</td> <td>openi-unguided/bertext-default</td> </tr> <tr> <td>BertAbs (Liu and Lapata, 2019)</td> <td>openi-unguided/bertabs-default</td> </tr> <tr> <td>GSum (Dou et al., 2021)</td> <td>openi-bertext-default-clip-k1/gsum-default</td> </tr> <tr> <td>GSum w/ LR-Approx</td> <td>openi-bertext-default-clip-lrapprox/gsum-default</td> </tr> <tr> <td>GSum w/ BERT-Approx</td> <td>openi-bertext-default-clip-bertapprox/gsum-default</td> </tr> <tr> <td>GSum w/ Thresholding</td> <td>openi-bertext-default-clip-threshold/gsum-default</td> </tr> <tr> <td>WGSum (Hu et al., 2021)</td> <td>openi-wgsum/wgsum-default</td> </tr> <tr> <td>WGSum+CL (Hu et al., 2022)</td> <td>openi-wgsum-cl/wgsum-cl-default</td> </tr> <tr> <td><em>Results from Table 3</em></td> </tr> <tr> <td>Fixed (k=1)</td> <td>openi-unguided/bertext-default-clip-k1</td> </tr> <tr> <td>LR-Approx</td> <td>openi-unguided/bertext-default-clip-lrapprox</td> </tr> <tr> <td>BERT-Approx</td> <td>openi-unguided/bertext-default-clip-bertapprox</td> </tr> <tr> <td>Thresholding</td> <td>openi-unguided/bertext-default-clip-threshold</td> </tr> <tr> <td>k = |OracleExt|</td> <td>openi-unguided/bertext-default-clip-oracle</td> </tr> <tr> <td><em>Results from Table 4</em></td> </tr> <tr> <td>Fixed (Dou et al., 2021)</td> <td>openi-bertext-default-clip-k1/gsum-default</td> </tr> <tr> <td>Oracle Length</td> <td>openi-bertext-default-clip-oracle/gsum-default</td> </tr> <tr> <td>Oracle Length + Content</td> <td>openi-oracle/gsum-default</td> </tr> <tr> <td><em>Results from Table 5</em></td> </tr> <tr> <td>BertExt w/ k=[1,5]</td> <td>openi-unguided/bertext-default-clip-k{1,2,3,4,5}</td> </tr> <tr> <td>GSum w/ k=[1,5]</td> <td>openi-bertext-default-clip-k{1,2,3,4,5}/gsum-default</td> </tr> </tbody> </table>