MUHAI Benchmark : Task 1 (Short story generation with Knowledge Graphs)

<p><strong>Meaning and Understanding in Human-Centric AI (MUHAI) Benchmark<br> Task 1 (Short story generation with Knowledge Graphs and Language Models)</strong>&nbsp;</p> <p>The dataset&nbsp;can be used to test understandability of text generated through the&nbsp;combination of&nbsp;knowledge graphs and language models without using knowledge graph embeddings.<br> <br> The task here is to generate 5-sentence stories from a set of <em>subject-predicate-object</em>&nbsp;triples that are extracted&nbsp;from&nbsp;a knowledge graph. Two steps need to be performed:</p> <p>1. Language model fine-tuning (SVO triple extraction + model fine-tuning)<br> 2. Story generation (knowledge enrichment + text generation)&nbsp;<br> <br> The submission includes the following data:</p> <ol> <li>Original ROC stories corpus (100 stories)</li> <li>ROC stories encoded&nbsp;with relevant triples&nbsp;(extracted through SpaCy, 2 versions, with and without coreference resolution)</li> <li>Stories generated by the pre-trained&nbsp;model (GPT2-simple)</li> <li>Stories generated by the fine-tuned model (DICE + ConceptNet + DBpedia&nbsp;)</li> <li>Stories generated by the fine-tuned model (DICE + ConceptNet + DBpedia + WordNet&nbsp;)</li> <li>Stories generated by the GPT-2-keyword-generation (an open-source software that uses&nbsp;GPT-2 to generate text pertaining to the specified keywords)</li> <li>Model results</li> <li>Evaluation metrics description</li> <li>User-evaluation questionnaire&nbsp;</li> </ol> <p>Code :&nbsp;https://github.com/kmitd/muhai-dice_story</p>