Skip to main content
zenodoopen

Dataset for Generation of multiple true false questions

<p><strong>Generation of multiple true-false questions</strong></p> <p>This project provides a Natural Language Pipeline for processing German Textbook sections as an input generating Multiple True-False Questions using GPT2.</p> <p>Assessments are an important part of the learning cycle and enable the development and promotion of competencies. However, the manual creation of assessments is very time-consuming. Therefore, the number of tasks in learning systems is often limited. In this repository, we provide an algorithm that can automatically generate an arbitrary number of German True False statements from a textbook using the GPT-2 model. The algorithm was evaluated with a selection of textbook chapters from four academic disciplines (see `data` folder) and rated by individual domain experts. One-third of the generated MTF Questions are suitable for learning. The algorithm provides instructors with an easier way to create assessments on chapters of textbooks to test factual knowledge.</p> <p>As a type of Multiple-Choice question, Multiple True False (MTF) Questions are, among other question types, a simple and efficient way to objectively test factual knowledge. The learner is challenged to distinguish between true and false statements. MTF questions can be presented differently, e.g. by locating a true statement from a series of false statements, identifying false statements among a list of true statements, or separately evaluating each statement as either true or false. Learners must evaluate each statement individually because a question stem can contain both incorrect and correct statements. Thus, MTF Questions as a machine-gradable format have the potential to identify learners&rsquo; misconceptions and knowledge gaps.</p> <p>Example MTF question:</p> <blockquote> <p>Check the correct statements:</p> <p>[&nbsp; ] All trees have green leafs.</p> <p>[&nbsp; ] Trees grow towards the sky.</p> <p>[&nbsp; ] Leafes can fall from a tree.</p> <p>&nbsp;</p> </blockquote> <p><strong>Features</strong></p> <p>- generation of false statements</p> <p>- automatic selection of true statements</p> <p>- selection of an arbitrary similarity for true and false statements as well as the number of false statements</p> <p>- generating false statements by adding or deleting negations as well as using a german gpt2</p> <p><strong>Setup</strong></p> <p><strong>Installation</strong></p> <p>1. Create a new environment: `conda create -n mtfenv python=3.9`</p> <p>2. Activate the environment: `conda activate mtfenv`</p> <p>3. Install dependencies using anaconda:</p> <p>```</p> <p>conda install -y -c conda-forge pdfplumber</p> <p>conda install -y -c conda-forge nltk</p> <p>conda install -y -c conda-forge pypdf2</p> <p>conda install -y -c conda-forge pylatexenc</p> <p>conda install -y -c conda-forge packaging</p> <p>conda install -y -c conda-forge transformers</p> <p>conda install -y -c conda-forge essential_generators</p> <p>conda install -y -c conda-forge xlsxwriter</p> <p>```</p> <p>3. Download spacy: `python3.9 -m spacy download de_core_news_lg`</p> <p><strong>Getting started</strong></p> <p>After installation, you can execute the bash script `bash run.sh` in the terminal to compile MTF questions for the provided textbook chapters.</p> <p>To create MTF questions for your own texts use the following command:</p> <p>`python3 main.py --answers 1 --similarity 0.66 --input ./&lt;path&gt;/&lt;to&gt;/&lt;your&gt;/&lt;textbook&gt;.txt`</p> <p>The parameter `answers` indicates how many false answers should be generated.</p> <p>By configuring the parameter `similarity` you can determine what portion of a sentence should remain the same. The remaining portion will be extracted and used to generate a false part of the sentence.</p> <p>&nbsp;</p> <p><strong>## History and roadmap </strong></p> <p>* Outlook third iteration: Automatic augmentation of text chapters with generated questions</p> <p>* Second iteration: Generation of multiple true-false questions with improved text summarizer and German GPT2 sentence generator</p> <p>* First iteration: Generation of multiple true false questions in the Bachelor thesis of Mirjam Wiemeler</p> <p>&nbsp;</p> <p><strong>Publications, citations, license</strong></p> <p><strong>Publications</strong></p> <ul> <li>Kasakowskij, R., Kasakowskij, T. &amp; Seidel, N., (2022). Generation of Multiple True False Questions. In: Henning, P. A., Striewe, M. &amp; W&ouml;lfel, M. (Hrsg.), 20. Fachtagung Bildungstechnologien (DELFI). Bonn: Gesellschaft f&uuml;r Informatik e.V.. (S. 147-152). DOI: [10.18420/delfi2022-026](https://dl.gi.de/handle/20.500.12116/38826)</li> </ul> <p>&nbsp;</p> <p><strong>Citation of the Dataset</strong></p> <ul> <li>Kasakowskij, R., Kasakowskij, T., &amp; Seidel, N. (2022). <em>Dataset for Generation of multiple true false questions</em>. Zenodo. https://doi.org/10.5281/zenodo.7303300</li> </ul> <p>The source code and data are maintained at GitHub: https://github.com/D2L2/multiple-true-false-question-generation</p> <p><strong>Contact</strong></p> <ul> <li>Regina Kasakowskij (M.A.) - regina.kasakowskij@fernuni-hagen.de</li> <li>Dr. Niels Seidel - niels.seidel@fernuni-hagen.de</li> </ul> <p><strong>License</strong> Distributed under the MIT License. See [LICENSE.txt](https://gitlab.pi6.fernuni-hagen.de/la-diva/adaptive-assessment/generationofmultipletruefalsequestions/-/blob/master/LICENSE.txt) for more information.</p> <p><strong>Acknowledgments</strong> This research was supported by CATALPA - Center of Advanced Technology for Assisted Learning and Predictive Analytics of the FernUniversit&auml;t in Hagen, Germany.</p> <p>This project was carried out as part of research in the CATALPA project [LA DIVA](https://www.fernuni-hagen.de/forschung/schwerpunkte/catalpa/forschung/projekte/la-diva.shtml)</p>

ShareScore

40/100

Overall dataset sharing score

Score breakdown

These five areas show where the dataset supports — or may limit — practical reuse.

Stewardship
8
Harmonization
4
Access
16
Reuse readiness
8
Engagement
4

Topics

Dataset details

Repository
zenodo
Published
September 12, 2022
License
cc-by-4.0
DOI
10.5281/zenodo.7303299

Scored with release 0.9.0

ScienceDex for faculty

Your academic CV and NIH biosketch

Make your CV in your school's format and your NIH biosketch from it, then download both as Word files.