Dataset for Generation of multiple true false questions
<p><strong>Generation of multiple true-false questions</strong></p> <p>This project provides a Natural Language Pipeline for processing German Textbook sections as an input generating Multiple True-False Questions using GPT2.</p> <p>Assessments are an important part of the learning cycle and enable the development and promotion of competencies. However, the manual creation of assessments is very time-consuming. Therefore, the number of tasks in learning systems is often limited. In this repository, we provide an algorithm that can automatically generate an arbitrary number of German True False statements from a textbook using the GPT-2 model. The algorithm was evaluated with a selection of textbook chapters from four academic disciplines (see `data` folder) and rated by individual domain experts. One-third of the generated MTF Questions are suitable for learning. The algorithm provides instructors with an easier way to create assessments on chapters of textbooks to test factual knowledge.</p> <p>As a type of Multiple-Choice question, Multiple True False (MTF) Questions are, among other question types, a simple and efficient way to objectively test factual knowledge. The learner is challenged to distinguish between true and false statements. MTF questions can be presented differently, e.g. by locating a true statement from a series of false statements, identifying false statements among a list of true statements, or separately evaluating each statement as either true or false. Learners must evaluate each statement individually because a question stem can contain both incorrect and correct statements. Thus, MTF Questions as a machine-gradable format have the potential to identify learners’ misconceptions and knowledge gaps.</p> <p>Example MTF question:</p> <blockquote> <p>Check the correct statements:</p> <p>[ ] All trees have green leafs.</p> <p>[ ] Trees grow towards the sky.</p> <p>[ ] Leafes can fall from a tree.</p> <p> </p> </blockquote> <p><strong>Features</strong></p> <p>- generation of false statements</p> <p>- automatic selection of true statements</p> <p>- selection of an arbitrary similarity for true and false statements as well as the number of false statements</p> <p>- generating false statements by adding or deleting negations as well as using a german gpt2</p> <p><strong>Setup</strong></p> <p><strong>Installation</strong></p> <p>1. Create a new environment: `conda create -n mtfenv python=3.9`</p> <p>2. Activate the environment: `conda activate mtfenv`</p> <p>3. Install dependencies using anaconda:</p> <p>```</p> <p>conda install -y -c conda-forge pdfplumber</p> <p>conda install -y -c conda-forge nltk</p> <p>conda install -y -c conda-forge pypdf2</p> <p>conda install -y -c conda-forge pylatexenc</p> <p>conda install -y -c conda-forge packaging</p> <p>conda install -y -c conda-forge transformers</p> <p>conda install -y -c conda-forge essential_generators</p> <p>conda install -y -c conda-forge xlsxwriter</p> <p>```</p> <p>3. Download spacy: `python3.9 -m spacy download de_core_news_lg`</p> <p><strong>Getting started</strong></p> <p>After installation, you can execute the bash script `bash run.sh` in the terminal to compile MTF questions for the provided textbook chapters.</p> <p>To create MTF questions for your own texts use the following command:</p> <p>`python3 main.py --answers 1 --similarity 0.66 --input ./<path>/<to>/<your>/<textbook>.txt`</p> <p>The parameter `answers` indicates how many false answers should be generated.</p> <p>By configuring the parameter `similarity` you can determine what portion of a sentence should remain the same. The remaining portion will be extracted and used to generate a false part of the sentence.</p> <p> </p> <p><strong>## History and roadmap </strong></p> <p>* Outlook third iteration: Automatic augmentation of text chapters with generated questions</p> <p>* Second iteration: Generation of multiple true-false questions with improved text summarizer and German GPT2 sentence generator</p> <p>* First iteration: Generation of multiple true false questions in the Bachelor thesis of Mirjam Wiemeler</p> <p> </p> <p><strong>Publications, citations, license</strong></p> <p><strong>Publications</strong></p> <ul> <li>Kasakowskij, R., Kasakowskij, T. & Seidel, N., (2022). Generation of Multiple True False Questions. In: Henning, P. A., Striewe, M. & Wölfel, M. (Hrsg.), 20. Fachtagung Bildungstechnologien (DELFI). Bonn: Gesellschaft für Informatik e.V.. (S. 147-152). DOI: [10.18420/delfi2022-026](https://dl.gi.de/handle/20.500.12116/38826)</li> </ul> <p> </p> <p><strong>Citation of the Dataset</strong></p> <ul> <li>Kasakowskij, R., Kasakowskij, T., & Seidel, N. (2022). <em>Dataset for Generation of multiple true false questions</em>. Zenodo. https://doi.org/10.5281/zenodo.7303300</li> </ul> <p>The source code and data are maintained at GitHub: https://github.com/D2L2/multiple-true-false-question-generation</p> <p><strong>Contact</strong></p> <ul> <li>Regina Kasakowskij (M.A.) - regina.kasakowskij@fernuni-hagen.de</li> <li>Dr. Niels Seidel - niels.seidel@fernuni-hagen.de</li> </ul> <p><strong>License</strong> Distributed under the MIT License. See [LICENSE.txt](https://gitlab.pi6.fernuni-hagen.de/la-diva/adaptive-assessment/generationofmultipletruefalsequestions/-/blob/master/LICENSE.txt) for more information.</p> <p><strong>Acknowledgments</strong> This research was supported by CATALPA - Center of Advanced Technology for Assisted Learning and Predictive Analytics of the FernUniversität in Hagen, Germany.</p> <p>This project was carried out as part of research in the CATALPA project [LA DIVA](https://www.fernuni-hagen.de/forschung/schwerpunkte/catalpa/forschung/projekte/la-diva.shtml)</p>
ShareScore
40/100
Overall dataset sharing score
Score breakdown
These five areas show where the dataset supports — or may limit — practical reuse.
- Stewardship
- 8
- Harmonization
- 4
- Access
- 16
- Reuse readiness
- 8
- Engagement
- 4
Topics
Dataset details
- Repository
- zenodo
- Published
- September 12, 2022
- License
- cc-by-4.0
- DOI
- 10.5281/zenodo.7303299
Scored with release 0.9.0
ScienceDex for faculty
Your academic CV and NIH biosketch
Make your CV in your school's format and your NIH biosketch from it, then download both as Word files.