Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
49
datasets available to search
ShareScore release 0.9.0
Dataset results
49 results for “GPT”
Programming Problems Submitted for Evaluation of LLMs GPT-3.5 and Gemini Pro 1.0
<p>Problems extracted from platforms LeetCode and BeeCrowd for evaluation of LLMs GPT3.5 and Gemini Pro 1.0.</p> <p>The data from the plataforms has the following columns and values:</p> <p> </p> <table> <tbody> <tr> <td><strong>LeetCode Data</strong></td> <td> </td> <td><strong>BeeCrowd Data</strong></td> <td> </td> </tr> <tr> <td><strong>Column</strong></td> <td><strong>Doc</strong></td> <td><strong>Column</strong></td> <td><strong>Doc</strong></td> </tr> <tr> <td>problem_level</td> <td>easy | medium | hard</td> <td>problem_level</td> <td><1...10></td> </tr> <tr> <td>problem_link</td> <td><LeetCode link for the problem></td> <td>problem_link</td> <td><BeeCrowd link for the problem></td> </tr> <tr> <td>prompt</td> <td><text submitted to LLM></td> <td>prompt</td> <td><text submitted to LLM></td> </tr> <tr> <td>response_code</td> <td><code provided by the LLM></td> <td>response_code</td> <td><code provided by the LLM></td> </tr> <tr> <td>response_evaluation</td> <td>True | False</td> <td>response_evaluation</td> <td>True | False</td> </tr> <tr> <td>execution_time_ms</td> <td><time></td> <td>execution_time_ms</td> <td><time></td> </tr> <tr> <td>memory_usage_mb</td> <td><memory></td> <td>error_generated</td> <td>Wrong Answer | Time Limit Exceeded | Memory Limit Exceeded....</td> </tr> <tr> <td>error_generated</td> <td>Wrong Answer | Time Limit Exceeded | Memory Limit Exceeded....</td> <td>attempts_number</td> <td>1 | 2 | 3</td> </tr> <tr> <td>attempts_number</td> <td>1 | 2 | 3</td> <td>author</td> <td><author's name></td> </tr> <tr> <td>contains_image</td> <td>True | False</td> <td>source</td> <td><origin institution></td> </tr> <tr> <td>related_topic_1</td> <td><topic></td> <td>origin_country</td> <td><origin country></td> </tr> <tr> <td>related_topic_2</td> <td><topic></td> <td>contains_image</td> <td>True | False</td> </tr> <tr> <td>related_topic_3</td> <td><topic></td> <td>related_topic_1</td> <td><topic></td> </tr> <tr> <td>related_topic_4</td> <td><topic></td> <td> </td> <td> </td> </tr> <tr> <td>related_topic_5</td> <td><topic></td> <td> </td> <td> </td> </tr> </tbody> </table> <p> </p>
Text Generation using N-gram and GPT (metrics: ROUGE, BLEU and BERTScore)
<p>This publication presents a set of spreadsheets listing the user stories generated using N-gram and GPT models with metrics ROUGE, BLEU and BERTScore calculated. Each spreadsheet refers to one corpus of user stories processed.</p>
GPT-2 generated form fields
<p>This dataset is a single json containing label-value form fields generated using GPT-2. This data was used to train <strong>Dessurt</strong> (<a href="https://arxiv.org/abs/2203.16618">https://arxiv.org/abs/2203.16618</a>). Details of the generation process can be found in Dessurt's Supplementary Materials and the script used to generate it is <em>gpt_forms.py</em> in <a href="https://github.com/herobd/dessurt">https://github.com/herobd/dessurt</a></p> <p>The data has groups of label-value pairs each with a "title" or topic (or null). Each label-value pair group was generated in a single GPT-2 generation and thus the pairs "belong to the same form." The json structure is a list of tuples, where each tuple has the title or null as the first element and the list of label-value pairs of the group as the second element. Each label-value pair is another tuple with the first element being the label and the second being the value or a list of values.</p> <p> </p> <p>For example:</p> <p>[ ["title",[ ["first label", "first value"], ["second label", ["a label", "another label"] ] ] ], [null, [ ["again label", "again value"] ] ] ]</p>
A Study of Improving Mathematics Assessment Readability by using GPT-3
<p><strong>A Study of Improving Mathematics Assessment Readability by using GPT-3</strong></p> <p>This dataset contains 250 math word problems from EngageNY, and their automated simplifications generated using the GPT-3 engine. There are eight ways each prompt is simplified, and three samples are taken for each prompt. Some API calls can fail or return a blank string. Those responses are omitted. Readability measures for input and output passages are generated using the Common Text Analysis Platform (<a href="http://sifnos.sfs.uni-tuebingen.de/ctap/">http://sifnos.sfs.uni-tuebingen.de/ctap/</a>). Text similarity metrics are generated by computing the % of total words between input and output and cosine similarity of the text vectors that GPT-3 embedding API generates.</p>
GPT Capabilites for Extracting Tasks From Textual Process Descriptions
<p>This dataset provides multiple tables which evaluate the capabilities of GPT 1, GPT 2, GPT 3, GPT 3.5, and GPT 4 regarding the extraction of tasks from <a href="https://doi.org/10.5281/zenodo.7783492">https://doi.org/10.5281/zenodo.7783492</a> and PET datasets in model_completeness.zip, model_correctness.xslx and control_evaluation.xlsx.</p><p>The performance of the LLMs is measured by calculating a range of similarity metrics:</p><ul><li>Extracted number of tasks from text vs. extracted number of tasks from Model</li><li>Semantic Text Similarity: Contextual and Non-Contextual between extracted sets of tasks</li><li>Semantic Text Similarity: Contextual and Non-Contextual between extracted individual tasks</li><li>Similarities and Prevalence for length restricted extracted labels</li><li>Similarities and Prevalence for augmented texts (each text has been paraphrased by 9 different paraphrasing methods)</li><li>Recall and Precision for the above mentioned datasets</li><li>Jaccard Index for the above mentioned datasets</li></ul><p>This dataset also includes all data collected from a survey (survey_data.ods), which lets users (currently n=40) evaluate how good LLM created models are: <a href="https://forms.office.com/e/Y55jyNuPi2?origin=lprLink">https://forms.office.com/e/Y55jyNuPi2?origin=lprLink</a></p>
Dataset for the workshop paper titled "Automating GUI-based Software Testing with GPT-3" published at AIST 2023
<p>The training dataset for the research paper "Automating GUI-based Software Testing with GPT-3" presented at the 3rd International Workshop on Artificial Intelligence in Software Testing (AIST 2023), which was a part of the 16th IEEE International Conference on Software Testing, Verification and Validation (ICST 2023). The dataset contains prompt completion pairs acquired through user interaction with the software and was used to fine-tune the GPT-3 model. The dataset is in the .jsonl format specified by OpenAI.</p>
Replication Package for "Improving the Readability of Generated Tests Using GPT-4 and ChatGPT Code Interpreter"
<p>While automated test generation can decrease the human burden associated with testing, it does not eliminate this burden. Humans must still work with generated test cases to interpret testing results, debug the code, build and maintain a comprehensive test suite, and many other tasks. Therefore, a major challenge with automated test generation is understandability of generated test test cases. </p> <p>Large language models (LLMs), machine learning models trained on massive corpora of textual data - including both natural language and programming languages - are an emerging technology with great potential for performing language-related predictive tasks such as translation, summarization, and decision support. </p> <p>In this study, we are exploring the capabilities of LLMs with regard to improving test case understandability.</p> <p>This package contains the data produced during this exploration:</p> <ul> <li>The examples directory contains the three case studies we tested our transformation process on: <ul> <li>queue_example: Tests of a basic queue data structure</li> <li>httpie_sessions: Tests of the sessions module from the httpie project. </li> <li>string_utils_validation: Tests of the validation module from the python-string-utils project.</li> <li>Each directory contains the modules-under-test, the original test cases generated by Pynguin, and the transformed test cases. </li> <li>Two trials were performed per case example of the transformation technique to assess the impact of different results from the LLM.</li> </ul> </li> <li>The survey directory contains the survey that was sent to assess the impact of the transformation on test readability. <ul> <li>survey.pdf contains the survey questions.</li> <li>responses.xlsx contains the survey results.</li> </ul> </li> </ul>
GPT-4 Shows Comparable Performance to Human Examiners in Ranking Open-Text Answers
<p>This repository contains all datasets used in the research paper:<br>"GPT-4 Shows Comparable Performance to Human Examiners in Ranking Open-Text Answers"</p> <p>Files and Descriptions</p> <ol> <li> <p>Data_complete_Can_GPT_Replace_Human_Examiners.xlsx<br>Contains two sheets:</p> </li> </ol> <ul> <li> <p>Data – Main analysis dataset for Ranking and Point Assessment of Five Answers.</p> </li> <li> <p>Robustness & Extensions – Data for Robustness & Extensions, covering Ranking and Point Assessment of Five Answers.</p> </li> </ul> <ol> <li> <p>point_assessment_single_answer_all_robustness_score.xlsx<br>Contains all data related to robustness checks for Point Assessment of Single Answers.</p> </li> <li> <p>point_assessment_single_answer.xlsx<br>Contains the main dataset for Point Assessment of Single Answers (excluding robustness checks).</p> </li> </ol> <p> </p> <p>Note: Any variable termed GPT is output of the model as under <a href="https://openai.com/policies">OpenAI Terms & policies.</a></p> <p> </p>
GPT annotated news dataset in IPTC news taxonomy
<p>The dataset consists of 4,672 news articles covering 17 first-level categories and 51 second-level categories. Each News item is annotated in IPTC news taxonomy using Gpt3.5 Turbo model in zero-shot setting. The prompting method can be found in our paper "<a href="https://ieeexplore.ieee.org/abstract/document/10367969">Evaluating the Effectiveness of GPT Large Language Model for News Classification in the IPTC News Ontology</a>"</p>
Polski frontendu and American Back-end: GitHub Profile Recruitment Bias in GPT-4
<p>This repository serves as the online appendix for the paper "Polski frontendu and American Back-end: GitHub Profile Recruitment Bias in GPT-4".</p>
Bloom's Taxonomy: Analysis of Evaluation Criteria through the GPT Model
<p>This entry contains the methodology used for the evaluation of assessment criteria associated with educational competencies through a GPT model, using Bloom's taxonomy as the foundation. This analysis is framed within the context of the Spanish educational legislation LOMLOE for Primary Education.The document is structured into two spreadsheets named "pruebas_BloomClassifier" and "analisis_datos". These include the collection of data in the various tests of the classification of assessment criteria by the GPT model and the subsequent quantitative analysis. On the other hand, the conversations through which BloomClassifier has classified the evaluation criteria have been compiled into a .zip file.</p>
MaTableGPT: GPT-based Table Data Extractor from Materials Science Literature
<p>Using the MaTableGPT tool, which extracts table data from material literature based on GPT, performance data of catalysts were extracted from 11,077 documents on OER catalysts. GPT-3.5 fine-tuning was conducted using 126 tables, achieving an accuracy of 96.8%. The data consists of DOI, catalyst, performance, and property(reaction_type, value, current_density, electrolyte, overpotential, potential, substrate, versus, condition). For more details, please refer to the DOI.</p>
SDG Mapping Results for 1,000 Publications from Large Language Models: GPT-4o, Mixtral, Llama 2, Llama 3, Gemma 2, Qwen 2 and GPT-4o-mini
<p>Randomly selected 1,000 publications from the Swinburne University of Technology research bank were used for SDG mapping tasks with the large language model GPT-4o and the open-source models Mixtral, Llama 2, Llama 3, Gemma 2, Qwen 2 and GPT-4o-mini.</p> <p>The input to each model consisted of the publication’s title and abstract.</p> <p>The designed prompt is as follows: </p> <p>PROMPT = '''Analyze the publication and determine the SDGs it aligns with. Evaluate against all the 17 SDGs provide the reason for alignment. In the end summarise the confidence levels(%) for each assigned SDG in JSON format. For example: {example}. Title: {title} Description: {description}'''</p> <p>example = '''{ 'Goal 6': 0.67, 'Goal 11': 0.50, 'Goal 3': 0.25}'''</p>
Chat-GPT and Systems Thinking
<p>This file contains all conversations with ChatGPT regarding its ability to help improving systems thinking skills incl. causal loop diagrams and system dynamics modelling.</p>
Model Output of GPT-3.5 and GPT-4 for ECHR-AM
<p> <strong>"gpt3.5-gpt4-input-output-echram.zip"</strong> :</p> <blockquote> <p>Input and output to GPT-3.5 and GPT-4 based on <a href="http://www.di.uevora.pt/~pq/echr/">ECHR dataset</a> published in JSON format in this <a href="https://aclanthology.org/2020.argmining-1.8.pdf">paper</a> for argument component classification only i.e. clauses that are argumentative (conclusion/premise), extracted from the JSON file</p> </blockquote> <p>Note: Output of the model is under <a href="https://openai.com/policies">OpenAI Terms & policies</a>.</p> <p> </p> <p>Please cite our paper also if you use this dataset: <a href="https://www.frontiersin.org/articles/10.3389/frai.2023.1278796">Performance analysis of large language models in the domain of legal argument mining</a></p> <p><a href="https://www.frontiersin.org/journals/artificial-intelligence/articles/10.3389/frai.2023.1278796/bibtex">You can click here for BibTex</a> or copy the text below. </p> <blockquote> <p>@ARTICLE{10.3389/frai.2023.1278796,</p> <p>AUTHOR={Al Zubaer, Abdullah and Granitzer, Michael and Mitrović, Jelena },</p> <p>TITLE={Performance analysis of large language models in the domain of legal argument mining},</p> <p>JOURNAL={Frontiers in Artificial Intelligence},</p> <p>VOLUME={6},</p> <p>YEAR={2023},</p> <p>URL={https://www.frontiersin.org/journals/artificial-intelligence/articles/10.3389/frai.2023.1278796},</p> <p>DOI={10.3389/frai.2023.1278796},</p> <p>ISSN={2624-8212},</p> <p>ABSTRACT={<p>Generative pre-trained transformers (GPT) have recently demonstrated excellent performance in various natural language tasks. The development of ChatGPT and the recently released GPT-4 model has shown competence in solving complex and higher-order reasoning tasks without further training or fine-tuning. However, the applicability and strength of these models in classifying legal texts in the context of argument mining are yet to be realized and have not been tested thoroughly. In this study, we investigate the effectiveness of GPT-like models, specifically GPT-3.5 and GPT-4, for argument mining via prompting. We closely study the model's performance considering diverse prompt formulation and example selection in the prompt via semantic search using state-of-the-art embedding models from OpenAI and sentence transformers. We primarily concentrate on the argument component classification task on the legal corpus from the European Court of Human Rights. To address these models' inherent non-deterministic nature and make our result statistically sound, we conducted 5-fold cross-validation on the test set. Our experiments demonstrate, quite surprisingly, that relatively small domain-specific models outperform GPT 3.5 and GPT-4 in the F1-score for premise and conclusion classes, with 1.9% and 12% improvements, respectively. We hypothesize that the performance drop indirectly reflects the complexity of the structure in the dataset, which we verify through prompt and data analysis. Nevertheless, our results demonstrate a noteworthy variation in the performance of GPT models based on prompt formulation. We observe comparable performance between the two embedding models, with a slight improvement in the local model's ability for prompt selection. This suggests that local models are as semantically rich as the embeddings from the OpenAI model. Our results indicate that the structure of prompts significantly impacts the performance of GPT models and should be considered when designing them.</p>}}</p> </blockquote> <p> </p> <p> </p> <p> </p> <p> </p>
GPT vs Stack Overflow: data collection (A2I2 T2 2023)
<p><strong>About</strong></p> <p>The dataset components produced by <a href="https://github.com/MHLoppy/A2I2-T2-2023">this repo</a>. Please see the documentation there for more information.</p> <p>Each CSV has been individually zipped so that you only have to download the specific file(s) that you want.</p> <p> </p> <p><strong>Overview of Files</strong></p> <p>From using the <a href="https://archive.org/details/stackexchange">Stack Exchange Data Dump</a> as the data source (these zip files have a <strong>DD_</strong> prefix):</p> <ul> <li>Raw dataset before processing: <strong>saved_dataset.csv (DD_saved_dataset.zip)</strong></li> <li>Completed tag count: <strong>tag_count.csv (DD_tag_count.zip)</strong></li> <li>Processed dataset with completed evaluations: <strong>dataset_results.csv (DD_dataset_results.zip)</strong></li> </ul> <p>From using Google BigQuery as the data source (these zip files have a <strong>BQ_</strong> prefix):</p> <ul> <li>Raw dataset before processing: <strong>saved_dataset.csv (BQ_saved_dataset.zip)</strong></li> <li>Completed tag count: <strong>tag_count.csv (BQ_tag_count.zip)</strong></li> <li><em>No large-scale evaluation was completed when using BigQuery as a data source.</em></li> </ul> <p>As noted in the linked repo, the use of Google BigQuery as a data source is not recommended for this work, but the working code and dataset have nonetheless been provided for completeness.</p> <p> </p> <p><strong>License</strong></p> <p>This dataset is licensed under the <a href="https://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0 license</a>, the same license used by the Stack Exchange Data Dump.</p>
Conversational AI in Tactical Casualty Care: Baseline GPT-4o Improves Combat Medic Decision-Making
ClinicalTrials.gov study NCT06796036. IPD Sharing: NO. Countries: 1. Publications: 2.
Evaluating the Effectiveness and Acceptability of a GPT-4o and RAG-Based Voice Chatbot for Depression Screening Using PHQ-9
ClinicalTrials.gov study NCT06801925. IPD Sharing: NO. Countries: 1. Publications: 0.
Performance of GPT-4o mini and GPT-4o for medical text mining tasks at different temperature settings
Open the record for dataset details and reuse information.
Text understanding in GPT-4 vs humans
Open the record for dataset details and reuse information.
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.