Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
167
datasets available to search
ShareScore release 0.7.1
Dataset results
167 results for “Large Language Model”
SDG Mapping Results for 1,000 Publications from Large Language Models: GPT-4o, Mixtral, Llama 2, Llama 3, Gemma 2, Qwen 2 and GPT-4o-mini
<p>Randomly selected 1,000 publications from the Swinburne University of Technology research bank were used for SDG mapping tasks with the large language model GPT-4o and the open-source models Mixtral, Llama 2, Llama 3, Gemma 2, Qwen 2 and GPT-4o-mini.</p> <p>The input to each model consisted of the publication’s title and abstract.</p> <p>The designed prompt is as follows: </p> <p>PROMPT = '''Analyze the publication and determine the SDGs it aligns with. Evaluate against all the 17 SDGs provide the reason for alignment. In the end summarise the confidence levels(%) for each assigned SDG in JSON format. For example: {example}. Title: {title} Description: {description}'''</p> <p>example = '''{ 'Goal 6': 0.67, 'Goal 11': 0.50, 'Goal 3': 0.25}'''</p>
Supplementary Material for "Investigating Software Development Teams Members' Perceptions of Data Privacy in the Use of Large Language Models (LLMs)"
<h3>ABSTRACT<strong>: </strong></h3> <p><strong>Context</strong>: Large Language Models (LLMs) have revolutionized natural language generation and understanding. However, they raise significant data privacy concerns, especially when sensitive data is processed and stored by third parties. <br><strong>Goal</strong>: This paper investigates the perception of software development teams members regarding data privacy when using LLMs in their professional activities. Additionally, we examine the challenges faced and the practices adopted by these practitioners. <br><strong>Method</strong>: We conducted a survey with 78 ICT practitioners from five regions of the country. <br><strong>Results</strong>: Software development teams members have basic knowledge about data privacy and LGPD, but most have never received formal training on LLMs and possess only basic knowledge about them. Their main concerns include the leakage of sensitive data and the misuse of personal data. To mitigate risks, they avoid using sensitive data and implement anonymization techniques. The primary challenges practitioners face are ensuring transparency in the use of LLMs and minimizing data collection. Software development teams members consider current legislation inadequate for protecting data privacy in the context of LLM use. <br><strong>Conclusions</strong>: The results reveal a need to improve knowledge and practices related to data privacy in the context of LLM use. According to software development teams members, organizations need to invest in training, develop new tools, and adopt more robust policies to protect user data privacy. They advocate for a multifaceted approach that combines education, technology, and regulation to ensure the safe and responsible use of LLMs.</p>
LLMs4OL 2024 Datasets: Toward Ontology Learning with Large Language Models
<p>Ontology learning (OL) from unstructured data has evolved significantly, with recent advancements integrating large language models (LLMs) to enhance various aspects of the process. The LLMs4OL 2024 datasets, were developed to benchmark and advance research in OL using LLMs. This dataset as a key component of the LLMs4OL Challenge, targets three primary OL tasks: Term Typing, Taxonomy Discovery, and Non-Taxonomic Relation Extraction. It encompasses seven domains, i.e. lexosemantics and biological functions, offering a comprehensive resource for evaluating LLM-based OL approaches Each task within the dataset is carefully crafted to facilitate both Few-Shot (FS) and Zero-Shot (ZS) evaluation scenarios, allowing for robust assessment of model performance across different knowledge domains to address a critical gap in the field by offering standardized benchmarks for fair comparison for evaluating LLM applications in OL. </p>
Geoparsing with Large Language Models: Leveraging the linguistic capabilities of generative AI to improve geographic information extraction
<h2>Geoparsing with Large Language Models</h2> <p>The .zip file included in this repository contains all the code and data required to reproduce the results from our paper. Note, however, that in order to run the OpenAI models, users will required an OpenAI API key and sufficient API credits.</p> <div> <h3>Data</h3> <p>The data used for the paper are in the <code>datasetst</code> and <code>results</code> folders.</p> <ul> <li> <p>**Datasets: **This contains the XML files (LGL and Geovirus) and Json files (News2024) used to benchmark the models. It also contains all the data used to fine-tune the gpt-3.5 model, the prompt templates sent to the LLMs, and other data used for mapping and data creation.</p> </li> <li> <p>**Results: **This contains the results for the models on the three datastes. The folder is separated by dataset, with a single <code>.csv</code> file giving the results for each model on each dataset separately. The <code>.csv</code> file is structured so that each row contains either a predicted toponym and an associated true toponym (along with assigned spatial coordinates), if the model correctly identified a toponym; otherwise the true toponym columns are empty for false positives and the predicted columns are empty for false negatives.</p> </li> </ul> <h3>Code</h3> <p>The code is split into two seperate folders <code>gpt_geoparser</code> and <code>notebooks</code>.</p> <ul> <li>**GPT_Geoparser: **this contains the classes and methods used process the XML and JSON articles (<code>data.py</code>), interact with the Nominatim API for geocoding (<code>gazetteer.py</code>), interact with the OpenAI API (<code>gpt_handler.py</code>), process the outputs from the GPT models (<code>geoparser.py</code>) and analyse the results (<code>analysis.py</code>).</li> <li><strong>Notebooks</strong>: This series of notebooks can be used to reproduce the results given in the paper. The file names a reasonably descriptive of what they do within the context of the paper.</li> </ul> <h3>Code/software</h3> <h3>Requirements</h3> <ul> <li>Numpy</li> <li>Pandas</li> <li>Geopy</li> <li>Scitkit-learn</li> <li>lxml</li> <li>openai</li> <li>matplotlib</li> <li>Contextily</li> <li>Shapely</li> <li>Geopandas</li> <li>tqdm</li> <li>huggingface_hub</li> <li>Gnews</li> </ul> <h3>Access information</h3> <p>Other publicly accessible locations of the data:</p> <ul> <li>The LGL and GeoVirus datasets can also be obtained <a href="https://github.com/milangritta/Pragmatic-Guide-to-Geoparsing-Evaluation" target="_blank" rel="noopener">here<span> (opens in new window)</span></a>.</li> </ul> <h3>Abstract</h3> <div> <p>Geoparsing- the process of associating textual data with geographic locations - is a key challenge in natural language processing. The often ambiguous and complex nature of geospatial language make geoparsing a difficult task, requiring sophisticated language modelling techniques. Recent developments in Large Language Models (LLMs) have demonstrated their impressive capability in natural language modelling, suggesting suitability to a wide range of complex linguistic tasks. In this paper, we evaluate the performance of four LLMs - GPT-3.5, GPT-4o, Llama-3.1-8b and Gemma-2-9b - in geographic information extraction by testing them on three geoparsing benchmark datasets: GeoVirus, LGL, and a novel dataset, News2024, composed of geotagged news articles published outside the models' training window. We demonstrate that, through techniques such as fine-tuning and retrieval-augmented generation, LLMs significantly outperform existing geoparsing models. The best performing models achieve a toponym extraction F1 score of 0.985 and toponym resolution accuracy within 161 km of 0.921. Additionally, we show that the spatial information encoded within the embedding space of these models may explain their strong performance in geographic information extraction. Finally, we discuss the spatial biases inherent in the models' predictions and emphasize the need for caution when applying these techniques in certain contexts.</p> </div> <h3>Methods</h3> <div> <p>This contains the data and codes required to reproduce the results from our paper. The LGL and GeoVirus datasets are pre-existing datasets, with references given in the manuscript. The News2024 dataset was constructed specifically for the paper. </p> <p>To construct the News2024 dataset, we first created a list of 50 cities from around the world which have population greater than 1000000. We then used the GNews python package <a href="https://pypi.org/project/gnews/" target="_blank" rel="noopener">https://pypi.org/project/gnews/<span> (opens in new window)</span></a> to find a news article for each location, published between 2024-05-01 and 2024-06-30 (inclusive). Of these articles, 47 were found to contain toponyms, with the three rejected articles referring to businesses which share a name with a city, and which did not otherwise mention any place names.</p> <p>We used a semi autonmous approach to geotagging the articles. The articles were first processed using a Distil-BERT model, fine tuned for named entity recognicion. This provided a first estimate of the toponyms within the text. A human reviewer then read the articles, and accepted or rejected the machine tags, and added any tags missing from the machine tagging process. We then used OpenStreetMap to obtain geographic coordinates for the location, and to identify the toponym type (e.g. city, town, village, river etc). We also flagged if the toponym was acting as a geo-political entity, as these were reomved from the analysis process. In total, 534 toponyms were identified in the 47 news articles. </p> </div> </div>
Advancing Large Language Models through Story Energy, Universal Harmony Energy, and SA-UUH-UPP
<p><span>In this groundbreaking exploration of AI, we unveil how Story Energy, Universal Harmony Energy, and the SA-UUH-UPP framework could revolutionize large language models (LLMs). Discover how these advanced concepts push AI beyond current boundaries, enabling deeper contextual understanding, energy-efficient models, and steps toward self-awareness. Whether you’re an AI researcher, developer, or enthusiast, this video provides insights that could redefine the future of AI. Watch now to dive into the next frontier of artificial intelligence!</span></p>
Official Code and Dataset of Table Tennis Coaching System Based on a Multimodal Large Language Model with Knowledge Base
<p>Official Code and Dataset of Table Tennis Coaching System Based on a Multimodal Large Language Model with Knowledge Base</p>
ThoughtSource: A central hub for large language model reasoning data (dataset snapshot)
<p><strong>ThoughtSource is a meta-dataset and software library for chain-of-thought reasoning in large language models (LLMs). </strong></p> <p><strong>This repository contains a snapshot of the openly available ThoughtSource datasets.</strong></p>
Results and log of LLM-KG-Bench runs described in article "Benchmarking the Abilities of Large Language Models for RDF Knowledge Graph Creation and Comprehension: How Well Do LLMs Speak Turtle?", Frey et al. 2023
<p>Results and log of LLM-KG-Bench runs described in article ""Benchmarking the Abilities of Large Language Models for RDF Knowledge Graph Creation and Comprehension: How Well Do LLMs Speak Turtle?", Frey et al. 2023, to appear in proceedings for workshop DL4KG@ISWC 2023.</p> <p>For data on task FactExtractStatic please contact authors.</p>
Data for paper on inductive reasoning in large language models
<p>This dataset contains all data that was used for:</p> <p>Han, S. J., Ransom, K. J., Perfors, A. & Kemp, C. (2023). Inductive reasoning in humans and large language models. <em>Cognitive Systems Research</em>.</p> <p>The code for this project can be found <a href="https://github.com/S-J-HAN/InductiveReasoningInLargeLanguageModels">here</a>.</p> <p> </p>
Automating pharmacovigilance evidence generation: Using large language models to produce context-aware SQL
Open the record for dataset details and reuse information.
Human review for post-training improvement of low-resource language performance in large language models
Open the record for dataset details and reuse information.
Incentivizing news consumption on social media platforms using large language models and realistic bot accounts
Open the record for dataset details and reuse information.
Data and code for "Large language models identify causal genes in complex trait GWAS"
<p><span>This file contains the data and scripts for the preprint "Large language models identify causal genes in complex trait GWAS"</span></p>
Supplementary Data for "Exploring the Integration of Large Language Models in Industrial Test Maintenance Processes"
<p>This package contains supplementary data not directly included in the paper, including per-commit results for each prototype and the prompts used in the proof-of-concept implementations.</p>
Characterizing Code Clones from Large Language Models Dataset and Scripts
<p>characterizing_code_clones_data.zip: <br><br>This dataset contains a collection of code snippets generated by Large Language Models (LLMs) such as GPT-3.5 and GPT-4 in response to specific programming prompts derived from LeetCode. Each sub-directory within the dataset corresponds to a particular LLM version and contains code snippets, preprocessed data, and SLACC input files. </p><p>characterizing_code_clones_project.zip: </p><p>This zipped directory encompasses the core scripts and results used in the "Characterizing Code Clones of LLMs" research. It features the Python script <strong>collect_samples.py</strong> for collecting LLM-generated code snippets, as well as a suite of scripts in the <strong>slacc_scripts</strong> sub-directory for processing and analyzing the data using SLACC. The directory also includes the results of the LeetCode test suites, providing insights into the correctness and efficiency of the code generated by GPT-3.5 and GPT-4. </p>
Dataset for "In-IDE Human-AI Experience in the Era of Large Language Models; A Literature Review" paper
Open the record for dataset details and reuse information.
Artifact of Program Selection from Large Language Models
<p>Artifact of <em>Program Selection from Large Language Models</em>, including documentation, source code, and experimental data.</p>
Replication Package of the Paper: "Using Large Language Models for Commit Message Generation: A Preliminary Study"
<p>This replication package contains the evaluation data and script files used in the paper "Using Large Language Models for Commit Message Generation: A Preliminary Study". We provide below a brief description of each folder:</p><ul><li><strong>experiment_data/human_evaluation</strong>: Human evaluation results from two participants.</li><li><strong>experiment_data/msg</strong>: Generated commit messages of each method (baselines & LLMs) and human-written commit messages.</li><li><strong>script</strong>: Evaluation metric script, parallel inference script (for using OpenAI API).</li></ul>
Graph Neural Network vs. Large Language Model: A Comparative Analysis for Bug Report Priority and Severity Prediction
Open the record for dataset details and reuse information.
Resources for the paper "Teaching a Multilingual Large Language Model to Understand Multilingual Speech via Multi-Instructional Training"
Open the record for dataset details and reuse information.
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.