Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
68
datasets available to search
ShareScore release 0.9.0
Dataset results
68 results for “Topic modelling”
Topic Labels of "Dynamic Topic Modelling for Exploring the Scientific Literature on Coronavirus: An Unsupervised Labelling Technique"
<p>These are the labels generated with the method proposed in the article <em>"Dynamic Topic Modelling for Exploring the Scientific Literature on Coronavirus: An Unsupervised Labelling Technique".</em> These labels are for the 100 and 200 DTM topic models, trained both with the whole corpus and with only the COVID-19 period data </p> <p> </p> <p>For the generation of these labels you can go to the original published work or to the linked Zenodo resource.</p>
Einführung ins Topic Model anhand des Beispiels Klimawandel
<p>Das Thema Klimawandel ist weder völlig neu, und genauso wenig sind es digitale Technologien und soziale Medien. Soziale Plattformen bieten eine Möglichkeit, sich über gesellschaftlich relevante Themen, unter anderem auch Klimawandel, auszutauschen, zu diskutieren und Anstöße für soziales und gesellschaftliches Umdenken zu erreichen (Uthirapathy & Sandanam, 2023).Um mit der großen Anzahl von Daten umgehen zu können, bieten Computational Social Sciences eine gute Möglichkeit damit umzugehen. Diese OER soll eine Einführung in das Topic Modelling am Beispiel des Subreddits "climate change" sein.</p> <p>Anhand eines ca. 15 minütigen Videos wird, besonders für Anfänger:innen, das Topic Model einfach, mit wenigen Fachbegriffen und ohne komplizierte Formeln, erklärt. Neben der theoretischen Einführung in das Thema, werden die für ein Topic Model typischen Durchführungsschritte anhand eines konkreten Beispiels - Subreddit "climate change" - erläutert. </p> <p>Eine schriftliche Fassung der Präsentation ist ebenso vorhanden und kann als Handreichung für Personen, die keine bis wenig Erfahrung mit dem Topic Model haben, verstanden werden. Die einzelnen Schritte der Durchführung des Topic Models werden detailliert aufgeschlüsselt und anhand des Beispiels Subreddit "climate change" dargelegt. Der gesamte Code, ohne Kommentierung, befindet sich am Ende des Verschriftlichung. </p>
SMART: Spatial transcriptomics deconvolution using marker-gene-assisted topic model
<p>Source code and simulated datasets used in manuscript "SMART: Spatial transcriptomics deconvolution using marker-gene-assisted topic model"</p>
Paper information in the topic of large language models
<p>This dataset supports the findings in the preprint 'Academic collaboration on large language model studies increases overall but varies across disciplines.' The study aims to explore the application of large language models (LLMs) in scientific disciplines and their implications for interdisciplinary collaboration.</p> <p>To build LLM paper group, we start with a broad search using general terms related to LLMs and popular models based on the MMLU benchmark spanning from October 2018 to September 2024. We apply this search to the title and abstract to avoid excessive noise in the dataset and then undergo a series of filtering steps<br>to enhance relevance and remove duplicates. The resulting dataset contains 59,293 papers.</p> <p>In addition to the paper group in the topic of LLMs, we establish two control groups. The first control group focuses on machine learning (ML) papers. We select ML as a control because it is a well-established field from which LLM emerged as a subfield. To construct this group, we collect a random sampling of 70,945 papers containing the phrase ''machine learning'' in either their title or abstract. To provide an even broader perspective beyond AI-related fields, we create a second control group consisting of a random sample of 73,110 papers from all other research categories---specifically, papers that belong neither to the ML nor LLM categories. </p> <p>The three files below contain the cleaned samples collected from OpenAlex, which are derived from the original files. </p> <ul> <li>LLM: llm-cleaned-samples.csv</li> <li>ML: ml-cleaned-samples.csv</li> <li>Non-LLM/ML: non-llm-cleaned-samples.csv</li> </ul> <p>The three zip files below contain author affiliation information (including departmental discipline) extracted by GPT-4o-mini to support the departmental analysis in the paper:</p> <ul> <li>LLM: llm-author-affiliations.zip</li> <li>ML: ml-author-affiliations.zip</li> <li>Non-LLM/ML: non-llm-author-affiliations.zip</li> </ul> <p>The three files below contain the paper information used to support all the analysis in our paper:</p> <ul> <li>LLM: llm-information-entropy.csv</li> <li>ML: ml-information-entropy.csv</li> <li>Non-LLM/ML: non-llm-information-entropy.csv</li> </ul> <p>If you have any additional questions, please feel free to contact <a rel="noreferrer">lingyaol@umich.edu or lydinh@usf.edu.</a></p>
Topic Model for English Wikipedia's Biographies with list of all 1.8M articles linked to Wikidata
<p>A Genism LDA Topic Model of English Wikipedia biographical articles with list of all 1.8M articles, and some associated Wikidata information</p> <p>The model has 150 Topics.</p> <p>This model was developed in the process of isolating a set of visual arts biographical articles, as described in "Clowns in the Visual Artists: Topic Modeling Wikipedia and Wikidata" in the Spring 2022 issue of <em>Art Documentation - </em><a href="https://doi.org/10.1086/719999">https://doi.org/10.1086/719999</a></p> <p>Because names, nationalities, and birthdays are so prominent in biographies, the stopwords list removed 170,000 names, surnames, city names, place names, countries, days, months and other time related words (https://github.com/mandiberg/Names-Surnames-and-Countries-for-Stopwords). We also directly removed each article subject’s given and surname, which were almost always the most frequently occurring words in any given article. Otherwise, the model just produced topics based on nationality, and common names and surnames.</p> <p><strong>Files:</strong></p> <p>all_enwiki_bios_from_wikidata.csv<br> The list of all Wikidata items for humans with an enwiki page (e.g biographical article) was extracted from Wikidata JSON dump; list includes gender, occupation, and nationality. This was joined with the converted plaintext from an English Wikipedia dump. This data was downloaded in March 2021.</p> <p>Wikipedia Biographies LDA Topic Model human readable summary.csv<br> A human readable file with the 150 topics ranked by count of articles per topic from the 1.8M corpus. The most popular topics have categorical descriptions of the occupations of each cluster. Some are marked as not an occupation cluster. </p> <p>BoW_corpus.mm*<br> model_lda_full_Sep2_150Tv2*<br> These six files comprise the topic model. The code to load them is present in the python files. </p> <p>dict_full_Aug-28-2021<br> processed_docs_full_Aug-28-2021.txt<br> processed_docs_1000_Aug-18-2021.txt<br> These are the dictionary and processed corpuses required to build and implement the model using this code. The corpus with the first 1000 items is meant to be used for testing, as the full one is quite large and takes a long time to complete. </p> <p>topic-model-wikipedia-sept2021.zip<br> The code and settings used for creating and implementing this model are included in this zip and are also available here: https://github.com/mandiberg/topic-model-wikipedia</p> <p>All-Wikipedia-Biographies-with-topic1.csv<br> All-Wikipedia-Biographies-with-topic1and2.csv<br> These are the list of 1.8M biographies matched to topics. The "topic1" file just includes the first topic, this is a slightly larger list. The "topic1and2" file is slightly smaller because about 2% articles do not match to a second topic.</p> <p>Analysis-for-Clowns-Visual-Arts.zip<br> These are the raw data and final data produced for the "Clowns in the Visual Artists." Please see the article for context.</p>
Deezer Podcast Dataset for Topic Modeling
<p>We release a new dataset consisting of podcast metadata (title and description) for 29 539 shows. This dataset can be used to reproduce the experiments from the article <em>Topic Modeling on Podcast Short-Text Metadata</em> accepted at the <a href="https://ecir2022.org">ECIR 2022</a> conference.</p> <p>More information about this data and how it should be used in experiments can be found in our paper and <a href="https://github.com/deezer/podcast-topic-modeling">GitHub repository</a>.</p> <p>Please cite our paper if you use the code or data.</p>
Exploratory Topic Modelling in Python Dataset - EHRI-3
<p>In the EHRI-3 project, we are investigating tools and methods that historical researchers and scholars can use to better understand, visualise, and interpret the material held by our partner archives. This dataset accompanies a tutorial exploring a technique called topic modelling in the context of a Holocaust-related historical collection.</p> <p>We were on the lookout for datasets that would be easily accessible and, for convenience, predominantly in English. One such dataset was the United States Holocaust Memorial Museum’s (USHMM) extensive collection of oral history testimonies, for which there are a considerable number of textual transcripts. The museum’s total collection consists of over 80,703 testimonies, 41,695 of which are available in English, with 2,894 of them listing a transcript.</p> <p>Since there is not yet a ready-to-download dataset that includes these transcripts, we had to construct our own. Using a web scraping tool, we managed to create a list of the links pointing to the metadata (including transcripts) of the testimonies that were of interest to us. After obtaining the transcript and other metadata of each of these testimonies, we were able to create our dataset and curate it to remove any unwanted entries. For example, we made sure to remove entries with restrictions on access or use. We also removed entries with transcripts that consisted only of some automatically generated headers and entries which turned out to be in languages other than English. The remaining 1,873 transcripts form the corpus of this tutorial — a small, but still decently sized dataset.</p> <p>The process that we followed to put together this dataset is detailed in the Jupyter Notebook accompanying this post, which can be found in this <a href="https://github.com/mdermentzi/ehri-topic-modelling-guide">Github repository</a>.</p> <p>In this Zenodo upload, the user can find two files, each of them containing a <a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_pickle.html">pickled pandas DataFrame</a> that was obtained at a different stage of the tutorial:</p> <p>"unrestricted_df.pkl" contains 1,946 entries of Oral Testimony transcripts and has five fields (RG_number, text, display_date, conditions_access, conditions_use)<br> "unrestricted_lemmatized_df.pkl" contains 1,873 entries of Oral Testimony transcripts and has six fields (RG_number, text, display_date, conditions_access, conditions_use, lemmas)</p> <p>Instructions on their intended use can be found in the accompanying Jupyter Notebook.</p> <p>Credits:</p> <p>The transcripts that form the corpus in this tutorial were obtained through the <a href="https://www.ushmm.org/">United States Holocaust Memorial Museum</a> (USHMM).</p>
Labeled data and models for COVID-19 vaccine related tweets with stance, location, and topics
<p>The dataset contains Tweet IDs along with the location and tweet timestamp. The tweets are labeled based on motivating/demotivating status, stance towards the COVID-19 vaccine, and topic in the tweet text. To comply with Twitter guidelines, we removed the tweet texts and author information. You can use Hydrator API to hydrate the tweets.</p> <p>The repository also contains the machine-learning models for topic modeling, de/motivation classifier, and stance detection from the tweets.</p>
A topic modelling application for violence against the women through text mining techniques
<p>This research was focused on a social problem that affects worldwide. Violence Against Women (VAW) and girls occurs in many forms with different levels of aggressiveness. There is a wide amount of data on the internet about real cases of VAW that can be gathered for further study. Many researchers have done studies on structured data sets which facilitates their manipulation and understanding. But in this work digital news (data source large non-structured text) was used.</p> <p>These are the data used for the development of the investigation.This file was extracted from our mongoDB database. The data set contains real news of violence against women, which were organized with their date, the title and the body of the news.</p>
Dataset Topic modelling De Gids
<p>Dataset accompanying 'Van Wissen, L. (2019). Topic Modelling “De Gids”: An Explorative Study into the Use of Topic Modelling on a Cultural Periodical. <em>Research Master Thesis, </em>Vrije Universiteit Amsterdam.' (included as pdf)</p> <p><strong>Usage</strong><br> See the 'experiment_1', 'experiment_2', and 'experiment_3' folders in the 'notebooks' folder. These correspond to the three experiments described in the thesis. </p> <p><strong>Corpus</strong><br> The corpus is not included in this repository because its licence does not allow for it, but part of it, if not all, is accessible through the website of the DBNL.</p>
Topic model of cerme
<p>Technical appendix for ESM paper. The list of topic distribution. The topic model visualization. The code that produced the model, for full transparency.</p>
OpenAlex Topic Classification v1 Model Artifacts and Training Data
<p>This is all data used to train the topic classification model and also the model artifacts to deploy the model. Please see the github repo for more information:</p> <p>https://github.com/ourresearch/openalex-topic-classification</p>
Topic modeling on articles about female librarians
<p>The dataset on topic modeling for articles about female librarians consists of a VOSviewer JSON file, Jupyter notebook files, and a search query.</p>
Topic Modeling Data
<p>This repository contains the data used to perform topic modeling on premodern texts; the article that discusses the application and the outcomes can be found here: https://doi.org/10.57813/20220623-153139-0</p>
Topic model of cerme
<p>Technical appendix for ESM paper. The list of topic distribution. The topic model visualization. The code that produced the model, for full transparency.</p>
Topic model of cerme
<p>Technical appendix for ESM paper. The list of topic distribution. The topic model visualization. The code that produced the model, for full transparency.</p>
Digital Twins: A Systematic Literature Review Based on Data Analysis and Topic Modeling
<p>The digital twin has recently become a popular topic in research related to manufacturing, such as Industry 4.0, the industrial internet of things, and cyber-physical systems. In addition, digital twins are the focus of several research areas: construction, urban management, digital transformation of the economy, medicine, virtual reality, software testing, and others. The concept is not yet fully defined, its scope seems unlimited, and the topic is relatively new; all this can present a barrier to research. The main goal of this paper is to develop a proper methodology for visualizing the digital-twin science landscape using modern bibliometric tools, text-mining and topic-modelling, based on machine learning models—Latent Dirichlet Allocation (LDA) and BERTopic (Bidirectional Encoder Representations from Transformers). The scope of the study includes 8693 publications on the topic selected from the Scopus database, published between January 1993 and September 2022. Keyword co-occurrence analysis and topic-modelling indicate that studies on digital twins are still in the early stage of development. At the same time, the core of the topic is growing, and some topic clusters are emerging. More than 100 topics can be identified; the most popular and fastest-growing topic is ‘digital twins of industrial robots, production lines and objects.’ Further efforts are needed to verify the proposed methodology, which can be achieved by analyzing other research fields.</p>
Topic Modeling for Multi-label Research Articles
<p>The abstract and title for a set of research articles, and their assigned topics. The research article abstracts and titles are sourced from the following 6 topics:</p> <ol> <li> <p>Computer Science</p> </li> <li> <p>Physics</p> </li> <li> <p>Mathematics</p> </li> <li> <p>Statistics</p> </li> <li> <p>Quantitative Biology</p> </li> <li> <p>Quantitative Finance</p> </li> </ol> <p>Note that a research article can possibly have more than 1 topic. For more details check out the paper of :</p> <p>"Evaluation of SVM Transformations for Multi-Label Research Article Classification"</p>
Dataset Similarity Artikel Topic Modeling
<p>Dataset Similarity Artikel Topic Modeling.</p> <p>Data artikel diperoleh dari Google Scholar menggunakan Publish or Perish. Query yang digunakan "Topic modeling" dengan batas waktu publikasi sejak 2019. Keyword diisi dengan Mendeley auto update dan sisanya dicari secara manual. Data berupa buku dan data tanpa keyword dibuang. Data dipasangkan secara kombinatorik dari 50 data menjadi 1225. Labelling manual.</p>
Topic Modeling der historischen Fachzeitschrift »Francia« 1973–2022
<p>Das Projekt »Topic Modeling der historischen Fachzeitschrift ›Francia‹ 1973–2022« untersucht, welche Trends und Inhalte sich in der Zeitschrift des Deutschen Historischen Instituts Paris (DHIP) in den 49 Jahrgängen seit ihrer Gründung durch ein Topic Modeling beobachten lassen.</p> <p>Die grundlegenden Ergebnisse der Studie sind veröffentlicht unter: Mareike König, Eike Löhden, Die »Francia« anders lesen. Was Topic Modeling über Schwerpunkte und Trends der Fachzeitschrift verrät, in: Francia 50 (2023), S. 13–54. Der Beitrag ist mit eine Moving Wall von einem Jahr ab Herbst 2024 online im Volltext zugänglich unter: <a href="https://journals.ub.uni-heidelberg.de/index.php/fr/https://journals.ub.uni-heidelberg.de/index.php/fr/.">https://journals.ub.uni-heidelberg.de/index.php/fr/https://journals.ub.uni-heidelberg.de/index.php/fr/.</a> </p> <p>Der vorliegende Datenreport enthält zur o.g. Publikation ergänzende Anmerkungen zu den Bereichen Korpuserstellung, -bereinigung und -auswertung der »Francia«. Er enthält außerdem eine Auswahl an weiterführenden Zahlen und Visualisierungen, auf die im o.g. publizierten Aufsatz Bezug genommen wird oder die dort als Visualisierung abgebildet sind.</p> <p>Die Zeitschrift ist im Volltext zugänglich unter: <a href="https://journals.ub.uni-heidelberg.de/index.php/fr/.">https://journals.ub.uni-heidelberg.de/index.php/fr/.</a> Die txt-Dateien der Zeitschrift sind daher nicht erneut Teil dieses Eintrags.</p> <p>Der für die Studie verwendete Code (Python) ist bei GitHub veröffentlicht: <a href="https://github.com/dhiparis/Francia_TM">https://github.com/dhiparis/Francia_TM</a>.</p> <p>Dieser Eintrag enthält die folgenden Dateien:</p> <p>-Datenreport mit zur Aufsatzpublikation ergänzenden Statistiken und Erläuterungen zum Vorgehen und zu den Ergebnissen (00_koenig_loehden_francia_datenreport_v1)</p> <p>-Excel-Datei mit der Zuordnung der Autorinnen und Autoren nach Geschlecht (01_authors_francia)</p> <p>-Excel-Datei mit der Zuordnung der Aufsätze nach Epoche und Rubrik für das deutschsprachige Korpus (02a_francia_dt_metadaten_epoche_rubrik). )</p> <p>-Excel-Datei mit der Zuordnung der Aufsätze nach Epoche und Rubrik für das französischsprachige Korpus (02b_francia_fr_metadaten_epoche_rubrik)</p> <p>-Excel-Datei mit der Zuordnung der Aufsätze nach Epoche und Rubrik für das englischsprachige Korpus (02c_francia_eng_metadaten_epoche_rubrik)</p> <p>-Excel-Datei mit den Ergebnissen aus Mallet für das deutschsprachige Korpus (Topics, Prozentanteil pro Jahrgang, Prozentanteil pro Aufsatz, Treemap der Topics und ihrer Anteile), 03a_francia_topic_model_30_dt_korpus</p> <p>-Excel-Datei mit den Ergebnissen aus Mallet für das französischsprachige Korpus (Topics, Prozentanteil pro Jahrgang, Prozentanteil pro Aufsatz, Treemap der Topics und ihrer Anteile), 03b_francia_topic_model_30_frz_korpus</p> <p>-die Ergebnisdateien des Topic Modeling in Mallet für das deutschsprachige Korpus (04a_francia_dt.txt, 04a_francia.comp_dt.txt, 04a_francia.keys_dt.txt, 04a_francia.diag_dt.xml)</p> <p>-die Ergebnisdateien des Topic Modeling in Mallet für das französischsprachige Korpus (04b_francia_fr.txt, 04b_francia.comp_fr.txt, 04b_francia.keys_fr.txt, 04b_francia.diag_fr.xml)</p> <p>-Excel-Datei zur Dokument-Topic-Korrelation für das deutschsprachige Korpus, 05a_francia_document_topic_correlation_dt_korpus</p> <p>--Excel-Datei zur Dokument-Topic-Korrelation für das französischsprachige Korpus, 05b_francia_document_topic_correlation_fr_korpus</p> <p>-die in Mallet verwendete Stoppwortliste (06_stopwords_francia.txt)</p> <p>-png-Dateien der Wortwolken für die im Aufsatz ausgewerteten 24 deutschsprachigen und die 23 französischsprachigen Topics</p> <p>-png-Dateien der Verlaufskurven für die im Aufsatz ausgewerteten 24 deutschsprachigen und die 23 französischsprachigen Topics</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.