Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
239
datasets available to search
ShareScore release 0.9.0
Dataset results
239 results for “Data-driven”
Models for "A data-driven approach to studying changing vocabularies in historical newspaper collections"
<p>NOTE: This is a badly rendered version of the README within the archive.</p> <p><strong>A data-driven approach to studying changing vocabularies in historical newspaper collections</strong></p> <p>Simon Hengchen,* Ruben Ros,** Jani Marjanen,*** Mikko Tolonen***</p> <p>*<a href="https://spraakbanken.gu.se/en/about/staff/simon">Språkbanken Text</a>, University of Gothenburg, Sweden and <a href="https://iguanodon.ai">iguanodon.ai</a>, Belgium: firstname.lastname@gu.se<br> **<a href="https://www.c2dh.uni.lu/people/ruben-ros">Centre for Contemporary and Digital History (C2DH)</a>, University of Luxembourg: firstname.lastname@uni.lu<br> ***<a href="https://www.helsinki.fi/en/researchgroups/computational-history">COMHIS</a>, University of Helsinki: <a href="mailto:firstname.lastname@helsinki.fi">firstname.lastname@helsinki.fi</a>;</p> <p>These are the supplementary materials for the DH2019 paper <em>A data-driven approach to the changing vocabulary of the ‘nation’ in English, Dutch, Swedish and Finnish newspapers, 1750-1950</em>, as well as the 2021 Digital Scholarship in the Humanities publication available in OpenAccess: <a href="https://academic.oup.com/dsh/article/36/Supplement_2/ii109/6421793">https://academic.oup.com/dsh/article/36/Supplement_2/ii109/6421793</a>. If you end up using whole or parts of this resource, please use the following citation(s):</p> <ul> <li>Hengchen, S., Ros, R., and Marjanen, J. (2019). A data-driven approach to the changing vocabulary of the 'nation' in English, Dutch, Swedish and Finnish newspapers, 1750-1950. In <em>Proceedings of the Digital Humanities (DH) conference 2019, Utrecht, The Netherlands</em></li> </ul> <p>and/or:</p> <ul> <li>Hengchen, S., Ros, R., Marjanen, J. and Tolonen, M., 2021. A data-driven approach to studying changing vocabularies in historical newspaper collections. Digital Scholarship in the Humanities, 36(Supplement_2), pp.ii109-ii126.</li> </ul> <p>or alternatively use one of the following <code>bib</code>s:</p> <pre><code>@inproceedings{hengchen2019nation, title="A data-driven approach to the changing vocabulary of the 'nation' in {E}nglish, {D}utch, {S}wedish and {F}innish newspapers, 1750-1950.", author={Hengchen, Simon and Ros, Ruben and Marjanen, Jani}, year={2019}, address = "Utrecht, The Netherlands", booktitle={Proceedings of the Digital Humanities (DH) conference 2019} }</code></pre> <pre><code>@article{hengchen2021data, title={A data-driven approach to studying changing vocabularies in historical newspaper collections}, author={Hengchen, Simon and Ros, Ruben and Marjanen, Jani and Tolonen, Mikko}, journal={Digital Scholarship in the Humanities}, volume={36}, number={Supplement\_2}, pages={ii109--ii126}, year={2021}, publisher={Oxford University Press} }</code></pre> <p> </p> <p>Files</p> <p>This archive contains two folders -- one per diachronic representation method -- as well as this README. The folders each contain four folders, which contain the models for their respective languages. As can be inferred from the small datasize, most of the earlier models are not reliable and should not be used, but are still made available. This work is licensed under a <a href="http://creativecommons.org/licenses/by-sa/4.0/">Creative Commons Attribution-ShareAlike 4.0 International License</a>.</p> <p><strong>Source material</strong></p> <p>Finnish:</p> <p>The models were created with data from the Finnish Sub-corpus of the Newspaper and Periodical Corpus of the National Library of Finland (National Library of Finland, 2011). We used everything in the corpus.</p> <p>Filesizes:</p> <pre><code>[simon@taito-login3 SGNS]$ du -h fi* 12M fi_1820_SGNS_corpus_file.gensim 89M fi_1840_SGNS_corpus_file.gensim 797M fi_1860_SGNS_corpus_file.gensim 7.0G fi_1880_SGNS_corpus_file.gensim 22G fi_1900_SGNS_corpus_file.gensim</code></pre> <p>Swedish:</p> <p>The models were created with data from the Kubhist 2 corpus (Språkbanken) -- more precisely, the data dumps available at <a href="https://spraakbanken.gu.se/lb/resurser/meningsmangder/">https://spraakbanken.gu.se</a>. After a manual evaluation of Swedish embeddings trained without pre-processing seemed to show that the embeddings were of low quality, we retrained models, only keeping sentences that were at least 10 tokens long and were constituted of at least 50% of lemmas as per the KORP processing pipeline (Borin et al, 2012).</p> <p>Filesizes:</p> <pre><code>[simon@taito-login3 SGNS]$ du -h sv* 1.6M sv_1740_SGNS_corpus_file.gensim 44M sv_1760_SGNS_corpus_file.gensim 124M sv_1780_SGNS_corpus_file.gensim 228M sv_1800_SGNS_corpus_file.gensim 678M sv_1820_SGNS_corpus_file.gensim 1.6G sv_1840_SGNS_corpus_file.gensim 4.5G sv_1860_SGNS_corpus_file.gensim 6.5G sv_1880_SGNS_corpus_file.gensim 113M sv_1900_SGNS_corpus_file.gensim</code></pre> <p>Dutch:</p> <p>The models were created with data from the Delpher newspaper archive (Royal Dutch Library, 2017), through data dumps for newspapers until and including 1876, and through API hits for articles from 1877 to 1899 (included).</p> <ul> <li>For anything pre-1877 we discarded full texts that had, in the metadata, anything else than exclusively <code>nl</code> or <code>NL</code> as a language tag.</li> <li>For the full texts between 1877 and 1899: we queried the API for all items in the “artikel” category that contained the determiner <code>de</code>.</li> </ul> <p>Our assumption was that most articles should contain <code>de</code> at least once, and those that didn't were too short to be deemed interesting. A subsequent study showed that was not exactly the case, but we were reassured by the fact that left-out articles were probably "shipping or financial reports" (thanks go to Melvin Wevers). We also did not include the colonial newspapers for our embeddings. This is motivated by our research questions. A list of removed newspapers is available on request.</p> <p>Filesizes:</p> <pre><code>[simon@taito-login3 SGNS]$ du -h nl* 6.8M nl_1620_SGNS_corpus_file.gensim 7.9M nl_1640_SGNS_corpus_file.gensim 43M nl_1660_SGNS_corpus_file.gensim 78M nl_1680_SGNS_corpus_file.gensim 138M nl_1700_SGNS_corpus_file.gensim 243M nl_1720_SGNS_corpus_file.gensim 287M nl_1740_SGNS_corpus_file.gensim 431M nl_1760_SGNS_corpus_file.gensim 825M nl_1780_SGNS_corpus_file.gensim 1.2G nl_1800_SGNS_corpus_file.gensim 1.8G nl_1820_SGNS_corpus_file.gensim 3.1G nl_1840_SGNS_corpus_file.gensim 5.2G nl_1860_SGNS_corpus_file.gensim 13G nl_1880_SGNS_corpus_file.gensim</code></pre> <p>English:</p> <p>The models were created with data from the British Library Newspapers collection (<a href="https://www.gale.com/intl/primary-sources/british-library-newspapers%5D">link</a>), the Nichols collection (<a href="https://www.gale.com/intl/c/17th-and-18th-century-burney-newspapers-collection">link</a>), and the Burney collection (<a href="https://www.gale.com/intl/c/17th-and-18th-century-nichols-newspapers-collection">link</a>). We used everything in the corpora. For English, only SGNS_ALIGN models are available. We thank Gale Cengage for their help with this project.</p> <p>Filesizes:</p> <pre><code>[simon@taito-login3 SGNS]$ du -h en* 4.3M en_1620_SGNS_corpus_file.gensim 11M en_1640_SGNS_corpus_file.gensim 11M en_1660_SGNS_corpus_file.gensim 106M en_1680_SGNS_corpus_file.gensim 409M en_1700_SGNS_corpus_file.gensim 1.7G en_1720_SGNS_corpus_file.gensim 834M en_1740_SGNS_corpus_file.gensim 2.4G en_1760_SGNS_corpus_file.gensim 5.3G en_1780_SGNS_corpus_file.gensim 5.5G en_1800_SGNS_corpus_file.gensim 15G en_1820_SGNS_corpus_file.gensim 42G en_1840_SGNS_corpus_file.gensim 65G en_1860_SGNS_corpus_file.gensim 88G en_1880_SGNS_corpus_file.gensim 26G en_1900_SGNS_corpus_file.gensim 21G en_1920_SGNS_corpus_file.gensim 6.3G en_1940_SGNS_corpus_file.gensim</code></pre> <p><strong>Word embeddings</strong></p> <p>For every language, we train diachronic embeddings as follows. We divide the data in 20-year time bins. We train SGNS_UPDATE and SGNS_ALIGN models. Current research on German (Schlechtweg et al, 2019) and English (Shoemark et al, 2019) indicates you should use the SGNS_ALIGN models. <strong>For EN, FI, NL, no tokens (including punctuation) were removed nor altered, aside from lowercasing</strong>. For SV, see above. Parameters are as follows: SGNS architecture (Mikolov et al 2013), window size of 5, frequency threshold of 100, 5 epochs, 300 dimensions (or 100 for EN).</p> <ul> <li>For SGNS_UPDATE: We first train a model for the first time bin <code>t</code>. To train the model for <code>t+1</code>, we use the <code>t</code> model to initialise the vectors for <code>t+1</code>, set the learning rate to correspond to the end learning rate of <code>t</code>, and continue training. This approach, closely following Kim et al (2014), has the advantage of avoiding the need for post-training vector space alignment.</li> </ul> <p>The Python snippet below, which makes use of gensim (Rehurek and Sojka, 2010), illustrates the approach. Special thanks go to Sara Budts.</p> <pre><code>## dict_files[key] is a dictionary with double decades as keys and a corresponding LineSentence object as value: https://radimrehurek.com/gensim/models/word2vec.html#gensim.models.word2vec.LineSentence count = 0 for key in sorted(list(dict_files.keys())): if count == 0: ## This is the first model. model = gensim.models.Word2Vec(corpus_file=dict_files[key], min_count=100, sg=1 ,size=300, workers=64, seed=1830, iter=5) model.save(os.path.join(data_path_final,"KIM",lang+"_"+str(timebin)+".w2v")) print("Model saved, on to the next\n") count += 1 if count > 0: ## this is for the subsequent models. print("model for double decade starting in",str(key)) model = gensim.models.Word2Vec.load(os.path.join(data_path_final,"KIM",lang+"_"+str(timebin-20)+".w2v")) print("previous model loaded") model.build_vocab(corpus_file=dict_files[key], update=True) model.train(corpus_file=dict_files[key], total_words = model.corpus_count, total_examples = model.corpus_count, start_alpha = model.alpha, end_alpha = model.min_alpha, epochs=model.epochs) model.save(os.path.join(data_path_final,"KIM",lang+"_"+str(timebin)+".w2v")) </code></pre> <ul> <li>For SGNS_ALIGN: We independently train models for all time bins. The models in this repository are <em>NOT</em> aligned, leaving you the choice of how to align them. For example, <a href="https://gist.github.com/quadrismegistus/09a93e219a6ffc4f216fb85235535faf">here</a> is a link to code by Ryan Heuser to do just that. Models were trained with the <code>count == 0</code> scenario in the snippet above.</li> </ul> <p><strong>Acknowledgments</strong></p> <p>This work has been supported by the European Union's Horizon 2020 research and innovation programme under grant 770299 <a href="https://www.newseye.eu/">NewsEye</a>. Specials thanks go to the data providers/collection-holding institutions: the Finnish Language Bank, the Swedish Language Bank, the Royal Dutch Library, and Gale Cengage.</p> <p>The authors would like to thank the following persons and group, listed alphabetically: Antoine Doucet, Antti Kanner, Axel-Jean Caurant, Dominik Schlechtweg, Eetu Mäkelä, Elaine Zosa, Estelle Bunout, Haim Dubossarsky, Joris van Eijnatten, Krister Lindén, Lars Borin, Lidia Pivovarova, Melvin Wevers, Nina Tahmasebi, Sara Budts, Senka Drobac, Tanja Säily, the COMHIS group, and Steven Claeyssens. Computational resources were provided by CSC – IT Center for Science Ltd.</p> <p><strong>References</strong></p> <p>Borin, L., Forsberg, M., Roxendal, J. (2012). Korp-the corpus infrastructure of Spräkbanken,in: LREC. pp. 474–478.</p> <p>Kim, Y., Chiu, Y.I., Hanaki, K., Hegde, D. and Petrov, S. (2014). Temporal Analysis of Language through Neural Language Models. <em>ACL 2014</em>, p.61.</p> <p>Mikolov, T., Chen, K., Corrado, G. and Dean, J. (2013). Efficient estimation of word representations in vector space. <em>arXiv preprint arXiv:1301.3781</em>.</p> <p>National Library of Finland (2011). <em>The Finnish Sub-corpus of the Newspaper and Periodical Corpus of the National Library of Finland, Kielipankki Version</em> [text corpus]. Kielipankki. Retrieved from <a href="http://urn.fi/urn:nbn:fi:lb-2016050302">http://urn.fi/urn:nbn:fi:lb-2016050302</a>.</p> <p>Rehurek, R. and Sojka, P. (2010). Software framework for topic modelling with large corpora. In <em>Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks</em>.</p> <p>Royal Dutch Library (2017). <em>Delpher open krantenarchief (1.0)</em>. Den Haag, 2017.</p> <p>Schlechtweg D., Hätty A, del Tredici M., and Schulte im Walde S. (2019). A Wind of Change: Detecting and Evaluating Lexical Semantic Change across Times and Domains. In <em>Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)</em>, Florence, Italy. ACL.</p> <p>Shoemark, P., Liza, F.F., Nguyen, D., Hale, S. and McGillivray, B. (2019). Room to Glo: A Systematic Comparison of Semantic Change Detection Approaches with Word Embeddings. In <em>Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (pp. 66-76)</em>, Hong Kong.</p> <p>Språkbanken. <em>The Kubhist Corpus</em>. Department of Swedish, University of Gothenburg. <a href="https://spraakbanken.gu.se/korp/?mode=kubhist">https://spraakbanken.gu.se/korp/?mode=kubhist</a>.</p>
A Data-driven Analysis of a Cloud Data Center: Statistical Characterization of Workload, Energy and Temperature
<p>A characterization of cloud data center logs, analyzing its workload, energy and thermal characteristics. For more details of the dataset, please read the following paper: <a href="http://hpc.ec.tuwien.ac.at/files/UCC_23_data_center_analysis.pdf">http://hpc.ec.tuwien.ac.at/files/UCC_23_data_center_analysis.pdf.</a></p><p> </p><p>If you use the dataset, please cite the following work:</p><p>Shashikant Ilager, Adel N. Toosi, Mayank Raj Jha, Ivona Brandic, Rajkumar Buyya, "A Data-driven Analysis of a Cloud Data Center: Statistical Characterization of Workload, Energy and Temperature", In Proceedings of the 16th IEEE/ACM International Conference on Utility and Cloud Computing (UCC2023), Messina, Italy, December 4-7, 2023.</p>
A consensus compound/bioactivity dataset for data-driven drug design and chemogenomics
<p>This is the <strong>updated version</strong> of the dataset from <strong>10.5281/zenodo.6320761</strong></p> <p><strong>Information</strong></p> <p>The diverse publicly available compound/bioactivity databases constitute a key resource for data-driven applications in chemogenomics and drug design. Analysis of their coverage of compound entries and biological targets revealed considerable differences, however, suggesting benefit of a consensus dataset. Therefore, we have combined and curated information from five esteemed databases (ChEMBL, PubChem, BindingDB, IUPHAR/BPS and Probes&Drugs) to assemble a consensus compound/bioactivity dataset comprising <a href="tel:1144648">1144648</a> compounds with 10915362 bioactivities on 5613 targets (including defined macromolecular targets as well as cell-lines and phenotypic readouts). It also provides simplified information on assay types underlying the bioactivity data and on bioactivity confidence by comparing data from different sources. We have unified the source databases, brought them into a common format and combined them, enabling an ease for generic uses in multiple applications such as chemogenomics and data-driven drug design.</p> <p>The consensus dataset provides increased target coverage and contains a higher number of molecules compared to the source databases which is also evident from a larger number of scaffolds. These features render the consensus dataset a valuable tool for machine learning and other data-driven applications in (de novo) drug design and bioactivity prediction. The increased chemical and bioactivity coverage of the consensus dataset may improve robustness of such models compared to the single source databases. In addition, semi-automated structure and bioactivity annotation checks with flags for divergent data from different sources may help data selection and further accurate curation.</p> <p>This dataset belongs to the publication: <a href="https://doi.org/10.3390/molecules27082513">https://doi.org/10.3390/molecules27082513</a><br> </p> <p><strong>Structure and content of the dataset</strong></p> <table align="left"> <caption><strong>Dataset structure</strong></caption> <thead> <tr> <th scope="col"> <p>ChEMBL</p> <p>ID</p> </th> <th scope="col"> <p>PubChem</p> <p>ID</p> </th> <th scope="col"> <p>IUPHAR</p> <p>ID</p> </th> <th scope="col">Target</th> <th scope="col"> <p>Activity</p> <p>type</p> </th> <th scope="col">Assay type</th> <th scope="col">Unit</th> <th scope="col">Mean C (0)</th> <th scope="col">...</th> <th scope="col">Mean PC (0)</th> <th scope="col">...</th> <th scope="col">Mean B (0)</th> <th scope="col">...</th> <th scope="col">Mean I (0)</th> <th scope="col">...</th> <th scope="col">Mean PD (0)</th> <th scope="col">...</th> <th scope="col">Activity check annotation</th> <th scope="col">Ligand names</th> <th scope="col">Canonical SMILES C</th> <th scope="col">...</th> <th scope="col">Structure check (Tanimoto)</th> <th scope="col">Source</th> </tr> </thead> <tbody> <tr> <td> </td> <td> </td> <td> </td> <td> </td> <td> </td> <td> </td> <td> </td> <td> </td> <td> </td> <td> </td> <td> </td> <td> </td> <td> </td> <td> </td> <td> </td> <td> </td> <td> </td> <td> </td> <td> </td> <td> </td> <td> </td> <td> </td> <td> </td> </tr> </tbody> </table> <p>The dataset was created using the Konstanz Information Miner (KNIME) (https://www.knime.com/) and was exported as a CSV-file and a compressed CSV-file.</p> <p>Except for the canonical SMILES columns, all columns are filled with the datatype ‘string’. The datatype for the canonical SMILES columns is the smiles-format. We recommend the <strong>File Reader</strong> node for using the dataset in KNIME. With the help of this node the data types of the columns can be adjusted exactly. In addition, only this node can read the compressed format.</p> <p>Column content:</p> <ul> <li>ChEMBL ID, PubChem ID, IUPHAR ID: chemical identifier of the databases</li> <li>Target: biological target of the molecule expressed as the HGNC gene symbol</li> <li>Activity type: for example, pIC<sub>50</sub></li> <li>Assay type: Simplification/Classification of the assay into cell-free, cellular, functional and unspecified</li> <li>Unit: unit of bioactivity measurement</li> <li>Mean columns of the databases: mean of bioactivity values or activity comments denoted with the frequency of their occurrence in the database, e.g. Mean C = 7.5 *(15) -> the value for this compound-target pair occurs 15 times in ChEMBL database</li> <li>Activity check annotation: a bioactivity check was performed by comparing values from the different sources and adding an activity check annotation to provide automated activity validation for additional confidence <ul> <li>no comment: bioactivity values are within one log unit;</li> <li>check activity data: bioactivity values are not within one log unit;</li> <li>only one data point: only one value was available, no comparison and no range calculated;</li> <li>no activity value: no precise numeric activity value was available;</li> <li>no log-value could be calculated: no negative decadic logarithm could be calculated, e.g., because the reported unit was not a compound concentration</li> </ul> </li> <li>Ligand names: all unique names contained in the five source databases are listed</li> <li>Canonical SMILES columns: Molecular structure of the compound from each database</li> <li>Structure check (Tanimoto): To denote matching or differing compound structures in different source databases <ul> <li>match: molecule structures are the same between different sources;</li> <li>no match: the structures differ. We calculated the Jaccard-Tanimoto similarity coefficient from Morgan Fingerprints to reveal true differences between sources and reported the minimum value;</li> <li>1 structure: no structure comparison is possible, because there was only one structure available;</li> <li>no structure: no structure comparison is possible, because there was no structure available.</li> </ul> </li> <li>Source: From which databases the data come from</li> </ul> <p> </p> <p> </p>
Data-Driven Computational Intelligence Applied to Dengue Outbreak Forecasting: a case study at the scale of the city of Natal, RN-Brazil
<p><strong>The dataset comprises survey data from the following sources:dengue_incidence_data.csv: public data provided by Municipal Health Department of Natal, State of Rio Grande do Norte, Brazil; and data of Brazilian Notifiable Diseases Information System (Sinan). The objective of this paper was to analyze incidence data of dengue cases registered in each neighborhood of Natal city, weekly sampled (52 epidemiological weeks a year) between 2016 – 2019). </strong></p>
Data-driven reconstructions of the Indian palaeomonsoon (1500–1995 CE)
<p>These datasets are fully described in the paper <strong>A novel explainable deep learning framework for reconstructing South Asian palaeomonsoons</strong>. <br><br><strong>Paper abstract:</strong><br>We present novel explainable deep learning techniques for reconstructing South Asian palaeomonsoon rainfall over the last 500 years, leveraging long instrumental precipitation records and palaeoenvironmental datasets from South and East Asia to build two types of model: dense neural networks (`timeline models') and convolutional neural networks (CNNs). The timeline models are trained individually on seven regional rainfall datasets and while they capture decadal-scale variability and significant droughts, they underestimate interannual variability. The CNNs, designed to account for spatial relationships in both predictor and target, demonstrate higher skill in reconstructing rainfall patterns and produce robust spatiotemporal reconstructions. The 19th and 20th centuries were characterised by marked inter-annual variability in the monsoon, but earlier periods were characterised by more decadal- to centennial-scale oscillations. Multidecadal droughts occurred in the mid-seventeenth and nineteenth centuries, while much of the eighteenth century (particularly the early part of the century) was characterised by above-average monsoon precipitation. Extreme droughts tend to be concentrated in south and west India and often coincide with recorded famines. Our findings offer insights into the historical variability of the Indian summer monsoon and highlight the potential of deep learning techniques in palaeoclimate reconstruction.<br><br><strong>Dataset description:</strong><br><em>regional-famines.csv:</em> Derived from https://en.wikipedia.org/wiki/Timeline_of_major_famines_in_India_prior_to_1765 and https://en.wikipedia.org/wiki/Timeline_of_major_famines_in_India_during_British_rule. Columns are the seven homogeneous rainfall regions of India, rows are years. Variable is 1 is a significant famine occurred somewhere in that region in that year, else 0.</p> <p><em>timeline-model-regional-prcp.csv: </em>Reconstructed seasonal (Jun–Sep) monsoon precipitation anomalies for each of the seven homogeneous rainfall regions of India, as well as all-India. Period covered is 1501–1995.</p> <p><em>cnn-ensemble-prcp.nc: </em> NetCDF file containing spatial maps of reconstructed seasonal monsoon precipitation anomalies from the CNN model. Available for each year from 1501–1995, at a resolution of 0.25°x0.25°. Two variables are included, the full ten-member ensemble and its mean.</p> <p><em>cnn-ensemble-mean-individual-years.zip: </em>A ZIP archive containing maps showing the CNN ensemble mean for each year.</p> <p><em>cnn-ensemble-mean-all-India.csv: </em>The all-India average of the CNN ensemble mean for each year, giving a yearly monsoon index from 1501–1995.</p> <p> </p>
MITgcm Dataset for paper: Sensitivity analysis of a data-driven model of ocean temperature
<p>MITgcm dataset used in paper, Sensitivity analysis of a data-driven model of ocean temperature, made available here. The dataset comes from running a sector config of the MITgcm model, briefly described in the paper. This dataset is used to train the regression model described in the paper.</p> <p>Updated to include ncra_cat_tave.nc file which was accidentally missed on first version.</p>
Dataset for "Fast creation of data-driven low-order predictive cardiac tissue excitation models from recorded activation patterns"
<p>This archive contains the source code and data sets presented in the publication "Fast creation of data-driven low-order predictive cardiac tissue excitation models from recorded activation patterns".</p> <p>Kabus, D., De Coster, T., de Vries, A. A., Pijnappels, D. A., & Dierckx, H. (2024). Fast creation of data-driven low-order predictive cardiac tissue excitation models from recorded activation patterns. <em>Computers in Biology and Medicine</em>, 107949. <a title="Persistent link using digital object identifier" href="https://doi.org/10.1016/j.compbiomed.2024.107949" target="_blank" rel="noreferrer noopener"><span>https://doi.org/10.1016/j.compbiomed.2024.107949</span></a></p>
Analysing the data-driven approach of dynamically estimating positioning accuracy (data)
<p>The train/validation/test sets used in the study <strong>"<em>Analysing the data-driven approach of dynamically estimating positioning accuracy</em>"</strong>.</p> <p>Preprint: <a href="https://arxiv.org/abs/2011.10478">https://arxiv.org/abs/2011.10478</a></p> <p>Published paper: <a href="https://ieeexplore.ieee.org/document/9500369">https://ieeexplore.ieee.org/document/9500369</a></p> <p>-----------------------------------------------------------------------------------------------------------------------------------------------------------------</p> <p>The full dataset used in this study, is the public dataset <em>lorawan_dataset_antwerp.csv </em> (and its related file <em>lorawan_antwerp_gateway_locations.json)</em> which can be access here:</p> <pre><a href="https://zenodo.org/record/3904158#.X4_h7y8RpQI">https://zenodo.org/record/3904158#.X4_h7y8RpQI</a></pre> <p>The above link is related to the following publication, in which the original dataset was published:</p> <p><a href="http://www.mdpi.com/2306-5729/3/2/13">http://www.mdpi.com/2306-5729/3/2/13</a></p> <p>The credit for the creation of the dataset goes to Aernouts, Michiel; Berkvens, Rafael; Van Vlaenderen, Koen; and Weyn, Maarten.</p>
SkinSource: A Data-Driven Toolbox for Predicting Touch-Elicited Skin Vibrations Across the Upper Limb
<p>The repository contains the data for the toolbox released as part of the publication “SkinSource: A Data-Driven Toolbox for Predicting Touch-Elicited Vibrations in the Upper Limb.” The toolbox and installation and usage instructions can be found on GitHub here: <a href="https://github.com/neelitummala/skinsource">https://github.com/neelitummala/skinsource</a>. If you use these data or our toolbox please cite our publication: <a href="https://doi.org/10.1109/HAPTICS59260.2024.10520852">https://doi.org/10.1109/HAPTICS59260.2024.10520852</a>.</p> <p>Full citation: “Tummala, N., Reardon, G., Fani, S., Goetz, D., Bianchi, M., and Visell, Y. (2024) SkinSource: A Data-Driven Toolbox for Predicting Touch-Elicited Vibrations in the Upper Limb. IEEE Haptics Symposium 2024. DOI: 10.1109/HAPTICS59260.2024.10520852” </p> <p> </p> <p><strong>Abstract From Manuscript</strong></p> <p>Vibrations transmitted throughout the hand and arm during touch contact play a central role in haptic science and engineering but are challenging to model or experimentally characterize. Here, we present SkinSource, a data-driven toolbox for predicting skin vibrations across the upper limb in response to user-specified input forces. The toolbox leverages impulse response measurements that encode the physics of vibration transmission across the hands and arms of four participants and provides software tools for analyzing the predicted skin responses. We show that the SkinSource predictions closely match experimental measurements and confirm the underlying assumption of linear vibration transmission in the skin. We also demonstrate through several usage examples how SkinSource can act as a versatile computational platform for haptic research applications, such as characterizing vibrotactile transmission in the skin, engineering haptic interfaces, and investigating touch perception.</p> <p><strong> </strong></p> <p><strong>Dataset Description</strong></p> <p>This dataset comprises experimental data of 3-axis surface acceleration at 72 locations on the skin in response to unit impulsive forces supplied at 20 different input locations on the palmar hand surface. For details on our experimental procedure, please see our publication. This data is intended to be used as part of the SkinSource toolbox, which can be found here: <a href="https://github.com/neelitummala/skinsource">https://github.com/neelitummala/skinsource</a>.</p> <p><strong> </strong></p> <p><strong>Data Fields</strong></p> <p>The data is provided as a .mat file. This file contains a single variable “dataTable” of variable type “table.” The table contains 80 rows, each corresponding to a unique experimental condition (4 participants x 20 input locations), and contains the following fields:</p> <p><strong>Data </strong>(522x72x3) - 3D array containing the 3-axis skin acceleration at 522 time points (impulse responses) for each of 72 accelerometers. Please see the GitHub code and documentation (<a href="https://github.com/neelitummala/skinsource">https://github.com/neelitummala/skinsource</a>) for the accelerometer locations on the dorsal surface of the upper limb.</p> <p><strong>Model </strong>- The upper limb model number. This number specifies the participant that data was taken on.</p> <p><strong>Location</strong> -<strong> </strong>Number designating which input location on the palmar hand surface the data corresponds to. Please see the GitHub code and documentation (<a href="https://github.com/neelitummala/skinsource">https://github.com/neelitummala/skinsource</a>) for input location number mapping.</p>
Data-driven physics-based modeling of pedestrian dynamics - dataset: Pedestrian trajectories at Eindhoven train station
<p>Pedestrian trajectories measured at train station Eindhoven Centraal (the Netherlands) on platform 2 with acces to tracks 3 and 4.</p> <p>The dataset is partitioned in files containing 10 consecutive days each, recording 4 data fields:</p> <ul> <li><strong>time_ms:</strong> Passed time since start of the measurements. Unit: milliseconds.</li> <li><strong>object_identifier:</strong> unique id identifying an object.</li> <li><strong>x_position_mm: </strong>coordinates of the object along the x-axis at the given time. Unit: millimeters.</li> <li><strong>y_position_mm:</strong> coordinates of the object along the y-axis at the given time. Unit: millimeters.</li> </ul> <p>Each object resembles a pedestrian on the train platform recorded with 10 frames per second. We deliberately removed exact date and time information for privacy reasons (see additional note). The data set consists of 60 consecutive days starting at an unkown time between 00:00 AM and 01:00 AM of a random date between April 1st and May 1st 2022. An overhead image of the platform is included showing train track 3 in the bottom and train track 4 in the top of the image.</p> <p>The data set is supplemented to the paper <a title="Data-driven physics-based modeling of pedestrian dynamics" href="https://doi.org/10.48550/arXiv.2407.20794" target="_blank" rel="noopener">Data-driven physics-based modeling of pedestrian dynamics</a> and can be processed by the associated <a title="Software: Data-driven physics-based modeling of pedestrian dynamics" href="https://github.com/c-pouw/physics-based-pedestrian-modeling" target="_blank" rel="noopener">Python implementation</a> to create pedestrian models. </p>
Dataset for Paper "Towards Increased Diversity in STEM Education: Five archetypes Derived through a Data-Driven Approach Examining a Computer Science Student Cohort
<p># Dataset for Paper "Towards Increased Diversity in STEM Education: Five archetypes Derived through a Data-Driven Approach Examining a Computer Science Student Cohort" - Rev #1</p> <p>This is the dataset for the paper titled "Towards Increased Diversity in STEM Education: Five archetypes Derived through a Data-Driven Approach Examining a Computer Science Student Cohort".</p> <p>In case of questions, feel free to contact the authors, *anonymised*, ORCID: https://orcid.org/*anonymised*, current affiliation and email: *anonymised*</p> <p>## Survey 2019 ##<br> The raw survey data for the initial 2019 survey is available in the file *survey2019_anon.csv*. Note that the data is anonymised as free-text comments have been removed. Explanations on the variables and their levels are given in the files *variables_survey2019.csv* and *values_survey2019.csv*.<br> The questionnaire for the 2019 survey is contained in *survey2019_instrument.pdf*.</p> <p>## Survey 2020 ##<br> The raw survey data for the 2020 survey is available in the file *rdata_anon_survey2020.csv*. Additional scripts are supplied to reproduce the exploratory factor analysis. The main entry is the file *EFA.R*, which imports the data. The file contains some comments on the process.<br> The questionnaire for the 2020 survey is contained in *survey2020_instrument.pdf*.</p> <p>## Interviews ##<br> The interview guide used for the five interviews is available in the file *interview_instrument.pdf*.</p>
Scanning the horizon for invasive plant threats using a data-driven approach
<p>This repository holds the data and code for the manuscript "Scanning the horizon for invasive plant threats using a data-driven approach". </p> <p><strong>Contents</strong></p> <ul> <li>code: descriptions below</li> <li>data: descriptions below</li> <li>intermediate-data: datasets produced by processing original data (see code) or produced through horizon scan process (descriptions below)</li> <li>fl-plants-horizon-scan.Rproj: RStudio project for running R scripts</li> </ul> <p> </p> <table> <thead> <tr> <th scope="col">code</th> <th scope="col">description</th> </tr> </thead> <tbody> <tr> <td>gcw_processing.R</td> <td>R script to format data downloaded from the Global Compendium of Weeds</td> </tr> <tr> <td>native_introduced_ranges.R</td> <td>R script to create map of native and introduced ranges of taxa on the final list</td> </tr> <tr> <td>random_draws_plant_families.R</td> <td>R script to evaluate over- and underrepresentation of plant families in initial and final list</td> </tr> <tr> <td>review_process_comparison.R</td> <td>R script to evaluate differences in scores before and after peer-review and consensus-building</td> </tr> <tr> <td>scores_certainty_pathways.R</td> <td>R script to create figures of scores, certainty, and pathways for final list</td> </tr> <tr> <td>risk_scores_analys.R</td> <td>R script to evaluate final risk scores</td> </tr> <tr> <td>pathways_process.R</td> <td>R script to process pathways to introduction data</td> </tr> <tr> <td>taxa_list_processing.R</td> <td>R script to create list used for rapid risk assessments from an initial list</td> </tr> </tbody> </table> <p> </p> <table> <thead> <tr> <th scope="col">data</th> <th scope="col">description</th> </tr> </thead> <tbody> <tr> <td>cab_list_full.csv</td> <td>list of potential invasive species to Florida generated by CABI Horizon Scan Tool on November 15, 2019</td> </tr> <tr> <td>GCW_full_list_020420.csv</td> <td>Global Compendium of Weeds downloaded on February 4, 2020</td> </tr> <tr> <td>PlantAtlasDataExport-20191211-194219.csv</td> <td>Atlas of Florida plants downloaded December 11, 2019</td> </tr> <tr> <td>Taxon_x_List_GloNAF_vanKleunenetal2018Ecology_121119.csv</td> <td>GloNAF 1.2 database downloaded December 11, 2019</td> </tr> <tr> <td>the-plant-list</td> <td>The Plant List Database downloaded August 3, 2021</td> </tr> </tbody> </table> <p> </p> <table> <thead> <tr> <th scope="col">intermediate-data</th> <th scope="col">description</th> </tr> </thead> <tbody> <tr> <td>federal_noxious_weed_list.csv</td> <td>manually formatted version of the USDA Federal Noxious Weed List downloaded March 16, 2020</td> </tr> <tr> <td>first_round_assessments_050120.csv</td> <td>rapid risk assessments for horizon scan pre-peer-review</td> </tr> <tr> <td>fl_prohibited_plants.csv</td> <td>manually compiled list of prohibited plants in Florida based on the Florida Noxious Weed List, Florida Prohibited Plants list, and Florida Invasive Species Council (all downloaded March 9, 2020)</td> </tr> <tr> <td>horizon_scan_plants_full_reviews_080321.csv</td> <td>rapid risk assessments for horizon scan post-peer-review and consensus-building</td> </tr> </tbody> </table> <p> </p>
Dataset: Stochastic data-driven parameterization of unresolved eddy effects in a baroclinic quasi-geostrophic model
<p>This dataset is used to reproduce the figures in the manuscript "Stochastic data-driven parameterization of unresolved eddy effects in a baroclinic quasi-geostrophic model"</p> <p>The figures can be created with the following Python scripts:</p> <p>.</p> <p> </p>
Data-Driven Study of Long-Term Gaming Experience (Tribalwars)
<p>This is the accompanying dataset for our Paper "Data-Driven Study of Long-Term Gaming Experience" at the 14th International Conference on Quality of Multimedia Experience (QoMEX) - Please city this work if you use this dataset.</p> <p>This dataset contains hourly crawled player metadata of the Tribalwars casual game game round "dep13".</p> <p>With this dataset and a set of possible tooling, we hope to provide a foundation into further data-driven QoE studies.</p> <p>You will find further information about prerequisites and the dataset within the <strong>eval.ipynb</strong>.</p>
Dataset supporting the paper: Deciphering Oxygen Distribution and Hypoxia Profiles in the Tumor Microenvironment: A Data-Driven Mechanistic Modeling Approach
<p>The necessary image files for the paper titled "Deciphering Oxygen Distribution and Hypoxia Profiles in the Tumor Microenvironment: A Data-Driven Mechanistic Modeling Approach"</p>
Precomputed data accompanying the paper "Reduced Data-Driven Turbulence Closure for Capturing Long-Term Statistics"
<p>Together with the python code in <a href="https://github.com/rik-stra/tau_orthogonal_method_for_2D_turbulence" target="_blank" rel="noopener">https://github.com/rik-stra/tau_orthogonal_method_for_2D_turbulence</a>, this dataset lets the user interact with and reproduce the results in the paper "Reduced Data-Driven Turbulence Closure for Capturing Long-Term Statistics".</p> <p> All data in this dataset is computed with the provided code.</p>
Data-driven criteria for quantum correlations
<p>Dataset of 3-qubit density matrices used for training deep-learning models for quantum correlations identicfication introduced in two papers:</p> <ul> <li> <div>M Krawczyk, J Pawłowski, MM Maśka, K Roszak Physical Review A 109, 022405 (2024).</div> <div> </div> </li> <li> <div>J Pawłowski, M Krawczyk, Physical Review Applied 22, 014068 (2024).</div> <div> </div> </li> </ul>
Data-driven model enhancement of late-life lithium-ion batteries
<p>Suplement dataset for the paper "Data-driven model enhancement of late-life lithium-ion batteries"</p>
HELIX: Data-driven characterization of Brazilian land snails
<p>The Subulinidae family of Brazilian land snails includes similar species with particular morphometrical characteristics. We surveyed the metro area of the Juiz de Fora/MG city and collected snails from a representative private property with centered at 21º42'31''S, 43º21'26'', alt. 795m, with red-yellow oxisol, 8.5 pH, and Cwa climate. We performed monthly collections in a ecosystem with <em>Pennisetum purpureum</em>, <em>Brachiaria mutica</em>, <em>Paspalum notatum</em>, <em>Bidens pilosa</em>, <em>Leucena leucocephala</em>, and <em>Ricinus communis</em> plants near a river flow in the 2008/Sep - 2009/Aug timespan. There, we defined an equally spaced transect of 200m with ten collection points and gathered 50 x 50 cm, 500 g litter-falls. Samples were sieved by 2.0 mm meshs, and living specimens were cleaned and fixed in a Railliet-Henry liquid, their shells removed, dried, and separated. Finally, we label the species and create the Helix dataset. We measured each shell <em>Height</em>, <em>Diameter</em>, <em>Spire height</em>, and <em>Aperture width</em> and <em>height</em> with a pachymeter. The dataset includes 518 instances labeled as <em><strong>Beckianum beckianum</strong></em> (28.8%), <em><strong>Dysopeas muibum</strong></em> (21.2%), <em><strong>Allopeas gracilis</strong></em> (9.7%), <em><strong>Leptinaria unilamellata </strong></em>(12%), and <em><strong>Subulina octona</strong></em> (28.3%). </p>
Software repository for data-driven reconstruction of doping profiles in semiconductors
<p>Datasets and code used described in paper: "Data-driven solutions of ill-posed inverse problems arising from doping reconstruction in semiconductors" [arXiv:2208.00742]</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.