Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
64
datasets available to search
ShareScore release 0.9.0
Dataset results
64 results for “data-driven model”
Models for "A data-driven approach to studying changing vocabularies in historical newspaper collections"
<p>NOTE: This is a badly rendered version of the README within the archive.</p> <p><strong>A data-driven approach to studying changing vocabularies in historical newspaper collections</strong></p> <p>Simon Hengchen,* Ruben Ros,** Jani Marjanen,*** Mikko Tolonen***</p> <p>*<a href="https://spraakbanken.gu.se/en/about/staff/simon">Språkbanken Text</a>, University of Gothenburg, Sweden and <a href="https://iguanodon.ai">iguanodon.ai</a>, Belgium: firstname.lastname@gu.se<br> **<a href="https://www.c2dh.uni.lu/people/ruben-ros">Centre for Contemporary and Digital History (C2DH)</a>, University of Luxembourg: firstname.lastname@uni.lu<br> ***<a href="https://www.helsinki.fi/en/researchgroups/computational-history">COMHIS</a>, University of Helsinki: <a href="mailto:firstname.lastname@helsinki.fi">firstname.lastname@helsinki.fi</a>;</p> <p>These are the supplementary materials for the DH2019 paper <em>A data-driven approach to the changing vocabulary of the ‘nation’ in English, Dutch, Swedish and Finnish newspapers, 1750-1950</em>, as well as the 2021 Digital Scholarship in the Humanities publication available in OpenAccess: <a href="https://academic.oup.com/dsh/article/36/Supplement_2/ii109/6421793">https://academic.oup.com/dsh/article/36/Supplement_2/ii109/6421793</a>. If you end up using whole or parts of this resource, please use the following citation(s):</p> <ul> <li>Hengchen, S., Ros, R., and Marjanen, J. (2019). A data-driven approach to the changing vocabulary of the 'nation' in English, Dutch, Swedish and Finnish newspapers, 1750-1950. In <em>Proceedings of the Digital Humanities (DH) conference 2019, Utrecht, The Netherlands</em></li> </ul> <p>and/or:</p> <ul> <li>Hengchen, S., Ros, R., Marjanen, J. and Tolonen, M., 2021. A data-driven approach to studying changing vocabularies in historical newspaper collections. Digital Scholarship in the Humanities, 36(Supplement_2), pp.ii109-ii126.</li> </ul> <p>or alternatively use one of the following <code>bib</code>s:</p> <pre><code>@inproceedings{hengchen2019nation, title="A data-driven approach to the changing vocabulary of the 'nation' in {E}nglish, {D}utch, {S}wedish and {F}innish newspapers, 1750-1950.", author={Hengchen, Simon and Ros, Ruben and Marjanen, Jani}, year={2019}, address = "Utrecht, The Netherlands", booktitle={Proceedings of the Digital Humanities (DH) conference 2019} }</code></pre> <pre><code>@article{hengchen2021data, title={A data-driven approach to studying changing vocabularies in historical newspaper collections}, author={Hengchen, Simon and Ros, Ruben and Marjanen, Jani and Tolonen, Mikko}, journal={Digital Scholarship in the Humanities}, volume={36}, number={Supplement\_2}, pages={ii109--ii126}, year={2021}, publisher={Oxford University Press} }</code></pre> <p> </p> <p>Files</p> <p>This archive contains two folders -- one per diachronic representation method -- as well as this README. The folders each contain four folders, which contain the models for their respective languages. As can be inferred from the small datasize, most of the earlier models are not reliable and should not be used, but are still made available. This work is licensed under a <a href="http://creativecommons.org/licenses/by-sa/4.0/">Creative Commons Attribution-ShareAlike 4.0 International License</a>.</p> <p><strong>Source material</strong></p> <p>Finnish:</p> <p>The models were created with data from the Finnish Sub-corpus of the Newspaper and Periodical Corpus of the National Library of Finland (National Library of Finland, 2011). We used everything in the corpus.</p> <p>Filesizes:</p> <pre><code>[simon@taito-login3 SGNS]$ du -h fi* 12M fi_1820_SGNS_corpus_file.gensim 89M fi_1840_SGNS_corpus_file.gensim 797M fi_1860_SGNS_corpus_file.gensim 7.0G fi_1880_SGNS_corpus_file.gensim 22G fi_1900_SGNS_corpus_file.gensim</code></pre> <p>Swedish:</p> <p>The models were created with data from the Kubhist 2 corpus (Språkbanken) -- more precisely, the data dumps available at <a href="https://spraakbanken.gu.se/lb/resurser/meningsmangder/">https://spraakbanken.gu.se</a>. After a manual evaluation of Swedish embeddings trained without pre-processing seemed to show that the embeddings were of low quality, we retrained models, only keeping sentences that were at least 10 tokens long and were constituted of at least 50% of lemmas as per the KORP processing pipeline (Borin et al, 2012).</p> <p>Filesizes:</p> <pre><code>[simon@taito-login3 SGNS]$ du -h sv* 1.6M sv_1740_SGNS_corpus_file.gensim 44M sv_1760_SGNS_corpus_file.gensim 124M sv_1780_SGNS_corpus_file.gensim 228M sv_1800_SGNS_corpus_file.gensim 678M sv_1820_SGNS_corpus_file.gensim 1.6G sv_1840_SGNS_corpus_file.gensim 4.5G sv_1860_SGNS_corpus_file.gensim 6.5G sv_1880_SGNS_corpus_file.gensim 113M sv_1900_SGNS_corpus_file.gensim</code></pre> <p>Dutch:</p> <p>The models were created with data from the Delpher newspaper archive (Royal Dutch Library, 2017), through data dumps for newspapers until and including 1876, and through API hits for articles from 1877 to 1899 (included).</p> <ul> <li>For anything pre-1877 we discarded full texts that had, in the metadata, anything else than exclusively <code>nl</code> or <code>NL</code> as a language tag.</li> <li>For the full texts between 1877 and 1899: we queried the API for all items in the “artikel” category that contained the determiner <code>de</code>.</li> </ul> <p>Our assumption was that most articles should contain <code>de</code> at least once, and those that didn't were too short to be deemed interesting. A subsequent study showed that was not exactly the case, but we were reassured by the fact that left-out articles were probably "shipping or financial reports" (thanks go to Melvin Wevers). We also did not include the colonial newspapers for our embeddings. This is motivated by our research questions. A list of removed newspapers is available on request.</p> <p>Filesizes:</p> <pre><code>[simon@taito-login3 SGNS]$ du -h nl* 6.8M nl_1620_SGNS_corpus_file.gensim 7.9M nl_1640_SGNS_corpus_file.gensim 43M nl_1660_SGNS_corpus_file.gensim 78M nl_1680_SGNS_corpus_file.gensim 138M nl_1700_SGNS_corpus_file.gensim 243M nl_1720_SGNS_corpus_file.gensim 287M nl_1740_SGNS_corpus_file.gensim 431M nl_1760_SGNS_corpus_file.gensim 825M nl_1780_SGNS_corpus_file.gensim 1.2G nl_1800_SGNS_corpus_file.gensim 1.8G nl_1820_SGNS_corpus_file.gensim 3.1G nl_1840_SGNS_corpus_file.gensim 5.2G nl_1860_SGNS_corpus_file.gensim 13G nl_1880_SGNS_corpus_file.gensim</code></pre> <p>English:</p> <p>The models were created with data from the British Library Newspapers collection (<a href="https://www.gale.com/intl/primary-sources/british-library-newspapers%5D">link</a>), the Nichols collection (<a href="https://www.gale.com/intl/c/17th-and-18th-century-burney-newspapers-collection">link</a>), and the Burney collection (<a href="https://www.gale.com/intl/c/17th-and-18th-century-nichols-newspapers-collection">link</a>). We used everything in the corpora. For English, only SGNS_ALIGN models are available. We thank Gale Cengage for their help with this project.</p> <p>Filesizes:</p> <pre><code>[simon@taito-login3 SGNS]$ du -h en* 4.3M en_1620_SGNS_corpus_file.gensim 11M en_1640_SGNS_corpus_file.gensim 11M en_1660_SGNS_corpus_file.gensim 106M en_1680_SGNS_corpus_file.gensim 409M en_1700_SGNS_corpus_file.gensim 1.7G en_1720_SGNS_corpus_file.gensim 834M en_1740_SGNS_corpus_file.gensim 2.4G en_1760_SGNS_corpus_file.gensim 5.3G en_1780_SGNS_corpus_file.gensim 5.5G en_1800_SGNS_corpus_file.gensim 15G en_1820_SGNS_corpus_file.gensim 42G en_1840_SGNS_corpus_file.gensim 65G en_1860_SGNS_corpus_file.gensim 88G en_1880_SGNS_corpus_file.gensim 26G en_1900_SGNS_corpus_file.gensim 21G en_1920_SGNS_corpus_file.gensim 6.3G en_1940_SGNS_corpus_file.gensim</code></pre> <p><strong>Word embeddings</strong></p> <p>For every language, we train diachronic embeddings as follows. We divide the data in 20-year time bins. We train SGNS_UPDATE and SGNS_ALIGN models. Current research on German (Schlechtweg et al, 2019) and English (Shoemark et al, 2019) indicates you should use the SGNS_ALIGN models. <strong>For EN, FI, NL, no tokens (including punctuation) were removed nor altered, aside from lowercasing</strong>. For SV, see above. Parameters are as follows: SGNS architecture (Mikolov et al 2013), window size of 5, frequency threshold of 100, 5 epochs, 300 dimensions (or 100 for EN).</p> <ul> <li>For SGNS_UPDATE: We first train a model for the first time bin <code>t</code>. To train the model for <code>t+1</code>, we use the <code>t</code> model to initialise the vectors for <code>t+1</code>, set the learning rate to correspond to the end learning rate of <code>t</code>, and continue training. This approach, closely following Kim et al (2014), has the advantage of avoiding the need for post-training vector space alignment.</li> </ul> <p>The Python snippet below, which makes use of gensim (Rehurek and Sojka, 2010), illustrates the approach. Special thanks go to Sara Budts.</p> <pre><code>## dict_files[key] is a dictionary with double decades as keys and a corresponding LineSentence object as value: https://radimrehurek.com/gensim/models/word2vec.html#gensim.models.word2vec.LineSentence count = 0 for key in sorted(list(dict_files.keys())): if count == 0: ## This is the first model. model = gensim.models.Word2Vec(corpus_file=dict_files[key], min_count=100, sg=1 ,size=300, workers=64, seed=1830, iter=5) model.save(os.path.join(data_path_final,"KIM",lang+"_"+str(timebin)+".w2v")) print("Model saved, on to the next\n") count += 1 if count > 0: ## this is for the subsequent models. print("model for double decade starting in",str(key)) model = gensim.models.Word2Vec.load(os.path.join(data_path_final,"KIM",lang+"_"+str(timebin-20)+".w2v")) print("previous model loaded") model.build_vocab(corpus_file=dict_files[key], update=True) model.train(corpus_file=dict_files[key], total_words = model.corpus_count, total_examples = model.corpus_count, start_alpha = model.alpha, end_alpha = model.min_alpha, epochs=model.epochs) model.save(os.path.join(data_path_final,"KIM",lang+"_"+str(timebin)+".w2v")) </code></pre> <ul> <li>For SGNS_ALIGN: We independently train models for all time bins. The models in this repository are <em>NOT</em> aligned, leaving you the choice of how to align them. For example, <a href="https://gist.github.com/quadrismegistus/09a93e219a6ffc4f216fb85235535faf">here</a> is a link to code by Ryan Heuser to do just that. Models were trained with the <code>count == 0</code> scenario in the snippet above.</li> </ul> <p><strong>Acknowledgments</strong></p> <p>This work has been supported by the European Union's Horizon 2020 research and innovation programme under grant 770299 <a href="https://www.newseye.eu/">NewsEye</a>. Specials thanks go to the data providers/collection-holding institutions: the Finnish Language Bank, the Swedish Language Bank, the Royal Dutch Library, and Gale Cengage.</p> <p>The authors would like to thank the following persons and group, listed alphabetically: Antoine Doucet, Antti Kanner, Axel-Jean Caurant, Dominik Schlechtweg, Eetu Mäkelä, Elaine Zosa, Estelle Bunout, Haim Dubossarsky, Joris van Eijnatten, Krister Lindén, Lars Borin, Lidia Pivovarova, Melvin Wevers, Nina Tahmasebi, Sara Budts, Senka Drobac, Tanja Säily, the COMHIS group, and Steven Claeyssens. Computational resources were provided by CSC – IT Center for Science Ltd.</p> <p><strong>References</strong></p> <p>Borin, L., Forsberg, M., Roxendal, J. (2012). Korp-the corpus infrastructure of Spräkbanken,in: LREC. pp. 474–478.</p> <p>Kim, Y., Chiu, Y.I., Hanaki, K., Hegde, D. and Petrov, S. (2014). Temporal Analysis of Language through Neural Language Models. <em>ACL 2014</em>, p.61.</p> <p>Mikolov, T., Chen, K., Corrado, G. and Dean, J. (2013). Efficient estimation of word representations in vector space. <em>arXiv preprint arXiv:1301.3781</em>.</p> <p>National Library of Finland (2011). <em>The Finnish Sub-corpus of the Newspaper and Periodical Corpus of the National Library of Finland, Kielipankki Version</em> [text corpus]. Kielipankki. Retrieved from <a href="http://urn.fi/urn:nbn:fi:lb-2016050302">http://urn.fi/urn:nbn:fi:lb-2016050302</a>.</p> <p>Rehurek, R. and Sojka, P. (2010). Software framework for topic modelling with large corpora. In <em>Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks</em>.</p> <p>Royal Dutch Library (2017). <em>Delpher open krantenarchief (1.0)</em>. Den Haag, 2017.</p> <p>Schlechtweg D., Hätty A, del Tredici M., and Schulte im Walde S. (2019). A Wind of Change: Detecting and Evaluating Lexical Semantic Change across Times and Domains. In <em>Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)</em>, Florence, Italy. ACL.</p> <p>Shoemark, P., Liza, F.F., Nguyen, D., Hale, S. and McGillivray, B. (2019). Room to Glo: A Systematic Comparison of Semantic Change Detection Approaches with Word Embeddings. In <em>Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (pp. 66-76)</em>, Hong Kong.</p> <p>Språkbanken. <em>The Kubhist Corpus</em>. Department of Swedish, University of Gothenburg. <a href="https://spraakbanken.gu.se/korp/?mode=kubhist">https://spraakbanken.gu.se/korp/?mode=kubhist</a>.</p>
MITgcm Dataset for paper: Sensitivity analysis of a data-driven model of ocean temperature
<p>MITgcm dataset used in paper, Sensitivity analysis of a data-driven model of ocean temperature, made available here. The dataset comes from running a sector config of the MITgcm model, briefly described in the paper. This dataset is used to train the regression model described in the paper.</p> <p>Updated to include ncra_cat_tave.nc file which was accidentally missed on first version.</p>
Dataset for "Fast creation of data-driven low-order predictive cardiac tissue excitation models from recorded activation patterns"
<p>This archive contains the source code and data sets presented in the publication "Fast creation of data-driven low-order predictive cardiac tissue excitation models from recorded activation patterns".</p> <p>Kabus, D., De Coster, T., de Vries, A. A., Pijnappels, D. A., & Dierckx, H. (2024). Fast creation of data-driven low-order predictive cardiac tissue excitation models from recorded activation patterns. <em>Computers in Biology and Medicine</em>, 107949. <a title="Persistent link using digital object identifier" href="https://doi.org/10.1016/j.compbiomed.2024.107949" target="_blank" rel="noreferrer noopener"><span>https://doi.org/10.1016/j.compbiomed.2024.107949</span></a></p>
Data-driven physics-based modeling of pedestrian dynamics - dataset: Pedestrian trajectories at Eindhoven train station
<p>Pedestrian trajectories measured at train station Eindhoven Centraal (the Netherlands) on platform 2 with acces to tracks 3 and 4.</p> <p>The dataset is partitioned in files containing 10 consecutive days each, recording 4 data fields:</p> <ul> <li><strong>time_ms:</strong> Passed time since start of the measurements. Unit: milliseconds.</li> <li><strong>object_identifier:</strong> unique id identifying an object.</li> <li><strong>x_position_mm: </strong>coordinates of the object along the x-axis at the given time. Unit: millimeters.</li> <li><strong>y_position_mm:</strong> coordinates of the object along the y-axis at the given time. Unit: millimeters.</li> </ul> <p>Each object resembles a pedestrian on the train platform recorded with 10 frames per second. We deliberately removed exact date and time information for privacy reasons (see additional note). The data set consists of 60 consecutive days starting at an unkown time between 00:00 AM and 01:00 AM of a random date between April 1st and May 1st 2022. An overhead image of the platform is included showing train track 3 in the bottom and train track 4 in the top of the image.</p> <p>The data set is supplemented to the paper <a title="Data-driven physics-based modeling of pedestrian dynamics" href="https://doi.org/10.48550/arXiv.2407.20794" target="_blank" rel="noopener">Data-driven physics-based modeling of pedestrian dynamics</a> and can be processed by the associated <a title="Software: Data-driven physics-based modeling of pedestrian dynamics" href="https://github.com/c-pouw/physics-based-pedestrian-modeling" target="_blank" rel="noopener">Python implementation</a> to create pedestrian models. </p>
Dataset: Stochastic data-driven parameterization of unresolved eddy effects in a baroclinic quasi-geostrophic model
<p>This dataset is used to reproduce the figures in the manuscript "Stochastic data-driven parameterization of unresolved eddy effects in a baroclinic quasi-geostrophic model"</p> <p>The figures can be created with the following Python scripts:</p> <p>.</p> <p> </p>
Dataset supporting the paper: Deciphering Oxygen Distribution and Hypoxia Profiles in the Tumor Microenvironment: A Data-Driven Mechanistic Modeling Approach
<p>The necessary image files for the paper titled "Deciphering Oxygen Distribution and Hypoxia Profiles in the Tumor Microenvironment: A Data-Driven Mechanistic Modeling Approach"</p>
Data-driven model enhancement of late-life lithium-ion batteries
<p>Suplement dataset for the paper "Data-driven model enhancement of late-life lithium-ion batteries"</p>
Data-driven plasma modelling: Fluorocarbon ICP data set
<p>This is an open source dataset of optical emission spectra and optical images in fluorocarbon plasmas, along with associated tool logs, captured from a Oxford Intruments Plasma Technology PP100 ICP etcher. The dataset consists of Ar, O<sub>2</sub>, Ar/O<sub>2</sub>, CF<sub>4</sub>/O<sub>2</sub> and SF<sub>6</sub>/O<sub>2</sub> gas mixtures etching Si wafers.</p> <p>The data has been split into chunks for uploading to Zenodo, to reconstruct them:</p> <p>$ cat generative_model-fluorocarbon_data_set.tar.xz* > generative_model-fluorocarbon_data_set.tar.xz </p> <p>$ tar -xvf generative_model-fluorocarbon_data_set.tar.xz </p> <p>You can find the code to train an autoencoder model using the optical emission spectra and optical images at our github repo, https://github.com/gregdaly/generative_modelling_for_optical_plasma_diagnostics </p>
Comparative Study of Data-driven Solar Coronal Field Models Using a Flux Emergence Simulation as a Ground-truth Data Set
<p>For a better understanding of magnetic field in the solar corona and dynamic activities such as flares and coronal mass ejections, it is crucial to measure the time-evolving coronal field and accurately estimate the magnetic energy. Recently, a new modeling technique called the data-driven coronal field model, in which the time evolution of magnetic field is driven by a sequence of photospheric magnetic and velocity field maps, has been developed and revealed the dynamics of flare-productive active regions. Here we report on the first qualitative and quantitative assessment of different data-driven models using a magnetic flux emergence simulation as a ground-truth (GT) data set. We compare the GT field with those reconstructed from the GT photospheric field by four data-driven algorithms. It is found that, at least, the flux rope structure is reproduced in all coronal field models. Quantitatively, however, the results show a certain degree of model dependence. In most cases, the magnetic energies and relative magnetic helicity are comparable to or at most twice of the GT values. The reproduced flux ropes have a sigmoidal shape (consistent with GT) of various sizes, a vertically-standing magnetic torus, or a packed structure with curled field lines. The observed discrepancies can be attributed to the highly non-force-free input photospheric field, from which the coronal field is reconstructed, and to the modeling constraints such as the treatment of background atmosphere, the bottom boundary setting, and the spatial resolution.</p>
On the estimation of landslide intensity, hazard and density via data-driven models.
<p>The geographic prediction of landslide occurrence is undertaken by assessing whether a slope may be stable or unstable. In other words, current practices treat slopes where a single landslide occurred in the same way as slopes where many landslides occurred. At the slope scale, this procedure inevitably underestimates the effect of multiple landslides.<br> Here we model the number of landslides per slope instead. Then, thanks to the close relation that the number of failures shows with respect to landslide size, we convert the estimated number of landslides into estimated landslide areas. Ultimately, we also estimate the expected proportion of a slope affected by landslides. This framework is more informative than the stable/unstable paradigm and may help landslide risk mitigation strategies.</p>
A physically interpretable data-driven surrogate model for wake steering
<p>PALM input files for the simulations performed in the study "A physically interpretable data-driven surrogate model for wake steering" by Sengers et al. (2022). </p> <p>The PALM code is available at <a href="https://palm.muk.uni-hannover.de/">https://palm.muk.uni-hannover.de</a><br> Additional information to the input files is given in the README file</p> <p>Cite this as:<br> B.A.M. Sengers (2022). Dataset: A physically interpretable data-driven surrogate model for wake steering. https://doi.org/10.5281/zenodo.6821164</p>
Data and Code for "Quantifying spatio-temporal risk of Harmful Algal Blooms and their impacts on bivalve shellfish mariculture using a data-driven modelling approach"
<p>This is a zipped file of all associated code and data for the submitted paper entitled "Quantifying spatio-temporal risk of Harmful Algal Blooms and their impacts on bivalve shellfish mariculture using a data-driven modelling approach".</p>
Data set, Model, and Catalog for: Data-driven stellar intrinsic colors and dust reddenings for spectro-photometric data
<p>Intrinsic colors of stars are essential for the studies on both stellar physics and dust reddening. In this work, we developed an XGBoost model to predict the stellar intrinsic colors with the atmospheric parameters, Teff , log g, and [M/H], which is an improvement of the widely used blue-edge method. The dust reddening toward each line-of-sight can then be calculated by the observed colors minus the derived intrinsic colors.</p> <p>Here we provide the related data sets:</p> <ul> <li>xgb_model.pkl: the trained XGBoost model.</li> <li>use_xgb.py: a simple script showing how to use the XGBoost model to predict intrinsic colors.</li> <li>data_set.fits: this fits file contains the training and test sets, separated into four data arrays: <ul> <li>X_train: X-data (teff,logg,mh) of the training set.</li> <li>X_test: X-data (teff,logg,mh) of the test set.</li> <li>y_train: y-data (BP-RP, BP-Ks, J-Ks) of the training set.</li> <li>y_test: y-data (BP-RP, BP-Ks, J-Ks) of the test set.</li> </ul> </li> <li>IC_catalog.csv: a catalog containing a representative set of intrinsic colors at three bands ('BPRP0', 'BPK0', and 'JK0' in columns) as a function of typical Teff, logg, and [M/H] ('teff', 'logg', and 'mh' in columns).</li> </ul> <p>With the above data and model, users can apply the trained XGBoost to new sources to predict their intrinsic colors and calculate their dust reddenings. One can further control the quality of the prediction by selecting training-like sources with the training set and estimating the <a>generalization error by the test set.</a></p>
Measured BSDF and data-driven model of a laser cut panel (LCP001)
<p>Model generated from the measured Bidirectional Scattering Distribution Function (BSDF) of a LCP. The RADIANCE tool-chain pabopto2bsdf, bsdf2ttree was employed. Initial tensor resolution set to 128x128 incident, and equal number of outgoing, scattered directions (corresponding to approx. 1.4 degree). Subsequent adaptive data reduction by approx. 98%. Photometric BSDF.</p> <p>The measured data is included as Differential Scattering Function (DSF): DSF = BSDF x cos(theta_s), where theta_s is the off-normal angle of the outgoing, scattered direction of light (the first column in the measured data). The data was measured on a scanning gonio-photometer (pab advanced technologies pgII) at the optical laboratory of CC Building Envelopes. A halogen lamp was employed, focud on the detector plane for maximum resolution, with a hot-mirror installed in the illuminator to block near infrared emission. The Si photocell of the detector was equipped with a weighing filter to match photometric response v(lambda). The profiles through the unobstructed beam are given for the phi=0°,180°, and phi=90°,270° planes.</p> <p>The angular coordinates theta=90, phi=0 correspond to the intended up direction when installed vertically, e.g. in a window.</p> <p>When publishing any work making use of this data-set, please reference either this data-set, including its Digital Object Identifier (DOI:<a href="https://doi.org/10.5281/zenodo.3375294">10.5281/zenodo.3375294</a>), or (preferred) by this article which describes sample, model and its exemplary application:</p> <p>Lars Oliver Grobe. Photon mapping in image-based visual comfort assessments with BSDF models of high resolution. Journal of Building Performance Simulation. DOI:10.1080/19401493.2019.1653994</p>
Data-driven physics-based modeling of pedestrian dynamics
<p>Python package to create physics-based pedestrian models from crowd measurements</p> <p>Github: <a href="https://github.com/c-pouw/physics-based-pedestrian-modeling">https://github.com/c-pouw/physics-based-pedestrian-modeling</a></p>
Development and Comparison of Model-Based and Data-Driven Approaches for the Prediction of the Mechanical Properties of Lattice Structures
<p>This dataset comes from the following paper:</p> <p>Chiara Pasini, Oscar Ramponi, Stefano Pandini, Luciana Sartore, Giulia Scalet, Development and Comparison of Model-Based and Data-Driven Approaches for the Prediction of the Mechanical Properties of Lattice Structures, J. of Materi Eng and Perform, 2024. <a href="https://doi.org/10.1007/s11665-024-10199-x">https://doi.org/10.1007/s11665-024-10199-x</a></p> <p>It contains:</p> <ul> <li>"Notes.pdf" describing all the files uploaded</li> <li>. m of the neural network</li> <li>. inp of the Abaqus finite element simulations</li> </ul>
Data for replication of the publication: Probabilistic leak localization in water distribution networks using a hybrid data-driven and model-based approach
<p>20 to 30% of drinking water produced is lost due to leaks in water distribution pipes. In times of water scarcity, losing so much treated water comes at a significant cost, both environmentally and economically. In this paper, we propose a hybrid leak localization approach combining both model-based and data-driven modeling. Pressure heads of leak scenarios are simulated using a hydraulic model, and then used to train a machine-learning based leak localization model. A key element of our approach is that discrepancies between simulated and measured pressures are accounted for using a dynamically calculated bias correction, based on historical pressure measurements. Data of in-field leak experiments in operational water distribution networks were produced to evaluate our approach on realistic test data. Two problematic settings for leak localization were examined. In the first setting, an uncalibrated hydraulic model was used. In the second setting, an extended version of the water distribution network was considered, where large parts of the network were insensitive to leaks. Our results show that the leak localization model is able to reduce the leak search region in parts of the network where leaks induce detectable drops in pressure. When this is not the case, the model still localizes the leak but is able to indicate a higher level of uncertainty with respect to its leak predictions.</p>
name lists for 'Detecting intersectionality in NER models: A data-driven approach'
<p>Name lists used for data augmentation for testing biases (in terms of error disparities) of Name Entity Recognition in Danish NLP pipelines.</p> <p>The following lists are from <a href="https://www.dst.dk/da/Statistik/emner/borgere/navne/navne-i-hele-befolkningen">Statistics Denmark</a>:</p> <ul> <li>majority_first_names_2023_men.csv</li> <li>majority_first_names_2023_women.csv</li> <li>majority_last_names_2023.csv</li> </ul> <p>The following lists are from Eva Villarsen Meldgaard. 2005. <a href="https://nors.ku.dk/publikationer/webpublikationer/muslimske_fornavne/">Muslimske fornavne i danmark</a>. Publisher: Københavns Universitet</p> <ul> <li>minority_first_names_men.csv</li> <li>minority_first_names_men.csv</li> </ul> <p>The list majority_unisex_names.csv is retrieved from <a href="https://familieretshuset.dk/navne/navne/godkendte-fornavne">The Agency of Family Law</a> in Denmark, and the numbers are retrieved from the above lists from Statistics Denmark.</p> <p>The list minority_last_names.csv is retrieved from <a href="https://www.familyeducation.com/baby-names/surname/origin/muslim">FamilyEducation</a>.</p> <p>The list overlapping_names.csv contains first names, which both occur on the list of majority names and the list of minority names.</p>
Validation of an interpretable data-driven wake model using lidar measurements from a field wake steering experiment
<p>Selection of the data in the following paper:<br> Sengers, B. A. M., Steinfeld, G., Hulsman, P., & Kuehn, M. (2023). Validation of an interpretable data-driven wake model using lidar measurements from a free-field wake steering experiment. Wind Energy Science Discussions, 1-32.</p> <p>This data subset provides input parameters commonly used in wake models, as well as ten-minuted averaged cross sections of the flow field at 4 rotor diameters downstream, as measured by a nacelle-mounted lidar. </p> <p>Cite this as:<br> B.A.M. Sengers (2023). Dataset: Validation of an interpretable data-driven wake model using lidar measurements from a field wake steering experiment. https://doi.org/10.5281/zenodo.7741395</p>
Supplementary data of article Integrating Data-Driven and Hydraulic Modelling with Acoustic Sensor Information for Improved Leak Location in Water Distribution Networks
<p>This dataset was generated within the research thesis of Axel Hutomo, under the supervision of Leonardo Alfonso and Ioana Popescu at IHE Delft, and it is published as supplementary data for the article <em>Integrating Data-Driven and Hydraulic Modelling with Acoustic Sensor Information for Improved Leak Location in Water Distribution Networks, </em>currently under review. </p> <p>The Excel sheet provides information about the datasets produced to integrate acoustic sensor data and hydraulic model output data, to be used by the Machine Learning model. The acoustic sensor data were obtained by extracting several features in time and frequency domains from each audio file coming from acoustic sensors, whereas hydraulic model data was obtained by modelling these leaks using a pressure-independent analysis.</p> <p>The Python code shows the building of the ANN for leakage modelling prediction, integrating the two datasets above, for different leak rates.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.