Find research datasets worth reusing

opencc-by-4.0Jul 2021View details →

Dron view of the 2021 field trial for BRESOV project in SERIDA

This video shows the field trial with 311 snap bean lines in organic conditions. The field trial had three plots per line with ten plants distributed in 1 m. This wide diversity is being evaluated considering morpho-agronomic characters. This work is part of the BRESOV project funded by the EU (Grant agreement ID: 774244 ). This video was recorded at the SERIDA facilities, Villaviciosa, Asturias, Spain in July 2021. DOI  10.5281/zenodo.5573917 Also available in the link:  https://www.youtube.com/watch?v=dJa9UDFQnOE&t=5s

opencc-by-4.0Nov 2024View details →

Public Utility Data Liberation Project (PUDL) Data Release

<h2>v2025.10.0 (2025-10-14)</h2> This is a regular monthly data release, primarily intended to ensure that PUDL has the most up-to-date EIA-860M data. It also happens to include final EIA-860 data for 2024, and some newly integrated EIA-923 financial data and PHMSA natural gas data. See below for details. <h3>Expanded Data Coverage</h3> <h4>EIA-860</h4> <ul> <li> Updated EIA-860 with final release data from 2024. See issue <a href="https://github.com/catalyst-cooperative/pudl/issues/4616">#4616</a> and PR <a href="https://github.com/catalyst-cooperative/pudl/pull/4617">#4617</a>. </li> </ul> <h4>EIA-860M</h4> <ul> <li> Updated EIA-860M monthly generator report with newly published data for August of 2025. See issue <a href="https://github.com/catalyst-cooperative/pudl/issues/4639">#4639</a> and PR <a href="https://github.com/catalyst-cooperative/pudl/pull/4638">#4638</a>. </li> </ul> <h3>New Data</h3> <h4>PHMSA</h4> <ul> <li> Added eight transformed table containing annual data from PHMSA natural gas distributors from 1970 to the present. Note that these containing mostly numeric values are named as <code>_core</code> - indicating that these tables have not been fully cleaned and validated. We’ve published these tables to make the 50+ years of PHMSA data we’ve extracted and mapped available for others to use and for contributors to more easily improve incrementally. See <a href="https://github.com/catalyst-cooperative/pudl/issues/3770">#3770</a> and <a href="https://github.com/catalyst-cooperative/pudl/pull/4005">#4005</a>. </li> <li> The first cleaned table, <code>core_phmsagas__distribution_operators</code> has been added to our PUDL database. Thanks to <a href="https://github.com/sponsors/seeess1">@seeess1</a> for all of your work on this! </li> </ul> <h4>EIA 923</h4> <ul> <li> Thanks to contributions from <a href="https://github.com/sponsors/alexclippinger">@alexclippinger</a>, we’ve added cleaned EIA923 Schedule 8B Financial Information to the PUDL database as <a href="https://catalystcoop-pudl.readthedocs.io/en/v2025.10.0/data_dictionaries/pudl_db.html#i-core-eia923-yearly-byproduct-expenses-and-revenues">_core_eia923__yearly_byproduct_expenses_and_revenues</a>. Once harvested, this table will be replaced with a well-normalized version of the same data, but it is being published in this form until then. See <a href="https://github.com/catalyst-cooperative/pudl/issues/4099">#4099</a> and <a href="https://github.com/catalyst-cooperative/pudl/issues/2448">#2448</a>, and <a href="https://github.com/catalyst-cooperative/pudl/pull/4636">#4636</a>. </li> </ul> <h3>Documentation</h3> <ul> <li> Added data source pages for: <ul> <li> <a href="https://catalystcoop-pudl.readthedocs.io/en/v2025.10.0/data_sources/censuspep.html">Population Estimates Program's (PEP) Federal Information Processing Series (FIPS) Codes</a>; see issue <a href="https://github.com/catalyst-cooperative/pudl/issues/4375">#4375</a> and PR <a href="https://github.com/catalyst-cooperative/pudl/pull/4622">#4622</a>. </li> <li> <a href="https://catalystcoop-pudl.readthedocs.io/en/v2025.10.0/data_sources/sec10k.html">U.S. Securities and Exchange Commission (SEC) Form 10-K</a>; see issue <a href="https://github.com/catalyst-cooperative/pudl/issues/4329">#4329</a>, <a href="https://github.com/catalyst-cooperative/pudl/issues/4347">#4347</a> and PR <a href="https://github.com/catalyst-cooperative/pudl/pull/4562">#4562</a>. </li> </ul> </li> </ul> <h3>New Data Tests & Data Validations</h3> <ul> <li> After investigating some modest discrepancies between our imputed hourly electricity demand and prior work by <a href="https://github.com/sponsors/truggles">@truggles</a> & <a href="https://github.com/sponsors/awongel">@awongel</a>, we’re removing the “EXPERIMENTAL” warning label that we had on those tables. See <a href="https://github.com/catalyst-cooperative/pudl-examples/pull/10">our discussion about the imputation results in the PUDL Examples repo</a>. The <a href="https://www.kaggle.com/code/catalystcooperative/06-pudl-imputed-electricity-demand">associated notebook is available on Kaggle</a> This relates to the PUDL imputed demand values in following tables: <ul> <li> <a href="https://catalystcoop-pudl.readthedocs.io/en/v2025.10.0/data_dictionaries/pudl_db.html#out-eia930-hourly-operations">out_eia930__hourly_operations</a> </li> <li> <a href="https://catalystcoop-pudl.readthedocs.io/en/v2025.10.0/data_dictionaries/pudl_db.html#out-eia930-hourly-subregion-demand">out_eia930__hourly_subregion_demand</a> </li> <li> <a href="https://catalystcoop-pudl.readthedocs.io/en/v2025.10.0/data_dictionaries/pudl_db.html#out-eia930-hourly-aggregated-demand">out_eia930__hourly_aggregated_demand</a> </li> </ul> </li> </ul> <h3>Deprecations</h3> <ul> <li> We have finally shut down our long-suffering <a href="https://datasette.io">Datasette</a> deployment, but are still working on achieiving feature parity in the new <a href="https://viewer.catalyst.coop">PUDL Data Viewer</a>. We have <a href="https://github.com/catalyst-cooperative/eel-hole/issues/36">an epic tracking our progress</a>. See issue <a href="https://github.com/catalyst-cooperative/pudl/issues/4481">#4481</a> and PR <a href="https://github.com/catalyst-cooperative/pudl/pull/4605">#4605</a> for the removal of Datasette references within the main PUDL repo. </li> </ul> <h2>Other PUDL v2025.10.0 Resources</h2> <ul> <li><a href="https://catalystcoop-pudl.readthedocs.io/en/v2025.10.0/data_dictionaries/pudl_db.html">PUDL v2025.10.0 Data Dictionary</a></li> <li><a href="https://catalystcoop-pudl.readthedocs.io/en/v2025.10.0/">PUDL v2025.10.0 Documentation</a></li> <li><a href="https://registry.opendata.aws/catalyst-cooperative-pudl/">PUDL in the AWS Open Data Registry</a></li> <li>PUDL v2025.9.1 in a free, public AWS S3 bucket: s3://pudl.catalyst.coop/v2025.10.0/</li> <li>PUDL v2025.9.1 in a requester-pays GCS bucket: gs://pudl.catalyst.coop/v2025.10.0/</li> <li><a href="https://doi.org/10.5281/zenodo.17352325">Zenodo archive of the PUDL GitHub repo for this release</a></li> <li><a href="https://github.com/catalyst-cooperative/pudl/releases/tag/v2025.10.0">PUDL v2025.10.0 release on GitHub</a></li> <li><a href="https://pypi.org/project/catalystcoop.pudl/2025.10.0">PUDL v2025.10.0 package in the Python Package Index (PyPI)</a></li> </ul> <h2>Contact Us</h2> If you're using PUDL, we would love to hear from you! Even if it's just a note to let us know that you exist, and how you're using the software or data. Here's a bunch of different ways to get in touch: <ul> <li><a href="https://github.com/catalyst-cooperative">Follow us on GitHub</a></li> <li>Use the <a href="https://github.com/catalyst-cooperative/pudl/issues">PUDL Github issue tracker</a> to let us know about any bugs or data issues you encounter</li> <li><a href="https://github.com/orgs/catalyst-cooperative/discussions">GitHub Discussions</a> is where we provide user support.</li> <li>Watch our <a href="https://github.com/orgs/catalyst-cooperative/projects/9">GitHub Project</a> to see what we're working on.</li> <li>Email us at <a href="mailto:hello@catalyst.coop">hello@catalyst.coop</a> for private communications.</li> <li>On Mastodon: <a href="https://mastodon.energy/@catalystcoop">@CatalystCoop@mastodon.energy</a></li> <li>On BlueSky: <a href="https://bsky.app/profile/catalyst.coop">@catalyst.coop</a></li> <li>On Twitter: <a href="https://twitter.com/CatalystCoop">@CatalystCoop</a></li> <li>Connect with us <a href="https://www.linkedin.com/company/catalyst-cooperative/">on LinkedIn</a></li> <li>Play with our data and notebooks <a href="https://www.kaggle.com/catalystcooperative">on Kaggle</a></li> <li>Combine our data with ML models <a href="https://huggingface.co/catalystcooperative">on HuggingFace</a></li> <li>Learn more about us on our website: <a href="https://catalyst.coop">https://catalyst.coop</a></li> <li>Subscribe to our announcements list for <a href="https://catalyst.coop/updates">email updates</a>.</li> </ul>

opencc-by-sa-4.0Oct 2018View details →

Vectors for Goode's Homolosine projection

This dataset contains useful vector maps to work with with Goode's Homolosine projection. The list of files included are: <ul> <li> CounterDomain.geojson - a polygonal approximation of the Homolosine projection counter-domain. This can be used to fix vectors wrongly  projected by programmes that consider the counter-domain to be infinite. It can also be used to represent the seas in global mapping.</li> <li> ParallelsMeridians.geojson - a set of meridians and parallels to be used in the creation of global maps.</li> <li>Homolosine.crs - the PROJ string defining the Homolosine projection (referenced by the GeoJSON slides)</li> <li>LICENCE - full text of the licence (EUPL-1.2)</li> </ul> These datasets were generated with the open souce programme homolosine-vectors, available at: <a href="https://gitlab.com/ldesousa/homolosine-vectors">https://gitlab.com/ldesousa/homolosine-vectors</a>

opencc-by-4.0Mar 2019View details →

Online Real-Time Delphi Survey for the research project "MENARA" - Compilation of all Comments to Closed and Open Questions

Looking into the Futures: Delphi Survey about the MENA region In order to get a more realistic overview of the situation and trends, of the potentials, problems and potentials of the countries of the MENA region a Real Time Delphi survey was conducted. This is an important tool of modern future research. It was managed by the IZT- Institute for Future Studies in Berlin. A group of 139 experts and researchers from different institutes and organizations were invited to participate at the Online Real-Time Delphi Survey (RTD) about possible and likely futures of the MENA region. The experts were asked to answer questions and provide their opinions on twelve topics such as social unrest, youth unemployment, urbanization, gender equality, security etc. In this dataset all comments to the closed and the open questions are compiled. The output was one of the basic material used for the creation of future regional scenarios for mid-term (2025) and long-term (2050) time horizons. Focus scenarios were produced in order to exemplify selected characteristic and important future options, in terms of chances and risks (e.g. energy futures).

opencc-by-4.0Oct 2022View details →

Survey for coordinators of agroecology research projects

This dataset (Project_coordinators_survey.csv) contains data related to the survey launched within the Task 1.3 of AE4EU project for the coordinators of agroecology research projects funded by European, transnational, and national programmes identified in the mapping activities. Together with the dataset, the structure of the questionnaire related to this survey is also provided (Project_coordinators_questionnaire_structure.pdf). Answers from respondents were anonymised before the publication. Informed consent was obtained from participants to the survey to use their answers and quotations for research and publication

opencc-by-4.0Oct 2022View details →

List of validated primers of gilthead sea bream (Sparus aurata) and European seabass (DIcentrarchus labrax) developed in PerformFISH project (D2.3)

The document contains all the primers identified for the screening of genes tested for their potential as biomarkers  to predict quality performance in gilthead sea bream and European sea bass larvae and juveniles in the context of PERFORMFISH (WP2). The spreadsheet has the following information: Pathway, phisiologic process in which the gene is involved; name of protein that  gene produces; gene code; acession nº, code given in the consulted databases and the sequence extracted for primer design; FW and RV primer, forward and reverse primer sequence specific for target gene; melt temperature,  optimized temperature that primers work at ; amplicon size, size in base pairs of the product produced  with the  primers; eff%, efficency of primers; r2; source, the origin of the primers, "in house" or "literature" (including available DOI.  Each pair of primers are classified using a "traffic light" system indicating their validation status.

WoS and Scopus records for the bibliometric analysis in the output D2.2 Digital transformation of research and innovation roadmap of the reSEArch-EU project

These files represent the exported WoS and Scopus records, used in the output D2.2 Digital transformation of research and innovation roadmap  of the Horizont project reSEArch-EU, implemented by the SEA-EU university alliance.

Report on a Survey among Organisers of Citizen Science Projects - Dataset and Report

In this publication you will find the report on a survey among organisers of citizen science projects developed by Michael Strähle & Christine Urban (alphabetical order), Wissenschaftsladen Wien - Science Shop Vienna. You will also find, the datasets which contain the responses obtained from the very short questionnaire "VSQ" and the responses used for the report.

[Dataset] Does Volunteer Engagement Pay Off? An Analysis of User Participation in Online Citizen Science Projects

Corresponding dataset for the publication "Does Volunteer Engagement Pay Off? An Analysis of User Participation in Online Citizen Science Projects", a conference paper for the conference CollabTech 2022: <a href="https://link.springer.com/book/10.1007/978-3-031-20218-6">Collaboration Technologies and Social Computing</a> and published as part of the <a href="https://link.springer.com/bookseries/558">Lecture Notes in Computer Science</a> book series (LNCS,volume 13632) <a href="https://link.springer.com/chapter/10.1007/978-3-031-20218-6_5">here</a>. Usernames have been anonymised. The structure of the dataset is as follows: Annotations  List of annotations made per day for each of the analysed projects. <code>annotations.csv </code> Comments  Total list of comments with several data fields (i.e., comment id, text, reply_user_id) <code>comments.csv </code> Rolechanges  List of roles per user to determine number of role changes  <code>478_rolechanges.csv</code> <code>1104_rolechanges.csv</code> <code>...</code> Totalnetworkdata  Network data (edge and node sets) for the given projects (without time slices). Edges  <ul> <li> <code>478_edges.csv</code> </li> <li> <code>1104_edges.csv</code> </li> </ul> Nodes  <ul> <li> <code>478_nodes.csv</code>  </li> <li> <code>1104_nodes.csv</code>  </li> </ul> Trajectories  Network data (edge and node sets) for the given projects and all time slices (Q1 2016 - Q4 2021) 478  <ul> <li>Edges  <ul> <li> <code>edges_4782016_q1.csv</code> </li> <li> <code>edges_4782016_q2.csv</code> </li> <li> <code>edges_4782016_q3.csv</code> </li> <li> <code>edges_4782016_q4.csv</code> </li> </ul> </li> <li> ... </li> <li>Nodes  <ul> <li><code>nodes_4782016_q1.csv</code></li> <li> <code>nodes_4782016_q4.csv</code> </li> <li> <code>nodes_4782016_q3.csv</code> </li> <li> <code>nodes_4782016_q2.csv</code> </li> <li> <code>...</code> </li> </ul> </li> </ul>   1104  <ul> <li> Edges  <ul> <li> <code>...</code> </li> </ul> </li> <li> Nodes  <ul> <li> <code>...</code> </li> </ul> </li> <li> <code>...</code> </li> </ul>

Icons of the tipping points in the Earth System from the H2020 COMFORT project (820989)

Triple threat processes and/or other forcings can lead to changes in the ocean happening fast and abruptly. These changes, referred to as “tipping points”, are critical thresholds in a marine system that, when exceeded, can lead to a significant change in the state of the system, which often can be irreversible. This product has been prepared with the financial support of Norges forskningsråd (Research Council of Norway) (309382) and the European Union’s Horizon 2020 research and innovation programme under grant agreement No 820989 (project COMFORT, Our common future ocean in the Earth system – quantifying coupled cycles of carbon, oxygen, and nutrients for determining and achieving safe operating spaces with respect to tipping points). The work reflects only the author’s/authors’ view; the European Commission and their executive agency are not responsible for any use that may be made of the information the work contains.

Availability of information on citizen science activities, checked against the Activities & Dimensions Grid of Citizen Science on the basis of some projects

The research resulting in this report aimed at answering the following questions: <ul> <li> Which information on citizen science activities is online available that matches the Activity & Dimension Grid of Citizen Science or goes beyond it?   </li> <li> Is there any contradictory information? </li> <li> What can be the reason for the availability or non-availability of information about citizen science activities? </li> <li> How does/could this impact on the CS Track’s recommendations? </li> </ul> The corresponding dataset consists of the results of a keyword-based search in the WP2 project database. The information retrieval resulted in 3318 projects on which information is available in German or English. More information on this research can be found in D2.2 section 3.2.

Demographic factors and the environmental Kuznets curve: global plastic pollution by 2050 could be 2 to 4 times worse than projected

These data are made of two files. One file provides the observed data we collected and cleaned from the World Bank database. The second file provides the simulation results from the STIRPAT model we designed based on the observed data abovementioned. Our results can be summarised as follows: Since 2015, the detrimental effects of plastic pollution have attracted media, public, and governmental attention. Considering economic growth is inevitable and a key driver of plastic contamination, it is worthwhile to analyze the environmental Kuznets curve (EKC) relationship between economic development and plastic pollution. To this end, we contribute by being the first to (i) use the Stochastic Impacts by Regression on Population, Affluence, and technology model (STIRPAT model) to investigate this EKC relationship; (ii) provide a comprehensive analysis of how demographic factors affect plastic pollution; and (iii) use panel model techniques to examine the drivers of plastic pollution. Our empirical results support an inverted U-shaped relationship between plastic pollution and income. They show that at current trends, global plastic pollution (that is, annual discard of inadequately managed plastic waste) is expected to grow from 52 million tons per year in 2020 to 257 million tons per year in 2050.

Analysis of the content of the H2020 project websites related to LEAs and IA.

This is the dataset used in the article entitled "The disconnect between the goals of trustworthy AI for law enforcement and the EU research agenda". You can find more information about the results obtained, as well as the methodology used in the paper.

Codes to replicate statistical analysis in UPLIFT project Deliverable 2.4 Synthesis report, Chapter 6

Policies attempting to mitigate the effects of urban inequality, often disregard affected citizens’ experiences, and thus fail to achieve maximum impact. By incorporating these perspectives into the policy design process, the project "Urban PoLicy Innovation to address inequality with and for Future generaTions" (UPLIFT), funded under the EU Horizon 2020 program aims to find innovative interventions in a bottom-up approach. The aims of UPLIFT project are to understand patterns and trends of inequality across Europe and to understand how individuals experience and adapt to inequality through participatory research. Moreover the project will together with the communities in four locations, co-design a policy tool aimed at addressing and reducing inequality and socio-economic divisions. The activity and results of the project can be followed at <a href="https://www.uplift-youth.eu/">https://www.uplift-youth.eu/</a>. Deliverable 2.4 (Synthesis report: socioeconomic inequalities in different urban contexts) is the final deliverable of work package 2 of the UPLIFT project, which aims to synthetize the main outcomes of the urban reports that described the policy environment around vulnerable individuals in the fields of education, employment and housing in 16 functional urban areas of the EU. In addition section 6.2 "Statistical analysis of linkages between economic development of cities, their public policy performance and inequality outcomes" of the report provides a statistical analysis of how local economic competitiveness and the local policy context affect urban deprivation and inequality among the young in European cities. The analysis is based on data from 2006, 2009, 2012, 2015 and 2019 of the Quality of Life in European Cities survey.

Data for Project 'Feasibility, Usability and Acceptance of a Newly Developed Exergame-Based Training Concept for Older Adults with Mild Neurocognitive Disorder - A Pilot Randomized Controlled Trial'

Data for Project 'Feasibility, Usability and Acceptance of a Newly Developed Exergame-Based Training Concept for Older Adults with Mild Neurocognitive Disorder - A Pilot Randomized Controlled Trial' (trial registered at clinicaltrials.gov (<a href="https://clinicaltrials.gov/ct2/show/NCT04996654">NCT04996654</a>; date of registration: 11 July 2021), consisting of: (1) the original and complete data set for all primary outcomes ('Data_Primary-Outcomes_Brain-IT-Pilot-Feasibility-RCT_for-publication.xlsx'); (2) the original and complete data set for all secondary outcomes ('Data_Secondary-Outcomes_Brain-IT-Pilot-Feasibility-RCT_for-publication.xlsx'); (3) the original and complete data set for all other outcomes (i.e. baseline factors (demographic data, type of usual care interventions) and training heart rate; 'Data_Other-Outcomes_Brain-IT-Pilot-Feasibility-RCT_for-publication.xlsx'); (4) folder including the raw and processed heart rate variability (HRV) and electroencephalography (EEG) data for all participants and measurements (HRV-and-EEG_raw-and-processed-data.zip); (5) a corresponding README file including (a) general information, (b) data and file overview, (c) sharing and access information, (d) methodological information, and (e) data-specific information.

OpenChart-SE: A corpus of artificial Swedish electronic health records for imagined emergency care patients written by physicians in a crowd-sourcing project

Electronic health records (EHRs) are a rich source of information for medical research and public health monitoring. Information systems based on EHR data could also assist in patient care and hospital management. However, much of the data in EHRs is in the form of unstructured text, which is difficult to process for analysis. Natural language processing (NLP), a form of artificial intelligence, has the potential to enable automatic extraction of information from EHRs and several NLP tools adapted to the style of clinical writing have been developed for English and other major languages. In contrast, the development of NLP tools for less widely spoken languages such as Swedish has lagged behind. A major bottleneck in the development of NLP tools is the restricted access to EHRs due to legitimate patient privacy concerns. To overcome this issue we have generated a citizen science platform for collecting artificial Swedish EHRs with the help of Swedish physicians and medical students. These artificial EHRs describe imagined but plausible emergency care patients in a style that closely resembles EHRs used in emergency departments in Sweden. In the pilot phase, we collected a first batch of 50 artificial EHRs, which has passed review by an experienced Swedish emergency care physician. We make this dataset publicly available as OpenChart-SE corpus (version 1) under an open-source license for the NLP research community. The project is now open for general participation and Swedish physicians and medical students are invited to submit EHRs on the project website (<a href="https://github.com/Aitslab/openchart-se">https://github.com/Aitslab/openchart-se</a>), where additional batches of quality-controlled EHRs will be released periodically.     Dataset content OpenChart-SE, version 1 corpus (txt files and and dataset.csv) The OpenChart-SE corpus, version 1, contains 50 artificial EHRs (note that the numbering starts with 5 as 1-4 were test cases that were not suitable for publication). The EHRs are available in two formats, structured as a .csv file and as separate textfiles for annotation. Note that flaws in the data were not cleaned up so that it simulates what could be encountered when working with data from different EHR systems. All charts have been checked for medical validity by a resident in Emergency Medicine at a Swedish hospital before publication.   Codebook.xlsx The codebook contain information about each variable used. It is in XLSForm-format, which can be re-used in several different applications for data collection.   suppl_data_1_openchart-se_form.pdf OpenChart-SE mock emergency care EHR form.   suppl_data_3_openchart-se_dataexploration.ipynb This jupyter notebook contains the code and results from the analysis of the OpenChart-SE corpus.   More details about the project and information on the upcoming preprint accompanying the dataset can be found on the project website (<a href="https://github.com/Aitslab/openchart-se">https://github.com/Aitslab/openchart-se</a>).

opencc-by-4.0Jan 2023View details →

Model agreement and trend analysis data associated to the publication: "Impact of climate change on site characteristics of eight major astronomical observatories using high-resolution global climate projections until 2050"

This dataset is associated with the following publication: Haslebacher, C., Demory, M.-E., Demory, B.-O., Sarazin, M., and Vidale, P. L., “Impact of climate change on site characteristics of eight major astronomical observatories using high-resolution global climate projections until 2050. Projected increase in temperature and humidity leads to poorer astronomical observing conditions”, Astronomy and Astrophysics, vol. 665, 2022. doi:10.1051/0004-6361/202142493. In the folder 'model_agreement', there are pickle files from which a python dictionary can be extracted with: <pre><code>with open('mypklfile.pkl', 'rb') as myfile: dload = pickle.load(myfile)</code></pre> Pickle files ending with '_d_obs_ERA5.pkl' contain in situ data and ERA5 data. Pickle files ending with 'd_model.pkl' contain PRIMAVERA model data. A few explanations: - 'ds_sel': contains monthly timeseries of selected intersecting data - 'ds_taylor': contains data used for the Taylor diagram (Figs. 4-10) - 'ds_mean_month': contains seasonal cycle for plotting (Figs. 4-10) - 'ds_mean_year': contains yearly timeseries for plotting (Figs. 4-10)  The subfolder 'median_nc_u_v_t' contains NETCDF files with the median and interquartile range of the wind speed in u and v direction, the temperature and geopotential height. This was used for Figs. G1-G8 and to calculate the refractive index structure constant Cn2. The subfolder 'skill_score_classification' contains csv files with the sorted skill score classifications. The column headers are: model_name, skill score, correlation coefficient, standard deviation, centred root mean square error. The folder 'trend_analysis' contains for each variable csv files of ERA5 and PRIMAVERA monthly time series used for trend analysis, pdf files of analysis summaries, csv files of Bayesian analysis results and png files of longitude-latitude maps of trends (analysed with linear regression). Additionally, there is a csv file of averaged in situ pressures. Code that generated and used this data is available on github: <a href="https://github.com/CarolineHaslebacher/Astroclimate-future-project">https://github.com/CarolineHaslebacher/Astroclimate-future-project</a>

opencc-by-4.0Jan 2023View details →

IMMERSE Horizon 2020 Project Downstream User Toolbox – data for tutorial on impact of wave coupling on surface particle dispersion simulations

Exemplary data for tutorial on impact of wave coupling on surface particle dispersal simulations <a href="https://github.com/immerse-project/Downstream-Users-Toolbox/tree/main/T8.3_WaveCoupling_ParticleTransport_UniU">https://github.com/immerse-project/Downstream-Users-Toolbox/tree/main/T8.3_WaveCoupling_ParticleTransport_UniU</a> created as part of the downstream user toolbox of the IMMERSE Horizon 2020 project (<a href="https://immerse-ocean.eu/">https://immerse-ocean.eu/</a>). In the tutorial the impact of new options for the representation of wave-current interactions in the NEMO ocean model (<a href="https://www.nemo-ocean.eu/">https://www.nemo-ocean.eu/</a>) on surface particle simulations are tested in a case study for the Mediterranean Sea. The tutorial consists of two jupyter notebooks: Parcels_CalcTraj.ipynb and CompTraj_uncoupledVScoupled.ipynb. Parcels_CalcTraj.ipynb calculates Lagrangian particle trajectories based on velocity output  from ocean only as well as coupled ocean-wave model simulation by making use of the OceanParcels software (<a href="https://oceanparcels.org/">https://oceanparcels.org/</a>). CompTraj_uncoupledVScoupled.ipynb compares dispersal statistics of Lagrangian particle trajectories calculated from ocean-only vs coupled ocean-wave model simulations. This repository contains the surface velocity and ocean model grid data needed to run Parcels_CalcTraj.ipynb, as well as the trajectory data produced by Parcels_CalcTraj.ipynb, which is needed to run CompTraj_uncoupledVScoupled.ipynb. The surface velocity data stems from two simulations with a regional high-resolution (1/24° horizontal resolution) model configuration for the Mediterranean Sea: a coupled ocean-wave model simulation and a complimentary ocean-only simulation. These model simulations make use of the NEMO v4.2-RC ocean model, the Wave Watch 3 v.6.07 wave model, the OASIS3-MCT coupler, and ECMWF atmospheric fields; they are described in detail in IMMERSE deliverable D5.7 “Assessment of wave-current effects on the circulation in theMed-MFC system”.

opencc-by-4.0Feb 2023View details →

The e-NDP project : collaborative digital edition of the Chapter registers of Notre-Dame of Paris (1326-1504). Ground-truth for handwriting text recognition (HTR) on late medieval manuscripts.

The <a href="https://endp.hypotheses.org/">e-NDP project</a>, funded by the ANR, is led by the <a href="https://lamop.hypotheses.org/6870">LaMOP</a> (Julie Claustre and Darwin Smith). The project's partners are the Archives nationales, the Bibliothèque nationale de France (Department of Manuscripts, Bibliothèque de l'Arsenal), the École nationale des chartes and the Bibliothèque Mazarine. The e-NDP project aims at renewing our knowledge on Notre-Dame de Paris cathedral through the creation of a collaborative digital edition of the registers of its Chapter (1326-1504, AN LL 105-128), the community of 51 canons meeting three times a week on set days to take all administrative, financial and practical decisions pertaining to the cathedral, its estate and the society living in its cloister. This corpus has never been the object of a comprehensive study to understand the workings and history of this urban enclave and powerful community. The collaborative digital edition is based on a process of handwriting text recognition (HTR), tested and supervised by scholars, researchers and engineers combining expertise in Medieval history, paleography, philology and digital humanities. The edition shall allow a better insight into the Chapter’s administration, into its economical and political power within Paris, and the relationships it maintained with other institutions in the city.   Section 1 : The e-NDP ground-truth dataset for Handwriting text recognition. The full e-NDP corpus kept today in the French National Archives and was entirely digitized and described in its <a href="https://www.siv.archives-nationales.culture.gouv.fr/siv/rechercheconsultation/consultation/ir/consultationIR.action?formCaller=GENERALISTE&irId=FRAN_IR_059635">catalog</a> in 2022. The first major goal of the e-NDP projet is to propose a first automatic transcription of the 14k pages composing the 26 chapter registers. To achieve this goal representative samples from each one of the volumes were selected and transcribed in order to train a specialized HTR model able to propose a high quality automatic transcription. The collected ground-truth released on this repository currently has 512 pages from the 26 registers of the cathedral chapter preserved in the National Archives (LL105 - LL128, 1326-1504). The transcriptions were manually completed in two rounds by a group of 12 contributors, historians and paleographers, over the course of 2021-2022 using <a href="https://escriptorium.paris.inria.fr/">eScriptorium </a>as annotation environment.     Ground-truth features : Number of hands : according to our estimates no fewer than 18 main hands were involved in the writing of the registers during the medieval period.  Language : More than 98% of the content of the registers was written in Latin, the rest in French. The exact percentage is hard to estimate because the vernacular language is often used in formulae, notes and comments. It is rare to find entire pages or blocks written in French.  Script family : The registers were written using a Cursive script (ca. late XIIIe - XVIe). Documental typology : The volumes containing the chapter conclusions were conceived to serve as memorial records, but above all as documents for regular use and consultation in the daily practice of administration and management. In diplomatics the notion of "documentary manuscripts" is used to describe this kind of sources also by opposition to books and litterary or normative manuscripts. <table align="center"> <caption>Ground truth statistics</caption> <tbody> <tr> <th>Text units</th> <th>Count</th> </tr> <tr> <td>Pages</td> <td>512</td> </tr> <tr> <td>Annotated regions (see section 2)</td> <td>2448</td> </tr> <tr> <td>Lines of text</td> <td>34231</td> </tr> <tr> <td>Tokens</td> <td>205083</td> </tr> <tr> <td>Characters</td> <td>3320407</td> </tr> </tbody> </table>   Rules of transcription : <ul> <li>The abbreviations have been resolved, both those by suspension (<code>facimꝰ</code> ---> <code>facimus</code>) and by contraction (<code>dñi</code> --> <code>domini</code>). Likewise, those using conventional signs (<code>⁊</code> --> <code>et</code> ; <code>ꝓ</code> --> <code>pro</code>) have been resolved. </li> <li>The named entities (names of persons, places and institutions) have been <code>capitalized</code>. The beginning of a block of text as well as the original capitals used by the notary are also capitalized.</li> <li>The consonantal <code>i</code> and <code>u</code> characters have been transcribed as <code>j</code> and <code>v</code> in both French and Latin.</li> <li>The punctuation marks used in the text: <code>.</code> and <code>/</code> have been transcribed, but the transcription has not been standardized with modern punctuation.</li> <li>Corrections and words that appear cancelled in the manuscript have been transcribed surrounded by the sign <code>$</code> at the beginning and at the end.</li> <li>More specific transcription rules can be found into the file <code>transcription_guidelines.pdf</code></li> </ul>   Section 2. e-NDP Layout Segmentation. Layout segmentation is a compulsory step before HTR recognition in order to distinguish sections and regions inside a document. This process intend to separate interdependant page zones to produce a recognition in a section-sequence order and not in a line-sequence order which mix textual and peri-textual content. The regions of 364 pages (see <code>GT-layout_list</code>) of the e-NDP corpus were annotated using a 5 sections vocabulary (see <code>endp_layout_regions</code>) in order to describe the page distribution in all the 26 volumes : <ol> <li>Block : All the central text blocks, that normally corresponds to the main content called "conclusions" in registers.</li> <li>Liste : List of names of the canons who were present during the meeting. Normally located before the conclusions.</li> <li>Entrée : Marginal notes or entries to inform about the content of conclusions.</li> <li>Date : Paragraph contending the date. Normally at the head of a conclusion, but separate of the main body.</li> <li>Numérotation : Page numbers in roman or arabic. Usually appear in the top corners of the pages.</li> </ol> <table align="center"> <caption>Layout GT statistics</caption> <tbody> <tr> <th>Region</th> <th>Count</th> </tr> <tr> <td>block</td> <td>833</td> </tr> <tr> <td>liste</td> <td>431</td> </tr> <tr> <td>date</td> <td>448</td> </tr> <tr> <td>entrée</td> <td>205</td> </tr> <tr> <td>numérotation</td> <td>531</td> </tr> </tbody> </table>   Section 3. The e-NDP HTR modeling. The e-NDP project has progressively trained several HTR models adapted to work on late medieval cursive in order to accelerate the production of ground truth. Currently the best model delivers an average CER (Character error ratio) of 9.7% in handwriting recognition on the 26 registers (see <code>endp_learning_curve</code>) and can serve as generalist model for other manuscripts of the same period and similar script family. These models and their training implementation details can be found in the project's github <a href="https://github.com/chartes/e-NDP_HTR">repository</a>.  Additionally, the automatic HTR transcriptions of the 26 registers (14k pages, 4.5M tokens) enriched with lexical and semantical information has been the subject of a first <a href="https://nosketch-engine.lamop.fr/#dashboard?corpname=endp">online publication</a> using the NoSketch engine that allows advanced data mining based on the combination of data, metadata and NLP features.    Section 4. Dataset content. This zip dataset contains : - <code>HTR_ground_truth</code> : Two folders containing the jpg / jpeg images and their curated transcriptions in PAGE XML format. - <code>images_docs</code> : 4 files illustrating the different phases of the project (list of GT for layout segmentation, layout ontologie, transcription guideline and HTR evaluation curves)