Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
1,454
datasets available to search
ShareScore release 0.9.0
Dataset results
1,454 results for “large scale”
Classification Of Large-Scale Environments That Drive The Formation Of Mesoscale Convective Systems Over Southern West Africa
<p>This is a set of datasets used for the publication of the article titled: Classification Of Large-Scale Environments That Drive The Formation Of Mesoscale Convective Systems Over Southern West Africa</p>
Loghub-2.0: a collection of large-scale datasets for log parsing
<p>See dataset details: <a href="https://github.com/logpai/LogPub">https://github.com/logpai/Loghub-2.0</a></p> <p>The datasets are freely available for research or academic work, subject to the following condition: For any usage or distribution of the LogPub datasets, please refer to the LogPub repository URL (<a href="https://github.com/logpai/LogPub">https://github.com/logpai/Loghub-2.0</a>) and cite the LogPub paper (<a href="https://arxiv.org/abs/2308.10828">A Large-scale Evaluation for Log Parsing Techniques: How Far are We?</a>) where applicable.</p>
Large-scale genomic data reveal the phylogeny and evolution of owlet moths (Noctuoidea)
<p>Owlet moths (Noctuoidea; ~43–45K described species; Nieukerken et al. 2011) are one of the most ecologically diverse and speciose superfamilies of animals. Moreover, they comprise some of the world's most notorious pests of agriculture and forestry. Despite their contributions to terrestrial biodiversity and impacts on ecosystems and economies, the evolutionary history of Noctuoidea remains unclear because the superfamily lacks a statistically robust phylogenetic and temporal framework. We reconstructed the phylogeny of Noctuoidea using data from 1,234 genes (946.4 kb of nucleotides) obtained from the genome and transcriptome sequences of 76 species representing all six families. Relationships among the families of Noctuoidea were well resolved and consistently recovered based on both concatenation and gene coalescence approaches, supporting the following relationships: Oenosandridae + (Notodontidae + (Erebidae + (Nolidae + (Euteliidae + Noctuidae)))). A Yule tree prior with three unlinked molecular clocks was identified as the preferred BEAST analysis using marginal-likelihood estimations. The crown age of Noctuoidea was estimated to be 74.5 Ma, with most families originating before the end of Paleogene (23 Ma). Our study provides the first statistically robust phylogenetic and temporal framework for Noctuoidea, including all families of owlet moths, based on large-scale genomic data.</p>
A Large-Scale AIS Datset from Finnish Water
<p>The proposed AIS dataset encompasses a substantial temporal span of 20 months, spanning from April 2021 to December 2022. This extensive coverage period empowers analysts to examine long-term trends and variations in vessel activities. Moreover, it facilitates researchers in comprehending the potential influence of external factors, including weather patterns, seasonal variations, and economic conditions, on vessel traffic and behavior within the Finnish waters.</p> <p>This dataset encompasses an extensive array of data pertaining to vessel movements and activities encompassing seas, rivers, and lakes. Anticipated to be comprehensive in nature, the dataset encompasses a diverse range of ship types, such as cargo ships, tankers, fishing vessels, passenger ships, and various other categories.</p> <p>The AIS dataset exhibits a prominent attribute in the form of its exceptional granularity with a total of 2 293 129 345 data points. The provision of such granular information proves can help analysts to comprehend vessel dynamics and operations within the Finnish waters. It enables the identification of patterns and anomalies in vessel behavior and facilitates an assessment of the potential environmental implications associated with maritime activities.</p> <p>Please cite the following publication when using the dataset:</p> <p><em>TBD</em></p> <p>The publication is available at: <em>TBD</em></p> <p>A preprint version of the publication is available at <em>TBD</em></p> <h3>csv file structure</h3> <h4><a href="#yyyy-mm-dd-locationcsv"></a>YYYY-MM-DD-location.csv</h4> <p>This file contains the received AIS position reports. The structure of the logged parameters is the following: <code>[timestamp, timestampExternal, mmsi, lon, lat, sog, cog, navStat, rot, posAcc, raim, heading]</code></p> <p><code><strong>timestamp</strong></code> I beleive this is the UTC second when the report was generated by the electronic position system (EPFS) (0-59, or 60 if time stamp is not available, which should also be the default value, or 61 if positioning system is in manual input mode, or 62 if electronic position fixing system operates in estimated (dead reckoning) mode, or 63 if the positioning system is inoperative).</p> <p><code><strong>timestampExternal</strong></code> The timestamp associated with the MQTT message received from <a href="http://www.digitraffic.fi/" target="_blank" rel="nofollow noreferrer noopener">www.digitraffic.fi</a>. It is assumed this timestamp is the Epoch time corresponding to when the AIS message was received by digitraffic.fi.</p> <p><code><strong>mmsi</strong></code> MMSI number, Maritime Mobile Service Identity (MMSI) is a unique 9 digit number that is assigned to a (Digital Selective Calling) DSC radio or an AIS unit. Check <a href="https://en.wikipedia.org/wiki/Maritime_Mobile_Service_Identity" target="_blank" rel="nofollow noreferrer noopener">https://en.wikipedia.org/wiki/Maritime_Mobile_Service_Identity</a></p> <p><code><strong>lon</strong></code> Longitude, Longitude in 1/10 000 min (+/-180 deg, East = positive (as per 2's complement), West = negative (as per 2's complement). 181= (6791AC0h) = not available = default)</p> <p><code><strong>lat</strong></code> Latitude, Latitude in 1/10 000 min (+/-90 deg, North = positive (as per 2's complement), South = negative (as per 2's complement). 91deg (3412140h) = not available = default)</p> <p><code><strong>sog</strong></code> Speed over ground in 1/10 knot steps (0-102.2 knots) 1 023 = not available, 1 022 = 102.2 knots or higher</p> <p><code><strong>cog</strong></code> Course over ground in 1/10 = (0-3599). 3600 (E10h) = not available = default. 3 601-4 095 should not be used</p> <p><code><strong>navStat</strong></code> Navigational status, 0 = under way using engine, 1 = at anchor, 2 = not under command, 3 = restricted maneuverability, 4 = constrained by her draught, 5 = moored, 6 = aground, 7 = engaged in fishing, 8 = under way sailing, 9 = reserved for future amendment of navigational status for ships carrying DG, HS, or MP, or IMO hazard or pollutant category C, high speed craft (HSC), 10 = reserved for future amendment of navigational status for ships carrying dangerous goods (DG), harmful substances (HS) or marine pollutants (MP), or IMO hazard or pollutant category A, wing in ground (WIG); 11 = power-driven vessel towing astern (regional use); 12 = power-driven vessel pushing ahead or towing alongside (regional use); 13 = reserved for future use, 14 = AIS-SART (active), MOB-AIS, EPIRB-AIS 15 = undefined = default (also used by AIS-SART, MOB-AIS and EPIRB-AIS under test)</p> <p><code><strong>rot</strong></code> ROTAIS Rate of turn</p> <ul> <li>0 to +126 = turning right at up to 708 deg per min or higher</li> <li>0 to -126 = turning left at up to 708 deg per min or higher</li> <li>Values between 0 and 708 deg per min coded by ROTAIS = 4.733 SQRT(ROTsensor) degrees per min where ROTsensor is the Rate of Turn as input by an external Rate of Turn Indicator (TI). ROTAIS is rounded to the nearest integer value.</li> <li>+127 = turning right at more than 5 deg per 30 s (No TI available)</li> <li>-127 = turning left at more than 5 deg per 30 s (No TI available)</li> <li>-128 (80 hex) indicates no turn information available (default).</li> </ul> <p>ROT data should not be derived from COG information.</p> <p><code><strong>posAcc</strong></code> Position accuracy, The position accuracy (PA) flag should be determined in accordance with the table below:</p> <ul> <li>1 = high (<= 10 m)</li> <li>0 = low (> 10 m)</li> <li>0 = default</li> </ul> <p>See <a href="https://www.navcen.uscg.gov/?pageName=AISMessagesA#RAIM" target="_blank" rel="nofollow noreferrer noopener">https://www.navcen.uscg.gov/?pageName=AISMessagesA#RAIM</a></p> <p><code><strong>raim</strong></code> RAIM-flag Receiver autonomous integrity monitoring (RAIM) flag of electronic position fixing device; 0 = RAIM not in use = default; 1 = RAIM in use. See Table <a href="https://www.navcen.uscg.gov/?pageName=AISMessagesA#RAIM" target="_blank" rel="nofollow noreferrer noopener">https://www.navcen.uscg.gov/?pageName=AISMessagesA#RAIM</a></p> <p>Check <a href="https://en.wikipedia.org/wiki/Receiver_autonomous_integrity_monitoring" target="_blank" rel="nofollow noreferrer noopener">https://en.wikipedia.org/wiki/Receiver_autonomous_integrity_monitoring</a></p> <p><code><strong>heading</strong></code> True heading, Degrees (0-359) (511 indicates not available = default)</p> <h4><a href="#yyyy-mm-dd-metadatacsv"></a>YYYY-MM-DD-metadata.csv</h4> <p>This file contains the received AIS metadata: the ship static and voyage related data. The structure of the logged parameters is the following: <code>[timestamp, destination, mmsi, callSign, imo, shipType, draught, eta, posType, pointA, pointB, pointC, pointD, name]</code></p> <p><code><strong>timestamp</strong></code> The timestamp associated with the MQTT message received from <a href="http://www.digitraffic.fi/" target="_blank" rel="nofollow noreferrer noopener">www.digitraffic.fi</a>. It is assumed this timestamp is the Epoch time corresponding to when the AIS message was received by digitraffic.fi.</p> <p><code><strong>destination</strong></code> Maximum 20 characters using 6-bit ASCII; @@@@@@@@@@@@@@@@@@@@ = not available For SAR aircraft, the use of this field may be decided by the responsible administration</p> <p><code><strong>mmsi</strong></code> MMSI number, Maritime Mobile Service Identity (MMSI) is a unique 9 digit number that is assigned to a (Digital Selective Calling) DSC radio or an AIS unit. Check <a href="https://en.wikipedia.org/wiki/Maritime_Mobile_Service_Identity" target="_blank" rel="nofollow noreferrer noopener">https://en.wikipedia.org/wiki/Maritime_Mobile_Service_Identity</a></p> <p><code><strong>callSign</strong></code> 7?=?6 bit ASCII characters, @@@@@@@ = not available = default Craft associated with a parent vessel, should use “A” followed by the last 6 digits of the MMSI of the parent vessel. Examples of these craft include towed vessels, rescue boats, tenders, lifeboats and liferafts.</p> <p><code><strong>imo</strong></code> 0 = not available = default – Not applicable to SAR aircraft</p> <ul> <li>0000000001-0000999999 not used</li> <li>0001000000-0009999999 = valid IMO number;</li> <li>0010000000-1073741823 = official flag state number.</li> </ul> <p>Check: <a href="https://en.wikipedia.org/wiki/IMO_number" target="_blank" rel="nofollow noreferrer noopener">https://en.wikipedia.org/wiki/IMO_number</a></p> <p><code><strong>shipType</strong></code></p> <ul> <li>0 = not available or no ship = default</li> <li>1-99 = as defined below</li> <li>100-199 = reserved, for regional use</li> <li>200-255 = reserved, for future use Not applicable to SAR aircraft</li> </ul> <p>Check <a href="https://www.navcen.uscg.gov/pdf/AIS/AISGuide.pdf" target="_blank" rel="nofollow noreferrer noopener">https://www.navcen.uscg.gov/pdf/AIS/AISGuide.pdf</a> and <a href="https://www.navcen.uscg.gov/?pageName=AISMessagesAStatic" target="_blank" rel="nofollow noreferrer noopener">https://www.navcen.uscg.gov/?pageName=AISMessagesAStatic</a></p> <p><code><strong>draught</strong></code> In 1/10 m, 255 = draught 25.5 m or greater, 0 = not available = default; in accordance with IMO Resolution A.851 Not applicable to SAR aircraft, should be set to 0</p> <p><code><strong>eta</strong></code> Estimated time of arrival; MMDDHHMM UTC</p> <ul> <li>Bits 19-16: month; 1-12; 0 = not available = default</li> <li>Bits 15-11: day; 1-31; 0 = not available = default</li> <li>Bits 10-6: hour; 0-23; 24 = not available = default</li> <li>Bits 5-0: minute; 0-59; 60 = not available = default</li> </ul> <p>For SAR aircraft, the use of this field may be decided by the responsible administration</p> <p><code><strong>posType</strong></code> Type of electronic position fixing device</p> <ul> <li>0 = undefined (default)</li> <li>1 = GPS</li> <li>2 = GLONASS</li> <li>3 = combined GPS/GLONASS</li> <li>4 = Loran-C</li> <li>5 = Chayka</li> <li>6 = integrated navigation system</li> <li>7 = surveyed</li> <li>8 = Galileo,</li> <li>9-14 = not used</li> <li>15 = internal GNSS</li> </ul> <p><code><strong>pointA</strong></code> Reference point for reported position.</p> <p>Also indicates the dimension of ship (m). For SAR aircraft, the use of this field may be decided by the responsible administration. If used it should indicate the maximum dimensions of the craft. As default should A = B = C = D be set to “0”</p> <p>Check: <a href="https://www.navcen.uscg.gov/?pageName=AISMessagesAStatic#_Reference_point_for" target="_blank" rel="nofollow noreferrer noopener">https://www.navcen.uscg.gov/?pageName=AISMessagesAStatic#_Reference_point_for</a></p> <p><code><strong>pointB</strong></code> See above</p> <p><code><strong>pointC</strong></code> See above</p> <p><code><strong>pointD</strong></code> See above</p> <p><code><strong>name</strong></code> Maximum 20 characters 6 bit ASCII "@@@@@@@@@@@@@@@@@@@@" = not available = default The Name should be as shown on the station radio license. For SAR aircraft, it should be set to “SAR AIRCRAFT NNNNNNN” where NNNNNNN equals the aircraft registration number.</p>
FiN-2: Larg-Scale Powerline Communication Dataset (Pt.1)
<p># FiN-2 Large-Scale Real-World PLC-Dataset</p> <p>## About<br> #### FiN-2 dataset in a nutshell:<br> FiN-2 is the first large-scale real-world dataset on data collected in a powerline communication infrastructure. Since the electricity grid is inherently a graph, our dataset could be interpreted as a graph dataset. Therefore, we use the word node to describe points (cable distribution cabinets) of measurement within the low-voltage electricity grid and the word edge to describe connections (cables) in between them. However, since these are PLC connections, an edge does not necessarily have to correspond to a real cable; more on this in our paper.<br> FiN-2 shows measurements that relate to the nodes (voltage, total harmonic distortion) as well as to the edges (signal-to-noise ratio spectrum, tonemap). In total, FiN-2 is distributed across three different sites with a total of 1,930,762,116 node measurements each for the individual features and 638,394,025 edge measurements each for all 917 PLC channels. All data was collected over a 25-month period from mid-2020 to the end of 2022.<br> We propose this dataset to foster research in the domain of grid automation and smart grid. Therefore, we provide different example use cases in asset management, grid state visualization, forecasting, predictive maintenance, and novelty detection. For more decent information on this dataset, please see our [paper](https://arxiv.org/abs/2209.12693).</p> <p>* * *<br> ## Content<br> FiN-2 dataset splits up into two compressed `csv-Files`: *nodes.csv* and *edges.csv*.</p> <p>All files are provided as a compressed ZIP file and are divided into four parts. The first part can be found in this repo, while the remaining parts can be found in the following:<br> - <a href="https://zenodo.org/record/8328105">https://zenodo.org/record/8328105</a><br> - <a href="https://zenodo.org/record/8328108">https://zenodo.org/record/8328108</a><br> - <a href="https://zenodo.org/record/8328111">https://zenodo.org/record/8328111</a><br> <br> ### Node data</p> <p>| id | ts | v1 | v2 | v3 | thd1 | thd2 | thd3 | phase_angle1 | phase_angle2 | phase_angle3 | temp |<br> |----|----|----|----|----|----|----|----|----|----|----|----|----|----|<br> |112|1605530460|236.5|236.4|236.0|2.9|2.5|2.4|120.0|119.8|120.0|35.3|<br> |112|1605530520|236.9|236.6|236.6|3.1|2.7|2.5|120.1|119.8|120.0|35.3|<br> |112|1605530580|236.2|236.4|236.0|3.1|2.7|2.5|120.0|120.0|119.9|35.5|</p> <p>- id / ts: Unique identifier of the node that is measured and timestemp of the measurement<br> - v1/v2/v3: Voltage measurements of all three phases<br> - thd1/thd2/thd3: Total harmonic distortion of all three phases<br> - phase_angle1/2/3: Phase angle of all three phases<br> - temp: Temperature in-circuit of the sensor inside a cable distribution unit (in °C)</p> <p>### Edge data<br> | src | dst | ts | snr0 | snr1 | snr2 | ... | snr916 |<br> |----|----|----|----|----|----|----|----|<br> |62|94|1605528900|70|72|45|...|-53|<br> |62|32|1605529800|16|24|13|...|-51|<br> |17|94|1605530700|37|25|24|...|-55|</p> <p>- src & dst & ts: Unique identifier of the source and target nodes where the spectrum is measured and time of measurement<br> - snr0/snr1/.../snr916: 917 SNR measurements in tenths of a decibel (e.g. 50 --> 5dB).</p> <p>### Metadata<br> Metadata that is provided along with the data covers:</p> <p>- Number of cable joints<br> - Cable properties (length, type, number of sections)<br> - Relative position of the nodes (location, zero-centered gps)<br> - Adjacent PV or wallbox installations<br> - Year of installation w.r.t. the nodes and cables</p> <p>Since the electricity grid is part of the critical infrastructure, it is not possible to provide exact GPS locations.</p> <p>* * *<br> ## Usage<br> Simple data access using pandas:</p> <p>```<br> import pandas as pd</p> <p>nodes_file = "nodes.csv.gz" # /path/to/nodes.csv.gz<br> edges_file = "edges.csv.gz" # /path/to/edges.csv.gz</p> <p># read the first 10 rows<br> data = pd.read_csv(nodes_file, nrows=10, compression='gzip')</p> <p># read the row number 5 to 15<br> data = pd.read_csv(nodes_file, nrows=10, skiprows=[i for i in range(1,6)], compression='gzip')</p> <p># ... same for the edges<br> ```</p> <p>Compressed csv-data format was used to make sharing as easy as possible, however it comes with significant drawbacks for machine learning. Due to the inherent graph structure, a single snapshot of the whole graph consists of a set of node and edge measurements. But due to timeouts, noise and other disturbances, nodes sometimes fail in collecting the data, wherefore the number of measurements for a specific timestamp differs. This, plus the high sparsity of the graph, leads to a high inefficiency when using the csv-format for an ML training.<br> To utilize the data in an ML pipeline, we recommend other data formats like [datadings](https://datadings.readthedocs.io/en/latest/) or specialized database solutions like [VictoriaMetrics](https://victoriametrics.com/).</p> <p><br> ### Example use case (voltage forecasting)</p> <p>Forecasting of the voltage is one potential use cases. The Jupyter notebook provided in the repository gives an overview of how the dataset can be loaded, preprocessed and used for ML training. Thereby, a MinMax scaling was used as simple preprocessing and a PyTorch dataset class was created to handle the data. Furthermore, a vanilla autoencoder is utilized to process and forecast the voltage into the future.</p> <p> </p> <p> </p>
Evaluate the Efficacy and Safety of Large-Scale Field-Directed Topical Therapy of Actinic Keratosis of the Chest w/Ingenol Mebutate 0.015%
ClinicalTrials.gov study NCT02446223. IPD Sharing: Not stated. Countries: 1. Publications: 1.
ClinSeq: A Large-Scale Medical Sequencing Clinical Research Pilot Study
ClinicalTrials.gov study NCT00410241. IPD Sharing: YES. Countries: 1. Publications: 5.
Data from "Quantification of major particulate matter species from a single filter type using infrared spectroscopy – Application to a large-scale monitoring network"
Open the record for dataset details and reuse information.
Longer is not always better: optimizing barcode length for large-scale species discovery and identification
Open the record for dataset details and reuse information.
Data from: Quantifying natural disturbances using a large-scale dendrochronological reconstruction to guide forest management
Open the record for dataset details and reuse information.
Large-scale genomic data reveal the phylogeny and evolution of owlet moths (Noctuoidea)
Open the record for dataset details and reuse information.
Data from: Ant communities respond to a large-scale disturbance along an elevational gradient in Puerto Rico, U.S.A.
Open the record for dataset details and reuse information.
Data from: Odor source distance is predictable from time-histories of odor statistics for large scale outdoor plumes
Open the record for dataset details and reuse information.
Large-scale restoration of species-rich dry grasslands on arable land: Environmental filtering drives successful species establishment over a period of 10 years
Open the record for dataset details and reuse information.
Allometric scaling of a superposition eye optimises sensitivity and acuity in large and small hawkmoths
Open the record for dataset details and reuse information.
Long-term responses to large-scale disturbances: Spatiotemporal variation in gastropod populations and communities
Open the record for dataset details and reuse information.
Dataset 1 for Large- and small-scale geographic structures affect genetic patterns across populations of an Alpine butterfly
Open the record for dataset details and reuse information.
Large-scale changes in macrobenthic biodiversity driven by mangrove afforestation
Open the record for dataset details and reuse information.
Analysing indoor mycobiomes through a large‐scale citizen science study in Norway
Open the record for dataset details and reuse information.
Large scale eDNA monitoring of multiple aquatic pathogens as a tool to provide risk maps for wildlife diseases
Open the record for dataset details and reuse information.
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.