Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
4,612
datasets available to search
ShareScore release 0.9.0
Dataset results
4,612 results for “labelled”
DOI's with SDG labels on Target level | 1.4M research articles (2009-2020) related to Sustainable Development Goals
<p>Table content: This data set contains 1.4 million publication DOI's related to the <a href="http://metadata.un.org/sdg/">Targets of the Sustainable Development Goals</a> in the period 2009 - 2020.</p> <p>Table dimensions: rows: 1.4 million, columns: 4 / rows: 1.4 million, columns: 180</p> <p>Table columns: <a href="https://en.wikipedia.org/wiki/Digital_object_identifier">doi</a> | date | <a href="http://metadata.un.org/sdg/ontology#Target">sdg_target</a> | <a href="http://metadata.un.org/sdg/ontology#Goal">sdg_goal</a> / <a href="https://en.wikipedia.org/wiki/Digital_object_identifier">doi</a> | date | <a href="http://metadata.un.org/sdg/ontology#Target">169 sdg_targets</a> | <a href="http://metadata.un.org/sdg/ontology#Goal">17 sdg_goalsl</a></p> <p>Table formats: <a href="https://en.wikipedia.org/wiki/Comma-separated_values">.csv</a> | <a href="https://en.wikipedia.org/wiki/Microsoft_Excel">.xlsx</a> | <a href="https://en.wikipedia.org/wiki/Apache_Parquet">.parquet</a></p> <p><em>How we made this data:</em></p> <p>We have made a search on <a href="https://scopus.com">Scopus </a>using the <a href="https://aurora-network-global.github.io/sdg-queries/">Aurora SDG queries version 5</a> for each of the targets, with a limited year range from 2009 till 2020.</p> <p>Good to know: don't be alarmed if you can find a doi that is labeled with more than one target (~16%). This is not a bug, this is a feature... We used 169 queries, one for each target, a publication can appear in more han one result set.</p> <p>Read this <a href="https://zenodo.org/record/4964606/files/Evaluation_on_accuracy_of_mapping_science_to_the_United_Nations__Sustainable_Development_Goals__SDGs__of_the_Aurora_SDG_queries.pdf?download=1">report to learn more about the accuracy</a> of the queries and the data result sets.</p> <p><em>How can you use this data:</em></p> <p>You can use this data to 1. quickly match your existing publication lists to this list to see how that your publications are related to the targets of the SDG's. 2. use these as a basis / seed set / gold set to train more advanced text / graph classifiers (after you have extracted title, abstract or even full-text using crossref.org, unpaywall.org, etc)</p> <p><em>How can you help:</em></p> <p><a href="https://sites.google.com/vu.nl/aurora-sdg-research-dashboard/sdg-knowledge-base#h.d2pd3c39k276">Let us know</a> how you use this data. We'll put your project on the list in our <a href="https://sites.google.com/vu.nl/aurora-sdg-research-dashboard/sdg-knowledge-base">SDG matching knowledge base.</a></p>
Dataset and Code for Manuscript "Multi-angle pulse shape detection of scattered light in flow cytometry for label-free cell cycle classification"
<p>Dataset of measurements for cell cycle analysis with description:</p> <ul> <li>ReadMe file with explanations on the data set and analysis</li> <li>exemplary Matlab script file for analysis</li> <li>binary data files conatining the pulse shapes in all channels</li> <li>FCS data files containing common flow cytometry parameters in each channel</li> </ul> <p>Data on unsorted HEK cells, HEK cells sorted for cell cycle phases, and unsorted Jurkat cell are included.</p>
Music Streams Labelled with Listening Situation - [User/Track/Device/Situation] Dataset
<p>This is a contextual music dataset labeled with the listening situation associated with each stream. Each stream is composed of the user, track, and device data labelled with a situation. The dataset is collected from Deezer for the period of August 2019 from France and Brazil. The dataset is composed of 3 subsets of situations corresponding to 4, 8, and 12 different situations. The situations are extracted based on keyword matching with the associated playlist title in the Deezer catalog. The full set of situational tags are: "<strong>work, gym, party, sleep | morning, run, night, dance | car, train, relax, club"</strong>.</p> <p>Each instance contains the track/user/deviice triplets, and a situational tag indicating that this user listens to the track in the associated situation wth the corresponding data recieved from the device. The device data contain: "l<strong>inear-time, linear-day, circular-time X, circular-time Y,circular-day X, circular-day Y, device-type, network-type</strong>". The users are represented as <strong>embeddings</strong> based on their listening history computed through the matrix factorization of the user/track matrix. Additionally, the users are also represented with their demographic data of : "<strong>age, country, gender</strong>".</p> <p>The creation of the dataset and our experimental results are described in the paper: Karim M. Ibrahim, Elena V. Epure, Geoffroy Peeters, and Gaël Richard. "Audio Autotagging as Proxy for Contextual MusicRecommendation" [Under Revision]. The source code of the paper is available here: <a href="https://github.com/KarimMibrahim/Situational_Session_Generator.git">https://github.com/KarimMibrahim/Situational_Session_Generator.git</a></p> <p>The dataset is composed of the media_id which is the ID of the track in the Deezer catalog. The 30 seconds track previews used to train the model in the paper can be accessed through the Deezer API: <a href="https://developers.deezer.com/api">https://developers.deezer.com/api</a>. Each user is represented with an <strong>anonymized</strong> <strong>user_id</strong> which is associated with the user embedding available in the user_embeddings.npy file. Note: The index of the embeddings in the user_embeddings arrary corresponds to the user_id, i.e. user_id = 100 have its embeddings at user_embeddings[100]. </p> <p>Finally, the dataset also contains the splits used in our experiments. Our splits were conditioned by one of three conditions: <em>ColdTrack</em> (no overlap of tracks between the splits), <em>ColdUser</em> (no overlap of users between the splits), and <em>WarmCase</em> (overlaps allowed). Each condition is split into 4 subsets for cross-validation marked with a "<strong>fold</strong>" number in each condition. </p>
Segmentation Labels for Emergency Response Imagery from Hurricane Barry, Delta, Dorian, Florence, Isaias, Laura, Michael, Sally, Zeta, and Tropical Storm Gordon
<p>The zip file here contains 1,179 pairs of human-generated segmentation labels and images from Emergency Response Imagery collected by US National Oceanic and Atmospheric Administration (NOAA) after Hurricane Barry, Delta, Dorian, Florence, Ida, Laura, Michael, Sally, Zeta, and Tropical Storm Gordon. A total of 1,054 unique images were labeled. 946 images were annotated by a single labeler. 95 images were annotated by two labelers. 11 images were annotated by three labelers. 2 images were annotated by five labelers. All authors contributed to labeling, and all labeling was done with an open-source labeling tool (Buscombe et al., 2022).</p> <p>All pixels in each image are labeled with one of four classes: 0 (water), 1 (bare sand), 2 (vegetation - both sparse and dense), 4 (the built environment - buildings, roads, parking lots, boats, etc.)</p> <p>The csv file provided here is a list of each image file name (which includes the anonymized labeler ID), the name of the image without the labeler ID, the name of the corresponding NOAA jpg, the NOAA flight name, the storm name, the latitude and longitude of the image, and a column stating if the image has been labeled multiple times. </p> <p>Images labeled here correspond to multiple NOAA flights — all listed in the csv file for each jpeg image. These jpeg images can be downloaded directly from NOAA (https://storms.ngs.noaa.gov/) or using Moretz et al. (2020a, 2020b). The images included in this data release correspond to original NOAA images that have been resized and then split into quadrants (using ImageMagick). The naming convention corresponds to the image quarter — the *-0.jpg is upper left, *-1.jpg is upper right, *-2.jpg is lower left, and *-3.jpg is the lower right.</p> <p><br> The resize command used was:</p> <p><br> #to resize and then quarter<br> #Dir structure is:<br> # --Desktop<br> # |- originals<br> # |- resized<br> # |- quarters</p> <p>`cd originals`<br> `mogrify -resize 2000x2000 -path ../resized *.jpg`</p> <p>#then quarter them<br> `cd ..`<br> `cd resized`</p> <p>`mogrify -crop 2x2@ +repage -path ../quarters *.jpg`</p> <p>For full size images, please download the jpegs directly from NOAA.</p>
Images and 4-class labels for semantic segmentation of Sentinel-2 and Landsat RGB, NIR, and SWIR satellite images of coasts (water, whitewater, sediment, other)
<p><em><strong>Images and 4-class labels for semantic segmentation of Sentinel-2 and Landsat RGB, NIR, and SWIR satellite images of coasts (water, whitewater, sediment, other)</strong></em></p> <p><strong>Description</strong></p> <p>579 images and 579 associated labels for semantic segmentation of Sentinel-2 and Landsat RGB satellite images of coasts. The 4 classes are 0=water, 1=whitewater, 2=sediment, 3=other</p> <p>These images and labels have been made using the Doodleverse software package, Doodler*. These images and labels could be used within numerous Machine Learning frameworks for image segmentation, but have specifically been made for use with the Doodleverse software package, Segmentation Gym**.</p> <p>Some (422) of these images and labels were originally included in the Coast Train*** data release, and have been modified from their original by reclassifying from the original classes to the present 4 classes.</p> <p>The label images are a subset of the following data release**** <a href="https://doi.org/10.5281/zenodo.7335647">https://doi.org/10.5281/zenodo.7335647</a></p> <p>Imagery comes from the following 10 sand beach sites:</p> <ol> <li>Duck, NC, Hatteras NC, USA</li> <li>Santa Cruz CA, USA</li> <li>Galveston TX, USA</li> <li>Truc Vert,France</li> <li>Sunset State Beach CA, USA</li> <li>Torrey Pines CA, USA</li> <li>Narrabeen, NSW, Australia</li> <li>Elwha WA, USA</li> <li>Ventura region, CA, USA</li> <li>Klamath region, CA USA</li> </ol> <p>Imagery are a mixture of 10-m Sentinel-2 and 15-m pansharpened Landsat 7, 8, and 9 visible-band imagery of various sizes. Red, Green, Blue, NIR, and SWIR bands only</p> <p><strong>File descriptions</strong></p> <ol> <li>classes.txt, a file containing the class names</li> <li>images.zip, a zipped folder containing the 3-band RGB images of varying sizes and extents</li> <li>nir.zip, a zipped folder containing the corresponding near-infrared (NIR) imagery</li> <li>swir.zip, a zipped folder containing the corresponding shortwave-infrared (SWIR) imagery</li> <li>labels.zip, a zipped folder containing the 1-band label images</li> <li>overlays.zip, a zipped folder containing a semi-transparent overlay of the color-coded label on the image (blue=0=water, red=1=whitewater, yellow=2=sediment, green=3=other)</li> <li>resized_images.zip, RGB images resized to 512x512x3 pixels</li> <li>resized_nir.zip, NIR images resized to 512x512x3 pixels</li> <li>resized_swir.zip, SWIR images resized to 512x512x3 pixels</li> <li>resized_labels.zip, label images resized to 512x512 pixels</li> </ol> <p><strong>References</strong></p> <p>*Doodler: Buscombe, D., Goldstein, E.B., Sherwood, C.R., Bodine, C., Brown, J.A., Favela, J., Fitzpatrick, S., Kranenburg, C.J., Over, J.R., Ritchie, A.C. and Warrick, J.A., 2021. Human‐in‐the‐Loop Segmentation of Earth Surface Imagery. Earth and Space Science, p.e2021EA002085<a href="https://doi.org/10.1029/2021EA002085">https://doi.org/10.1029/2021EA002085</a>. See <a href="https://github.com/Doodleverse/dash_doodler">https://github.com/Doodleverse/dash_doodler.</a></p> <p>**Segmentation Gym: Buscombe, D., & Goldstein, E. B. (2022). A reproducible and reusable pipeline for segmentation of geoscientific imagery. Earth and Space Science, 9, e2022EA002332. <a href="https://doi.org/10.1029/2022EA002332">https://doi.org/10.1029/2022EA002332</a> See: <a href="https://github.com/Doodleverse/segmentation_gym">https://github.com/Doodleverse/segmentation_gym</a></p> <p>***Coast Train data release: Wernette, P.A., Buscombe, D.D., Favela, J., Fitzpatrick, S., and Goldstein E., 2022, Coast Train--Labeled imagery for training and evaluation of data-driven models for image segmentation: U.S. Geological Survey data release, <a href="https://doi.org/10.5066/P91NP87I">https://doi.org/10.5066/P91NP87I</a>. See <a href="https://coasttrain.github.io/CoastTrain/">https://coasttrain.github.io/CoastTrain/ </a>for more information</p> <p>**** Buscombe, Daniel, Goldstein, Evan, Bernier, Julie, Bosse, Stephen, Colacicco, Rosa, Corak, Nick, Fitzpatrick, Sharon, del Jesús González Guillén, Anais, Ku, Venus, Paprocki, Julie, Platt, Lindsay, Steele, Bethel, Wright, Kyle, & Yasin, Brandon. (2022). Images and 4-class labels for semantic segmentation of Sentinel-2 and Landsat RGB satellite images of coasts (water, whitewater, sediment, other) (v1.0) [Data set]. Zenodo. <a href="https://doi.org/10.5281/zenodo.7335647">https://doi.org/10.5281/zenodo.7335647</a></p>
Images and 4-class labels for semantic segmentation of Sentinel-2 and Landsat RGB satellite images of coasts (water, whitewater, sediment, other)
<p><strong>Description</strong></p> <p>1018 images and 1018 associated labels for semantic segmentation of Sentinel-2 and Landsat RGB satellite images of coasts. The 4 classes are 0=water, 1=whitewater, 2=sediment, 3=other</p> <p>These images and labels have been made using the Doodleverse software package, Doodler*. These images and labels could be used within numerous Machine Learning frameworks for image segmentation, but have specifically been made for use with the Doodleverse software package, Segmentation Gym**.</p> <p>Some (473) of these images and labels were originally included in the Coast Train*** data release, and have been modified from their original by reclassifying from the original classes to the present 4 classes.</p> <p>Imagery comes from the following 10 sand beach sites:</p> <ol> <li>Duck, NC, Hatteras NC, USA</li> <li>Santa Cruz CA, USA</li> <li>Galveston TX, USA</li> <li>Truc Vert,France</li> <li>Sunset State Beach CA, USA</li> <li>Torrey Pines CA, USA</li> <li>Narrabeen, NSW, Australia</li> <li>Elwha WA, USA</li> <li>Ventura region, CA, USA</li> <li>Klamath region, CA USA</li> </ol> <p>Imagery are a mixture of 10-m Sentinel-2 and 15-m pansharpened Landsat 7, 8, and 9 visible-band imagery of various sizes. Red, Green, and Blue bands only</p> <p><strong>File descriptions</strong></p> <ol> <li>classes.txt, a file containing the class names</li> <li>images.zip, a zipped folder containing the 3-band images of varying sizes and extents</li> <li>labels.zip, a zipped folder containing the 1-band label images</li> <li>overlays.zip, a zipped folder containing a semi-transparent overlay of the color-coded label on the image (blue=0=water, red=1=whitewater, yellow=2=sediment, green=3=other)</li> <li>resized_images.zip, RGB images resized to 512x512x3 pixels</li> <li>resized_labels.zip, label images resized to 512x512 pixels</li> </ol> <p><strong>References</strong></p> <p>*Doodler: Buscombe, D., Goldstein, E.B., Sherwood, C.R., Bodine, C., Brown, J.A., Favela, J., Fitzpatrick, S., Kranenburg, C.J., Over, J.R., Ritchie, A.C. and Warrick, J.A., 2021. Human‐in‐the‐Loop Segmentation of Earth Surface Imagery. Earth and Space Science, p.e2021EA002085<a href="https://doi.org/10.1029/2021EA002085">https://doi.org/10.1029/2021EA002085</a>. See <a href="https://github.com/Doodleverse/dash_doodler">https://github.com/Doodleverse/dash_doodler.</a></p> <p>**Segmentation Gym: Buscombe, D., & Goldstein, E. B. (2022). A reproducible and reusable pipeline for segmentation of geoscientific imagery. Earth and Space Science, 9, e2022EA002332. <a href="https://doi.org/10.1029/2022EA002332">https://doi.org/10.1029/2022EA002332</a> See: <a href="https://github.com/Doodleverse/segmentation_gym">https://github.com/Doodleverse/segmentation_gym</a></p> <p>***Coast Train data release: Wernette, P.A., Buscombe, D.D., Favela, J., Fitzpatrick, S., and Goldstein E., 2022, Coast Train--Labeled imagery for training and evaluation of data-driven models for image segmentation: U.S. Geological Survey data release, <a href="https://doi.org/10.5066/P91NP87I">https://doi.org/10.5066/P91NP87I</a>. See <a href="https://coasttrain.github.io/CoastTrain/">https://coasttrain.github.io/CoastTrain/ </a>for more information</p> <p> </p>
Images and 2-class labels for semantic segmentation of Sentinel-2 and Landsat RGB satellite images of coasts (water, other)
<p><em><strong>Images and 2-class labels for semantic segmentation of Sentinel-2 and Landsat RGB satellite images of coasts (water, other)</strong></em></p> <p>Images and 2-class labels for semantic segmentation of Sentinel-2 and Landsat RGB satellite images of coasts (water, other)</p> <p><strong>Description</strong></p> <p>4088 images and 4088 associated labels for semantic segmentation of Sentinel-2 and Landsat RGB satellite images of coasts. The 2 classes are 1=water, 0=other. Imagery are a mixture of 10-m Sentinel-2 and 15-m pansharpened Landsat 7, 8, and 9 visible-band imagery of various sizes. Red, Green, Blue bands only</p> <p>These images and labels could be used within numerous Machine Learning frameworks for image segmentation, but have specifically been made for use with the Doodleverse software package, Segmentation Gym**.</p> <p>Two data sources have been combined</p> <p><strong>Dataset 1</strong></p> <ul> <li>1018 image-label pairs from the following data release**** https://doi.org/10.5281/zenodo.7335647</li> <li>Labels have been reclassified from 4 classes to 2 classes.</li> <li>Some (422) of these images and labels were originally included in the Coast Train*** data release, and have been modified from their original by reclassifying from the original classes to the present 2 classes.</li> <li>These images and labels have been made using the Doodleverse software package, Doodler*.</li> </ul> <p><strong>Dataset 2</strong></p> <ul> <li>3070 image-label pairs from the Sentinel-2 Water Edges Dataset (SWED)***** dataset, https://openmldata.ukho.gov.uk/, described by Seale et al. (2022)******</li> <li>A subset of the original SWED imagery (256 x 256 x 12) and labels (256 x 256 x 1) have been chosen, based on the criteria of more than 2.5% of the pixels represent water</li> </ul> <p><strong>File descriptions</strong></p> <ul> <li> classes.txt, a file containing the class names</li> <li> images.zip, a zipped folder containing the 3-band RGB images of varying sizes and extents</li> <li> labels.zip, a zipped folder containing the 1-band label images</li> <li> overlays.zip, a zipped folder containing a semi-transparent overlay of the color-coded label on the image (red=1=water, bllue=0=other)</li> <li> resized_images.zip, RGB images resized to 512x512x3 pixels</li> <li> resized_labels.zip, label images resized to 512x512x1 pixels</li> </ul> <p><strong>References</strong></p> <p>*Doodler: Buscombe, D., Goldstein, E.B., Sherwood, C.R., Bodine, C., Brown, J.A., Favela, J., Fitzpatrick, S., Kranenburg, C.J., Over, J.R., Ritchie, A.C. and Warrick, J.A., 2021. Human‐in‐the‐Loop Segmentation of Earth Surface Imagery. Earth and Space Science, p.e2021EA002085https://doi.org/10.1029/2021EA002085. See https://github.com/Doodleverse/dash_doodler.</p> <p>**Segmentation Gym: Buscombe, D., & Goldstein, E. B. (2022). A reproducible and reusable pipeline for segmentation of geoscientific imagery. Earth and Space Science, 9, e2022EA002332. https://doi.org/10.1029/2022EA002332 See: https://github.com/Doodleverse/segmentation_gym</p> <p>***Coast Train data release: Wernette, P.A., Buscombe, D.D., Favela, J., Fitzpatrick, S., and Goldstein E., 2022, Coast Train--Labeled imagery for training and evaluation of data-driven models for image segmentation: U.S. Geological Survey data release, https://doi.org/10.5066/P91NP87I. See https://coasttrain.github.io/CoastTrain/ for more information</p> <p>****Buscombe, Daniel, Goldstein, Evan, Bernier, Julie, Bosse, Stephen, Colacicco, Rosa, Corak, Nick, Fitzpatrick, Sharon, del Jesús González Guillén, Anais, Ku, Venus, Paprocki, Julie, Platt, Lindsay, Steele, Bethel, Wright, Kyle, & Yasin, Brandon. (2022). Images and 4-class labels for semantic segmentation of Sentinel-2 and Landsat RGB satellite images of coasts (water, whitewater, sediment, other) (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7335647</p> <p>*****Seale, C., Redfern, T., Chatfield, P. 2022. Sentinel-2 Water Edges Dataset (SWED) https://openmldata.ukho.gov.uk/</p> <p>******Seale, C., Redfern, T., Chatfield, P., Luo, C. and Dempsey, K., 2022. Coastline detection in satellite imagery: A deep learning approach on new benchmark data. Remote Sensing of Environment, 278, p.113044.</p>
Labeled dataset of IEEE 802.11 probe requests
<p><strong>Introduction</strong> <br> <br> The 802.11 standard includes several management features and corresponding frame types. One of them are probe requests (PR). They are sent by mobile devices in the unassociated state to search the nearby area for existing wireless networks. The frame part of PRs consists of variable length fields called information elements (IE). IE fields represent the capabilities of a mobile device, such as data rates. <br> The dataset includes PRs collected in a controlled rural environment and in a semi-controlled indoor environment under different measurement scenarios. <br> It can be used for various use cases, e.g., analysing MAC randomization, determining the number of people in a given location at a given time or in different time periods, analysing trends in population movement (streets, shopping malls, etc.) in different time periods, etc.</p> <p> <br> <strong>Measurement setup</strong> <br> <br> The system for collecting PRs consists of a Raspberry Pi 4 (RPi) with an additional WiFi dongle to capture Wi-Fi signal traffic in monitoring mode. Passive PR monitoring is performed by listening to 802.11 traffic and filtering out PR packets on a single WiFi channel.<br> The following information about each PR received is collected: MAC address, Supported data rates, extended supported rates, HT capabilities, extended capabilities, data under extended tag and vendor specific tag, interworking, VHT capabilities, RSSI, SSID and timestamp when PR was received.<br> The collected data was forwarded to a remote database via a secure VPN connection. A Python script was written using the Pyshark package for data collection, preprocessing and transmission.</p> <p><br> <strong>Data preprocessing</strong></p> <p>The gateway collects PRs for each consecutive predefined scan interval (10 seconds). During this time interval, the data are preprocessed before being transmitted to the database.<br> For each detected PR in the scan interval, IEs fields are saved in the following JSON structure:<br> PR_IE_data =<br> {<br> 'DATA_RTS': {'SUPP': DATA_supp , 'EXT': DATA_ext},<br> 'HT_CAP': DATA_htcap,<br> 'EXT_CAP': {'length': DATA_len, 'data': DATA_extcap},<br> 'VHT_CAP': DATA_vhtcap,<br> 'INTERWORKING': DATA_inter,<br> 'EXT_TAG': {'ID_1': DATA_1_ext, 'ID_2': DATA_2_ext ...},<br> 'VENDOR_SPEC': {VENDOR_1:{<br> 'ID_1': DATA_1_vendor1,<br> 'ID_2': DATA_2_vendor1<br> ...},<br> VENDOR_2:{<br> 'ID_1': DATA_1_vendor2,<br> 'ID_2': DATA_2_vendor2<br> ...}<br> ...}<br> }</p> <p> <br> Supported data rates and extended supported rates are represented as arrays of values that encode information about the rates supported by a mobile device. The rest of the IEs data is represented in hexadecimal format. Vendor Specific Tag is structured differently than the other IEs. This field can contain multiple vendor IDs with multiple data IDs with corresponding data. Similarly, the extended tag can contain multiple data IDs with corresponding data. <br> Missing IE fields in the captured PR are not included in <em>PR_IE_DATA</em>.</p> <p>When a new MAC address is detected in the current scan time interval, the data from PR is stored in the following structure:</p> <p>{'MAC': MAC_address, 'SSIDs': [ SSID ], 'PROBE_REQs': [PR_data] },</p> <p>where <em>PR_data</em> is structured as follows:<br> {<br> 'TIME': [ DATA_time ],<br> 'RSSI': [ DATA_rssi ],<br> 'DATA': PR_IE_data<br> }.</p> <p>This data structure allows storing only <em>TOA</em> and <em>RSSI</em> for all PRs originating from the same MAC address and containing the same <em>PR_IE_data</em>. All SSIDs from the same MAC address are also stored. <br> The data of the newly detected PR is compared with the already stored data of the same MAC in the current scan time interval. <br> If identical PR's IE data from the same MAC address is already stored, then only data for the keys <em>TIME</em> and <em>RSSI</em> are appended.<br> If no identical PR's IE data has yet been received from the same MAC address, then PR_data structure of the new PR for that MAC address is appended to <em>PROBE_REQs</em> key. <br> The preprocessing procedure is shown in Figure ./Figures/Preprocessing_procedure.png <br> At the end of each scan time interval, all processed data is sent to the database along with additional metadata about the collected data e.g. wireless gateway serial number and scan start and end timestamps. For an example of a single PR captured, see the ./Single_PR_capture_example.json file.</p> <p><br> <strong>Environments description</strong> <br> <br> We performed measurements in a controlled rural outdoor environment and in a semi-controlled indoor environment of the Jozef Stefan Institute.<br> See the Excel spreadsheet Measurement_informations.xlsx for a list of mobile devices tested. <br> <br> <strong>Indoor environment </strong><br> <br> We used 3 RPi's for the acquisition of PRs in the Jozef Stefan Institute. They were placed indoors in the hallways as shown in the ./Figures/RPi_locations_JSI.png. Measurements were performed on weekend to minimize additional uncontrolled traffic from users' mobile devices. While there is some overlap in WiFi coverage between the devices at the location 2 and 3, the device at location 1 has no overlap with the other two devices.</p> <p> <strong>Rural environment outdoors</strong> <br> <br> The three RPi's used to collect PRs were placed at three different locations with non-overlapping WiFi coverage, as shown in ./Figures/RPi_locations_rural_env.png. Before starting the measurement campaign, all measured devices were turned off and the environment was checked for active WiFi devices. We did not detect any unknown active devices sending WiFi packets in the RPi's coverage area, so the deployment can be considered fully controlled.<br> All known WiFi enabled devices that were used to collect and send data to the database used a global MAC address, so they can be easily excluded in the preprocessing phase. MAC addresses of these devices can be found in the ./Measurement_informations.xlsx spreadsheet.<br> Note: The Huawei P20 device with ID 4.3 was not included in the test in this environment.</p> <p><br> <strong>Scenarios description</strong> <br> <br> We performed three different scenarios of measurements. <br> <br> <strong>Individual device measurements</strong><br> <br> For each device, we collected PRs for one minute with the screen on, followed by PRs collected for one minute with the screen off. In the indoor environment the WiFi interfaces of the other devices not being tested were disabled. In rural environment other devices were turned off. Start and end timestamps of the recorded data for each device can be found in the ./Measurement_informations.xlsx spreadsheet under the <em>Indoor environment of Jozef Stefan Institute</em> sheet and the <em>Rural environment</em> sheet.</p> <p> <strong>Three groups test</strong></p> <p>In this measurement scenario, the devices were divided into three groups. The first group contained devices from different manufacturers. The second group contained devices from only one manufacturer (Samsung). Half of the third group consisted of devices from the same manufacturer (Huawei), and the other half of devices from different manufacturers. The distribution of devices among the groups can be found in the ./Measurement_informations.xlsx spreadsheet. <br> <br> The same data collection procedure was used for all three groups. Data for each group were collected in both environments at three different RPis locations, as shown in ./Figures/RPi_locations_JSI.png and ./Figures/RPi_locations_rural_env.png. <br> At each location, PRs were collected from each group for 10 minutes with the screen on. Then all three groups switched locations and the process was repeated. Thus, the dataset contains measurements from all three RPi locations of all three groups of devices in both measurement environments. The group movements and the timestamps for the start and end of the collection of PRs at each loacation can be found in spreadsheet ./Measurement_informations.xlsx.</p> <p> <strong>One group test</strong> <br> <br> In the last measurement scenario, all devices were grouped together. In rural evironement we first collected PRs for 10 minutes while the screen was on, and then for another 10 minutes while the screen was off. In indoor environment data were collected at first location with screens on for 10 minutes. Then all devices were moved to the location of the next RPi and PRs were collected for 5 minutes with the screen on and then for another 5 minutes with the screen off.</p> <p><strong>Folder structure</strong> <br> <br> The root directory contains two files in JSON format for each of the environments where the measurements took place (Data_indoor_environment.json and Data_rural_environment.json). Both files contain collected PRs for the entire day that the measurements were taken (12:00 AM to 12:00 PM) to get a sense of the behaviour of the unknown devices in each environment. The spreadsheet ./Measurement_informations.xlsx. contains three sheets. <em>Devices description</em> contains general information about the tested devices, RPis, and the assigned group for each device. The sheets <em>Indoor environment of Jozef Stefan Institute</em> and <em>Rural environment</em> contain the corresponding timestamps for the start and end of each measurement scenario. For the scenario where the devices were divided into groups, additional information about the movements between locations is included. The location names are based on the RPi gateway ID and may differ from those on the figures showing the locations of the RPIs for each environment.<br> The ./Figures folder contains the figures already mentioned above.</p>
Detecting coarse beach sediment using remotely sensed imagery at the FRF, Duck, NC, USA: Labeled images, deep learning model, testing data, and predictions.
<p>This data record contains 5 zip files all used to build and use a semantic segmentation model to operate on beach imagery taken at the Field Research Facility (FRF) in Duck, North Carolina, USA. All data is from 2015-2021</p> <p>The `training_data.zip` contains all data used to train the ML model. All images come from the north facing (c1) camera. This zip file includes: a list of classes used to label the imagery, and folders of 107 images, 107 sparse annotations (doodles), 107 labels, and 107 overlays. All labeling was done with the open-source labeling tool ‘Doodler (Buscombe et al., 2021).</p> <p>The `model.zip` file contains the ML model, and associated metadata. This includes: a JSON model configuration file, a figure showing model training statistics, an `.npz` file of model training output, a list of training and validation files, the model as an h5 file and in the Tensorflow ‘saved model’ format. All modeling was done with Segmentation Gym (Buscombe & Goldstein 2022).</p> <p>The `test_data_c6.zip` file contains all data from the south facing (c6) camera to test the ML model. This includes: a list of classes used to label the imagery, and folders of 10 images, 10 sparse annotations (doodles), 10 labels, and 10 overlays. All labeling was done with the open-source labeling tool ‘Doodler (Buscombe et al., 2021). Testing the model with this data was done with codes in: https://github.com/ebgoldstein/FRF_GrainSize</p> <p>The `test_data_c1.zip` file contains all data from the north facing (c1) camera to test the ML model. This includes: a list of classes used to label the imagery, and folders of 10 images, 10 sparse annotations (doodles), 10 labels, and 10 overlays. All labeling was done with an open-source labeling tool ‘Doodler (Buscombe et al., 2021). Testing the model with this data was done with codes in: https://github.com/ebgoldstein/FRF_GrainSize</p> <p>The `predictions.zip` file contains 4418 images from the north facing (c1) camera that were run through the trained segmentation model as well as the resulting output (presented as side-by-side image and overlays). These images were created using codes in Segmentation Gym (Buscombe & Goldstein 2022).</p>
STORM imaging of Bacillus subtilis labeled by fluorescent d-amino acids
<p>Bacillus subtilus cells were labeled by fluorescent d-amino acids, followed by STORM super-resolution imaging.</p> <p>The wide-field image and STORM imaging stack are uploaded.</p>
Multi-label Tweet Dataset for Textual Propaganda Detection related to anti-CAA protest in India (2019-2021)
<p>This is a collection of English Tweets about the Citizenship (Amendment) Bill protests that occurred in India in 2019-2021. The dataset contains tweet instances multiple labels for identified propaganda techniques. The data set consists of tweet ids, hashtags used, and corresponding propaganda techniques. Labels have been automatically generated using Weak Supervision. </p> <p>As of 2023, there are very limited textual propaganda detection dataset for Tweets. This dataset is released to facilitate future research as propaganda has become omnipresent in modern social media. </p> <p> </p> <p> </p>
Image and label patches used to train GLASS-AI
<p>This archive contains the paired image and label patches used to train our machine learning pipeline, Grading of Lung Adenocarcinoma with Simultaneous Segmentation by Artificial Intelligence (GLASS-AI). </p> <p>Image patches were generated from whole slide images of H&E-stained sections using an Aperio ScanScope AT2 Slide Scanner (Leica) at 20x magnification with a 0.5022 microns/pixel resolution. The individual tumors and airways were annotated by an expert human before being divided into 224x224 pixel patches of the H&E image and paired annotation layer.</p> <p>For more details regarding how these data were used to train GLASS-AI, please see our forthcoming manuscript. </p>
CTU-SME-11: a labeled dataset with real benign and malicious network traffic mimicking a small medium-size enterprise environment
<p>As technology advances, the number and complexity of cyber-attacks increase, forcing defense techniques to be updated and improved. To help develop effective tools for detecting security threats it is essential to have reliable and representative security datasets. Many existing security datasets have limitations that make them unsuitable for research, including lack of labels, unbalanced traffic, and outdated threats.</p> <p>CTU-SME-11 is a labeled network dataset designed to address the limitations of previous datasets. The dataset was captured in a real network that mimics a small-medium enterprise setting. Raw network traffic (packets) was captured from 11 devices using tcpdump for a duration of 7 days, from 20th to 26th of February, 2023 in Prague, Czech Republic. The devices were chosen based on the enterprise setting and consists of IoT, desktop and mobile devices, both bare metal and virtualized. The devices were infected with malware or exposed to Internet attacks, and factory reset to restore benign behavior. </p> <p>The raw data was processed to generate network flows (Zeek logs) which were analyzed and labeled. The dataset contains two types of levels, a high level label and a descriptive label, which were put by experts. The former can take three values, benign, malicious or background. The latter contains detailed information about the specific behavior observed in the network flows. The dataset contains 99 million labeled network flows. The overall compressed size of the dataset is 80GB and the uncompressed size is 170GB.</p>
Labeled sampling of related discussion posts
<p>This is a labeled sampling of (uN)related discussion posts. Each file contains master_discussion_id;target_discussion_id;R or N</p> <p>The label R means that the master_discussion and target_discussion are related.</p> <p>The label N means that the master_discussion and target_discussion are unrelated.</p> <p>Projects: Gatsby, Homebrew, and Next.js </p> <p>https://github.com/gatsbyjs/gatsby/discussions/</p> <p>https://github.com/orgs/Homebrew/discussions/</p> <p>https://github.com/vercel/next.js/discussions/</p> <p> </p>
June 2023 Supplement Images and 4-class labels for semantic segmentation of Sentinel-2 and Landsat RGB, NIR, and SWIR satellite images of coasts (water, whitewater, sediment, other)
<p><strong>June 2023 Supplement of Images and 4-class labels for semantic segmentation of Sentinel-2 and Landsat RGB, NIR, and SWIR satellite images of coasts (water, whitewater, sediment, other)</strong></p> <p><strong>Description</strong></p> <p>Supplementary dataset to:</p> <p>Buscombe, Daniel, Goldstein, Evan, Bernier, Julie, Bosse, Stephen, Colacicco, Rosa, Corak, Nick, Fitzpatrick, Sharon, del Jesús González Guillén, Anais, Ku, Venus, Paprocki, Julie, Platt, Lindsay, Steele, Bethel, Wright, Kyle, & Yasin, Brandon. (2022). Images and 4-class labels for semantic segmentation of Sentinel-2 and Landsat RGB satellite images of coasts (water, whitewater, sediment, other) (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7335647</p> <p>This supplemental dataset consists of 283 RGB images and 283 associated labels for semantic segmentation of Sentinel-2 and Landsat RGB satellite images of coasts. Of these, 77 images-label pairs also have a corresponding NIR and SWIR satellite image. The 4 classes are 0=water, 1=whitewater, 2=sediment, 3=other</p> <p>These images and labels have been made using the Doodleverse software package, Doodler*. These images and labels could be used within numerous Machine Learning frameworks for image segmentation, but have specifically been made for use with the Doodleverse software package, Segmentation Gym**.</p> <p>Imagery are a mixture of 10-m Sentinel-2 and 15-m pansharpened Landsat 7, 8, and 9 visible-band imagery of various sizes. NIR, SWIR, Red, Green, and Blue bands only</p> <p><strong>File descriptions</strong></p> <ol> <li>classes.txt, a file containing the class names</li> <li>images.zip, a zipped folder containing the 3-band images of varying sizes and extents</li> <li>labels.zip, a zipped folder containing the 1-band label images</li> <li>overlays.zip, a zipped folder containing a semi-transparent overlay of the color-coded label on the image (blue=0=water, red=1=whitewater, yellow=2=sediment, green=3=other)</li> <li>nir.zip</li> <li>swir.zip</li> </ol> <p><strong>References</strong></p> <p>Buscombe, Daniel, Goldstein, Evan, Bernier, Julie, Bosse, Stephen, Colacicco, Rosa, Corak, Nick, Fitzpatrick, Sharon, del Jesús González Guillén, Anais, Ku, Venus, Paprocki, Julie, Platt, Lindsay, Steele, Bethel, Wright, Kyle, & Yasin, Brandon. (2022). Images and 4-class labels for semantic segmentation of Sentinel-2 and Landsat RGB satellite images of coasts (water, whitewater, sediment, other) (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7335647</p> <p>*Doodler: Buscombe, D., Goldstein, E.B., Sherwood, C.R., Bodine, C., Brown, J.A., Favela, J., Fitzpatrick, S., Kranenburg, C.J., Over, J.R., Ritchie, A.C. and Warrick, J.A., 2021. Human‐in‐the‐Loop Segmentation of Earth Surface Imagery. Earth and Space Science, p.e2021EA002085<a href="https://doi.org/10.1029/2021EA002085">https://doi.org/10.1029/2021EA002085</a>. See <a href="https://github.com/Doodleverse/dash_doodler">https://github.com/Doodleverse/dash_doodler.</a></p> <p>**Segmentation Gym: Buscombe, D., & Goldstein, E. B. (2022). A reproducible and reusable pipeline for segmentation of geoscientific imagery. Earth and Space Science, 9, e2022EA002332. <a href="https://doi.org/10.1029/2022EA002332">https://doi.org/10.1029/2022EA002332</a> See: <a href="https://github.com/Doodleverse/segmentation_gym">https://github.com/Doodleverse/segmentation_gym</a></p> <p> </p>
Deeptangle Dataset: Labelled Experimental and Synthetic Videos of Swimming and Overlapping C. elegans worms
<p>This repository contains the dataset employed in the paper <a href="https://arxiv.org/abs/2301.04460">Fast spline detection in high density microscopy data</a>.</p> <p>Three files are provided:</p> <p>1. <em>videos.zip</em>: raw experimental videos.<br> 2. <em>labeled_data.zip</em>: labelled sections of experimental videos used for evaluation<br> 3. <em>syntehthic_dataset.zip</em>: synthetic dataset used for training</p> <p><strong>Labelled data</strong></p> <p>Sections are named as VIDEONAME_FRAMENUMBER_SECTIONID.<br> All labels are in labels.json and correspond to the middle frame of the clip (05.png).<br> Example plotting script is provided (<em>plot_data.py</em>).</p> <p><strong>Synthetic data</strong></p> <p>Clips are named as NUMBEROFWORMS_ID.<br> Labels (for all frames) are stored in labels.npy.<br> Example plotting script is provided (plot_data.py).</p> <p> </p> <p><strong>Related</strong></p> <p>Paper: <a href="https://arxiv.org/abs/2301.04460">https://arxiv.org/abs/2301.04460</a></p> <p>Deeptangle code: <a href="https://github.com/kirkegaardlab/deeptangle">https://github.com/kirkegaardlab/deeptangle</a></p> <p>Labelling tool: <a href="https://github.com/kirkegaardlab/deeptanglelabel">https://github.com/kirkegaardlab/deeptanglelabel</a></p> <p>---</p> <p>If used, please cite</p> <p><em>Albert Alonso & Julius B. Kirkegaard. Fast spline detection in high density microscopy data. 2023.</em></p>
NPM3D dataset with instance labels used in paper "Toward Accurate Instance Segmentation in Large-scale LiDAR Point Clouds"
<p>NPM3D (https://npm3d.fr/paris-carla-3d) consists of mobile laser scanning (MLS) point clouds collected in four different regions in the French cities of Paris and Lille, where each point has been annotated with two labels: one that assigns it to one out of 10 semantic categories and another one that assigns it to an object instance. When inspecting the data, we found 9 cases where multiple tree instances had not been separated correctly (i.e., they had the same ground truth instance label). These cases were manually corrected using the CloudCompare software (https://www.cloudcompare.org), and 35 individual tree instances were obtained. Our variant of the dataset with 10 semantic categories and enhanced instance labels is publicly available.</p>
Dataset for "Blue-shift photoconversion of near-infrared fluorescent proteins for labeling and tracking in living cells and organisms"
<p>Dataset that supports the observation, characterization and application of the blue-shift photoconversion of the near infrared proteins, miRFPs, reported in the manuscript: "Blue-shift photoconversion of near-infrared fluorescent proteins for labeling and tracking in living cells and organisms". The data references to the specific figures and graphs in the manuscript.</p>
Data on soil compounds, respiration and incorporation of 13C-labeled substrate
<p>Root exudation increases the concentration of readily available carbon (C) compounds in its immediate environment. This creates ‘hotspots’ of microbial activity characterized by accelerated soil organic matter turnover with direct implications for nutrient availability for plants. However, we still lack a deeper understanding of the microbial metabolic processes that occur in the immediate vicinity of the roots during and after a root exudation event. Even though theoretical concepts have been developed, the direct consequences of root exudation on microbial metabolism and nutrient availability have never been measured in their immediate environment in intact soil.</p> <p>Here, we used reverse microdialysis to simulate root exudation by releasing a <sup>13</sup>C-labelled mix of low-molecular-weight organic C compounds at discrete, mm-sized locations in undisturbed soil in combination with <sup>13</sup>C stable isotope tracing. This approach allowed us to investigate the fine-scale temporal and spatial response of microbial metabolism and soil chemistry to root exudation at the mm-scale, and to trace microbial respiration and uptake of exuded compounds.</p> <p>Our results show that a 9-hour simulated root exudation pulse leads to i) a large local respiration event and ii) alteration of the temporal dynamics of soil metabolites over the following twelve days right at the spot of exudate release. Notably, we observed an approximately threefold increase in ammonium concentrations twelve hours after the pulse and increased nitrate concentrations five days after the pulse. We also observed an increase of various short-chain fatty acids, such as acetate, propionate and formate over the following days, indicating altered microbial metabolic pathways and activity. Phospholipid and neutral lipid fatty acids (PLFAs and NLFAs) of all major microbial groups were significantly enriched in <sup>13</sup>C within a radius of 5 mm around the microdialysis probes, but not beyond. The highest relative <sup>13</sup>C enrichment was observed in fungal NLFAs, indicating that a significant proportion of the exuded compounds had been incorporated into fungal storage compounds.</p> <p>Our findings indicate that the punctual release of low-molecular weight organic C compounds into intact soil significantly changes microbial metabolism and activity in its immediate surroundings, which lead to enhanced mineralisation of native organic nitrogen (N). Our observations emphasise the versatility of microbial metabolic pathways that underlie the response of soil microbes to rapidly altered C availability. They furthermore demonstrate the effectiveness of this response, as triggered by root exudation pulses, to increase nutrient availability for plants around the root.</p>
Input and output data (images + boulder labels, model setup, model weights and more) for the manuscript "Automatic characterization of boulders on planetary surfaces from high-resolution satellite images"
<p><strong>File 1:</strong> raw_data_BOULDERING.zip</p> <p><strong>Size:</strong> 8.8 GB</p> <p><strong>Summary: </strong>It contains all of the rasters (planetary images) and labeled boulders (raw data):</p> <ul> <li> <p>a boulder-mapping file, which is the manually digitized outline of boulders.</p> </li> <li> <p>a ROM file (stands for Region of Mapping), which depicts the image patches on which the boulder mapping has been conducted.</p> </li> <li> <p>a global-tiles file, which shows all of the image patches within a raster.</p> </li> </ul> <p>There are multiple locations/images per planetary body.</p> <p><strong>Structure:</strong></p> <pre>. └── raw_data/ ├── earth/ │ └── image_name/ │ ├── shp/ │ │ ├── <image_name>-ROM.shp │ │ ├── <image_name>-boulder-mapping.shp │ │ └── <image_name>-global-tiles.shp │ └── raster/ │ └── <image_name>.tif ├── mars/ │ └── image_name/ │ ├── shp/ │ │ ├── <image_name>-ROM.shp │ │ ├── <image_name>-boulder-mapping.shp │ │ └── <image_name>-global-tiles.shp │ └── raster/ │ └── <image_name>.tif └── moon/ └── image_name/ ├── shp/ │ ├── <image_name>-ROM.shp │ ├── <image_name>-boulder-mapping.shp │ └── <image_name>-global-tiles.shp └── raster/ └── <image_name>.tif</pre> <p> </p> <p><strong>File 2:</strong> best_model.zip</p> <p><strong>Size:</strong> 624.7 MB</p> <p><strong>Summary:</strong></p> <p>This zip file contains all of the inputs and outputs required/obtained from the training of the BoulderNet Mask R-CNN model (model setup, augmentation pipeline, model weights, log during training, logged metrics):</p> <ul> <li> <p>augmentation_pipeline.json (required as inputs for the training of the algorithm to apply augmentations). See <a href="https://github.com/astroNils">https://github.com/astroNils</a> and the MLtools repository for more information.</p> </li> </ul> <ul> <li> <p>Base-RCNN-FPN.yaml (base model setup file).</p> </li> <li> <p>config.yaml (complete model setup file, merge of the base and Mars-Moon-Earth setup file).</p> </li> <li> <p>Mars-MoonEarth-v050...yaml (model setup file).</p> </li> <li> <p>log.txt (log during training of the algorithm).</p> </li> <li> <p>model_0055999.pth (model weights at second last saving step)</p> </li> <li> <p>model_0063999.pth (model weights at last saving step)</p> </li> </ul> <p>We advice the use of model weights model_0055999.pth (to avoid slight overfitting).</p> <p><strong>File 3:</strong> Apr2023-Mars-Moon-Earth-mask-5px.zip (pre-processed input images)</p> <p><strong>Size:</strong> 252.8 MB</p> <p><strong>Summary:</strong></p> <p>This zip files contains the input data (images and boulder outlines) for the train, validation and test datasets. See <a href="https://github.com/astroNils">https://github.com/astroNils</a> and the MLtools repository for more information in how-to-use the different files.</p> <ul> <li> <p>The json folder contains json files that can be given as input (as a custom dataset) to the Detectron2 platform. The only differences between the two files is how the bounding boxes around masks have been generated. We advised to use "Apr2023-Mars-Moon-Earth-mask-5px.json".</p> </li> <li> <p>The pkl folder and pickle file includes some informations about the 950 image patches in our boulder dataset.</p> </li> <li> <p>The pre-processing folder contains all of the training, validation and test image patches and corresponding shapefiles.</p> </li> <li> <p>The shapefile folder is actually empty (it should not be there!).</p> </li> </ul> <p><strong>Structure:</strong></p> <pre>. └── preprocessed_inputs/ ├── json ├── pkl ├── preprocessing/ │ ├── train/ │ │ ├── images │ │ └── labels │ ├── validation/ │ │ ├── images │ │ └── labels │ └── test/ │ ├── images │ └── labels └── shp</pre> <p> </p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.