Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
1,832
datasets available to search
ShareScore release 0.9.0
Dataset results
1,832 results for “.NET”
Annual Net Primary Production on the Main Cropping System Experiment at the Kellogg Biological Station, Hickory Corners, MI (1990 to 2018)
Dataset Abstract Aboveground annual net primary production (ANPP) has been measured on the LTER main site since 1990 and on the successional and forested sites since 1993. ANPP is measured at peak biomass for a given treatment. In some systems with multiple harvests or complex communities that have peaks occurring at different times of the year, measurements are taken at multiple times per year. Additional ANPP measurements are made where appropriate using leaf litter traps, estimates of diameter from tree basal diameter and for the poplar treatment occasional destructive harvests. See the ANPP protocol for descriptions of the sampling and measurement methods for each of the treatments. original data source http://lter.kbs.msu.edu/datasets/22
KKE02 The Konza-Kruger Experiment: Net primary production data (2010-2011)
The distribution, structure and function of mesic savanna grasslands are strongly driven by fire regimes, grazing by large herbivores, and their interactions. This research addresses a general question about our understanding of savanna grasslands globally: Is our knowledge of fire and grazing sufficiently general to enable us to make accurate predictions of how these ecosystems will respond to changes in these drivers over time? Some evidence suggests that fire and grazing influence savanna grassland structure and function differently in South Africa (SA) compared to North America (NA). These differences have been attributed to the contingent factors of greater biome age, longer evolutionary history with fire and grazing, reduced soil fertility, and greater diversity of plants and large herbivores in SA. An alternative hypothesis is that differences in methods and approaches used to study these systems have led to differing perspectives on the role of these drivers. If the impacts of shared ecosystem drivers truly differ between NA and SA, this calls into question the generality of our understanding of these ecosystems and our ability to forecast how changes in key drivers will affect savanna grasslands globally. Since 2006, an explicitly comparative research program has been conducted to determine the degree of convergence in ecosystem (productivity, N and C cycling) and plant community (composition, diversity, dynamics) responses to fire and grazing in SA and NA. Thus far, initial support has been found for convergence at the ecosystem level and divergence at the community level in response to alterations in both fire regimes and grazing. However, there have also been two unexpected findings (1) the ways in which fire and grazing interact differed between NA and SA, and (2) the rate of change in communities when grazers were removed was much greater in NA than in SA. These unexpected findings raise a number of important new questions: (Q1) Will exclusion of grazi
PBB01 Belowground Plot Experiment: Aboveground net primary productivity of tallgrass prairie based on accumulated plant biomass
To address the potential interactive effects of fire, aboveground biomass removal, and nutrient amendments on above- and belowground responses, a long-term field experiment was initiated in 1986 as part of the Konza Prairie Long-Term Ecological Research (LTER) program. The general goals of this experiment are: 1) to document both short- and long-term responses of plants and soils to fire, aboveground biomass removal (a surrogate for grazing in these small plots), and nutrient amendments (additions of N and/or P); and 2) to provide a better understanding of the mechanisms underlying tallgrass prairie responses to fire, aboveground biomass removal and nutrient enrichment. Peak foliage biomass is measured annually in late fall (September to October) on the 64 belowground plots. Effects of burning, mowing and N + P additions on aboveground NPP are measured. Two 0.1m2 quadrats harvested per plot). 2003 was the last year the mowing treatment was implemented.
PEB01 Aboveground net primary productivity of tallgrass prairie based on accumulated plant biomass in grazing exclsoures on bison-grazed watersheds
Data set contains estimates of end-of-season standing crop biomass (grams per square meter) of live graminoids, forbs, woody plants, and previous year's dead vegetation in grazing exclosures. Date from exclosures is used to determine long-term effects of bison grazing on aboveground net primary productivity.
PTN01 Aboveground net primary productivity along transects spanning topographic gradients on an annually burned and unburned watershed at Konza Prairie
In 1989, single transects spanning upland-lowland-upland topographic positions were established in a long-term unburned (0020B) and an annually burned (001D) watershed. Standing crop biomass data were collected in late season at 11 sites along each transect and sorted into live graminoids, forbs and woody plants, current year's dead, and previous years dead vegetation. Four 0.1 m2 quadrats were harvested at each of the 11 sites per watershed and all data except previous years' dead were combined to provide an estimate of aboveground NPP. In 1993, soil moisture measurements began along each transect at 15 and 30 cm depths (where possible) with a Time Domain Reflectomerty (TDR) system. Measurements were made twice a month from March - October and intermittently during the winter months.
PAB02 Biweekly measurement of aboveground net primary productivity on an unburned and annually burned watershed at Konza Prairie
Data set contains estimates of standing crop biomass (grams per square meter) of live graminoids, forbs, woody plants, current year's dead, and previous year's dead vegetation. Twenty quadrats (0.1 square meters) are harvested for each watershed (001a and 020a) on each sample date.
PAB05 Aboveground net primary productivity of tallgrass prairie based on accumulated plant biomass on the LTER fire reversal experiment watersheds
Data set contains estimates of end-of-season standing crop biomass (grams per square meter) of live graminoids, forbs, woody plants, and previous year's dead vegetation for 2 soil types (shallow and deep) on the four Fire Reversal Experiment watersheds. This experiment is based on reversing fire treatments on four watersheds, two of which had a history of annual spring burning and two of which had a history of long-term fire suppression. The dataset includes both pre- and post-fire treatments.
Bird abundance-mist nets in the Luquillo Forest Dynamics Plot and Nora Devoe Cut Plots in the Luquillos Experimental Forest
This project seeks to measure long-term trends and variability in bird populations in tabonuco forest at El Verde. Repeated measures of bird populations using mist nets are conducted annually in the Luquillo Forest Dynamics Plot. For each bird caught, I obtain physical measurements (weight, length of wing, length of tail), and estimates of fat, age, and breeding condition. Captured birds are marked to estimate mortality and longevity. Data on canopy openness are also collected annually at each mist net as part of this project. Support for this work was provided by grants BSR-8811902, DEB-9411973, DEB-9705814 , DEB-0080538, DEB-0218039 , DEB-0620910 , DEB-1239764, DEB-1546686, and DEB-1831952 from the National Science Foundation to the University of Puerto Rico as part of the Luquillo Long-Term Ecological Research Program. Additional support provided by the University of Puerto Rico and the International Institute of Tropical Forestry, USDA Forest Service.
Ash-free dry mass data from drift net experiments in glacial-melt streams of Fryxell Basin, McMurdo Dry Valleys, Antarctica, during the 2022-2023 austral summer
During the 2022-2023 austral summer, 12 drift net experiments were conducted in four glacial-melt streams—Aiken Creek, Green Creek, Lost Seal Stream, and Von Guerard Stream—to quantify particulate organic matter (POM) flux from streams to downstream lakes. Streams are located in Fryxell Basin, within Taylor Valley in the McMurdo Dry Valleys of Antarctica. Drift nets were deployed near the mouth of each stream and sampled every 4 to 12 hours to collect POM. Collected as part of the McMurdo Dry Valleys LTER project, these data provide insight into the temporal variability of organic material transport in polar desert ecosystems.
Net community production (NCP) and gross oxygen production (GOP), based on oxygen-argon ratios and triple oxygen isotopes, from seasonal NES-LTER Transect cruises in 2020
This data package provides net community production (NCP) and gross oxygen production (GOP, a measure of gross primary production) for the winter, spring, and summer Northeast U.S. Shelf Long-Term Ecological Research (NES-LTER) Transect cruises in 2020. Two tables are provided: a high-frequency table with NCP rates calculated from measurements of O2/Ar made continuously by an at-sea equilibrator inlet mass spectrometer (EIMS), and a low-frequency table with both NCP and GOP rates calculated for discrete samples measured post-cruise. The GOP rates were calculated from triple O2 isotopic (TOI) ratios. These data are derived from the EIMS and TOI data for the NES-LTER Transect cruises in EDI data package knb-lter-nes.6.3.
Net community production (NCP) and gross oxygen production (GOP), based on oxygen-argon ratios and triple oxygen isotopes, from seasonal NES-LTER Transect cruises in 2021
This data package provides net community production (NCP) and gross oxygen production (GOP, a measure of gross primary production) for the winter and summer Northeast U.S. Shelf Long-Term Ecological Research (NES-LTER) Transect cruises in 2021. Two tables are provided: a high-frequency table with NCP rates calculated from measurements of O2/Ar made continuously by an at-sea equilibrator inlet mass spectrometer (EIMS), and a low-frequency table with both NCP and GOP rates calculated for discrete samples measured post-cruise. The GOP rates were calculated from triple O2 isotopic (TOI) ratios. These data are derived from the EIMS and TOI data for the NES-LTER Transect cruises in EDI data package knb-lter-nes.6.3.
Net community production (NCP) and gross oxygen production (GOP), based on oxygen-argon ratios and triple oxygen isotopes, from seasonal NES-LTER Transect cruises in 2022
This data package provides net community production (NCP) and gross oxygen production (GOP, a measure of gross primary production) for the winter and summer Northeast U.S. Shelf Long-Term Ecological Research (NES-LTER) Transect cruises in 2022. Two tables are provided: a high-frequency table with NCP rates calculated from measurements of O2/Ar made continuously by an at-sea equilibrator inlet mass spectrometer (EIMS), and a low-frequency table with both NCP and GOP rates calculated for discrete samples measured post-cruise. The GOP rates were calculated from triple O2 isotopic (TOI) ratios. These data are derived from the EIMS and TOI data for the NES-LTER Transect cruises in EDI data package knb-lter-nes.6.3.
Larval euphausiids collected using a 1 x 1 m square frame net with 333-μm mesh aboard Palmer LTER annual cruises off the coast of the Western Antarctic Peninsula, 1993-2013
Euphausiids (krill) are abundant along the Western Antarctic Peninsula where they have important impacts on the marine ecosystem and biogeochemical cycling. Euphausiids develop through a series of morphologically distinct larval stages within the first year of their life cycle. The calyptopis stages are followed by the furcilia stages before individuals recruit to the post-larval population. Larvae collected during Janury were most likely spawned during the preceding weeks or months of Antarctic spring/summer. Euphausia superba and Thysanoessa macrura are the most abundant euphausiid species along the Western Antarctic Peninsula. Euphausia crystallorophias, Euphausia frigida, and Euphausia triacantha are present but less abundant in the region. Samples were collected with a 1 x 1 m square frame net with 333-μm mesh towed obliquely to a depth of typically 300 m. Density of total calyptopis and furcilia larvae (all species combined) was determined for a subset of samples collected on the annual Palmer LTER cruises to cover the latitudinal and cross-shelf gradients of the study region. Larval euphausiid abundance varies spatially as spawning output is not homogeneous across the region. Larval euphausiid abundance also varies year-to-year due to changes in population demographics and environmental conditions.
Mesozooplankton taxonomic density collected using a 1-m diameter ring net with 200-μm mesh at Palmer Station, Antarctica during Palmer LTER field seasons, 2017-2020
Zooplankton are a morphologically and taxonomically diverse group of animals. Many zooplankton feed on phytoplankton in surface waters and thus provide a link between primary producers and higher trophic levels. The numerical density of common mesozooplankton taxa was determined at Palmer LTER Stations B and E. Samples were collected with a 1-m diameter, 200-μm mesh ring net towed obliquely from the surface to a target depth of 50 m and back. Tows were conducted during daytime, and sampling frequency was nominally twice weekly while personnel were at Palmer Station between the months of November and March. The preserved samples were size-fractionated with nested sieves into five size classes (0.2−0.5, 0.5−1, 1−2, 2−5, and >5 mm) prior to microscopic enumeration. Data are provided for the following taxa: copepods Oithona spp., Calanoides acutus (>1 mm only), Calanus propinquus (>1 mm only), Rhincalanus gigas (>1 mm only), and small calanoids (0.2−1 mm), chaetognaths, asteroid larvae, nemertean larvae, and foraminifera (not quantified in all years). Individual size fractions were split and subsampled such that at least 100 individuals of the most abundant taxon were present. Density varies across taxa, seasonally, among years, and between sampling stations. Units of density are individuals per cubic meter.
Macrozooplankton taxonomic density collected using a 1 x 1 m square net with 700-μm mesh at Palmer Station, Antarctica during Palmer LTER field seasons, 2017-2020
Zooplankton are a morphologically and taxonomically diverse group of animals. Many zooplankton feed on phytoplankton in surface waters and thus provide a link between primary producers and higher trophic levels. The numerical density of common macrozooplankton taxa was determined at Palmer LTER Stations B and E. Samples were collected with a 1 x 1 m square, 700-μm mesh Metro net towed obliquely from the surface to a target depth of 50 m and back. Duplicate tows typically were conducted at each sampling site. Tows were conducted during daytime, and sampling frequency was nominally twice weekly while personnel were at Palmer Station between the months of November and March. The catch was sorted and counted live. Data are provided for the following taxa, which dominated biomass: the euphausiids Euphausia superba and Thysanoessa macrura, the thecosome pteropod Limacina rangii, gymnosome pteropods, the salp Salpa thompsoni, amphipods, and larval fishes. Density varies across taxa, seasonally, among years, and between sampling stations. Units of density are individuals per cubic meter.
Size-fractionated zooplankton dry weight collected using a 1-m diameter ring net with 200-μm mesh at Palmer Station, Antarctica during Palmer LTER field seasons, 2017-2020
Zooplankton are a morphologically and taxonomically diverse group of animals. Many zooplankton feed on phytoplankton in surface waters and thus provide a link between primary producers and higher trophic levels. The density of zooplankton dry weight for five size fractions was determined at Palmer LTER Stations B and E. Samples were collected with a 1-m diameter, 200-μm mesh ring net towed obliquely from the surface to a target depth of 50 m and back. Tows were conducted during daytime, and sampling frequency was nominally twice weekly while personnel were at Palmer Station between the months of November and March. One-half of the catch was size-fractionated with nested sieves into the following five size classes for biomass analysis: 0.2−0.5, 0.5−1, 1−2, 2−5, and >5 mm. Individual size fractions were concentrated on preweighed 200 μm mesh filters and frozen at −20°C until analysis. Samples were thawed, weighed to determine wet biomass, dried at 60°C for at least 24 h, and weighed again to determine dry biomass. Zooplankton density varies across size groups, seasonally, among years, and between sampling stations. Units of biomass density are milligrams dry weight per cubic meter.
Net Primary Productivity (NPP) Weight Data at the Sevilleta National Wildlife Refuge, New Mexico
Several long-term studies at the Sevilleta LTER measure net primary production (NPP) across ecosystems and treatments. Net primary production is a fundamental ecological variable that quantifies rates of carbon consumption and fixation. Estimates of NPP are important in understanding energy flow at a community level as well as spatial and temporal responses to a range of ecological processes. Above-ground net primary production (ANPP) is the change in plant biomass, including loss to death and decomposition, over a given period of time. To measure this change, vegetation variables, including species composition and the cover and height of individuals, are sampled up to three times yearly (winter, spring, and fall) at permanent plots within a study site. The weight data presented here is obtained by harvesting a series of covers for species observed during plot sampling. These species are always harvested from habitat comparable to the plots in which they were recorded. This data is then used to make volumetric measurements of species and build regressions correlating biomass and volume. From these calculations, seasonal biomass and seasonal and annual NPP are determined.
Core Site Grid Quadrat Data for the Net Primary Production Study at the Sevilleta National Wildlife Refuge, New Mexico
Begun in spring 2013, this project is part of a long-term study at the Sevilleta LTER measuring net primary production (NPP) across three distinct ecosystems: creosote-dominant shrubland (Site C), black grama-dominant grassland (Site G), and blue grama-dominant grassland (Site B). Net primary production is a fundamental ecological variable that quantifies rates of carbon consumption and fixation. Estimates of NPP are important in understanding energy flow at a community level as well as spatial and temporal responses to a range of ecological processes. Above-ground net primary production is the change in plant biomass, represented by stems, flowers, fruit and foliage, over time and incorporates growth as well as loss to death and decomposition. To measure this change the vegetation variables in this dataset, including species composition and the cover and height of individuals, are sampled twice yearly (spring and fall) at permanent 1m x 1m plots within each site. A third sampling at Site C is performed in the winter. The data from these plots is used to build regressions correlating biomass and volume via weights of select harvested species obtained in SEV999, "Net Primary Productivity (NPP) Weight Data." This biomass data is included in SEV999, "Seasonal Biomass and Seasonal and Annual NPP for Core Grid Research Sites."
Long-term dynamics of soil organic matter and aboveground net primary production in a Chihuahuan Desert Grassland at the Sevilleta National Wildlife Refuge, New Mexico (1989-2014)
Drylands contain a third of the organic carbon stored in global soils; however, the long-term dynamics of soil organic carbon and soil organic matter (SOM) in drylands remain poorly understood relative to dynamics of the vegetation carbon pool. We examined long-term patterns in SOM against both climate and prescribed fire in a Chihuahuan Desert grassland in central New Mexico, USA. SOM was measured each spring and fall for 25 years (1989–2014) in unburned desert grassland and from 2003 to 2014 following a prescribed fire. SOM concentration from 0-20 cm depth did not show a clear long-term trend but fluctuated seasonally at both burned and unburned sites, ranging from a minimum of 0.9% to a maximum of 3.3%. SOM concentration declined nonlinearly in wet seasons and peaked in dry seasons. These results not only contrast with the positive relationships between aboveground net primary production and precipitation for this region, but also with previous reports of greater SOM in wetter sites across drylands globally, suggesting that space is not a good substitute for time in predicting the dynamics of dryland SOM. We suggest that declines in SOM in wet periods are caused by increased soil respiration, runoff, leaching, and soil erosion. In addition to tracking natural variability in climate, SOM concentration also decreased by 14% following prescribed fire, a response that magnified over time and has persisted for nearly a decade due to the slow recovery of primary production. Our results document the surprisingly dynamic nature of soil organic matter and its high sensitivity to climate and fire in this dryland ecosystem.
Block-wise sparse matrix-vector product dataset and convolutional neural nets for estimating the run time and energy consumption of the sparse matrix-vector product
<p><strong>Introduction</strong></p> <p><strong>SpMV-CNN</strong> is a set of Convolutional Neural Networks (CNNs) that provide accurate estimations of the performance and energy consumption of the SpMV kernel. The proposed CNN-based models use a block-wise approach to make the CNN architecture independent of the matrix size. These models cat be trained to estimate run time as well as total, package and DRAM energy consumption at different processor frequencies.</p> <p><strong>Prerequisites</strong></p> <p><strong>SpMV-CNN</strong> requires Python3 with the following packages:</p> <pre><code>keras==2.1.6 tensorflow==1.8.0 h5py==2.7.1 matplotlib==2.1.1 scikit-learn==0.19.1 </code></pre> <p><strong>Obtaining the dataset</strong></p> <p>The execution time and energy consumption data corresponding to the SpMV operation on a set of sparse matrices from the SuiteSparse Matrix Collection have been obtained on an Intel Xeon E5-2630 core running at frequencies 1.2, 1.6, 2.0, 2.4 GHz. The energy consumption measurements are obtained via the Intel RAPL interface and gathered at three different levels (total, package and DRAM, where total = package + DRAM) for this specific processor.</p> <p>The <code>spmv-cnn-dataset.tgz</code> archive contains the whole dataset, including the following HDF5 files:</p> <pre><code>$ tree . |-- test | |-- f_1200000_b250 | | |-- output_2cubes_sphere_1200000.h5 | | |-- output_apache2_1200000.h5 | | |-- output_bcsstk36_1200000.h5 | | |-- output_cfd1_1200000.h5 | | |-- output_cfd2_1200000.h5 | | |-- output_ct20stif_1200000.h5 | | |-- output_denormal_1200000.h5 | | |-- output_Dubcova2_1200000.h5 | | |-- output_Dubcova3_1200000.h5 | | |-- output_ecology2_1200000.h5 | | |-- output_gyro_1200000.h5 | | |-- output_gyro_k_1200000.h5 | | |-- output_msc10848_1200000.h5 | | |-- output_msc23052_1200000.h5 | | |-- output_nasasrb_1200000.h5 | | |-- output_nd3k_1200000.h5 | | |-- output_offshore_1200000.h5 | | |-- output_oilpan_1200000.h5 | | |-- output_olafu_1200000.h5 | | |-- output_parabolic_fem_1200000.h5 | | |-- output_qa8fm_1200000.h5 | | |-- output_raefsky4_1200000.h5 | | |-- output_s3dkq4m2_1200000.h5 | | |-- output_s3dkt3m2_1200000.h5 | | |-- output_ship_001_1200000.h5 | | |-- output_ship_003_1200000.h5 | | |-- output_shipsec1_1200000.h5 | | |-- output_shipsec5_1200000.h5 | | |-- output_shipsec8_1200000.h5 | | |-- output_smt_1200000.h5 | | |-- output_thermomech_dM_1200000.h5 | | |-- output_thread_1200000.h5 | | `-- output_vanbody_1200000.h5 | |-- f_1600000_b250 | | |-- output_2cubes_sphere_1600000.h5 | | |—- ... | | `-- output_vanbody_1600000.h5 | |-- f_2000000_b250 | | |-- output_2cubes_sphere_2000000.h5 | | |—- ... | | `-- output_vanbody_2000000.h5 | `-- f_2400000_b250 | |-- output_2cubes_sphere_2400000.h5 | |—- ... | `-- output_vanbody_2400000.h5 |-- test_pagerank | |-- f_1200000_b250 | | |-- output_adaptive_1200000.h5 | | |-- output_cit-HepPh_1200000.h5 | | |-- output_delaunay_n22_1200000.h5 | | |-- output_email-Enron_1200000.h5 | | |-- output_email-EuAll_1200000.h5 | | |-- output_europe_osm_1200000.h5 | | |-- output_hugebubbles-00020_1200000.h5 | | |-- output_rgg_n_2_24_s0_1200000.h5 | | |-- output_road_usa_1200000.h5 | | |-- output_Stanford_1200000.h5 | | |-- output_wb-edu_1200000.h5 | | |-- output_web-BerkStan_1200000.h5 | | |-- output_web-Google_1200000.h5 | | |-- output_web-NotreDame_1200000.h5 | | |-- output_wiki-Talk_1200000.h5 | | `-- output_wiki-Vote_1200000.h5 | |-- f_1600000_b250 | | |-- output_adaptive_1600000.h5 | | |—- ... | | `-- output_wiki-Vote_1600000.h5 | |-- f_2000000_b250 | | |-- output_adaptive_2000000.h5 | | |—- ... | | `-- output_wiki-Vote_2000000.h5 | `-- f_2400000_b250 | |-- output_adaptive_2400000.h5 | |—- ... | `-- output_wiki-Vote_2400000.h5 `-- train |-- merged_energy_train_shuffle_f1200000_250.h5 |-- merged_energy_train_shuffle_f1600000_250.h5 |-- merged_energy_train_shuffle_f2000000_250.h5 `-- merged_energy_train_shuffle_f2400000_250.h5 </code></pre> <p>The matrices contained in the merged training files (<code>merged_energy_train_shuffle_fXX00000_250.h5</code>) are the following:</p> <pre><code>$ tree . |-- output_af_0_k101_1200000.h5 |-- output_af_1_k101_1200000.h5 |-- output_af_2_k101_1200000.h5 |-- output_af_3_k101_1200000.h5 |-- output_af_4_k101_1200000.h5 |-- output_af_5_k101_1200000.h5 |-- output_af_shell10_1200000.h5 |-- output_af_shell1_1200000.h5 |-- output_af_shell2_1200000.h5 |-- output_af_shell3_1200000.h5 |-- output_af_shell4_1200000.h5 |-- output_af_shell5_1200000.h5 |-- output_af_shell6_1200000.h5 |-- output_af_shell7_1200000.h5 |-- output_af_shell8_1200000.h5 |-- output_af_shell9_1200000.h5 |-- output_atmosmodd_1200000.h5 |-- output_atmosmodj_1200000.h5 |-- output_atmosmodl_1200000.h5 |-- output_audikw_1_1200000.h5 |-- output_BenElechi1_1200000.h5 |-- output_bmw3_2_1200000.h5 |-- output_bmw7st_1_1200000.h5 |-- output_bmwcra_1_1200000.h5 |-- output_bone010_1200000.h5 |-- output_boneS01_1200000.h5 |-- output_boneS10_1200000.h5 |-- output_bundle_adj_1200000.h5 |-- output_cage14_1200000.h5 |-- output_cage15_1200000.h5 |-- output_circuit5M_1200000.h5 |-- output_circuit5M_dc_1200000.h5 |-- output_CO_1200000.h5 |-- output_consph_1200000.h5 |-- output_CoupCons3D_1200000.h5 |-- output_crankseg_1_1200000.h5 |-- output_crankseg_2_1200000.h5 |-- output_CurlCurl_2_1200000.h5 |-- output_CurlCurl_3_1200000.h5 |-- output_CurlCurl_4_1200000.h5 |-- output_dielFilterV2real_1200000.h5 |-- output_dielFilterV3real_1200000.h5 |-- output_Emilia_923_1200000.h5 |-- output_ESOC_1200000.h5 |-- output_F1_1200000.h5 |-- output_F2_1200000.h5 |-- output_Fault_639_1200000.h5 |-- output_Freescale1_1200000.h5 |-- output_Freescale2_1200000.h5 |-- output_FullChip_1200000.h5 |-- output_G3_circuit_1200000.h5 |-- output_Ga10As10H30_1200000.h5 |-- output_Ga19As19H42_1200000.h5 |-- output_Ga3As3H12_1200000.h5 |-- output_Ga41As41H72_1200000.h5 |-- output_Ge87H76_1200000.h5 |-- output_Ge99H100_1200000.h5 |-- output_Geo_1438_1200000.h5 |-- output_gsm_106857_1200000.h5 |-- output_Hardesty3_1200000.h5 |-- output_hood_1200000.h5 |-- output_Hook_1498_1200000.h5 |-- output_human_gene1_1200000.h5 |-- output_human_gene2_1200000.h5 |-- output_inline_1_1200000.h5 |-- output_JP_1200000.h5 |-- output_kkt_power_1200000.h5 |-- output_ldoor_1200000.h5 |-- output_Long_Coup_dt0_1200000.h5 |-- output_Long_Coup_dt6_1200000.h5 |-- output_mat_104_10000_1200000.h5 |-- output_mat_104_1000_1200000.h5 |-- output_mat_104_5000_1200000.h5 |-- output_mat_112_10000_1200000.h5 |-- output_mat_112_1000_1200000.h5 |-- output_mat_112_5000_1200000.h5 |-- output_mat_120_10000_1200000.h5 |-- output_mat_120_1000_1200000.h5 |-- output_mat_120_5000_1200000.h5 |-- output_mat_128_10000_1200000.h5 |-- output_mat_128_1000_1200000.h5 |-- output_mat_128_5000_1200000.h5 |-- output_mat_16_10000_1200000.h5 |-- output_mat_16_1000_1200000.h5 |-- output_mat_16_5000_1200000.h5 |-- output_mat_24_10000_1200000.h5 |-- output_mat_24_1000_1200000.h5 |-- output_mat_24_5000_1200000.h5 |-- output_mat_32_10000_1200000.h5 |-- output_mat_32_1000_1200000.h5 |-- output_mat_32_5000_1200000.h5 |-- output_mat_40_10000_1200000.h5 |-- output_mat_40_1000_1200000.h5 |-- output_mat_40_5000_1200000.h5 |-- output_mat_48_10000_1200000.h5 |-- output_mat_48_1000_1200000.h5 |-- output_mat_48_5000_1200000.h5 |-- output_mat_56_10000_1200000.h5 |-- output_mat_56_1000_1200000.h5 |-- output_mat_56_5000_1200000.h5 |-- output_mat_64_10000_1200000.h5 |-- output_mat_64_1000_1200000.h5 |-- output_mat_64_5000_1200000.h5 |-- output_mat_72_10000_1200000.h5 |-- output_mat_72_1000_1200000.h5 |-- output_mat_72_5000_1200000.h5 |-- output_mat_80_10000_1200000.h5 |-- output_mat_80_1000_1200000.h5 |-- output_mat_80_5000_1200000.h5 |-- output_mat_8_10000_1200000.h5 |-- output_mat_8_1000_1200000.h5 |-- output_mat_8_5000_1200000.h5 |-- output_mat_88_10000_1200000.h5 |-- output_mat_88_1000_1200000.h5 |-- output_mat_88_5000_1200000.h5 |-- output_mat_96_10000_1200000.h5 |-- output_mat_96_1000_1200000.h5 |-- output_mat_96_5000_1200000.h5 |-- output_memchip_1200000.h5 |-- output_ML_Laplace_1200000.h5 |-- output_mouse_gene_1200000.h5 |-- output_msdoor_1200000.h5 |-- output_m_t1_1200000.h5 |-- output_nd12k_1200000.h5 |-- output_nd24k_1200000.h5 |-- output_nd6k_1200000.h5 |-- output_nlpkkt120_1200000.h5 |-- output_nlpkkt80_1200000.h5 |-- output_PFlow_742_1200000.h5 |-- output_pwtk_1200000.h5 |-- output_rajat31_1200000.h5 |-- output_RM07R_1200000.h5 |-- output_Rucci1_1200000.h5 |-- output_Serena_1200000.h5 |-- output_Si34H36_1200000.h5 |-- output_Si41Ge41H72_1200000.h5 |-- output_Si87H76_1200000.h5 |-- output_SiO2_1200000.h5 |-- output_sls_1200000.h5 |-- output_StocF-1465_1200000.h5 |-- output_TEM152078_1200000.h5 |-- output_TEM181302_1200000.h5 |-- output_thermal2_1200000.h5 |-- output_tmt_sym_1200000.h5 |-- output_torso1_1200000.h5 |-- output_Transport_1200000.h5 |-- output_TSOPF_FS_b300_c2_1200000.h5 |-- output_TSOPF_FS_b300_c3_1200000.h5 |-- output_TSOPF_RS_b2383_1200000.h5 |-- output_TSOPF_RS_b2383_c1_1200000.h5 |-- output_TSOPF_RS_b678_c2_1200000.h5 `-- output_x104_1200000.h5 f_1600000_b250 |-- output_af_0_k101_1600000.h5 |—- ... `-- output_x104_1600000.h5 f_2000000_b250 |-- output_af_0_k101_2000000.h5 |—- ... `-- output_x104_2000000.h5 f_2400000_b250 |-- output_af_0_k101_2400000.h5 |—- ... `-- output_x104_2400000.h5 </code></pre> <p><strong>Creating your own dataset</strong></p> <p>If you wish to create your own training/testing dataset on a different target architecture you need to take the following steps:</p> <ol> <li> <p>Build the SpMV driver:</p> <ol> <li> <p>Go to <code>cd SpMV-driver/src</code></p> </li> <li> <p>Edit makefile and set the PAPI and HDF5 install prefixes.</p> </li> <li> <p>Build the driver via <code>make.</code></p> </li> </ol> </li> <li> <p>Run the SpMV driver: </p> <p><code>./driver <arg0> <arg1> ...</code></p> <p>List of driver arguments:</p> <pre><code>matrix = audikw_1.rb # Input matrix in rb format reps = 10000 # Number of repetitions of the operation to avoid overhead block_size_ini = 250 # Minimum block size block_size_end = 1000 # Maximum block size increment = 250 # Increment between block sizes base = 0 # Starting nnz of the matrix freq = [2400000, 2000000, 1600000, 1200000] # Operating frequency sym = 1 # If 1 the matrix is symmetric. If 0 the matrix is no-symmetric.</code></pre> <p>Example:</p> <p><code>numactl --membind 0 taskset -c 0 ./src/driver audikw_1.rb 10000 250 1000 250 0 2400000</code></p> <p>Note that <code>numactl</code> and <code>taskset</code> utilities are used to guarantee both NUMA and process-to-core affinity.</p> </li> <li> <p>Generating the dataset:</p> <ol> <li> <p>Edit the <code>SpMV-driver/run_all.sh</code> and uncomment the line <code>matrices =</code> in order to launch the driver for Train_symmetric / Train_noSymmetric / Test_symmetric / Test_noSymmetric matrices.</p> </li> <li> <p>Edit the 3rd parameter in the command SpMV-driver/run_driver.sh: 1 for symmetric matrices, 2 for unsymmetric matrices.</p> </li> <li> <p>Edit the command in SpMV-driver/run_driver.sh to select the input parameters of the driver as explained before.</p> </li> <li> <p>Run <code>SpMV-driver/run_all.sh</code> to obtain <code>hdf5</code> files that will create the dataset.</p> </li> </ol> </li> <li> <p>Merging the dataset:</p> <p>Run the script</p> <p><code>python3 SpMV-driver/merge_train_matrices.py /path/to/hdf5/matrix/files /output/path</code></p> <p>to obtain a single <code>hdf5</code> file containing all data from individual <code>hdf5</code> files obtained in the previous step. This merged file is the training dataset.</p> </li> </ol> <p><strong>Hyperparameter search</strong></p> <p>The script <code>spmv_cnn_hyperas.py</code> performs the hyperparameter search via the Hyperas tool. This script requires the hdf5 file dataset in the directory <code>dataset/train/</code> and produces both a <code>best_model_*.json</code> and <code>best_run_*.json </code>files in the <code>results/models/</code> directory containing the model structure and hyperparameters of the best performing configuration.</p> <p>This script can be invoked in the following way:</p> <p><code>python3 spmv_cnn_hyper.py 2400000 Time</code></p> <p>where <code>2400000</code> is the operating processor frequency (2.4 GHz) at which the dataset was generated and <code>Time</code> the modeled metric. According to the labels in the dataset, the hyperparameter search can also be performed with the <code>Energy</code>, <code>EPKG</code> and <code>EDRAM</code> metrics, corresponding to the energy measured by the Intel RAPL counters from our Intel Xeon Haswell core. In our case, however, we only search hyperparameters for the <code>Time</code> and <code>Energy</code> metrics at 2.4 GHz. Other metrics and frequencies inherit the best performing model and settings from the previous configuration.</p> <p><strong>Training</strong></p> <p>The script <code>spmv_cnn_train.py</code> performs the training on the best performing models obtained on the previous step. For that, it uses both the <code>best_model_*.json</code> and <code>best_run_*.json</code> files obtained in the hyperparameter search.</p> <p>This script can be invoked in the following way:</p> <p><code>python3 spmv_cnn_train.py 2400000 Time</code></p> <p>where <code>2400000</code> is the operating processor frequency (2.4 GHz) and <code>Time</code> the modeled metric. The training should be performed per metric and frequency. The training produces a file that contains the trained weights, so the model is ready for performing inference (testing).</p> <p><strong>Testing</strong></p> <p>The script <code>spmv_cnn_test.py</code> performs the test on the set of testing matrices involved in the SpMV operation.</p> <p>This script can be invoked in the following way:</p> <p><code>python3 spmv_cnn_test.py 2400000 Time</code></p> <p>where <code>2400000</code> is the operating processor frequency (2.4 GHz) and <code>Time</code> the modeled metric. The test should be performed per metric and frequency. The training produces two files in the <code>results/tests/</code> directory:</p> <ul> <li><code>Pred_*.txt</code>: This file contains the real measurements and the predictions obtained by the CNN for the individual vpos blocks of the testing matrices.</li> <li><code>Test_*.txt</code>: This file summarizes the information of <code>Pred_*.txt</code> file, showing the average relative error among the blocks of each test matrix and the total relative error, which is computed by summing up the real measurements and the predictions for all the blocks of a same matrix and computing the relative error upon those values.</li> </ul> <p><em>Note that this testing step and the two previous steps (hyperparameter search and training) can be performed at once using the <code>run.sh</code> script.</em></p> <p><strong>References</strong></p> <p>Publications describing <strong>SpMV-CNN-Model</strong>:</p> <ul> <li>Barreda, M., Dolz, M.F., Castaño, M.A. et al. Performance modeling of the sparse matrix–vector product via convolutional neural networks. J Supercomputing (2020). <a href="https://doi.org/10.1007/s11227-020-03186-1">https://doi.org/10.1007/s11227-020-03186-1</a></li> </ul> <p><strong>Acknowledgments</strong></p> <p>The <strong>SpMV-CNN-Model</strong> research has been partially supported by:</p> <ul> <li> <p>Project TIN2017-82972-R <strong>“Agorithmic Techniques for Energy-Aware and Error-Resilient High Performance Computing”</strong> funded by the Spanish Ministry of Economy and Competitiveness (2018-2020).</p> </li> <li> <p>Project CDEIGENT/2017/04 <strong>“High Performance Computing for Neural Networks” </strong>funded by the Valencian Government.</p> </li> <li> <p>Project UJI-A2019-11 <strong>“Energy-Aware High Performance Computing for Deep Neural Networks”</strong> funded by the Universitat Jaume I.</p> </li> </ul>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.