Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
10
datasets available to search
ShareScore release 0.9.0
Dataset results
10 results for “MapReduce”
Stable Modeling on Resource Usage Parameters of MapReduce Application-Figure 4. Statistical Metrics distribution of model on RIO as response of Terasort application
<p>Figure 4 shows the statistical metrics distribution of regression model on read rate as the response of Terasort application. The filled triangle point-up indicates the minimum stable sampling time for statistical metrics. The top-half of figure 4 shows the residual standard error (RSE) distribution as training data size increase. The remaining half is for the distribution of R2.</p>
Stable Modeling on Resource Usage Parameters of MapReduce Application-Figure 3. Estimate coefficients distribution of model on read rate as response of Terasort application
<p>Figure 3 shows estimate coefficients distribution of model on read rate as the response of Terasort application. The filled triangle point-up indicates the position of the minimum stable sampling time for the corresponding estimate coefficients as well as the number above it shows the exact position value. The dashed line represents the average estimated coefficient of the regression model.</p>
Stable Modeling on Resource Usage Parameters of MapReduce Application-Figure 1. ACF and PACF plot of TeraSort application
<p>In Figure 1, the autocorrelation plots of all resource usage parameters reveal non- randomness because of the corresponding autocorrelations, denoted by the circle, violate the dashed lines (95% confidence boundary) and are statistically significant for lags up to 100. The filled triangle point-up in partial autocorrelation plots marks the largest partial autocorrelation of usage parameters as well as the corresponding lag number.</p>
Stable Modeling on Resource Usage Parameters of MapReduce Application-Figure 6. RSE and R2 of regression models of MapReduce applications
<p>Figure 6 shows the fit quality of regression models. It is following.The left panel and the right panel of figure 6 show the residual standard error (RSE) distribution and R2 distribution of each application. The good fit quality corresponds to a taller R2 bar and a shorter RSE bar. The R2 almost 1 and small RSE show the best fit quality of the regression models on memory usage as the response. The overall higher RSE and lower R2 of regression models on CPU as the response show the worse quality of fitting goodness. The regression models on read rate as the response also show a moderate fitting quality. For the regression models on write rate as the response, Terasort application exhibits the best quality and Teragen application as well. Others show the worse fitting quality. The results show that the regression models on intensive usage parameters as response exhibit the good fitting quality.</p>
Stable Modeling on Resource Usage Parameters of MapReduce Application-Figure 2. ACF plot of Residuals of Regression Model of Terasort
<p>, are uncorrelated. If the error term is uncorrelated, it proves that there exists strong randomness in residuals of the model and provides the evidence for the unbiased estimate for the true standard error. The autocorrelation plot is used to check this assumption. Figure 2 shows the autocorrelation plot of residuals of regression models of TeraSort application. In Figure 2, the horizontal axis represents lag time and the vertical axis indicates the autocorrelation between residual at time t and residual at other lag time. At lag 0, autocorrelation is always equal to 1 and represents time series itself. Most of the autocorrelation at other lag time fall into the 95% confidence interval, only few of them violate the dashed line. Such a shape of ACF plot proves that residuals are uncorrelated and respects to the independent assumption of the linear regression model residuals.</p>
Stable Modeling on Resource Usage Parameters of MapReduce Application-Figure 8. Minimum sample time of statistical metrics of MapReduce applications
<p>Figure 8 presents the minimum sampling time distribution of statistic metrics which ensures the stable modeling. Overall, the minimum sampling time of statistic metrics is smaller than sampling time of estimated coefficients. For different applications, a time-consuming application like Terasort needs the largest sampling time to tend to be stable. The Pi application shows the smallest minimum sampling time to reach stability.</p>
Stable Modeling on Resource Usage Parameters of MapReduce Application-Figure 7. Minimum sampling time of MapReduce applications on various response variable
<p>Pi application needs the smallest sampling time. The remaining applications need the similar minimum sampling time. Overall, the stable regression models on memory usage as response show the least need for sampling time. The results show that various applications have different minimum sampling time to get stable. The application which performs more read/write operations shows larger sampling time need.</p>
Stable Modeling on Resource Usage Parameters of MapReduce Application-Department of Networked Systems and Services, Budapest University of Technology and Economics, Budapest, Hungary
<p>In Figure 5, the positive dependency of different strength between each resource usage parameter and the corresponding previous usage parameter is exhibited for all MapReduce applications. It indicates that all current resource usage parameters are positively dependent on the previous values to some extent degree. Except for these common dependencies, there exist some special dependencies for different applications. On the top-left panel of Figure 5, CPU usage of Pi application shows the strongest positive dependency to lagged CPU usage, the Teragen application had the weakest positive dependency, and others exhibit the moderate positive dependency. </p>
Efficient Large-Scale Trace Checking Using MapReduce
<p>The problem of checking a logged event trace against a temporal logic specification arises in many practical cases. Unfortunately, known algorithms for an expressive logic like MTL (Metric Temporal Logic) do not scale with respect to two crucial dimensions: the length of the trace and the size of the time interval for which logged events must be buffered to check satisfaction of the specification. The former issue can be addressed by distributed and parallel trace checking algorithms that can take advantage of modern cloud computing and programming frameworks like MapReduce. Still, the latter issue remains open with current state-of-the-art approaches. </p> <p>In this paper we address this memory scalability issue by proposing a new semantics for MTL, called lazy semantics. This semantics can evaluate temporal formulae and boolean combinations of temporal-only formulae at any arbitrary time instant. We prove that lazy semantics is more expressive than standard point-based semantics and that it can be used as a basis for a correct parametric decomposition of any MTL formula into an equivalent one with smaller, bounded time intervals. We use lazy semantics to extend our previous distributed trace checking algorithm for MTL. We evaluate the proposed algorithm in terms of memory scalability and time/memory tradeoffs.</p>
Supplemental material for: Automatic Debugging of Design Faults in MapReduce applications
<p>This is the supplemental material of the paper titled as “Automatic Debugging of Design Faults in MapReduce applications” published in IEEE Transactions on Software Engineering. <a href="https://doi.org/10.1109/TSE.2024.3369766" target="_blank" rel="noopener">Link</a></p> <p> </p> <p>It contains both the test cases used in the evaluation and the statistical analysis to reproduce the experiments. The supplemental material contains the following files:</p> <ul> <li>1_testCases.zip: all test cases randomly generated for the experiments. The description of the test cases is in ./1_testCases/README.txt</li> <li>2_executionTestCases.zip: the aggregated data obtained after the execution of the test cases in the debugging techniques: fault localization technique (MRDebug-FL), input reduction technique (MRDebug-IR) and the combination of both techniques (MRDebug-IR-FL). The folder contains csv with the results the experimentation unit, and they are detailed in the ./2_executionTestCases/README.pdf file.</li> <li>3_notebook.zip: jupyter notebook that contains the analysis done in the experiments. This notebook allows the interactive execution of statistical test and plots. The instructions to install the notebook are in the file 3_notebook.zip/installation.txt.</li> </ul> <p> </p> <p>To cite this work: </p> <p>J. Morán, A. Bertolino, C. de la Riva and J. Tuya, "Automatic Debugging of Design Faults in MapReduce Applications," in <em>IEEE Transactions on Software Engineering</em>, vol. 50, no. 4, pp. 956-978, April 2024, doi: 10.1109/TSE.2024.3369766</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.