Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
6
datasets available to search
ShareScore release 0.9.0
Dataset results
6 results for “task scheduling”
Production line dataset for task scheduling and energy optimization - Demand Response Participation
<p>Using the previous dataset at <<a href="https://zenodo.org/record/4106746">https://zenodo.org/record/4106746</a>> it was simulated an announcement of a demand response program at period 757, describing a demand response event from period 937 (Friday at 21:00h) to 960 (Friday at 23:00h) , where each period represents five minutes. The demand response program imposed a limit consumption, during its event, of 2.5 kWh. The announcement of the demand response allowed the use of the proposed solution to limit the energy consumption. For that, the algorithm described in section 3.3 was executed at period 769 (Friday at 7:00h).</p> <p>The API can be found at <<a href="http://www.gecad.isep.ipp.pt/api/spear/%3E">http://www.gecad.isep.ipp.pt/api/spear/</a>></p> <p>File Description:</p> <ul> <li>Input_JSON_Demand_Response_Optimization - JSON input data for the demand response participation</li> <li>Output_JSON_Demand_Response_Optimization - JSON output data for the demand response participation</li> <li>Output_Statistics_Demand_Response_Optimization - Excel output demand response participation statistics</li> <li>Comparison_Output_Statistics_Demand_Response - Excel output statistics comparing the before and after the demand response participation</li> </ul>
Production line dataset for task scheduling and energy optimization - Schedule Optimization
<p>The case study of this dataset uses real production data, provided by a textile company that manufactures hang tags. Their working schedule is from 7h00 of Monday to 23h00 of Saturday. This dataset uses a period of 5 minutes for all task durations and energy data. The case study considers a six-day period from 7h00 of Monday to 23h00 of Saturday. The scheduling algorithm was used for three machines that share the same cell.<br> <br> The API can be found at <http://www.gecad.isep.ipp.pt/api/spear/><br> <br> File Description:</p> <ul> <li>Input_JSON_Schedule_Optimization - JSON input data for the schedule optimization</li> <li>Output_JSON_Schedule_Optimization - JSON output data for the schedule optimization</li> <li>Output_Statistics_Schedule_Optimization - Excel output schedule optimization statistics</li> </ul>
Task Scheduler Performance Survey Results
<p><strong>Task scheduler performance survey</strong></p> <p>This dataset contains results of task graph scheduler performance survey.<br> The results are stored in the following files, which correspond to simulations performed on <br> the `elementary`, `irw` and `pegasus` task graph datasets published at https://doi.org/10.5281/zenodo.2630384.</p> <ul> <li>elementary-result.zip</li> <li>irw-result.zip</li> <li>pegasus-result.zip</li> </ul> <p>The files contain compressed pandas dataframes in CSV format, it can be read with the following Python code:<br> ```python<br> import pandas as pd<br> frame = pd.read_csv("elementary-result.zip")<br> ```</p> <p>Each row in the frame corresponds to a single instance of a task graph that<br> was simulated with a specific configuration (network model, scheduler etc.).<br> The list below summarizes the meaning of the individual columns.</p> <ul> <li><strong>graph_name</strong> - name of the benchmarked task graph</li> <li><strong>graph_set</strong> - name of the task graph dataset from which the graph originates</li> <li><strong>graph_id</strong> - unique ID of the graph</li> <li><strong>cluster_name</strong> - type of cluster used in this instance the format is <number-of-workers>x<number-of-cores>; 32x16 means 32 workers, each with 16 cores</li> <li><strong>bandwidth</strong> - network bandwidth [MiB]</li> <li><strong>netmodel</strong> - network model (simple or maxmin)</li> <li><strong>scheduler_name</strong> - name of the scheduler</li> <li><strong>imode</strong> - information mode</li> <li><strong>min_sched_interval</strong> - minimal scheduling delay [s]</li> <li><strong>sched_time</strong> - duration of each scheduler invocation [s]</li> <li><strong>time</strong> - simulated makespan of the task graph execution [s]</li> <li><strong>execution_time </strong>- real duration of all scheduler invocations [s]</li> <li><strong>total_transfer</strong> - amount of data transferred amongst workers [MiB]</li> </ul> <p>The file `charts.zip` contains charts obtained by processing the datasets.<br> On the X axis there is always bandwidth in [MiB/s].<br> There are the following files:</p> <ul> <li>[DATASET]-schedulers-time - Absolute makespan produced by schedulers [seconds] </li> <li>[DATASET]-schedulers-score - The same as above but normalized with respect to the best schedule (shortest makespan) for the given configuration.</li> <li>[DATASET]-schedulers-transfer - Sums of transfers between all workers for a given configuration [MiB]</li> <li>[DATASET]-[CLUSTER]-netmodel-time - Comparison of netmodels, absolute times [seconds]</li> <li>[DATASET]-[CLUSTER]-netmodel-score - Comparison of netmodels, normalized to the average of model "simple"</li> <li>[DATASET]-[CLUSTER]-netmodel-transfer - Comparison of netmodels, sum of transfered data between all workers [MiB]</li> <li>[DATASET]-[CLUSTER]-schedtime-time - Comparison of MSD, absolute times [seconds]</li> <li>[DATASET]-[CLUSTER]-schedtime-score - Comparison of MSD, normalized to the average of "MSD=0.0" case</li> <li>[DATASET]-[CLUSTER]-imode-time - Comparison of Imodes, absolute times [seconds]</li> <li>[DATASET]-[CLUSTER]-imode-score - Comparison of Imodes, normalized to the average of "exact" imode</li> </ul> <p><strong>Reproducing the results</strong></p> <p><em>1. Download and install Estee (https://github.com/It4innovations/estee)</em></p> <p>$ git clone https://github.com/It4innovations/estee<br> $ cd estee<br> $ pip install .<br> <br> <em>2. Generate task graphs</em><br> You can either use the provided script `benchmarks/generate.py` to generate graphs<br> from three categories (elementary, irw and pegasus):</p> <p>$ cd benchmarks<br> $ python generate.py elementary.zip elementary<br> $ python generate.py irw.zip irw<br> $ python generate.py pegasus.zip pegasus<br> <br> or use our task graph dataset that is provided at https://doi.org/10.5281/zenodo.2630384.</p> <p><em>3. Run benchmarks</em><br> To run a benchmark suite, you should prepare a JSON file describing the benchmark.<br> The file that was used to run experiments from the paper is provided in<br> `benchmark.json`. Then you can run the benchmark using this command:<br> <br> $ python pbs.py compute benchmark.json</p> <p>The benchmark script can be interrupted at any time (for example using Ctrl+C).<br> When interrupted, it will store the computed results to the result file and restore<br> the computation when launched again.</p> <p><em>3. Visualizing results</em><br> <br> $ python view.py --all <result-file><br> <br> The resulting plots will appear in a folder called `outputs`.</p>
Task graphs for benchmarking schedulers
<p><strong>Workflow Task Graph Dataset</strong></p> <p>This dataset contains three sets of task graphs representing different types of task workflows:</p> <ul> <li><em>Elementary</em> - contains trivial graph shapes, such as tasks with no dependencies or simple fork-join graphs. This set should test how the scheduler heuristics react to basic graph scenarios that frequently form parts of larger workflows.</li> <li><em>IRW</em> - is inspired by real-world workflows, such as machine learning cross-validation or map-reduce.</li> <li> <em>Pegasus</em> - is derived from graphs created by Pegasus Synthetic Workflow Generators (https://github.com/pegasus-isi/WorkflowGenerator)</li> </ul> <p>All of the provided task graphs are generated and compatible with ESTEE (https://github.com/It4innovations/estee) that allows to simulate their execution on a distributed system using various scheduling heuristics and environment conditions.</p> <p><strong>Data Format</strong></p> <p>Task graphs are stored in {elementary, irw, pegasus}.zip files that contain JSON representation of respective task graphs with the following fields:</p> <ul> <li>`graph_name` - Task graph name</li> <li>`graph_id` - Unique task graph identifier</li> <li> `graph` - Task graph representation - list of tasks where each task is represented as a dictionary with the following keys:</li> <li> `d`: Actual task duration in seconds (float value)</li> <li> `e_d`: User estimated task duration in seconds (float value)</li> <li> `cpus`: Task CPU core requirements (integer value)</li> <li> `outputs`: List of task outputs (list of integers indicating sizes of task outputs in MiB)</li> <li> `inputs`: List of task inputs in format of list [task\_id, output\_index]}. Output index is zero-based.</li> </ul> <p>For example this task graph:</p> <p>[{'d': 200, 'e_d': 180, 'cpus': 1, 'outputs': [100], 'inputs': []},</p> <p>{'d': 50, 'e_d': 60, 'cpus': 2, 'outputs': [], 'inputs': [[0, 0]]}]</p> <p>contains two tasks. One requiring no input, single CPU core with estimated duration 180s, actual duration 200s and producing a single output of 100 MiB. And another one requiring as an input task0's 0-th output, requiring 2 CPU cores, producing no output with estimated duration 60s and actual duration 50s.</p> <p> </p> <p><strong>Parsing the data</strong></p> <p>In Python, to load the elementary task graph set run the following snippet:</p> <pre><code class="language-python">import pandas as pd graphs = pd.read_json("./elementary.zip")</code></pre> <p> </p> <p>If you have Estee installed, you can use its provided `json_deserialize`</p> <p>function to parse the JSON encoded graphs into Estee TaskGraph data structure.</p> <p> </p> <pre><code class="language-python">from estee.serialization.dask_json import json_deserialize graph_json = graphs.loc[0, "graph"] graph = json_deserialize(graph)</code></pre> <p> </p>
Parameterized Task Graph Scheduling Algorithm for Comparing Algorithmic Components - All Figures and Datasets
<p>Figures and Datasets for "Parameterized Task Graph Scheduling Algorithm for Comparing Algorithmic Components"</p> <ul> <li>dataset.zip: All datasets evaluated</li> <li>figures.zip: All plots (1080) for all results for our paper</li> </ul>
The Effect of Different Schedules of Functional Task Practice for Improving Hand and Arm Function After Stroke
ClinicalTrials.gov study NCT00361660. IPD Sharing: Not stated. Countries: 0. Publications: 0.
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.