Find research datasets worth reusing

opencc-by-4.0Mar 2020View details →

Deep reinforcement learning for the control of microbial co-cultures in bioreactors

Data for the figures in the paper: <a href="https://www.biorxiv.org/content/10.1101/457366v2">https://www.biorxiv.org/content/10.1101/457366v2</a> (in press PLoS Comp Biol.) Abstract: Multi-species microbial communities are widespread in natural ecosystems. When employed for biomanufacturing, engineered synthetic communities have shown increased productivity in comparison with monocultures and allow for the reduction of metabolic load by compartmentalising bioprocesses between multiple sub-populations. Despite these benefits, co-cultures are rarely used in practice because control over the constituent species of an assembled community has proven challenging. Here we demonstrate, in silico, the efficacy of an approach from artificial intelligence – reinforcement learning – for the control of co-cultures within continuous bioreactors. We confirm that feedback via reinforcement learning can be used to maintain populations at target levels, and that model-free performance with bang-bang control can outperform a traditional proportional integral controller with continuous control, when faced with infrequent sampling. Further, we demonstrate that a satisfactory control policy can be learned in one twenty-four hour experiment by running five bioreactors in parallel. Finally, we show that reinforcement learning can directly optimise the output of a co-culture bioprocess. Overall, reinforcement learning is a promising technique for the control of microbial communities.

opencc-by-4.0Jun 2021View details →

Dataset: Reinforcing Cybersecurity Hands-on Training With Adaptive Learning

This repository contains supplementary materials for the following conference paper: Pavel Seda, Jan Vykopal, Valdemar &Scaron;vábenský, Pavel Čeleda. Reinforcing Cybersecurity Hands-on Training With Adaptive Learning. In Proceedings of the 51st IEEE Frontiers in Education Conference (FIE 2021). <a href="https://doi.org/10.1109/FIE49875.2021.9637252">https://doi.org/10.1109/FIE49875.2021.9637252</a> Preprint available at: <a href="https://arxiv.org/abs/2201.01574">https://arxiv.org/abs/2201.01574</a> How to cite If you use or build upon the materials, please use the BibTeX entry below to cite the original paper (not only this web link). Some of the linked repositories have their separate citation entry; please use that one as well, if possible. <pre><code>@inproceedings{Seda2021reinforcing, author = {Seda, Pavel and Vykopal, Jan and \v{S}v\'{a}bensk\'{y}, Valdemar and \v{C}eleda, Pavel}, title = {{Reinforcing Cybersecurity Hands-on Training With Adaptive Learning}}, booktitle = {Proceedings of the 51st IEEE Frontiers in Education Conference}, series = {FIE '21}, location = {Lincoln, NE, USA}, publisher = {IEEE}, address = {New York, NY, USA}, month = {10}, year = {2021}, pages = {1--9}, numpages = {9}, isbn = {978-1-6654-3851-3}, url = {https://doi.org/10.1109/FIE49875.2021.9637252}, doi = {10.1109/FIE49875.2021.9637252}, }</code></pre>

opencc-by-4.0Apr 2023View details →

Inverse design of metal-organic frameworks for direct air capture of CO2 via deep reinforcement learning

The combination of several interesting characteristics makes metal-organic frameworks (MOFs) a highly sought-after class of nanomaterials for a broad range of applications like gas storage and separation, catalysis, drug delivery, and so on. However, the ever-expanding and nearly infinite chemical space of MOFs makes it extremely challenging to identify the most optimal materials for a given application. In this work, we present a novel approach using deep reinforcement learning for the inverse design of MOFs, our motivation being designing promising materials for the important environmental application of direct air capture of CO2 (DAC). We demonstrate that the reinforcement learning framework can successfully design MOFs with critical characteristics important for DAC. Our top-performing structures populate two separate subspaces of the MOF chemical space: the subspace with high CO2 heat of adsorption and the subspace with preferential adsorption of CO2 from humid air, with few structures having both characteristics. Our model can thus serve as an essential tool for the rational design and discovery of materials for different target properties and applications.

openmit-licenseJul 2023View details →

lilGym: Natural Language Visual Reasoning with Reinforcement Learning, model files

Baselines models for the paper <a href="https://lil.nlp.cornell.edu/lilgym">lilGym: Natural Language Visual Reasoning with Reinforcement Learning</a>.

opencc-by-4.0May 2023View details →

Enhanced Westermo dataset - Transformed and Modified for Test case Selection and Priorotization in the context of Continuous Integration and Reinforcement Learning.

Overview This repository contains a modified version of the existing, recently published dataset, Westermo. The initial dataset was gathered at Westermo Network Technologies AB, located in Västerås, Sweden. It encompasses over 1 Million verdicts obtained from testing embedded systems, collected over a span of more than 500 consecutive days of nightly testing. The dataset has been transformed and tailored specifically to cater to the research community, particularly for addressing challenges such as regression test selection, identification of flaky tests, and visualization of test results. The original dataset can be accessed through the reference provided in [1]. The Westermo dataset offers valuable historical information regarding the execution of test cases and their corresponding results. It serves as a valuable resource for evaluating and comparing different Test case Selection and Prioritization (TSP) techniques, enabling researchers to identify test cases that are more likely to fail during subsequent executions. Test cases in the dataset are characterized by attributes such as execution duration, previous last execution time, and the results of their recent executions. This dataset offers valuable historical information regarding the execution of test cases and their corresponding results. It serves as a valuable resource for evaluating and comparing different test case prioritization and selection techniques, enabling researchers to identify test cases that are more likely to fail during subsequent executions. Test cases in the dataset are characterized by attributes such as execution duration, previous last execution time, and the results of their recent executions. <table align="left"> <caption>Table 1: Dataset Overview</caption> <tbody> <tr> <td>Test Cases</td> <td>1855</td> </tr> <tr> <td>CI Cycles</td> <td>15,197</td> </tr> <tr> <td>Verdict</td> <td>1,036,818</td> </tr> <tr> <td>Failed</td> <td>5.03%</td> </tr> </tbody> </table>               However, the diversity and multitude of the features in the dataset can be irrelevant to some TSP approaches. This led us to perform a dataset conversion, where we customized Westermo to have the same features from Paint Control and IOF/ROL, two widely used datasets in Reinforcement Learning based TSP approaches. This conversion required the combination of multiple variables and generating the target ones. When it comes to generating the “LastResults” and “Cycle” values, further analysis was required and the data handling needed an in-depth understanding of how the nightly testing was conducted. This led us to investigate what a CI cycle is in their context, and we followed their definition of a session, stating that “a session is when we run a suite of tests on one test system with a certain software version and testware version”. When splitting the data according to the 9 different systems used, we were able to generate 9 different sub-sets that fit the CI context.   File Format The compressed .zip file contains 9 files, each one corresponding to each of the 9 systems. The datasets are available in CSV format, with the semicolon (;) serving as the delimiter. The columns included are represented in the table below along with their descriptions. <table> <caption>Table 2: Parameters of the dataset</caption> <thead> <tr> <th scope="col">Column Name</th> <th scope="col">Content</th> </tr> </thead> <tbody> <tr> <td>Id</td> <td>Unique numeric identifier of the test execution </td> </tr> <tr> <td>Name</td> <td>Unique numeric identifier of the test case</td> </tr> <tr> <td>Duration</td> <td>Approximated runtime of the test case</td> </tr> <tr> <td>CalcPrio</td> <td>Priority of the test case, calculated by the prioritization algorithm (output column, initially 0)</td> </tr> <tr> <td>LastRun</td> <td>Previous last execution of the test case as date-time-string (Format: YYYY-MM-DD HH:ii )</td> </tr> <tr> <td>LastResults</td> <td>List of previous test results (Failed: 1, Passed: 0), ordered by ascending age. Lists are delimited by [ ].</td> </tr> <tr> <td>Verdict</td> <td> Test verdict of this test execution (Failed: 1, Passed: 0) </td> </tr> <tr> <td>Cycle</td> <td>The number of the CI cycle this test execution belongs to.</td> </tr> </tbody> </table>   The implications of this conversion are important as it can help the previous works to re-assess their approaches and have more data for training and testing, as well as opening a broader data spectrum for future researchers in this field to find ready-to-use, rich datasets, on which they could evaluate their approaches and contribute to the TSP community. This also addresses the limitations in the field discussed in the systematic literature review [2], stating that future research on TSP techniques should focus on collecting data from more recent subjects in a CI context with varying failure rates and larger execution times, as reproducible studies with appropriate datasets are needed to develop a usable body of knowledge regarding TSP over time. We believe that this conversion of the Westermo dataset is our contribution to alleviating the gap for the RL-based approaches. The original dataset can be found <a href="https://sites.mdu.se/aidoart/results/open-source/test-results-dataset-westermo">here.</a>

opencc-by-4.0Nov 2020View details →

Reinforcement Learning Control of a Biomechanical Model of the Upper Extremity

This dataset contains evaluation data of the paper "Reinforcement Learning Control of a Biomechanical Model of the Upper Extremity". Motivation We address the question whether the assumptions of signal-dependent and constant motor noise in a full skeletal model of the human upper body, together with the objective of movement time minimization, can predict reaching movements. For evaluation of the learned policy, two tasks are defined in the paper: a Fitts' Law type task and an elliptic via-point task. General description of the dataset <ul> <li>Fitts' Law Type Task <ul> <li>This dataset incorporates detailed information of all 6500 synthesized movements generated in the Fitts' Law type task (following the ISO 9241-9 standard).</li> <li> For each of the 10 task conditions differing in distance between targets ("dist<xxx>" in filename) and ID ('ID<xxx>' in filename), there are two .csv-files: - one with detailed trajectory information on a sample-to-sample basis ("ISO_SAMPLES" in filename), and - one with aggregated movement information on an episode basis ("ISO_METRICS" in filename). </li> <li> In addition, for each task condition and each of the 13 movement directions in the Fitts' Law type task, we include 6 figures: Position, Velocity, and Acceleration Profiles, as well as 3D movement path, Phasespace, and Hooke plots. Apart from the 3D plots, all figures use centroid projections of the respective trajectory onto the vector between initial and target position. The first integer in the file name denotes the movement direction number, starting with "0" for movements between the targets 1 and 2, "1" for movements between the targets 2 and 3 etc. The file "6_distance0.35_ID2_policy2100000_phasespace.png", e.g., shows velocity plotted againt position for all 50 movements between the targets 7 and 8 in the task condition with ID 2 and 35cm diameter of the target circle (see Fig 2 in Paper). </li> </ul> </li> <li> Elliptic Task <ul> <li> This dataset also contains two CSV-files with data of the trajectory generated by the final policy in the elliptic task: - one with detailed trajectory information on a sample-to-sample basis ("ELLIPSE_SAMPLES" in filename), and - one with aggregated movement information on an episode basis, where a new episode starts whenever the target on the ellipse given to the policy switches ("ELLIPSE_METRICS" in filename). </li> </ul> </li> </ul> Description of the .csv-files <ul> <li>SAMPLES Files <ul> <li>"time": time after reaching the initial target (target 1 in Fig 2) for the first time (in seconds)</li> <li>"elv_angle_pos" - "flexion_pos": angle of respective independent DOF (in radians) *</li> <li>"elv_angle_vel" - "flexion_vel": angular velocity of respective independent DOF (in radians/s) *</li> <li>"end-effector_xpos_x" - "end-effector_xpos_z": 3D position of end-effector in global coordinates (in meters) *</li> <li>"target_xpos_x" - "target_xpos_z": 3D position of target sphere in global coordinates (in meters) *</li> <li>"end-effector_xvelp_x" - "end-effector_xvelp_z": positional velocities of end-effector (in meters/s) *</li> <li>"target_xvelp_x" - "target_xvelp_z": positional velocities of target sphere (in meters/s) *</li> <li>"accsensor_end-effector_x" - "accsensor_end-effector_z": positional acceleration of end-effector (in meters/s^2) *</li> <li>"E_elv_angle" - "E_flexion": activation of respective independent DOF *</li> <li>"E_elv_angle_derivative" - "E_flexion_derivative": derivative of activation of respective independent DOF *</li> <li>"difference_vec_x" - "difference_vec_z": vector between the end-effector attached to the index finger and the target, pointing towards the target (in meters) *</li> <li>"centroid_vel_projection": projection of end-effector velocity towards target (in meters/s) *</li> <li>"target_width": radius (!) of the target sphere (in meters) *</li> <li>"A_elv_angle" - "A_flexion": action vector</li> <li>"thorax_tx_frc" - "wrist_hand_r3_frc": net external force at respective DOF (including dependent and fixed DOFs such as "thorax_tx" (thorax translation))</li> <li>"reward": reward obtained in this step</li> <li>"step_type": 0=initial step of episode, 1=intermediate step of episode, 2=terminal step of episode</li> <li>"target_switch": whether the target switched in this step</li> <li>"discount": internal value of tf-agents (does not correspond to the discount factor gamma, which is additionally applied!)</li> <li>"thorax_tx_pos" - "wrist_hand_r3_pos": angle of respective dependent DOF (in radians)</li> <li>"thorax_tx_vel" - "wrist_hand_r3_vel": angular velocity of respective dependent DOF (in radians)</li> </ul> </li> <li>METRICS Files <ul> <li>Index: episode ID</li> <li>"Init_X" - "Init_Z": initial position in global coordinates (in meters)</li> <li>"Init_X" - "Init_Z": target position in global coordinates (in meters)</li> <li>"Init_Distance": distance between last target (i.e., desired initial position) and current target (in meters)</li> <li>"Target_Width_Diameter": target width diameter (in meters)</li> <li>"Movement_ID": Index of Difficulty of current movement (using the Shannon Formulation) (in bits)</li> <li>"target_accuracy": 1 - (<remaining distance to target center at the end of the episode>/<target radius>) if end-effector is inside target, 0 else</li> <li>"movement_time": duration of the episode (in seconds)</li> <li>"episode_successful": whether episode terminated successfully within the permitted 1.5 seconds</li> <li>"dist2target": remaining distance to target center at the end of the episode (in meters)</li> </ul> </li> </ul> ------------------------------------- * included in state space

opencc-by-4.0Dec 2023View details →

Datasets: Enhancing Anger Management via Reinforcement Learning: A Comparative Analysis of the PPO Algorithm with Optimised Hyperparameters

Open the record for dataset details and reuse information.

opencc-by-4.0Nov 2024View details →

Reproduction code and data for the plot of "Synthesizing survival robot behavior through reinforcement learning for homeostasis"

<pre># Reproduction code and data for the plot of "Synthesizing survival robot behavior through reinforcement learning for homeostasis" Author: Naoto Yoshida How to use: 1. Clone https://github.com/ugo-nama-kun/journalpaper_robot_2024 from github. 2. Extract data_20241119.zip in the cloned repository. 3. Run each plot_Fig*.py</pre>

opencc-by-4.0Aug 2022View details →

Source codes and datasets for the paper "DRLComplex: Reconstruction of protein quaternary structures using deep reinforcement learning"

This contains the reproducible source code and dataset for the paper "DRLComplex : Reconstruction of protein quaternary structures using deep reinforcement learning paper"

opencc-by-4.0Aug 2024View details →

Acquiring musculoskeletal skills with curriculum-based reinforcement learning - model weights

Acquiring musculoskeletal skills with curriculum-based reinforcement learning, Neuron 2024 Here we provide the weights of the neural network policies used for the analysis presented in our article. The archives whose names start with a number (01 - 32) correspond to the 32 curriculum steps to train the Baoding Balls policy which ranked first at the MyoChallenge 2022. The code used for the training and which can be used to test the policies can be found at https://github.com/amathislab/myochallenge. The archives hand_pose, hand_reach, pen and reorient correspond to the other policies used in the article. They were developed in the paper Latent exploration for reinforcement learning, Chiappa et al., NeurIPS 2023. They can be loaded and tested with the code at https://github.com/amathislab/lattice. The archive datasets includes three subfolders: rollouts, umap and csi. <ul> <li>The files in rollouts are the datasets of transitions resulting from the interaction between a policy and the environment. </li> <li>The files in umap are the pre-computed projections of specific subsets fo the datasets included in rollouts using UMAP.</li> <li>The files in csi report the performance of the policies described in our paper when applying Control Subspace Inactivation (CSI).</li> </ul> These datasets are necessary to run the notebooks to reproduce the paper's figures and main results, with the code at https://github.com/amathislab/MyoChallengeAnalysis If you find these weights useful, please cite: <div> <div>@article{chiappa2024acquiring, title = {Acquiring musculoskeletal skills with curriculum-based reinforcement learning}, journal = {Neuron}, volume = {112}, number = {23}, pages = {3969-3983.e5}, year = {2024}, issn = {0896-6273}, doi = {https://doi.org/10.1016/j.neuron.2024.09.002}, url = {https://www.sciencedirect.com/science/article/pii/S0896627324006500}, author = {Alberto Silvio Chiappa and Pablo Tano and Nisheet Patel and Abigaïl Ingster and Alexandre Pouget and Alexander Mathis}, keywords = {motor control, motor learning, reinforcement learning, curriculum learning, motor skills, musculoskeletal control, muscle synergies}, }</div> <div> </div> <div>@article{chiappa2024latent,</div> <div>title={Latent exploration for reinforcement learning},</div> <div>author={Chiappa, Alberto Silvio and Marin Vargas, Alessandro and Huang, Ann and Mathis, Alexander},</div> <div>journal={Advances in Neural Information Processing Systems},</div> <div>volume={36},</div> <div>year={2024}</div> <div>}</div> </div>

opencc-by-4.0Jan 2023View details →

Dataset for "Deep reinforcement learning for the olfactory search POMDP: a quantitative benchmark"

Dataset containing the results shown in Fig. 6 of "Deep reinforcement learning for the olfactory search POMDP: a quantitative benchmark".

opencc-by-4.0Sep 2022View details →

Deep Reinforcement Learning for Data-Driven Adaptive Scanning in Ptychography

These are the data sets used for the publication https://arxiv.org/abs/2203.15413. In more detail, the DataFile.pkl files in the training_data and testing_data folders are the actual data, including diffraction patterns, the corresponding reconstructions and the used illumination probe. The folders dwcnt_training_data and dwcnt_testing_data include the data sets of the simulated double-walled carbon nanotube. The reconstruction folder includes the source code of the reconstruction algorithm ROP and the used parameter files. The results folder includes the updated weights of the network model and the reconstructed potentials for the 25 test data sets used in the comparison of the publication. The sequence_grid.npy file stores the sparse grid scanning sequence used for initialization and comparison as described in the publication. The dwcnt_sequence_grid.npy file is the corresponding file for the double-walled carbon nanotube data. If you want more information, please contact the corresponding author of the publication, at schlozma@hu-berlin.de.

openapache2.0May 2023View details →

Deep Reinforcement Learning Enables Better Bias Control in Benchmark for Virtual Screening

This compressed file contains all datasets made for the validation of MUBDsyn.<ul><li>datasets_int_val: 17 cases in this folder are derived from <a href="https://github.com/jwxia2014/ULS-UDS">MUBD for GPCRs</a>. MUBDreal was made by <a href="https://github.com/jwxia2014/MUBD-DecoyMaker2.0">MUBD-DecoyMaker2.0</a> and MUBDsyn was made by <a href="https://github.com/taoshen99/MUBDsyn">MUBD-DecoyMakersyn</a>.</li><li>datasets_ext_val_classical_VS: Five cases in this folder are derived from the shared cases of MUV and DUD-E. The active sets of MUV were taken as the input to make corresponding MUBD datasets. Files in SBVS are raw molecular docking results by smina.</li><li>datasets_ext_val_SI_classical_VS: DeepCoy and TocoDecoy were used to make the datasets corresponding to the same five cases above. The data of DeepCoy was directly retrieved from <a href="https://opig.stats.ox.ac.uk/resources">DeepCoy resources at OPIG</a> while topology decoys of TocoDecoy_9W were made based on the scripts provided at <a href="https://github.com/5AGE-zhang/TocoDecoy">TocoDecoy GitHub Repository</a>. Files in SBVS are raw molecular docking results by smina.</li><li>datasets_ext_val_ML_VS: Ten cases in this folder are derived from <a href="http://nrlist.drugdesign.fr/">NRLiSt-BDB</a>. Corresponding MUBD datasets were made as described above.</li></ul>All these datasets can be used for the reproduction of validation performed in the manuscript or to benchmark various virtual screening methods.

opencc-by-4.0Dec 2022View details →

Learning to Do or Learning While Doing: Reinforcement Learning and Bayesian Optimisation for Online Continuous Tuning

Dataset of optimisation runs performed for a study comparing reinforcement learning and Bayesian optimisation for online continuous tuning at the example of a linear particle accelerator tuning task.   Abstract of the Paper on the Study Online tuning of real-world plants is a complex optimisation problem that continues to require manual intervention by experienced human operators. Autonomous tuning is a rapidly expanding field of research, where learning-based methods, such as Reinforcement Learning-trained Optimisation (RLO) and Bayesian optimisation (BO), hold great promise for achieving outstanding plant performance and reducing tuning times. Which algorithm to choose in different scenarios, however, remains an open question. Here we present a comparative study at the example of a routine task on a real particle accelerator, showing that RLO generally outperforms BO, but is not always the best choice. Based on the study’s results, we provide a clear set of criteria to guide the choice of algorithm for a given tuning task. These can ease the adoption of learning-based autonomous tuning solutions to the operation of complex real-world plants, ultimately improving the availability and pushing the limits of operability of these facilities, thereby enabling scientific and engineering advancements.

opencc-zeroJul 2023View details →

Dynamic robotic tracking of underwater targets using reinforcement learning

To realize the potential of autonomous underwater robots that scale up our observational capacity in the ocean, new approaches and techniques are needed. Fleets of autonomous robots could be used to study complex marine systems and animals with either new imaging configurations or by tracking tagged animals to study their behavior. These activities can then inform and create new policies for community conservation. The role of animal connectivity via active movement of animals represents a major knowledge gap related to the distribution of deep ocean populations. Tracking underwater targets represents a major challenge for observing biological processes in situ, and methods to robustly respond to a changing environment during monitoring missions are needed. Analytical techniques for optimal sensor placement and path planning to locate underwater targets are not straightforward in such cases. The aim of this study is to investigate the use of deep reinforcement learning as a tool for range-only underwater target tracking optimization, whose promising capabilities have been demonstrated in terrestrial scenarios. To evaluate its usefulness, a reinforcement learning method was implemented as a path planning system for an autonomous surface vehicle while tracking an underwater mobile target. A complete description of an open-source model, performance metrics in simulated environments, and evaluated algorithms based on more than 15 hours of at-sea field experiments are presented. These efforts demonstrate that deep reinforcement learning is a powerful approach that enhances the abilities of autonomous robots in the ocean and encourages the deployment of algorithms like these for monitoring marine biological systems in the future.

publicDec 2024View details →

Data from: An opponent striatal circuit for distributional reinforcement learning

Open the record for dataset details and reuse information.

publicJun 2024View details →

Robotic manipulation datasets for offline compositional reinforcement learning

Open the record for dataset details and reuse information.

publicJul 2023View details →

Dynamic robotic tracking of underwater targets using reinforcement learning

Open the record for dataset details and reuse information.