Task-driven neural network models predict neural dynamics of proprioception: Experimental data, activations and predictions of neural network models

<p>#############</p> <p>Task-driven neural network models predict neural dynamics of proprioception, Cell 2024</p> <p>#############</p> <p>Authors: Marin Vargas, Alessandro (orcid=0000-0001-7073-4120) and Bisi, Axel (orcid=0009-0006-8602-7555) and Chiappa, Alberto Silvio (orcid=0009-0001-2764-6552) and Versteeg, Christopher (orcid=0000-0002-4269-5109) and Miller, Lee E. (orcid=0000-0001-8675-7140) and Mathis, Alexander (orcid=0000-0002-3777-2202)</p> <p>Affiliation: EPFL</p> <p>Date: January, 2024</p> <p>Link to the Cell article:</p> <p><a href="https://www.cell.com/cell/pdf/S0092-8674(24)00239-3.pdf">https://www.cell.com/cell/pdf/S0092-8674(24)00239-3.pdf</a></p> <p>--------------------------------</p> <p>Here we provide the neural data, activation and predictions for the best models and result dataframes of our article "Task-driven neural network models predict neural dynamics of proprioception".</p> <p>It contains the behavioral and neural experimental data (cuneate nucleus and somatosensory recordings from the Miller Lab, Northwestern University), the result dataframes for task-driven and untrained models, the activations and predictions for the *best models for all tasks* for active and passive movements and the predictions for linear models for active and passive movements.&nbsp;</p> <p>Note, the predictions of other models can be computed from the network weights that were deposited for all trained models.&nbsp;</p> <p>The overall structure of the data is:</p> <p>└── exp_analysis<br>&nbsp; &nbsp; ├── results &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; - Contains the result dataframe of the predictions for all models, tasks and primates<br>&nbsp; &nbsp; ├── activations<br>&nbsp; &nbsp; │ &nbsp; ├── active &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp; - Contains activations related to active movements<br>&nbsp; &nbsp; │ &nbsp; └── passive &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp; - Contains activations related to passive movements<br>&nbsp; &nbsp; ├── predictions<br>&nbsp; &nbsp; │ &nbsp; ├── active &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; - Contains predictions related to active movements<br>&nbsp; &nbsp; │ &nbsp; └── passive &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; - Contains predictions related to passive movements<br>&nbsp; &nbsp; └── beh_exp_datasets<br>&nbsp; &nbsp; &nbsp; &nbsp; ├── matlab_data &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; - Contains raw behavioral and neural data<br>&nbsp; &nbsp; &nbsp; &nbsp; ├── MonkeyAlignedDatasets_new &nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp; - Contains padded test behavioral input for generating network activations<br>&nbsp; &nbsp; &nbsp; &nbsp; ├── MonkeyDatasets&nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; - Contains not aligned padded test behavioral input for generating network activations<br>&nbsp; &nbsp; &nbsp; &nbsp; ├── MonkeySpikeRegressDatasets &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; - Contains datasets for training data-driven models<br>&nbsp; &nbsp; &nbsp; &nbsp; ├── MonkeySpikeRegressDatasets_new &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; - Contains trial index for regression splits&nbsp;<br>&nbsp; &nbsp; &nbsp; &nbsp; └── new_beh_exp_dataframe &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;&nbsp; - Contains pre-processed behavioral and neural data</p> <p>--------------------------------</p> <p>The activations and predictions for the best 3 models and for all tasks are stored in experiments folder (in .h5 format) that follows the same name convention of the checkpoints.</p> <p>The checkpoints are stored in experiment folders (experiment_***) that follow this scheme:<br>- Task: &nbsp; &nbsp; &nbsp; &nbsp; shallow exp id, &nbsp; &nbsp; deep TCNs exp id, &nbsp; &nbsp; LSTM id.</p> <p>Experiment IDs for each task:</p> <p>- Untrained: &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp; &nbsp;&nbsp; &nbsp;&nbsp;&nbsp; 15, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 115, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 45<br>- Classification: &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;&nbsp; 4015, &nbsp; 5015, &nbsp; 4045</p> <p>- Torque: &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp; 8015, &nbsp; 8030, &nbsp; 8045</p> <p>- Regress joint pos: &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp; &nbsp; 17016, &nbsp;17031, &nbsp;17046<br>- Regress joint vel: &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp; 17216, &nbsp;17231, &nbsp;17246<br>- Regress joint pos &amp; vel:: &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;&nbsp; &nbsp; &nbsp; &nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp; 17416, &nbsp;17431, &nbsp;17446<br>- Regress joint pos &amp; vel &amp; acc:: &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp; &nbsp;&nbsp;&nbsp; 20516, &nbsp;20531, &nbsp;20546</p> <p>- Regress hand pos: &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp; &nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 4016, &nbsp; 5016, &nbsp; 4046<br>- Regress hand vel: &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 17316, &nbsp;17331, &nbsp;17346<br>- Regress hand pos &amp; vel: &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 17516, &nbsp;17531, &nbsp;17546<br>- Regress hand pos &amp; vel &amp; acc: &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp; &nbsp; &nbsp; &nbsp;&nbsp; 20416, &nbsp;17831, &nbsp;17846</p> <p>- Regress hand and elbow pos: &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 20016, &nbsp;20031, &nbsp;20046<br>- Regress hand and elbow vel: &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 20916, &nbsp;20931, &nbsp;20946<br>- Regress hand and elbow pos &amp; vel: &nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp; 20616, &nbsp;20631, &nbsp;20646<br>- Regress hand and elbow pos &amp; vel &amp; acc:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 20816, &nbsp;20831, &nbsp;20846</p> <p>- Redundancy reduction: &nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 10020, &nbsp;10035, &nbsp;10050<br>- Autoencoder &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp; 20716 &amp; 20717, 20731 &amp; 20732, &nbsp; X</p> <p>&nbsp;</p> <p>The code to process the behavioral data is available at: <a href="https://github.com/amathislab/Task-driven-Proprioception/tree/master/exp_data_processing">https://github.com/amathislab/Task-driven-Proprioception/tree/master/exp_data_processing</a><br>The code to load and use the models to generate activations and predictions is available at: <a href="https://github.com/amathislab/Task-driven-Proprioception/tree/master/neural_prediction">https://github.com/amathislab/Task-driven-Proprioception/tree/master/neural_prediction</a></p> <p>To reproduce the results, it is possible to reproduce the main figures using the result dataframe. See our repository for more details.&nbsp;</p> <p>--------------------------------</p> <p>The datasets, weights, activations and predictions are released with Creative Commons Attribution 4.0 license.</p> <p>The code is released under the MIT license, see <a href="https://github.com/amathislab/Task-driven-Proprioception">https://github.com/amathislab/Task-driven-Proprioception</a></p> <p>If you find our code, weights, predictions or ideas useful, please cite:</p> <p>@article{vargas2024task,<br>&nbsp; title={Task-driven neural network models predict neural dynamics of proprioception},<br>&nbsp; author={{Marin Vargas}, Alessandro and Bisi, Axel and Chiappa, Alberto S and Versteeg, Chris and Miller, Lee E and Mathis, Alexander},<br>&nbsp; journal={Cell},<br>&nbsp; year={2024},<br>&nbsp; publisher={Elsevier}<br>}</p>