Find research datasets worth reusing

Bio-logger Ethogram Benchmark: A benchmark for computational analysis of animal behavior, using animal-borne tags

<p>This repository contains the datasets and experiment results presented in our <a href="https://arxiv.org/abs/2305.10740">arxiv paper</a>:</p> <blockquote> <p>B. Hoffman, M. Cusimano, V. Baglione, D. Canestrari, D. Chevallier, D. DeSantis, L. Jeantet, M. Ladds, T. Maekawa, V. Mata-Silva, V. Moreno-Gonz&aacute;lez, A. Pagano, E. Trapote, O. Vainio, A. Vehkaoja, K. Yoda, K. Zacarian, A. Friedlaender, "A benchmark for computational analysis of animal behavior, using animal-borne tags," 2023.</p> </blockquote> <p>Standardized code to implement, train, and evaluate models can be found at <a href="https://github.com/earthspecies/BEBE/">https://github.com/earthspecies/BEBE/</a>.&nbsp;</p> <p>Please note the licenses in each dataset folder.</p> <p><strong>Zip folders beginning with "formatted":</strong> These are the datasets we used to run the experiments reported in the benchmark paper.&nbsp;</p> <p><strong>Zip folders beginning with "raw": </strong>These are the unprocessed datasets used in BEBE. Code to process&nbsp;these raw datasets into the formatted ones used by BEBE can be found at&nbsp;<a href="https://github.com/earthspecies/BEBE-datasets/">https://github.com/earthspecies/BEBE-datasets/</a>.</p> <p><strong>Zip folders beginning with "experiments": </strong>Results of the cross-validation experiments reported in the paper, as well as hyperparameter optimization. Confusion matrices for all experiments can also be found here. Note that dt, rf, and svm refer to the feature set from Nathan et al., 2012.</p> <p><em>Results used in Fig. 4 of <a href="https://arxiv.org/abs/2305.10740">arxiv paper</a> (deep neural networks vs. classical models)</em><br>{dataset}_ harnet_nogyr<br>{dataset}_CRNN<br>{dataset}_CNN<br>{dataset}_dt<br>{dataset}_rf<br>{dataset}_svm<br>{dataset}_wavelet_dt<br>{dataset}_wavelet_rf<br>{dataset}_wavelet_svm</p> <p><em>Results used in Fig. 5D of <a href="https://arxiv.org/abs/2305.10740">arxiv paper</a> (full data setting)<br></em>If dataset contains gyroscope (HAR, jeantet_turtles, vehkaoja_dogs):<br>{dataset}_harnet_nogyr<br>{dataset}_harnet_random_nogyr<br>{dataset}_harnet_unfrozen_nogyr<br>{dataset}_RNN_nogyr<br>{dataset}_CRNN_nogyr<br>{dataset}_rf_nogyr<br><br>Otherwise:<br>{dataset}_harnet_nogyr<br>{dataset}_harnet_unfrozen_nogyr<br>{dataset}_harnet_random_nogyr<br>{dataset}_RNN_nogyr<br>{dataset}_CRNN<br>{dataset}_rf</p> <p><em>Results used in Fig. 5E of <a href="https://arxiv.org/abs/2305.10740">arxiv paper</a> (reduced data setting)<br></em>If dataset contains gyroscope (HAR, jeantet_turtles, vehkaoja_dogs):<br>{dataset}_harnet_low_data_nogyr<br>{dataset}_harnet_random_low_data_nogyr<br>{dataset}_harnet_unfrozen_low_data_nogyr<br>{dataset}_RNN_low_data_nogyr<br>{dataset}_wavelet_RNN_low_data_nogyr<br>{dataset}_CRNN_low_data_nogyr<br>{dataset}_rf_low_data_nogyr</p> <p>Otherwise:<br>{dataset}_harnet_low_data_nogyr<br>{dataset}_harnet_random_low_data_nogyr<br>{dataset}_harnet_unfrozen_low_data_nogyr<br>{dataset}_RNN_low_data_nogyr<br>{dataset}_wavelet_RNN_low_data_nogyr<br>{dataset}_CRNN_low_data<br>{dataset}_rf_low_data<br><br></p> <p><strong>CSV files</strong>: we also include summaries of the experimental results in experiments_summary.csv, experiments_by_fold_individual.csv, experiments_by_fold_behavior.csv.&nbsp;</p> <p><em>experiments_summary.csv - results averaged over individuals and behavior classes<br></em>dataset (str): name of dataset<br>experiment (str): name of model with experiment setting&nbsp;<br>fig4 (bool): True if dataset+experiment was used in figure 4 of&nbsp;<a href="https://arxiv.org/abs/2305.10740">arxiv paper</a><br>fig5d (bool): True if dataset+experiment was used in figure 5d of&nbsp;<a href="https://arxiv.org/abs/2305.10740">arxiv paper</a><br>fig5e (bool): True if dataset+experiment was used in figure 5e of&nbsp;<a href="https://arxiv.org/abs/2305.10740">arxiv paper</a><br>f1_mean (float): mean of macro-averaged F1 score, averaged over individuals in test folds<br>f1_std (float): standard deviation of macro-averaged F1 score, computed over individuals in test folds<br>prec_mean, prec_std (float): analogous for precision<br>rec_mean, rec_std (float): analogous for recall<em><br><br>experiments_by_fold_individual.csv - results per individual in the test folds<br></em>dataset (str): name of dataset<br>experiment (str): name of model with experiment setting&nbsp;<br>fig4 (bool): True if dataset+experiment was used in figure 4 of&nbsp;<a href="https://arxiv.org/abs/2305.10740">arxiv paper</a><br>fig5d (bool): True if dataset+experiment was used in figure 5d of&nbsp;<a href="https://arxiv.org/abs/2305.10740">arxiv paper</a><br>fig5e (bool): True if dataset+experiment was used in figure 5e of&nbsp;<a href="https://arxiv.org/abs/2305.10740">arxiv paper</a><br>fold (int): test fold index<br>individual (int): individuals are numbered zero-indexed, starting from fold 1<br>f1 (float): macro-averaged f1 score for this individual<br>precision (float): macro-averaged precision for this individual<br>recall (float): macro-averaged recall for this individual<em><br></em></p> <p><em>experiments_by_fold_behavior.csv - results per behavior class, for each test fold<br></em>dataset (str): name of dataset<br>experiment (str): name of model with experiment setting&nbsp;<br>fig4 (bool): True if dataset+experiment was used in figure 4 of&nbsp;<a href="https://arxiv.org/abs/2305.10740">arxiv paper</a><br>fig5d (bool): True if dataset+experiment was used in figure 5d of&nbsp;<a href="https://arxiv.org/abs/2305.10740">arxiv paper</a><br>fig5e (bool): True if dataset+experiment was used in figure 5e of&nbsp;<a href="https://arxiv.org/abs/2305.10740">arxiv paper</a><br>fold (int): test fold index<br>behavior_class (str): name of behavior class<br>f1 (float): f1 score for this behavior, averaged over individuals in the test fold<br>precision (float): precision for this behavior, averaged over individuals in the test fold<br>recall (float): recall for this behavior, averaged over individuals in the test fold<br>train_ground_truth_label_counts (int): number of timepoints labeled with this behavior class, in the training set<em><br></em></p>