Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
47
datasets available to search
ShareScore release 0.9.0
Dataset results
47 results for “hidden Markov model”
Accompanying simulated data for "Go multivariate: a Monte Carlo study of a multilevel hidden Markov model with categorical data of varying complexity"
<p>The multilevel hidden Markov model (MHMM) is a promising vehicle to investigate latent dynamics over time in social and behavioral processes. By including continuous individual random effects, the model accommodates variability between individuals, providing individual-specific trajectories and facilitating the study of individual differences. However, the performance of the MHMM has not been sufficiently explored. Currently, there are no practical guidelines on the sample size needed to obtain reliable estimates related to categorical data characteristics We performed an extensive simulation to assess the effect of the number of dependent variables (1-4), the number of individuals (5-90), and the number of observations per individual (100-1600) on the estimation performance of group-level parameters and between-individual variability on a Bayesian MHMM with categorical data of various levels of complexity. We found that using multivariate data generally alleviates the sample size needed and improves the stability of the results. Regarding the estimation of group-level parameters, the number of individuals and observations largely compensate for each other. Meanwhile, only the former drives the estimation of between-individual variability. We conclude with guidelines on the sample size necessary based on the complexity of the data and the study objectives of the practitioners.</p> <p>This repository contains data generated for the manuscript: "Go multivariate: a Monte Carlo study of a multilevel hidden Markov model with categorical data of varying complexity". It comprehends: (1) model outputs (maximum a posteriori estimates) for each repetition (n=100) of each scenario (n=324) of the main simulation, (2) complete model outputs (including estimates for 4000 MCMC iterations) for two chains of each repetition (n=3) of each scenario (n=324). Please note that the empirical data used in the manuscript is not available as part of this repository. A subsample of the data used in the empirical example are openly available as an example data set in the R package <a href="https://cran.r-project.org/web/packages/mHMMbayes/index.html">mHMMbayes on CRAN</a>. The full data set is available on request from the authors.</p>
Feature Extraction Using Hidden Markov Model for a Phonetic Process
<p>Speech is one of the primary forms of communication among humans. In real life, a dictionary is used to seek the pronunciation of a complex word; but, for computers, this look-up table is called a phonetic dictionary. A speech recognition process tags a word-utterance to its phoneme structure, thereby returning the grapheme representation. However, the speech recognition process is challenging because of the contextual relationship between words and sentences, dependent on speakers’ intentions. Further, factors influencing time, accents, noisy environment, and data security impose accuracy threats. The present research study proposes a new hybrid speech recognition model by considering three significant aspects: sound generation through phonetic representation, sound acoustics for transmission, and sound reception on how the sound is received. These steps are achieved through a speech-to-text model divided into various stages such as noise removal, speech-pause detection, feature extraction through framing, and windowing by adopting Hidden Markov Model (HMM). The implementation is performed on a phonetic tool, Praat. The robustness of the model is estimated using evaluation metrics such as f-measure and accuracy, resulting in 98% and 99% scores, respectively. Thus, the proposed approach efficiently transforms the spoken words into their corresponding text.</p>
Figure 5. A chromosome structure in case we have 2 visible states and 3 invisible states-Neuroevolution Mechanism for Hidden Markov Model
<p>Generating a population of size n of HMMs at random can be performed with some<br> restrictions:<br> - The weights representing the input layer in the chromosome should be always negligible as<br> initial values.<br> - The weights which are involved in summation of 1.0 in the hidden layer part of the<br> chromosome should be exactly 1.0.<br> Let us assume the following case<br> Visible states are 2 and invisible states (observations) are 3, , then we shall have a<br> chromosome as shown in Figure 5.</p>
Figure 6. Two point crossover of HMM chromosomes.-Neuroevolution Mechanism for Hidden Markov Model
<p>Two point Crossover<br> For the two point crossover we get two parent HMMs and choose at random two cutting points for<br> the weights that have a sum of 1.0 and swap the contents between the crossing points. This is<br> illustrated in the example shown in Figure 6.</p>
Figure 4. A chromosome structure for HMM shown in Figure 2.-Neuroevolution Mechanism for Hidden Markov Model
<p>The chromosome which represents the HMM can be extracted from its corresponding neural<br> network. The general structure of the chromosome is divided into two sections, input layer and<br> hidden layer. Each section contains many slots, and each slot represents a weight from one node in<br> that layer to a node in the next layer (from input to hidden and from hidden to output). The number<br> of slots in the input layer is the same number of input nodes in the neural network. In the hidden<br> layer, number of slots is equal to nodes in the output layer multiplied by the nodes in the hidden<br> layer.</p>
Figure 3. Neural network representation for HMM given in Figure 2.-Neuroevolution Mechanism for Hidden Markov Model
<p>In our proposed structure, we injected a hidden layer to have a multilayer perceptron which<br> is more efficient than single layer perceptron.<br> To make this process clear, Figure 3 shows the neural networks for the HMM presented in<br> Figure 2.</p>
Figure 9. Adding-Reducing mutation.-Neuroevolution Mechanism for Hidden Markov Model
<p>Adding a small value from one weight and decrement that value to another weight. The chosen<br> weights should be involved in summation of 1.0. Figure 9 shows an example, we add 0.001 from<br> one weight and decrement the same value from another weight.</p>
Figure 1. HMM to describe a relation between the states Med. and High with the observations (invisible states) cold and hot.-Neuroevolution Mechanism for Hidden Markov Model
<p>The advantage of using this technique is that MCPRs are very useful in real time<br> applications and can be adapted over time based on the obtained experience of the networking<br> working process. Again Hewahi[6] proposed a mechanism (algorithm) to evolve and select the best<br> suitable HMM for a given problem using GA, this mechanism lacks to the training process that can<br> be of great usefulness in finding the best HMM.<br> Based on the above mentioned research, the importance of using HMM is increasing<br> rapidly.<br> Let us consider the HMM presented in Figure 1.</p>
Figure 2. HMM with weights and necessary conditions on top of edges-Neuroevolution Mechanism for Hidden Markov Model
<p>Based on the HMM structure in Figure 2, we can perform the following steps:<br> 1. Make the number of nodes of inputs in the input layer of the NN as the number of states<br> (visible states not the observations). Each input node represents one state.<br> 2. Number of nodes in the output layer in the NN is equal to the number of states and<br> observations (visible and invisible states), where each node corresponds to one state (visible<br> or invisible).<br> 3. We construct a hidden layer in NN with n number of nodes, where n is the same number of<br> nodes in the input layer.<br> 4. We make a connection from every input to every hidden layer node with a very negligible<br> weight.<br> 5. Connect every hidden node in the hidden layer to every node in the output layer.<br> 6. Assign weights from the hidden layer to output layer in a way that as every node in the<br> hidden layer corresponding to input state. The weight on top of the link between the hidden<br> node to the output node is the probability value between the states in the HMM.<br> In our proposed structure, we injected a hidden layer to have a multilayer perceptron which<br> is more efficient than single layer perceptron.<br> To make this process clear, Figure 3 shows the neural networks for the HMM presented in<br> Figure 2.</p>
Figure 6. One chromosome from the population and the five chromosomes existing in the evaluation partition.-Genetic Algorithms Principles Towards Hidden Markov Model
<p>For example comparing the<br> chromosome given in Figure 6 with the first chromosome in the evaluation partition, the<br> difference between the relation Med-Med and Med-High as a pair is 0.0 and the difference<br> between the relation High-High and High-Med as a pair is 0.1. Similarly the difference between<br> the relation Med-Cold and Med-Hot as a pair is 0.1 and the difference between the relation<br> High-Cold and High-Hot as a pair is 0.2. We sum all these differences to get the value of<br> compare(i,j), the sum value is 0+0.1+0.1+0.2 = 0.4. Using the same approach we compute the<br> compare function with the other four chromosomes and we get values 0.4, 0.5,0.4 and 0.6. Now<br> we sum the five values 0.4 + 0.4 + 0.5+ 0.4 +0.6 = 2.3. The fitness value is then 1/ 2.3 = 0.434.<br> The highest is the fitness value, the better is the performance of the chromosome.</p>
Figure 4. Incorrect crossover operation. The High-High and High-Med probability values summation should be 1.-Genetic Algorithms Principles Towards Hidden Markov Model
<p>In this genetic operator, we choose two chromosomes at random and apply crossover between<br> them. Figure 3 shows the proposed crossover. We choose a crossing cut site at random. It is to be<br> noted that the crossing cut site should be even number. We should have two crossing cut sites. If<br> we make crossing cut site odd number, the resultant child will not have a correct value of<br> probability. The incorrect crossover is shown in Figure 4.</p>
Figure 5. Mutation process. This is happened by decreasing 0.2 from Med-Cold probability and adding 0.2 to Med- Hot.-Genetic Algorithms Principles Towards Hidden Markov Model
<p>Figure 5 illustrates an example of mutation process. In Figure 5, Med-Cold:0.9 and Med-Hot:0.1<br> before mutation and become Med-Cold:0.7 and Med-Hot:0.3 after mutation. This is done by<br> decreasing 0.2 from Med-Cold probability and adding 0.2 to Med-Hot probability.</p>
Figure 11. Group mutation-Neuroevolution Mechanism for Hidden Markov Model
<p>This happens by swapping two complete groups with summation of 1.0 with the same criteria.<br> Figure 10 shows a case of this.</p>
Figure 7. One point crossover of HMM chromosomes.-Neuroevolution Mechanism for Hidden Markov Model
This crossover is performed in the input layer part only. We choose a crossing cut point in the input layer part of the chromosome, and exchange everything before it. This is illustrated in Figure 7.
Figure 3. The crossover operation between two HMM chromosomes Figure-Genetic Algorithms Principles Towards Hidden Markov Model
<p>Crossover<br> In this genetic operator, we choose two chromosomes at random and apply crossover between<br> them. Figure 3 shows the proposed crossover. We choose a crossing cut site at random. It is to be<br> noted that the crossing cut site should be even number. We should have two crossing cut sites. If<br> we make crossing cut site odd number, the resultant child will not have a correct value of<br> probability. The incorrect crossover is shown in Figure 4.</p>
Figure 2. The general structure of the proposed approach-Genetic Algorithms Principles Towards Hidden Markov Model
<p>The chromosome contains 8 genes, each is represented by the relation between two states<br> accompanied with a probability value. The genes should be formed in this way because this is<br> important in the crossover operation as to be explained later. The most important thing is that each<br> two genes has the probability summation of 1.0. For example Med-Med:02 and Med-High:08 have<br> the summation of 1.0. Similarly High-High:0.6 and High-Med:0.4 have the summation of 1.0. Each<br> two genes with summation of 1.0 should be neighbors.</p>
Figure 1. HMM to describe a relation between the states Med. and High with the observations (invisible states) cold and hot.-Genetic Algorithms Principles Towards Hidden Markov Model
<p>Hewahi [4] presented a modified version of Censored Production Rule (CPR) called<br> Modified Censored Production Rules (MCPR). CPR is proposed by Michalski and Winston [6 ] to<br> capture real time situations. MCPR can fit with hidden Markov model and present a scheme to<br> compute the certainty values of the obtained conclusions out of the induced rules. To compute the<br> certainty values for the rule actions (conclusions), the approach exploited only the probability<br> values associated with the hidden Markov model without using any of the other well known<br> certainty computation approaches. Hewahi [3] also proposed an intelligent networking<br> management system based on the induced MCPRs extracted from a networking structure based on<br> HMM. The advantage of using this technique is that MCPRs are very useful in real time<br> applications and can be adapted over time based on the obtained experience of the networking<br> working process.<br> Let us consider the HMM presented in Figure 1.</p>
Accompanying empirical data for Kirchherr et al., 2023, "Bayesian multilevel hidden Markov models identify stable state dynamics in longitudinal recordings from macaque primary motor cortex"
<p>This repository contains data accompanying: Kirchherr et al., 2023, "Bayesian multilevel hidden Markov models identify stable state dynamics in longitudinal recordings from macaque primary motor cortex".</p> <p>Data collection methods:</p> <p>Two adult female rhesus macaques (Macaca mulatta) trained on a reaching, and grasping, and placing task served as the subjects. The animal handling as well as surgical and experimental procedures complied with European guideline (2010/63/UE) and authorized by the French Ministry for Higher Education and Research (project # 2016112713202878) in force on the care and use of laboratory animals, and were approved by the ethics committee CELYNE (comité d’éthique Lyonnais pour les neurosciences expérimentale, C2EA 42). After initial training, we performed a sterile surgery to implant six floating multielectrode arrays (FMA, Microprobes for Life Science, Gaithersburg, MD, USA) in the right (monkey 1) or left (monkey 2) cortical hemisphere. Each array was comprised of 32 platinum/iridium electrodes (impedance 0.5 MΩ at 1 kHz) with lengths ranging from 1 to 6 mm, and with an inter-electrode spacing of 400 μm. One electrode array was implanted in the primary motor cortex (M1), two were implanted in the ventral premotor cortex (F5), one in the dorsal premotor cortex (F2), and two in the prefrontal cortex (45a and 46/12r), as estimated according to a previous magnetic resonance imaging scan. For the purposes of this study, we analyzed data from the M1 array of each monkey.</p> <p>The wideband neural signal (bandpass filtered at 0.1 to 7500 kHz) was recorded at 30 kS/s, and amplified and digitized (16-bit; 0.192 μV resolution) with an Intan Tech-based (Intan Technologies, Los Angeles, CA, USA) open source acquisition system (Open Ephys; Siegle et al. 2017). This system uses a 256-channel Intan RHD2000 series acquisition board and 32-channel headstages (RHD2132). Spike detection was performed offline using Trisdesclous (Garcia & Pouzat,2015). The common reference was removed to reduce ambient noise. Spikes were then detected from each electrode using a threshold of 2 times the median absolute deviation (MAD), and analyzed as multi-unit activity (MUA) in 10 ms bins. All electrodes in which at least one well-isolated spike waveform was detected were selected for the following analyses. We thus used a sample of 21 electrodes out of 32 for monkey 1, and 25 out of 32 electrodes for monkey 2. Custom made detection panels were used to record the moments when the monkey’s hand released the handle, the hand contacted the target object, and when the object was placed in the groove. An Omniplex 16-channel recording system (Plexon, Dallas, TX, USA) was used to simultaneously record these behavioral events. Trials were discarded if the response time (time between the go signal and handle release) was less than 100 or greater than 1500 ms, the reach duration (time between handle release and object contact) was less than 100 or greater than 1000 ms, or the placing duration (time between object contact and placing the object in the groove) was less than 100 or greater than 1200 ms, leaving 19 - 68 trials per day for monkey 1 (M = 43.9, SD = 15.46, N = 439; left: M = 14.8, SD = 5.74; center: M = 14.4, SD = 5.15; right: M = 14.7, SD = 7.73), and 23 - 49 per day for monkey 2 (M = 38.3, SD = 9.87, N = 383; left: M = 14.2, SD = 3.91; center: M = 10.8, SD = 3.55; right: M = 13.3, SD = 3.37).</p> <p><br> Abstract:</p> <p>Neural populations, rather than single neurons, may be the fundamental unit of cortical computation. Analyzing chronically recorded neural population activity is challenging not only because of the high dimensionality of activity in many neurons, but also because of changes in the recorded signal that may or may not be due to neural plasticity. Hidden Markov models (HMMs) are a promising technique for analyzing such data in terms of discrete, latent states, but previous approaches have either not considered the statistical properties of neural spiking data, have not been adaptable to longitudinal data, or have not modeled condition specific differences. We present a multilevel Bayesian HMM which addresses these shortcomings by incorporating multivariate Poisson log-normal emission probability distributions, multilevel parameter estimation, and trial-specific condition covariates. We applied this framework to multi-unit neural spiking data recorded using chronically implanted multi-electrode arrays from macaque primary motor cortex during a cued reaching, grasping, and placing task. We show that the model identifies latent neural population states which are tightly linked to behavioral events, despite the model being trained without any information about event timing. We show that these events represent specific spatiotemporal patterns of neural population activity and that their relationship to behavior is consistent over days of recording. The utility and stability of this approach is demonstrated using a previously learned task, but this multilevel Bayesian HMM framework would be especially suited for future studies of long-term plasticity in neural populations.</p>
A Bayesian Phylogenetic Hidden Markov Model for B Cell Receptor Sequence Analysis
<p>simulation and PC64/VRC01 input/output data files</p>
From pup to predator; generalized hidden Markov models reveal rapid development of movement strategies in a naïve long‐lived vertebrate
<p>Rapid development of a successful foraging strategy is critical for juvenile survival, especially for naïve animals that receive no parental guidance. However, this process is poorly understood for many species. Although observation of early-life movements is increasingly possible with miniaturisation of animalborne telemetry devices, analytical limitations remain. Here, we tracked 29 recently-weaned, grey seal <em>Halichoerus grypus</em> pups from colonies in two geographically distinct regions of the United Kingdom. We analysed at-sea movements of pups throughout their initial months of nutritional independence to investigate the ontogeny of behaviour-specific (foraging and travelling) movement patterns. Using generalized hidden Markov models (HMMs), we extended the conventional HMM framework to account for temporal changes in putative foraging and travelling movement characteristics, and investigate the effects of intrinsic (sex) and extrinsic (environment) factors on this process. Putative foraging behaviour became more tortuous with time, and travelling became faster and more directed, suggesting a reduction in search scale and an increase in travel efficiency as pups shifted from exploration to an adult-like repeatable foraging strategy. Sex differences in movement characteristics were evident from colony departure, but sex-specific activity budgets were only detected in one region. We show that sex-specific behavioural strategies emerge before sexual size dimorphism in grey seals, and suggest that this phenomenon may occur in other long-lived species. Our results also indicate that environmental variation may affect the emergence of sex-specific foraging behaviour, highlighting the need to consider interacting intrinsic and extrinsic factors in shaping movement strategies of long-lived vertebrates. Moreover, comparing the behavioural state estimations to those of a conventional HMM (no variation in statespecific movement parameters) revealed differences in the amount and location of foraging activity, with implications for spatial conservation management. Overlooking intrinsic and extrinsic variation in movement processes could distort our understanding of foraging ecology, population dynamics, and conservation requirements.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.