The Experience Sampling Method (ESM) is one of the most reliable procedures for studying daily lives and people’s experiences, activities, and feelings throughout the day. With the great idea of Kurt Lewin (1935) to create a topology of daily activities and the ambitious techniques created by Roger Barker and Herbert Wright, ESM studies have become incredibly popular today.

Each research method and findings benefit science, for sure. From longitudinal studies to archival research, adding more to the existing knowledge is always beneficial. However, ESM has numerous advantages when compared to other techniques in the field of biology, psychology, and medicine. For instance, while laboratory experiments limit research with the artificiality of some procedures, ESM do not take subjects out of their natural environment. In fact, that’s why ESM studies are called “ecological” – because all research and observation take place within natural settings, in “real-time” and “real-life” situations (Schiffman, Stone, and Hufford, 2008). On the other hand, while observational studies cannot eliminate the influence of the observer completely, ESM rely on self-reports and people’s active participation. In other words, ESM studies give researchers the chance to implement controlled and natural settings at the same time.

ESM studies are applicable to a wide range of scientific topics, such as panic attacks, medical interventions, pain management, reflecting on feelings, etc., they can be used within organizational settings as well. As mentioned earlier, ESM studies are among the most valuable research assets to science.

2.2.1. Research Hypothesis:

After the hard work, it’s time for a more precise design. You’re the designer, so the beauty of your research lies in your hands. Formulate a hypothesis – a hypothesis that’s in fact testable. In order to do that, systematize all variables that might have an impact on the study. Decide if you want to study behaviors, such as drinking or eating, experiences, such as mood or cognition, or physiological functions, such as breathing or hormonal levels. Having a clear operational definition is important because – by having a clearer idea on what’s being studied – you have more control on the process.

2.2.2. Define Your Variables:

As you know, the relation between the variables of interest will be tested in your hypothesis. Hypotheses usually focus on various dynamic within-person interactions. The best way to achieve a meaningful result is to study variables that change over a short period of time (Conner & Lehman, 2012). Do not forget that having a null hypothesis is also important. Last but not least, remember that even by rejecting your hypothesis, you contribute to science.

2.2.3. Statistical Procedures:

Having a clear idea on how to analyze the collected data is also very important. Simply because sometimes we might have data, which we cannot use, or too many figures we do not know how to analyze. Collecting, recording, interpreting and presenting data is vital, and research should be factual and based on statistical procedures.

Note that ESM can be a great quantitative method. Simply because all techniques provide large numbers of data. A method that includes multiple questions, multiple contexts and multiples times every single day is definitely a research technique that is a dream.

On the other hand, ESM studies can be used as a wonderful qualitative method because each participant self-reports their feelings, behaviors, and thoughts in their everyday life.

Do not forget to include charts and tables for the visual representation of your data.

2.2.4. ESM Design:

Then, it’s time for another major aspect of your ESM study: choose an ESM design. If having a hypothesis and data is needed in each experiment, ESM designs are a unique characteristic applicable only to ESM studies. We’ll discuss this section and the different types of ESM design later.

2.2.5. Reporting Methods:

Just like ESM designs, the different reporting methods used in ESM studies are also unique. When one starts designing a study, they are often overwhelmed by ideas and expected results and applications. However, the practical side of your study should also be considered and planned in details. In addition, you have to decide if your study will benefit from an active or passive reporting method.  An example of an active reporting method is daily diaries, while a passive method is neuroendocrine sampling, for example.  Choosing the right reporting method is crucial. Note that Smartphone Apps engage participants the most, and in today’s society, even older participants have no problems using technology. This section will also be discussed at a later stage.

Another important aspect of the practical side of your ESM study is to create meaningful and engaging questions. Note that questions for ESM studies should not be time-consuming. As participants will be asked to report behavior, thoughts, and other issues several times per day (for one or more weeks), questions should not become a burden (emotionally or cognitively).

When deciding on the number of questions needed, researchers should focus on how motivated their participants are, how long the study is going to take, and how much effort the testing itself requires. Note that researchers have accepted a burden cap of 20 minutes per day (Conner, 2015).  Formulate the questions correctly – ESM studies should not ask people for their experiences in general, but for current events. Usually, researchers can convert a traditional study into an ESM measure to ensure good validity and reliability.

The actual text of the questions should be short, especially if small screens of mobile phones or personal digital assistants (PDAs) are used as preferred reporting methods.

2.3.1. Is It Reliable?

Well done on the hard job of creating a questionnaire! However, hold on for a second. It’s time to ask yourself: Are the ESM scales used in the study reliable? Note that some researchers claim that reliability matters only in the service of validity (Bergkvist & Rossiter, 2009). Still, experts are concerned the most about the following problem: Do ESM items measure between-person variations only, instead of within-person factors (Shrout & Lane, 2011)?

The first step to check if the created questionnaire shows good reliability is to report Cronbach’s alpha (especially when multi-item scales are present). When we create questions, we should remember that at least three items for each ESM construct are needed (Shrout and Lane, 2011). Single items work fine as well, for example, a 7- to 10-point scale or a 0-100 slider scale, which in addition is a method that can be used to increase variance (Van Hooff, Geurts, Kompier, and Taris, 2007). However, complicated constructs that change over a longer time span should be assessed with multi-item scales.

2.4. Participants:

After that, it’s time to focus on the most important aspect of each research – participants. We have to agree that building up on knowledge for the pure purpose of knowledge is intriguing. However, the human factor is the most important part of a research. By applying the existing knowledge to everyday life, scientists can improve people’s well-being and interactions. And that’s what matters the most.

Researchers might face various difficulties when experts choose participants. Factors, such as sample size and demographics, are very important. Usually, ESM studies test the same participants and their daily lives, with variables that fluctuate over the short term. You might consider even recruiting and motivating your research participants (Conner & Lehman, 2011). Having intrinsically and extrinsically motivated participants is one of the factors that will influence your results positively. Rewarding your participants might be extremely helpful.

Note that when it comes to participants and ESM studies, researchers worry if constant self-reporting can affect respondents’ perceptions. However, we can look at this effect as one of the main advantages of ESM testing. Simply because frequent self-monitoring can be used as a therapeutic intervention, creating reactivity (Barta, Tennen, & Litt, 2011).

2.5. Pilot Testing

When all research plans are finalized, it’s time to give your research endeavors a go. Pilot testing is more than beneficial. Choose a group of people similar to the intended participants of your ESM study. However, note that human beings are unique and what works fine for one sample might go wrong for another.

If you think that pilot testing is just a waste of time, you should think twice. Did you know that Schneider and Waite (2005) did three pilot-tests of their national study of families? Pilot testing can give you valuable insights on question-wording, length and other crucial aspects of your ESM questionnaire. Not only that, though. The post-collection procedures and analysis of data, such as descriptive statistics, can show researchers some problematic parameters of the questionnaire, such as coding, that was not obvious during administration.

Most of all, pilot testing is a great way to connect to all participants. The size of the pilot sample depends on the purposes of the full-scale study. In some cases, 10-20 pilot subjects are enough, in others, smaller samples are also acceptable. Although ESM studies don’t take too much time, real testing might interrupt some important daily activities, which as a result may affect the intrinsic motivation of the subjects. Thus, listening to people’s feedback and analyzing their notes taken during the study is important as it can help researchers improve their ESM study design. For instance, always ask if: response options were meaningful; signals were clear; study situations were difficult to understand; testing procedure was frustrating;, etc. From giving instructions to signaling methods, pilot-tests contain a lot of useful information and may give some humane touch to the ESM design.

Also remember that when you thrive for perfection, you should embrace errors. Focus on missing or noncompliant reports and adopt procedures to battle similar problems that may arise in the future. Ask participants if items were too irrelevant or too confusing. Make sure that the procedure used in the study wasn’t too difficult to administer, or check if signals were unclear. At the same time, create a more realistic testing environment: encourage pilot subjects to give false data or to miss a response from time to time. All these problematic situations are things that could happen even to the best ESM study designs.

Do not forget that research assistants also have to undergo some training so they can relate to their participants and understand the testing procedure better. In other words, even the “pilots” of the study have to go through the pilot testing procedure.

Also, when you choose a tech device for a reporting method, have a spare in case something fails or breaks. The signaling equipment and the procedures for dealing with data are also a subject of errors. Mind that plans don’t always go smoothly and learn from results!

3.1. Signal-Contingent Studies

Signal-contingent studies involve the use of a signal after which the participants respond to questions. Signal-contingent ESM studies request participants’ reports at varying times each day. Signals can be given by a beeper or a notification on the phone. Not only the device but the intervals should be considered: fixed times, regular repeating times, random times, or semi-random times. Signal-contingent designs are great for measuring constantly changing variables, such as depressive mood or fatigue. However, studies should not last for more than 1-2 weeks as they might become too burdensome for the participants.

As signal-based studies depend on many factors, such as the frequency of signaling, the duration of the signals and of course, the length of the study itself, we should go into details and explain a little bit more about the signal-contingent method. First of all, researchers should consider the main aspect of each ESM study: their participants and response rate. Note that in studies where participants were signaled 1-2 a day, the response rate was around 95%. In comparison, studies with signals more than eight times a day showed that participants tend to respond less, with a response rate ranging between 60% and 80% (Conner Christensen et al., 2003). The duration of the ESM study is also crucial for the design and the planning of all signaling procedures. If your testing forms require more than 3 minutes, do not expect people to fill them out more than six times a day as you’ll easily reach the burden cap in ESM research.

Also, consider all plans for any statistical analysis you may compute. Snijders and Bosker (1999) suggest that one can compute a multilevel power analysis to predict the number of cases necessary to measure within-subjects variance. Do not forget that the statistical power of the ESM study is a major factor to consider.

3.2. Interval-Contingent Studies

Interval-contingent reporting requires participants to respond at certain times every day. In other words, interval-contingent studies require reports at the same time each day. This design is perfect for routine activities (e.g., how much water one drinks), or things that are easy to remember. As they don’t require too much time or effort, these studies usually last about 1-2 weeks. However, some researchers worry that such studies are focused on a particular context. Let’s say that you test participants only at 10 A.M., when they are at work. Subject A shows higher levels of depression, while subject B seems more relaxed. So here researchers might face a problem: Can we implement these findings into everyday life and family life, for example? Is subject A always more depressed? We can’t be sure, so instead of making fake generalizations, we should consider combined methods that might be helpful.

Note that daily diaries are a type of interval-contingent study, with answering questions once a day, mainly at the end of the day. This is a great option for asking reflective questions. Studies using daily diaries can last longer as they are not so demanding, they last even about 1-4 weeks. We’ll present a daily diary study at the end of the guide.

3.3. Event-Contingent Studies

Event-contingent ESM studies require from participants to respond to questions related to a certain event (for instance, smoking) (Moskowitz & Sadikaj, 2011). They are applied to social interactions, anger management, incidents of conflict, and stress. This ESM design is the most reliable option for special events, which are not frequent. Thus, these studies can last longer – from 1 week to several months. As events are unpredictable, signaling schedules are not needed. However, some researchers find it beneficial to signal participants just to remind them to respond (Cote & Moskowitz, 1998).

3.4. Continuous Reporting

Continuous reporting involves monitoring research variables all the time. Although this method can be employed for studying some factors over short periods of time, it is generally used for physiological variables, such as heart rate or temperature. This design can require a lot from participants or can generate large amounts of data (especially when it comes to passive reporting, such as heart rate). Thus, studies that rely on this design usually last not more than a week.

As mentioned above, another alternative is to combine some of these schedules. For instance, one might combine event-contingent reports with a time-based schedule (Shiffman, 2007).

3.5. In a nutshell

To sum up, let’s present the pros and cons of the three main scheduling methods: event contingent, signal contingent, and interval contingent, as systematized by Fisher and March (2012).

Interval-contingent schedules can be conducted with simple technology. Paper-pencil or computer, this type of research allows the use of various reporting methods. Also, as participants know which times of the day they are expected to respond, they might find this method less frustrating when compared to other ESM studies. However, as mentioned earlier, as interval-contingent studies focus only on certain times, they might miss some crucial information about people’s feelings, behaviors and experiences that happen at other times. A suggestion for researchers is to use a signal just as a reminder throughout the day. In addition, to avoid memory decay, have participants reporting morning experiences reported at noon, for example.

Signal-contingent reports capture momentary changes and experiences without memory decay and even ten signals per day might be used in signal-based ESM studies. However, the unpredictable side of the signaling methods might become too burdensome to participants, and also the presence of some kind of signaling technology can be ineffective and even unappropriated (for the workplace, for example). A suggestion for researchers: design the given signals by basing them on the time it takes for each variable to fluctuate.

Event-contingent reports focus on certain events and also eliminate memory errors.  However, researchers have no control over the reporting methods. Let’s say that a participant decided to light a cigarette without reporting the event. How can you know about it? Well, a tip is to train participants to recognize which events should be reported and motivate them. As mentioned above, remuneration should be considered.

Emoticons have become an inseparable part of virtual communication. Although people often associate emoticons with online chats or teen forums, emoticons can be a scientific tool. Some studies even implement the use of emoticons to present people’s feelings. In fact, assessing emotions when using technology is difficult, so a new and challenging approach is to use emoticons on mobile devices. One of the main advantages is that emoticons can save space and create positive user experience.

As mentioned earlier, ESM studies are among the most reliable approaches to study participants’ everyday lives and experiences.

ESM studies are used in a wide range of settings; from studying individual differences to assessing various treatment programs, ESM studies have a lot of applications.

Individual Differences: First of all, ESM studies explore people’s feelings, behaviors, and routines. On top of that, it’s been proven that ESM studies are highly reliable and valid methods. For instance, eliminating recall bias makes ESM really effective, something that can be achieved by using a digital reporting method.

Natural History: It’s not only that, ESM designs can elaborate natural history, with independent variable (IV) time and dependent variable (DV) subject’s intrinsic variation, for instance. That said, ESM designs can be used to study withdrawal symptoms in ex-smokers (McCarthy et al., 2006).

Temporal Sequences: ESM studies can help researchers explore various temporal sequences. In other words, ESM studies also show the connection between dynamic processes and situational cases focusing on the sequence of events. For instance, ESM has been used a lot to study smoking cessation and relapse (Shiffman at al., 1997).

Contextual Associations: One of the many applications is that ESM experiments study the connection between events that are occurring simultaneously, not only their sequence. An interesting application is ESM in studying schizophrenia and the relation between emotions and stressful events (Myin-Germeys et al., 2001). Other symptoms of psychopathology, bipolar disorders, in particular, are also linked to stress (Stiglmayr et al., 2005), and can be studied in depth with ESM studies.

Treatment: ESM studies are very popular within clinical settings. ESM studies can be used for ongoing assessment during treatment of illnesses, such as depression (Kramer et al., 2014). Also, ESM studies are extremely effective when studying bipolar disorders and the dyadic relationships of illnesses and events (Ebner-Priemer et al., 2009).

Intervention: As mentioned earlier, ESM is a revolutionary approach that can be used not only in treatment but intervention programs as well. Some of the applications that ESM studies have can be applied to panic disorders (Newman et al., 1997), addictions (Riley at al., 2002), eating disorders (Norton et al., 2003) and many other health problems.

Last but not least, let’s focus on a classical ideographic study (Allport 1962) as these studies have influenced today’s ESM designs: Lorraine’s Week as presented by Larson and Csikszentmihalyi.

Lorraine (a pseudonym) was a high school senior. She had a psychological trauma during the week of her ESM reports. Lorraine couldn’t realize her plans to go to Spain, and what’s more, she couldn’t join her friends on a trip. Before that, according to her self-reports, Lorraine was very positive – talking to people, watching TV, and going out. After the bad news, her mood changed dramatically and for a short period of time, between 12.15 P.M. and 1.30 P.M., she went from happy to extremely sad. After those traumatic events, her mood stayed low, and she even reported that the pager used for signaling was a nuisance. In fact, she was happy only when she was with someone else. In the interviews after the study, she reported, ‘‘When I was with people, it was better.”

This study took a week, but still, it gave a clear picture of this girl’s everyday life and her experiences. The findings from her daily diary might seem too individual, but that’s not 100% true. In fact, they show a lot about people’s interactions. On top of that, some findings can be generalized and applied to other teenagers – listening to music and going out is something that most teens do.

In the end, by being statistically powerful, each ESM method has a lot of practical strength, and the results can be generalized.

To sum up, one of the biggest advantages of all ESM studies is that many researchers can achieve their goals within various clinical and organizational settings. Having the chance to work across various contexts is beneficial. On top of that, ESM studies have high ecological validity (Csikszentmihalyi et al. 2103; Shiffman et al. 2008). By designing a study that relies on technology, Smartphone apps, in particular, scientists have the chance to approach more people (Intille, 2007). ESM studies collect not only data – they can be used for sharing vital information, such as videos and geographical location.

       The only requirement is to keep ESM studies short as they can become too time-consuming. Also, they should not always focus on negative feelings as that negativity might trigger emotional pain or relapse (in the case of addictions). Follow-up interviews can help researchers eliminate this problem. Last but not least, a major concern of many researchers is acceptance and compliance. Thus, training and reward are important.

Interestingly enough, some findings show that participants are more open to ESM studies, especially when the testing methods are based on technology such as smartphone apps, when compared to clinicians and researchers. Thus, having a motivated and trained team is crucial for each ESM study and research in general. In the end, researchers and participants work together and form a single happy team.

Following simple steps can help researchers organize their thoughts and work. Doing research, reporting design, recruiting participants, conducting pilot-tests, focusing on data collection and implementation of findings and results, these are all fundamental steps for the successful design of your ESM study. ESM is a powerful methodology, which on top of that is becoming highly accessible, thanks to the use of Smartphones and apps.

And of course, do not forget about the specific ingredients of each scientific recipe: a drop of creativity and a lot of eagerness to learn from mistakes.

Allport, G. (1962). The general and the unique in psychological science. Journal of Personality.

Alpers (2009). Ambulatory assessment in panic disorder and specific phobia. Psychol Assess, 4, p. 476-485.

Barta, W., Tennen, H., & Litt, M. (2012). Measurement reactivity in diary research. In M. R. Mehl & T. S. Conner (Eds.), Handbook of research methods for studying daily life. New York, NY: Guilford Press. In press.

Bergkvist, L., & Rossiter, J. R. (2009). Tailor-made single-item measures of doubly concrete constructs. International Journal of Advertising, 28 (4), p. 607-621.

Bolger, N., Davis, A., & Rafaeli, E. (2003). Diary methods: Capturing life as it us lived.  Annual Review of Psychology, p. 579–616.

Collins, R.L., Morsheimer, E.T., Shiffman, S., Paty, J.A., Gnys, M., & Papandonatos, G. (1998). Ecological momentary assessment in a behavioral drinking moderation program. Experimental and Clinical Psychopharmacology, p.306–315.

Conner, T.S. (2015). Experience Sampling Methodology for Psychological Scientists. Workshop presented at the meeting of the Association for Psychological Science, New York, NY.

Conner, T. S., & Lehman, B. (2012). Getting started: Launching a study in daily life.  In M. R. Mehl and T. S. Conner (Eds.), Handbook of research methods for studying daily life (89 –107). New York, New York: Guilford Press

Conner Christensen, Wood, & Barrett: (2003). Remembering everyday events through the prism of self-esteem. Personality and Social Psychology Bulletin, p. 51–62.

Côté, S., & Moskowitz, D. S. (1998). On the dynamic covariation between interpersonal behavior and affect: Prediction from neuroticism, extraversion, and agreeableness.  Journal of Personality and Social Psychology, 75, p.1032‐1046.

Csikszentmihalyi M, Mehl, M.R., & Conner, T.S. (2013). Handbook of research methods for studying daily life.   New York, NY: Guilford Publications.

Ebner-Priemer, U.W., Eid, M., Stabenow, S., Kleindienst, N., & Trull, T. (2008). Analytic strategies for understanding affective (in)stability and other dynamic processes in psychopathology. Journal of Abnormal Psychology, p.195–202.

Fisher, C., & March, L. (2012). Using experience sampling methodology in organizational behavior.  Journal of Organizational Psychology.

Hektner, J.M., Schmidt, J, & Csikszentmihalyi, M. (2007).  Experience sampling method: Measuring the quality of everyday life.  Thousand Oaks, CA: Sage Publications.

Intille, S. S. (2007). Technological  innovations enabling automatic, context-sensitive ecological momentary assessment. In A. A. Stone, S. Shiffman, A. A. Atienza, & L. Nebeling (Eds.), The science of real-time data capture (pp. 308–337). New York, NY: Oxford University Press.

Jonasson, G., Carlsen, K., Sodal, A., Jonasson, C., & Mowinckel, P. (1999). Patient compliance in a clinical trial with inhaled budesonide in children with mild asthma. Eur. Respir. p.150–154.

Kamarck, T. W., Shiffman, S.M., Smithline, L., Goodie, J.L., Thompson, H.S., Ituarte, P.H.G., Jong, J.Y., Pro, V., Paty, J.A., Kassel, J.D., Gnys, M., et al. (1998).  The Diary of Ambulatory Behavioral States: a new approach to the assessment pf psychosocial influences on ambulatory cardiovascular activity. In D. S. Krantz, & A. Baum, (Ed.), Perspectives in behavioral medicine: technology and methods in behavioral medicine (pp. 163-194). Mahwah, NJ: Lawrence Erlbaum.

Lewin, K. (1935). A Dynamic Theory of Personality. New York: McGraw-Hill.

McCarthy, D.E., Piasecki, T.M., Fiore, M.C., & Baker, T.B. (2006).  Life before and after quitting smoking: an electronic diary study. J. Abnorm. Psychol. 115:454–66.

Myin-Germeys, I., Birchwood, M., & Kwapil, T. (2011). From environment to therapy in psychosis: a real-world momentary assessment approach.  Schizophr Bull; 37:244–7.

Moskowitz, D.S., & Sadikaj, G. (2011). Event-contingent reporting.  In M. R. Mehl, & T. A. Conner (Eds.), Handbook of research methods for studying daily life (pp. 160–175). New York, NY: Guilford Press.

Newman, M.G., Kenardy, J., Herman, S., & Taylor, C.B. (1997). Comparison of cognitive behavioral treatment of panic disorder with computer-assisted brief cognitive behavior treatment. J. Consult. Clin. Psychol. 65:178–83.

Norton, M., Wonderlich, S.A., Myers, T., Mitchell, J.E., & Crosby R.D. (2003). The use of palmtop computers in the treatment of bulimia nervosa.  Eur. Eat. Disord., 11:231–42.

Riley, W.T., Jerome, A., Behar, A., & Weil, J. (2002). Computer and manual self-help behavioral strategies for smoking reduction: initial feasibility and one-year follow-up.   Nicotine Tobacco Res. 4:S183–88.

Schneider, B., & Waite, L. (Eds.). (2005). Being together, working apart: Dual career families and the work-life balance.  Cambridge, UK: Cambridge University Press.

Shiffman, S. (2007). Designing protocols for ecological momentary assessment. In: Stone AA, Shiffman S, Atienza A, Nebeling L, editors. The Science of real-time data capture: Self-reports in health research. New York: Oxford University Press; p. 27–53.

Shiffman, S., Engberg, J., Paty, J.A., Perz,W., Gnys, M., et al. (1997). A day at a time: predicting smoking lapse from daily urge.  J. Abnorm. Psychol. 106:104–16.

Shrout, P. E., & Lane, S. P. (2011). Psychometrics.  In M. R. Mehl, & T. A. Conner (Eds.), Handbook of research methods for studying daily life (pp. 302–320). New York, NY: Guilford Press

Snijders & Bosker (1999). Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling.  Thousand Oaks, CA: Sage.

Stiglmayr, C., Grathwol, T., Linehan, M., Ihorst, G., Fahrenberg, J., & Bohus, M. (2005). Aversive tension in patients with borderline personality disorder: a computer-based controlled field study.  Acta Psychiatrica Scandinavica, p. 372–379.

Taylor, B.C., Fried, L., & Kenardy, J. (1990). The use of a real-time diary for data acquisition and processing. Behav. Res. Ther. 28:92–97.

Trull, T.J., Solhan, M.B., Tragesser, S.L., Jahng, S., Wood, P.K., Piasecki, T.M., & Watson, D. (2008). Affective instability: Measuring a core feature of borderline personality disorder with ecological momentary assessment.   Journal of Abnormal Psychology.

Van Hooff, M.L.M., Geurts, S.A.E., Kompier, M.A.J. & Taris, T.W. (2007). Workdays, in-between workdays, and the weekend: A diary study on effort and recovery. International Archives of Occupational and Environmental Health, 80, p. 599-613.

Wheeler, L. & Reis, H. (1991). Self-recording of everyday life events: Origins, types, and uses. Journal of Personality, 59, p. 339–354.