The Experience Sampling Method (ESM) is one of the most reliable procedures for studying daily lives and people’s experiences, activities, and feelings throughout the day. With the great idea of Kurt Lewin (1935) to create a topology of daily activities and the ambitious techniques created by Roger Barker and Herbert Wright, ESM studies have become incredibly popular today.

Each research method and findings benefit science, for sure. From longitudinal studies to archival research, adding more to the existing knowledge is always beneficial. However, ESM has numerous advantages when compared to other techniques in the field of biology, psychology, and medicine. For instance, while laboratory experiments limit research with the artificiality of some procedures, ESM do not take subjects out of their natural environment. In fact, that’s why ESM studies are called “ecological” – because all research and observation take place within natural settings, in “real-time” and “real-life” situations (Schiffman, Stone, and Hufford, 2008). On the other hand, while observational studies cannot eliminate the influence of the observer completely, ESM rely on self-reports and people’s active participation. In other words, ESM studies give researchers the chance to implement controlled and natural settings at the same time.

ESM studies are applicable to a wide range of scientific topics, such as panic attacks, medical interventions, pain management, reflecting on feelings, etc., they can be used within organizational settings as well. As mentioned earlier, ESM studies are among the most valuable research assets to science.

2.2.1. Research Hypothesis:

After the hard work, it’s time for a more precise design. You’re the designer, so the beauty of your research lies in your hands. Formulate a hypothesis – a hypothesis that’s in fact testable. In order to do that, systematize all variables that might have an impact on the study. Decide if you want to study behaviors, such as drinking or eating, experiences, such as mood or cognition, or physiological functions, such as breathing or hormonal levels. Having a clear operational definition is important because – by having a clearer idea on what’s being studied – you have more control on the process.

2.2.2. Define Your Variables:

As you know, the relation between the variables of interest will be tested in your hypothesis. Hypotheses usually focus on various dynamic within-person interactions. The best way to achieve a meaningful result is to study variables that change over a short period of time (Conner & Lehman, 2012). Do not forget that having a null hypothesis is also important. Last but not least, remember that even by rejecting your hypothesis, you contribute to science.

2.2.3. Statistical Procedures:

Having a clear idea on how to analyze the collected data is also very important. Simply because sometimes we might have data, which we cannot use, or too many figures we do not know how to analyze. Collecting, recording, interpreting and presenting data is vital, and research should be factual and based on statistical procedures.

Note that ESM can be a great quantitative method. Simply because all techniques provide large numbers of data. A method that includes multiple questions, multiple contexts and multiples times every single day is definitely a research technique that is a dream.

On the other hand, ESM studies can be used as a wonderful qualitative method because each participant self-reports their feelings, behaviors, and thoughts in their everyday life.

Do not forget to include charts and tables for the visual representation of your data.

2.2.4. ESM Design:

Then, it’s time for another major aspect of your ESM study: choose an ESM design. If having a hypothesis and data is needed in each experiment, ESM designs are a unique characteristic applicable only to ESM studies. We’ll discuss this section and the different types of ESM design later.

2.2.5. Reporting Methods:

Just like ESM designs, the different reporting methods used in ESM studies are also unique. When one starts designing a study, they are often overwhelmed by ideas and expected results and applications. However, the practical side of your study should also be considered and planned in details. In addition, you have to decide if your study will benefit from an active or passive reporting method.  An example of an active reporting method is daily diaries, while a passive method is neuroendocrine sampling, for example.  Choosing the right reporting method is crucial. Note that Smartphone Apps engage participants the most, and in today’s society, even older participants have no problems using technology. This section will also be discussed at a later stage.

Another important aspect of the practical side of your ESM study is to create meaningful and engaging questions. Note that questions for ESM studies should not be time-consuming. As participants will be asked to report behavior, thoughts, and other issues several times per day (for one or more weeks), questions should not become a burden (emotionally or cognitively).

When deciding on the number of questions needed, researchers should focus on how motivated their participants are, how long the study is going to take, and how much effort the testing itself requires. Note that researchers have accepted a burden cap of 20 minutes per day (Conner, 2015).  Formulate the questions correctly – ESM studies should not ask people for their experiences in general, but for current events. Usually, researchers can convert a traditional study into an ESM measure to ensure good validity and reliability.

The actual text of the questions should be short, especially if small screens of mobile phones or personal digital assistants (PDAs) are used as preferred reporting methods.

2.3.1. Is It Reliable?

Well done on the hard job of creating a questionnaire! However, hold on for a second. It’s time to ask yourself: Are the ESM scales used in the study reliable? Note that some researchers claim that reliability matters only in the service of validity (Bergkvist & Rossiter, 2009). Still, experts are concerned the most about the following problem: Do ESM items measure between-person variations only, instead of within-person factors (Shrout & Lane, 2011)?

The first step to check if the created questionnaire shows good reliability is to report Cronbach’s alpha (especially when multi-item scales are present). When we create questions, we should remember that at least three items for each ESM construct are needed (Shrout and Lane, 2011). Single items work fine as well, for example, a 7- to 10-point scale or a 0-100 slider scale, which in addition is a method that can be used to increase variance (Van Hooff, Geurts, Kompier, and Taris, 2007). However, complicated constructs that change over a longer time span should be assessed with multi-item scales.

2.4. Participants:

After that, it’s time to focus on the most important aspect of each research – participants. We have to agree that building up on knowledge for the pure purpose of knowledge is intriguing. However, the human factor is the most important part of a research. By applying the existing knowledge to everyday life, scientists can improve people’s well-being and interactions. And that’s what matters the most.

Researchers might face various difficulties when experts choose participants. Factors, such as sample size and demographics, are very important. Usually, ESM studies test the same participants and their daily lives, with variables that fluctuate over the short term. You might consider even recruiting and motivating your research participants (Conner & Lehman, 2011). Having intrinsically and extrinsically motivated participants is one of the factors that will influence your results positively. Rewarding your participants might be extremely helpful.

Note that when it comes to participants and ESM studies, researchers worry if constant self-reporting can affect respondents’ perceptions. However, we can look at this effect as one of the main advantages of ESM testing. Simply because frequent self-monitoring can be used as a therapeutic intervention, creating reactivity (Barta, Tennen, & Litt, 2011).

2.5. Pilot Testing

When all research plans are finalized, it’s time to give your research endeavors a go. Pilot testing is more than beneficial. Choose a group of people similar to the intended participants of your ESM study. However, note that human beings are unique and what works fine for one sample might go wrong for another.

If you think that pilot testing is just a waste of time, you should think twice. Did you know that Schneider and Waite (2005) did three pilot-tests of their national study of families? Pilot testing can give you valuable insights on question-wording, length and other crucial aspects of your ESM questionnaire. Not only that, though. The post-collection procedures and analysis of data, such as descriptive statistics, can show researchers some problematic parameters of the questionnaire, such as coding, that was not obvious during administration.

Most of all, pilot testing is a great way to connect to all participants. The size of the pilot sample depends on the purposes of the full-scale study. In some cases, 10-20 pilot subjects are enough, in others, smaller samples are also acceptable. Although ESM studies don’t take too much time, real testing might interrupt some important daily activities, which as a result may affect the intrinsic motivation of the subjects. Thus, listening to people’s feedback and analyzing their notes taken during the study is important as it can help researchers improve their ESM study design. For instance, always ask if: response options were meaningful; signals were clear; study situations were difficult to understand; testing procedure was frustrating;, etc. From giving instructions to signaling methods, pilot-tests contain a lot of useful information and may give some humane touch to the ESM design.

Also remember that when you thrive for perfection, you should embrace errors. Focus on missing or noncompliant reports and adopt procedures to battle similar problems that may arise in the future. Ask participants if items were too irrelevant or too confusing. Make sure that the procedure used in the study wasn’t too difficult to administer, or check if signals were unclear. At the same time, create a more realistic testing environment: encourage pilot subjects to give false data or to miss a response from time to time. All these problematic situations are things that could happen even to the best ESM study designs.

Do not forget that research assistants also have to undergo some training so they can relate to their participants and understand the testing procedure better. In other words, even the “pilots” of the study have to go through the pilot testing procedure.

Also, when you choose a tech device for a reporting method, have a spare in case something fails or breaks. The signaling equipment and the procedures for dealing with data are also a subject of errors. Mind that plans don’t always go smoothly and learn from results!