Why ESM?……………………………..

  1. Design Your ESM Study…………..

2.1. Aim………………………………………….

2.1.1Aim for an Aim

2.1.2. Do Your Research

2.2. Designer’s Choice………………………

2.2.1 Research Hypothesis

2.2.2. Define Your Variables

2.2.3 Statistical Procedures

2.2.4 ESM Designs

2.2.5 Reporting Methods

2.3. Questions…………………………………..

2.3.1Is It Reliable?

2.4. Participants…………………………………

2.5. Pilot Study………………………………….

  1. ESM Designs…………………………….

3.1. Signal-Contingent Studies…………….

3.2. Interval-Contingent Studies…………..

3.3. Event-Contingent Studies……………..

3.4. Continuous Reporting…………………..

3.5 In a Nutshell

4: Reporting Methods…………………….

4.1 Emoticons & Research……………………….

  1. Applications of ESM………………….
  2. Classical Ideographic study
  3. Pros & Cons………………………………………
  4. References………………………………………………..


Why Experience Sampling Methods?

The Experience Sampling Method (ESM) is one of the most reliable procedures for studying daily lives and people’s experiences, activities, and feelings throughout the day. With the great idea of Kurt Lewin (1935) to create a topology of daily activities and the ambitious techniques created by Roger Barker and Herbert Wright, ESM studies have become incredibly popular today.

Each research method and findings benefit science, for sure. From longitudinal studies to archival research, adding more to the existing knowledge is always beneficial. However, ESM has numerous advantages when compared to other techniques in the field of biology, psychology, and medicine. For instance, while laboratory experiments limit research with the artificiality of some procedures, ESM do not take subjects out of their natural environment. In fact, that’s why ESM studies are called “ecological” – because all research and observation take place within natural settings, in “real-time” and “real-life” situations (Schiffman, Stone, and Hufford, 2008). On the other hand, while observational studies cannot eliminate the influence of the observer completely, ESM rely on self-reports and people’s active participation. In other words, ESM studies give researchers the chance to implement controlled and natural settings at the same time.

ESM studies are applicable to a wide range of scientific topics, such as panic attacks, medical interventions, pain management, reflecting on feelings, etc., they can be used within organizational settings as well. As mentioned earlier, ESM studies are among the most valuable research assets to science.

Design Your ESM Study

2.1. Aim

You’re probably already familiar with ESM studies and their advantages, and what you actually need is some practical help with designing an ESM study. When doing that, few basic steps should be followed:

2.1.1. Aim for an Aim:

Having a clear goal is the first step that can take your experiment towards success. In other words, having an identified aim can give a powerful kick to your research journey. In the midst of scientific ideas, experimental curiosities and philosophical mysteries revolving around the human nature, having research questions that can actually be tested is crucial. Being realistic and specific is the only way to get real results with some statistical strength.

2.1.2. Do Your Research:

Having an aim will open the door to research. Conducting in-depth research is crucial. From online sources to journals, our world is full of information. Do not set any limits. Literature research might be time-consuming but expand on your study by focusing on references and related topics. Background research can help scientists generate more ideas and investigate new aspects of the topic. That said, brainstorming ideas within academia and other communities might also contribute to the field of research.

2.2. Designer’s Choice:

2.2.1. Research Hypothesis:

After the hard work, it’s time for a more precise design. You’re the designer, so the beauty of your research lies in your hands. Formulate a hypothesis – a hypothesis that’s in fact testable. In order to do that, systematize all variables that might have an impact on the study. Decide if you want to study behaviors, such as drinking or eating, experiences, such as mood or cognition, or physiological functions, such as breathing or hormonal levels. Having a clear operational definition is important because – by having a clearer idea on what’s being studied – you have more control on the process.

2.2.2. Define Your Variables:

As you know, the relation between the variables of interest will be tested in your hypothesis. Hypotheses usually focus on various dynamic within-person interactions. The best way to achieve a meaningful result is to study variables that change over a short period of time (Conner & Lehman, 2012). Do not forget that having a null hypothesis is also important. Last but not least, remember that even by rejecting your hypothesis, you contribute to science.

2.2.3. Statistical Procedures:

Having a clear idea on how to analyze the collected data is also very important. Simply because sometimes we might have data, which we cannot use, or too many figures we do not know how to analyze. Collecting, recording, interpreting and presenting data is vital, and research should be factual and based on statistical procedures.

Note that ESM can be a great quantitative method. Simply because all techniques provide large numbers of data. A method that includes multiple questions, multiple contexts and multiples times every single day is definitely a research technique that is a dream.

On the other hand, ESM studies can be used as a wonderful qualitative method because each participant self-reports their feelings, behaviors, and thoughts in their everyday life.

Do not forget to include charts and tables for the visual representation of your data.

2.2.4. ESM Design:

Then, it’s time for another major aspect of your ESM study: choose an ESM design. If having a hypothesis and data is needed in each experiment, ESM designs are a unique characteristic applicable only to ESM studies. We’ll discuss this section and the different types of ESM design later.

2.2.5. Reporting Methods:

Just like ESM designs, the different reporting methods used in ESM studies are also unique. When one starts designing a study, they are often overwhelmed by ideas and expected results and applications. However, the practical side of your study should also be considered and planned in details. In addition, you have to decide if your study will benefit from an active or passive reporting method.  An example of an active reporting method is daily diaries, while a passive method is neuroendocrine sampling, for example.  Choosing the right reporting method is crucial. Note that Smartphone Apps engage participants the most, and in today’s society, even older participants have no problems using technology. This section will also be discussed at a later stage.

2.3. Questions:

Another important aspect of the practical side of your ESM study is to create meaningful and engaging questions. Note that questions for ESM studies should not be time-consuming. As participants will be asked to report behavior, thoughts, and other issues several times per day (for one or more weeks), questions should not become a burden (emotionally or cognitively).

When deciding on the number of questions needed, researchers should focus on how motivated their participants are, how long the study is going to take, and how much effort the testing itself requires. Note that researchers have accepted a burden cap of 20 minutes per day (Conner, 2015).  Formulate the questions correctly – ESM studies should not ask people for their experiences in general, but for current events. Usually, researchers can convert a traditional study into an ESM measure to ensure good validity and reliability.

The actual text of the questions should be short, especially if small screens of mobile phones or personal digital assistants (PDAs) are used as preferred reporting methods.

2.3.1. Is It Reliable?

Well done on the hard job of creating a questionnaire! However, hold on for a second. It’s time to ask yourself: Are the ESM scales used in the study reliable? Note that some researchers claim that reliability matters only in the service of validity (Bergkvist & Rossiter, 2009). Still, experts are concerned the most about the following problem: Do ESM items measure between-person variations only, instead of within-person factors (Shrout & Lane, 2011)?

The first step to check if the created questionnaire shows good reliability is to report Cronbach’s alpha (especially when multi-item scales are present). When we create questions, we should remember that at least three items for each ESM construct are needed (Shrout and Lane, 2011). Single items work fine as well, for example, a 7- to 10-point scale or a 0-100 slider scale, which in addition is a method that can be used to increase variance (Van Hooff, Geurts, Kompier, and Taris, 2007). However, complicated constructs that change over a longer time span should be assessed with multi-item scales.

2.4. Participants:

After that, it’s time to focus on the most important aspect of each research – participants. We have to agree that building up on knowledge for the pure purpose of knowledge is intriguing. However, the human factor is the most important part of a research. By applying the existing knowledge to everyday life, scientists can improve people’s well-being and interactions. And that’s what matters the most.

Researchers might face various difficulties when experts choose participants. Factors, such as sample size and demographics, are very important. Usually, ESM studies test the same participants and their daily lives, with variables that fluctuate over the short term. You might consider even recruiting and motivating your research participants (Conner & Lehman, 2011). Having intrinsically and extrinsically motivated participants is one of the factors that will influence your results positively. Rewarding your participants might be extremely helpful.

Note that when it comes to participants and ESM studies, researchers worry if constant self-reporting can affect respondents’ perceptions. However, we can look at this effect as one of the main advantages of ESM testing. Simply because frequent self-monitoring can be used as a therapeutic intervention, creating reactivity (Barta, Tennen, & Litt, 2011).

2.5. Pilot Testing

When all research plans are finalized, it’s time to give your research endeavors a go. Pilot testing is more than beneficial. Choose a group of people similar to the intended participants of your ESM study. However, note that human beings are unique and what works fine for one sample might go wrong for another.

If you think that pilot testing is just a waste of time, you should think twice. Did you know that Schneider and Waite (2005) did three pilot-tests of their national study of families? Pilot testing can give you valuable insights on question-wording, length and other crucial aspects of your ESM questionnaire. Not only that, though. The post-collection procedures and analysis of data, such as descriptive statistics, can show researchers some problematic parameters of the questionnaire, such as coding, that was not obvious during administration.

Most of all, pilot testing is a great way to connect to all participants. The size of the pilot sample depends on the purposes of the full-scale study. In some cases, 10-20 pilot subjects are enough, in others, smaller samples are also acceptable. Although ESM studies don’t take too much time, real testing might interrupt some important daily activities, which as a result may affect the intrinsic motivation of the subjects. Thus, listening to people’s feedback and analyzing their notes taken during the study is important as it can help researchers improve their ESM study design. For instance, always ask if: response options were meaningful; signals were clear; study situations were difficult to understand; testing procedure was frustrating;, etc. From giving instructions to signaling methods, pilot-tests contain a lot of useful information and may give some humane touch to the ESM design.

Also remember that when you thrive for perfection, you should embrace errors. Focus on missing or noncompliant reports and adopt procedures to battle similar problems that may arise in the future. Ask participants if items were too irrelevant or too confusing. Make sure that the procedure used in the study wasn’t too difficult to administer, or check if signals were unclear. At the same time, create a more realistic testing environment: encourage pilot subjects to give false data or to miss a response from time to time. All these problematic situations are things that could happen even to the best ESM study designs.

Do not forget that research assistants also have to undergo some training so they can relate to their participants and understand the testing procedure better. In other words, even the “pilots” of the study have to go through the pilot testing procedure.

Also, when you choose a tech device for a reporting method, have a spare in case something fails or breaks. The signaling equipment and the procedures for dealing with data are also a subject of errors. Mind that plans don’t always go smoothly and learn from results!

ESM Study Designs

So you have your hypothesis, constructs to measure and samples of interest. As mentioned above, it’s time for you to find the right ESM design. There are two major types: event-based and interval-based designs (Shiffman et al. 2007; Wheeler and Reis, 1991). ESM designs are a crucial part in designing an ESM study.

Before you choose a protocol, try to familiarize yourself with the different types of ESM design in order to choose the most appropriate protocol for your study.

Event-based studies focus on various episodes in people’s lives and are effective for the measurement of events, such as panic attacks (Taylor et al., 1990), intake of medicines (Jonasson et al., 1999), cravings (Shiffman et al., 1997), etc.

On the other hand, time-based studies test people’s experiences over various events and are effective for the measurement of pain, blood pressure (Kamarck et al., 1998), etc. A lot of medical researchers use ESM designs, finding these studies effective for the measurement of various physiological changes.

Of course, there are combined methods that can be used effectively in some complicated research topics.

In fact, ESM designs are always sophisticated. We can expand on the topic by dividing the types of design into interval-contingent reporting, signal-contingent reporting, event-contingent reporting, continuous reporting and combinations of these approaches (Bolger, Davis, & Rafaeli, 2003).